zig-mobile-runner 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +484 -0
- package/CONTRIBUTING.md +42 -0
- package/FEATURES.md +112 -0
- package/LICENSE +21 -0
- package/README.md +255 -0
- package/SECURITY.md +34 -0
- package/build.zig +38 -0
- package/build.zig.zon +7 -0
- package/clients/README.md +144 -0
- package/clients/go/README.md +24 -0
- package/clients/go/examples/fake-session/main.go +93 -0
- package/clients/go/go.mod +3 -0
- package/clients/go/zmr/client.go +432 -0
- package/clients/kotlin/README.md +35 -0
- package/clients/kotlin/build.gradle.kts +35 -0
- package/clients/kotlin/settings.gradle.kts +15 -0
- package/clients/kotlin/src/main/kotlin/dev/zmr/FakeSession.kt +86 -0
- package/clients/kotlin/src/main/kotlin/dev/zmr/ZmrClient.kt +67 -0
- package/clients/python/README.md +29 -0
- package/clients/python/examples/fake_session.py +48 -0
- package/clients/python/pyproject.toml +13 -0
- package/clients/python/zmr_client.py +202 -0
- package/clients/rust/Cargo.lock +107 -0
- package/clients/rust/Cargo.toml +10 -0
- package/clients/rust/README.md +19 -0
- package/clients/rust/examples/fake_session.rs +70 -0
- package/clients/rust/src/lib.rs +461 -0
- package/clients/swift/Package.swift +16 -0
- package/clients/swift/README.md +36 -0
- package/clients/swift/Sources/ZMRClient/ZMRClient.swift +114 -0
- package/clients/swift/Sources/ZMRFakeSession/main.swift +86 -0
- package/clients/typescript/README.md +34 -0
- package/clients/typescript/examples/fake-session.mjs +36 -0
- package/clients/typescript/index.d.ts +144 -0
- package/clients/typescript/index.mjs +192 -0
- package/clients/typescript/package.json +8 -0
- package/docs/adr/0001-agent-native-runner-boundary.md +31 -0
- package/docs/adr/0002-app-local-zmr-contract.md +39 -0
- package/docs/adr/0003-ios-simulator-xctest-shim.md +41 -0
- package/docs/adr/0004-benchmark-claims-and-baseline-collection.md +37 -0
- package/docs/adr/README.md +12 -0
- package/docs/ai-agents.md +156 -0
- package/docs/app-integration.md +316 -0
- package/docs/benchmarking.md +275 -0
- package/docs/client-installation.md +141 -0
- package/docs/clients.md +98 -0
- package/docs/config.md +175 -0
- package/docs/demo.md +259 -0
- package/docs/dsl.md +57 -0
- package/docs/install.md +233 -0
- package/docs/market-positioning.md +70 -0
- package/docs/npm.md +359 -0
- package/docs/protocol-fixtures/README.md +8 -0
- package/docs/protocol-fixtures/core-session.requests.jsonl +8 -0
- package/docs/protocol-fixtures/core-session.responses.jsonl +8 -0
- package/docs/protocol-versioning.md +65 -0
- package/docs/protocol.md +560 -0
- package/docs/publication.md +77 -0
- package/docs/release-audit.md +99 -0
- package/docs/release-candidate.md +111 -0
- package/docs/release-evidence.md +188 -0
- package/docs/release-notes-template.md +58 -0
- package/docs/roadmap.md +334 -0
- package/docs/scenario-authoring.md +88 -0
- package/docs/shipping.md +170 -0
- package/docs/trace-privacy.md +88 -0
- package/docs/troubleshooting.md +256 -0
- package/examples/android-app-auth-probe.json +89 -0
- package/examples/android-app-error-state.json +13 -0
- package/examples/android-app-login-smoke.json +192 -0
- package/examples/android-app-onboarding.json +12 -0
- package/examples/android-app-referral-deep-link.json +12 -0
- package/examples/android-shim-smoke.json +19 -0
- package/examples/demo-failure.json +12 -0
- package/examples/demo-fake.json +14 -0
- package/examples/ios-dev-client-open-link.json +26 -0
- package/examples/ios-dev-client-route-snapshot.json +24 -0
- package/examples/ios-shim-smoke.json +23 -0
- package/examples/ios-smoke.json +9 -0
- package/go.work +3 -0
- package/npm/agents.mjs +183 -0
- package/npm/app-config.mjs +95 -0
- package/npm/build-zmr.mjs +21 -0
- package/npm/commands.mjs +104 -0
- package/npm/generated-files.mjs +50 -0
- package/npm/index.mjs +75 -0
- package/npm/init-app.mjs +80 -0
- package/npm/package-scripts.mjs +72 -0
- package/npm/postinstall.mjs +21 -0
- package/npm/scaffold.mjs +179 -0
- package/npm/scenarios.mjs +93 -0
- package/npm/setup.mjs +69 -0
- package/npm/wizard.mjs +117 -0
- package/npm/zmr.mjs +23 -0
- package/package.json +114 -0
- package/prebuilds/darwin-arm64/zmr +0 -0
- package/prebuilds/darwin-x64/zmr +0 -0
- package/prebuilds/linux-arm64/zmr +0 -0
- package/prebuilds/linux-x64/zmr +0 -0
- package/schemas/README.md +26 -0
- package/schemas/action-result.schema.json +27 -0
- package/schemas/capabilities-output.schema.json +98 -0
- package/schemas/devices-output.schema.json +25 -0
- package/schemas/doctor-output.schema.json +51 -0
- package/schemas/explain-output.schema.json +51 -0
- package/schemas/import-output.schema.json +23 -0
- package/schemas/init-output.schema.json +71 -0
- package/schemas/json-rpc.schema.json +55 -0
- package/schemas/release-manifest.schema.json +43 -0
- package/schemas/release-readiness-output.schema.json +127 -0
- package/schemas/run-output.schema.json +43 -0
- package/schemas/scenario.schema.json +128 -0
- package/schemas/schemas-output.schema.json +26 -0
- package/schemas/semantic-snapshot.schema.json +116 -0
- package/schemas/snapshot.schema.json +60 -0
- package/schemas/trace-event.schema.json +14 -0
- package/schemas/trace-manifest.schema.json +59 -0
- package/schemas/validate-output.schema.json +42 -0
- package/schemas/version-output.schema.json +23 -0
- package/schemas/zmr-config.schema.json +75 -0
- package/scripts/android-emulator.sh +126 -0
- package/scripts/assert-ios-physical-ready.sh +213 -0
- package/scripts/benchmark-command.sh +307 -0
- package/scripts/benchmark.sh +359 -0
- package/scripts/benchmark_gate.py +117 -0
- package/scripts/benchmark_result_row.py +88 -0
- package/scripts/compare-benchmarks.py +288 -0
- package/scripts/create-android-demo-app.sh +342 -0
- package/scripts/create-ios-demo-app.sh +261 -0
- package/scripts/demo-android-real.sh +232 -0
- package/scripts/demo-ios-real.sh +270 -0
- package/scripts/demo.sh +464 -0
- package/scripts/device-matrix.sh +338 -0
- package/scripts/ensure-ios-shim-target.rb +237 -0
- package/scripts/install-android-shim.sh +281 -0
- package/scripts/install-ios-shim.sh +589 -0
- package/scripts/pilot-gate.sh +560 -0
- package/scripts/release-readiness.py +838 -0
- package/scripts/release-readiness.sh +91 -0
- package/scripts/run-android-pilot.sh +561 -0
- package/scripts/run-ios-pilot.sh +509 -0
- package/shims/android/README.md +21 -0
- package/shims/android/ZMRShimInstrumentedTest.java +152 -0
- package/shims/android/protocol.md +18 -0
- package/shims/ios/README.md +50 -0
- package/shims/ios/ZMRShim.swift +110 -0
- package/shims/ios/ZMRShimUITestCase.swift +475 -0
- package/shims/ios/protocol.md +74 -0
- package/skills/zmr-mobile-testing/SKILL.md +127 -0
- package/src/android.zig +344 -0
- package/src/android_device_info.zig +99 -0
- package/src/android_emulator.zig +154 -0
- package/src/android_screen_recording.zig +112 -0
- package/src/android_shell.zig +112 -0
- package/src/bundle.zig +124 -0
- package/src/bundle_redaction.zig +272 -0
- package/src/bundle_tar.zig +123 -0
- package/src/cli_devices.zig +97 -0
- package/src/cli_doctor.zig +114 -0
- package/src/cli_import.zig +70 -0
- package/src/cli_info.zig +39 -0
- package/src/cli_init.zig +72 -0
- package/src/cli_output.zig +467 -0
- package/src/cli_run.zig +259 -0
- package/src/cli_serve.zig +287 -0
- package/src/cli_trace.zig +111 -0
- package/src/cli_validate.zig +41 -0
- package/src/command.zig +211 -0
- package/src/config.zig +305 -0
- package/src/config_diagnostics.zig +212 -0
- package/src/config_paths.zig +49 -0
- package/src/device_registry.zig +37 -0
- package/src/doctor.zig +412 -0
- package/src/doctor_hints.zig +52 -0
- package/src/errors.zig +55 -0
- package/src/fake_device.zig +163 -0
- package/src/health.zig +28 -0
- package/src/importer.zig +343 -0
- package/src/importer_json.zig +100 -0
- package/src/importer_model.zig +103 -0
- package/src/ios.zig +399 -0
- package/src/ios_devices.zig +219 -0
- package/src/ios_lifecycle.zig +72 -0
- package/src/ios_shim.zig +242 -0
- package/src/ios_snapshot.zig +20 -0
- package/src/json_fields.zig +80 -0
- package/src/json_rpc.zig +150 -0
- package/src/json_rpc_methods.zig +318 -0
- package/src/json_rpc_observation.zig +31 -0
- package/src/json_rpc_params.zig +52 -0
- package/src/json_rpc_protocol.zig +110 -0
- package/src/json_rpc_trace.zig +73 -0
- package/src/main.zig +135 -0
- package/src/mcp.zig +234 -0
- package/src/mcp_protocol.zig +64 -0
- package/src/mcp_trace.zig +83 -0
- package/src/report.zig +346 -0
- package/src/report_html.zig +63 -0
- package/src/report_values.zig +27 -0
- package/src/run_options.zig +152 -0
- package/src/runner.zig +280 -0
- package/src/runner_actions.zig +109 -0
- package/src/runner_config.zig +6 -0
- package/src/runner_diagnostics.zig +268 -0
- package/src/runner_events.zig +170 -0
- package/src/runner_native.zig +88 -0
- package/src/runner_waits.zig +300 -0
- package/src/scaffold.zig +472 -0
- package/src/scenario.zig +346 -0
- package/src/scenario_fields.zig +50 -0
- package/src/schema_registry.zig +53 -0
- package/src/selector.zig +84 -0
- package/src/semantic.zig +171 -0
- package/src/trace.zig +315 -0
- package/src/trace_json.zig +340 -0
- package/src/trace_summary.zig +218 -0
- package/src/trace_summary_diagnostic.zig +202 -0
- package/src/types.zig +120 -0
- package/src/uiautomator.zig +164 -0
- package/src/validation.zig +187 -0
- package/src/version.zig +22 -0
- package/viewer/app.js +373 -0
- package/viewer/index.html +126 -0
- package/viewer/parser.js +233 -0
- package/viewer/styles.css +585 -0
package/docs/roadmap.md
ADDED
|
@@ -0,0 +1,334 @@
|
|
|
1
|
+
# ZMR Product Roadmap
|
|
2
|
+
|
|
3
|
+
**Goal:** Make Zig Mobile Runner a shippable open-source mobile test runner for AI agents, with Android production support first and iOS production support next.
|
|
4
|
+
|
|
5
|
+
**Product Thesis:** ZMR should feel like Playwright for mobile agents: fast typed control, rich observations, deterministic traces, and reliable isolation. It should beat shell-driven mobile automation by moving orchestration, waits, retries, traces, and protocol handling into a small Zig core while delegating platform-specific state and interaction to focused native adapters.
|
|
6
|
+
|
|
7
|
+
**Current State:** `0.1.0` local dev preview. Android pilot flows exist. iOS simulator lifecycle, snapshot, and selector actions are supported through `simctl` plus the XCTest shim. Coverage gate is above 90%. Demo traces can be generated without an emulator.
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## Definition Of Shippable
|
|
12
|
+
|
|
13
|
+
ZMR is shippable when a new developer can install it, run the no-device demo, connect an Android emulator, run the Android pilot wrapper, connect a booted iOS simulator, run the iOS smoke wrapper, inspect actionable traces, and integrate an external AI agent through JSON-RPC without manual code changes or hidden setup.
|
|
14
|
+
|
|
15
|
+
The first public product should be a **developer preview**, not a broad device-farm platform. It should be honest about limitations and strong on repeatability.
|
|
16
|
+
|
|
17
|
+
## Product Principles
|
|
18
|
+
|
|
19
|
+
- **Typed agent protocol first:** agents should consume structured snapshots and action results, not parse terminal output.
|
|
20
|
+
- **Deterministic traces:** every run should leave enough evidence to debug failure without rerunning immediately.
|
|
21
|
+
- **Fast local loop:** fake-device tests, fake demos, and simulator/emulator flows must run locally.
|
|
22
|
+
- **Adapter boundaries:** Android, iOS, and future platforms should share runner semantics without sharing platform hacks.
|
|
23
|
+
- **No embedded LLM:** ZMR is the control plane; external agents decide.
|
|
24
|
+
- **Security by default:** persisted JSON traces redact common secrets; publishing real visual artifacts requires explicit sanitizer work.
|
|
25
|
+
|
|
26
|
+
## Release Tracks
|
|
27
|
+
|
|
28
|
+
### Track A: `0.1.x` Dev Preview Hardening
|
|
29
|
+
|
|
30
|
+
Purpose: make the current repo credible for early users.
|
|
31
|
+
|
|
32
|
+
Ship criteria:
|
|
33
|
+
- `README.md` has install, demo, Android pilot, iOS smoke, app integration, and troubleshooting instructions.
|
|
34
|
+
- `docs/protocol.md` documents every JSON-RPC method, params, response shape, and error shape.
|
|
35
|
+
- `docs/shipping.md` has one copy-paste release gate.
|
|
36
|
+
- `scripts/demo.sh` passes on a clean macOS machine with Zig and no emulator.
|
|
37
|
+
- `scripts/coverage.sh` stays at or above 90%.
|
|
38
|
+
- Release archives include binary, README, license, protocol docs, demo scenarios, checksum file, and verifier.
|
|
39
|
+
|
|
40
|
+
Work:
|
|
41
|
+
- [x] Add `zmr doctor` to detect Zig, ADB, `xcrun`, emulator/simulator state, Android SDK paths, and missing permissions.
|
|
42
|
+
- [x] Add `zmr init` to scaffold a starter scenario.
|
|
43
|
+
- [x] Add stable JSON schemas for scenario files, snapshots, action results, and JSON-RPC payloads.
|
|
44
|
+
- [x] Add app-local `.zmr/config.json` schema and CLI default loading.
|
|
45
|
+
- [x] Add public error code taxonomy instead of exposing raw Zig error names.
|
|
46
|
+
- [x] Add install instructions for manual tarball/source install.
|
|
47
|
+
- [x] Add release notes template and changelog.
|
|
48
|
+
|
|
49
|
+
Verification:
|
|
50
|
+
- `zig fmt --check build.zig src`
|
|
51
|
+
- `bash -n scripts/*.sh tests/*.sh`
|
|
52
|
+
- `zig test src/main.zig -target aarch64-macos.15.0`
|
|
53
|
+
- `./scripts/demo.sh`
|
|
54
|
+
- `./scripts/coverage.sh`
|
|
55
|
+
- `./scripts/build-release.sh`
|
|
56
|
+
|
|
57
|
+
### Track B: Android Production Adapter
|
|
58
|
+
|
|
59
|
+
Purpose: move Android from useful pilot to product-grade reliability.
|
|
60
|
+
|
|
61
|
+
Ship criteria:
|
|
62
|
+
- Android sample app auth probe and login smoke flows pass repeatedly on a clean emulator.
|
|
63
|
+
- Failure traces identify selector misses, active package/activity, visible text, screenshots, logs, and timing.
|
|
64
|
+
- A native Android shim can provide hierarchy and interaction when ADB shell/UI Automator is too slow or flaky.
|
|
65
|
+
- Runner can install, launch, stop, clear state, deep link, tap, type, swipe, press back, wait, assert, and snapshot reliably.
|
|
66
|
+
|
|
67
|
+
Work:
|
|
68
|
+
- [x] Add Android shim project under `shims/android/`.
|
|
69
|
+
- [x] Implement faster hierarchy retrieval through Android accessibility/UI Automator APIs.
|
|
70
|
+
- [x] Implement reliable tap/type/swipe through the shim with ADB shell fallback.
|
|
71
|
+
- [x] Add app idle/settle detection instead of fixed sleeps where possible.
|
|
72
|
+
- [x] Add local emulator helper script for boot, wait-ready, snapshot save/load, and kill.
|
|
73
|
+
- [x] Add in-run emulator lifecycle orchestration: create, boot, wait-ready, reset, snapshot restore.
|
|
74
|
+
- [x] Pilot wrapper can reset the target emulator and boot an AVD from a named snapshot.
|
|
75
|
+
- [x] `zmr run` can boot an AVD, restore a snapshot, reset, and wait-ready before scenario execution.
|
|
76
|
+
- [x] `zmr run` can create a missing AVD from an installed Android system image.
|
|
77
|
+
- [x] Add Android artifact capture: screenshot, hierarchy, logcat window, package/activity, display metrics, optional screen recording.
|
|
78
|
+
- [x] Pilot wrapper can capture an optional MP4 screen recording for visual flake triage.
|
|
79
|
+
- [x] Snapshots include Android display density when available.
|
|
80
|
+
- [x] Traced Android runs can capture opt-in MP4 screen recordings.
|
|
81
|
+
- [x] Add Android flake harness that repeats pilot scenarios and emits pass rate/duration summaries.
|
|
82
|
+
|
|
83
|
+
Acceptance:
|
|
84
|
+
- `examples/android-app-auth-probe.json` passes 20/20 local runs.
|
|
85
|
+
- `examples/android-app-login-smoke.json` passes 20/20 local runs.
|
|
86
|
+
- Repeated-run reports show stable pass rate, useful failure diagnostics, and acceptable runtime on a clean emulator.
|
|
87
|
+
- All Android adapter unit/fake tests pass without an emulator.
|
|
88
|
+
|
|
89
|
+
### Track C: iOS Production Adapter
|
|
90
|
+
|
|
91
|
+
Purpose: keep the iOS simulator adapter at agent-grade quality and expand the supported surface deliberately.
|
|
92
|
+
|
|
93
|
+
Ship criteria:
|
|
94
|
+
- iOS supports structured hierarchy, selector matching, tap, type, swipe, press home/back-equivalent navigation, launch, stop, clear state, deep link, screenshot, logs, and snapshots.
|
|
95
|
+
- iOS implementation uses XCTest/XCUIAutomation boundaries instead of fragile coordinate-only shelling.
|
|
96
|
+
- iOS simulator smoke flow runs with the same scenario semantics as Android where platform concepts overlap.
|
|
97
|
+
|
|
98
|
+
Work:
|
|
99
|
+
- [x] Create app-local XCTest shim source and installer under `shims/ios/`.
|
|
100
|
+
- [x] Define a small local shim protocol for hierarchy, element query, tap, type, swipe, keyboard, and app state.
|
|
101
|
+
- [x] Add Zig-side iOS shim command and snapshot mapping tests.
|
|
102
|
+
- [x] Add Zig-side iOS shim process control for one-command local shim execution.
|
|
103
|
+
- [x] Map XCUIElement snapshots into `UiNode` with stable ids, labels, identifiers, values, enabled/visible flags, and bounds.
|
|
104
|
+
- [x] Add iOS selector tests using fake shim responses.
|
|
105
|
+
- [x] Add real simulator E2E smoke scenario with install/launch/openLink/snapshot/tap/type using the generated XCTest shim.
|
|
106
|
+
- [x] Add `examples/ios-dev-client-open-link.json` for Expo dev-client
|
|
107
|
+
simulator flows that need to open a local Metro bundle before selector waits.
|
|
108
|
+
- [x] Decide and document iOS clear-state semantics: best-effort app uninstall by bundle id; a missing app is already clean.
|
|
109
|
+
|
|
110
|
+
Acceptance:
|
|
111
|
+
- `examples/ios-smoke.json` passes on a real booted simulator with a test app.
|
|
112
|
+
- At least one selector-driven iOS flow passes repeatedly without manual interaction.
|
|
113
|
+
- iOS unsupported-action errors disappear for normal UI actions.
|
|
114
|
+
- iOS traces include screenshot, hierarchy, logs, active app id, timings, and assertions.
|
|
115
|
+
|
|
116
|
+
### Track D: Protocol And SDKs
|
|
117
|
+
|
|
118
|
+
Purpose: make ZMR easy for external AI agents and test harnesses to consume.
|
|
119
|
+
|
|
120
|
+
Ship criteria:
|
|
121
|
+
- JSON-RPC is stable and versioned.
|
|
122
|
+
- Clients can generate valid scenarios and validate snapshots using schemas.
|
|
123
|
+
- At least one reference client exists.
|
|
124
|
+
|
|
125
|
+
Work:
|
|
126
|
+
- [x] Add `schemas/` with JSON Schema files for scenario, snapshot, action result, trace event, and JSON-RPC messages.
|
|
127
|
+
- [x] Add protocol compatibility tests that load fixtures and assert exact response shapes.
|
|
128
|
+
- [x] Add a TypeScript reference client package in `clients/typescript/`.
|
|
129
|
+
- [x] Add a Python reference client package in `clients/python/`.
|
|
130
|
+
- [x] Add a Go reference client package in `clients/go/`.
|
|
131
|
+
- [x] Add a Rust reference client package in `clients/rust/`.
|
|
132
|
+
- [x] Add streaming observation/event mode for long-running agent sessions.
|
|
133
|
+
- [x] Add `trace.export` support for live RPC sessions, not only scenario runs.
|
|
134
|
+
- [x] Add semantic protocol version policy.
|
|
135
|
+
|
|
136
|
+
Acceptance:
|
|
137
|
+
- Reference clients can run demo scenarios through stdio.
|
|
138
|
+
- Protocol fixtures are committed and checked in CI.
|
|
139
|
+
- Breaking changes require protocol version changes and changelog entries.
|
|
140
|
+
|
|
141
|
+
### Track E: Trace Viewer And Diagnostics
|
|
142
|
+
|
|
143
|
+
Purpose: make failures easy to understand, not just logged.
|
|
144
|
+
|
|
145
|
+
Ship criteria:
|
|
146
|
+
- A failed run produces a portable trace bundle that a human can inspect quickly.
|
|
147
|
+
- Trace viewer shows timeline, screenshots, hierarchy, selected nodes, action inputs, logs, and assertion failures.
|
|
148
|
+
|
|
149
|
+
Work:
|
|
150
|
+
- [x] Define `trace.json` manifest for each run.
|
|
151
|
+
- [x] Add trace bundle export as `.zmrtrace` tar/zip.
|
|
152
|
+
- [x] Build a local static trace viewer under `viewer/`.
|
|
153
|
+
- [x] Add screenshot + UI tree side-by-side inspection.
|
|
154
|
+
- [x] Add replay controls for snapshot-linked trace frames.
|
|
155
|
+
- [x] Add selector miss diagnostics: nearest text matches, hidden/disabled candidates, offscreen hints.
|
|
156
|
+
- [x] Add artifact redaction options for screenshots/XML before sharing.
|
|
157
|
+
- [x] Add benchmark report viewer for duration and flake trends. V1: `zmr report`.
|
|
158
|
+
- [x] Add local device-matrix runner for multi-device smoke gates before cloud
|
|
159
|
+
device farm certification.
|
|
160
|
+
|
|
161
|
+
Acceptance:
|
|
162
|
+
- `scripts/demo.sh` produces a trace that opens in the viewer.
|
|
163
|
+
- Selector timeout traces show enough context to fix the selector without rerunning.
|
|
164
|
+
- Public demo traces contain no real app secrets.
|
|
165
|
+
|
|
166
|
+
### Track F: Scenario Authoring
|
|
167
|
+
|
|
168
|
+
Purpose: make scenario files easy to create, review, and maintain inside `.zmr/`.
|
|
169
|
+
|
|
170
|
+
Ship criteria:
|
|
171
|
+
- Users can author scenarios by hand, generate them from examples, and keep app-local runner state under `.zmr/`.
|
|
172
|
+
- Scenario errors are precise and actionable.
|
|
173
|
+
|
|
174
|
+
Work:
|
|
175
|
+
- [x] Add `zmr validate <scenario.json>` with semantic checks.
|
|
176
|
+
- [x] Add line/field-aware parse errors.
|
|
177
|
+
- [x] Add `zmr explain <trace-dir>` for a concise failure summary.
|
|
178
|
+
- [x] Add examples for auth, onboarding, referral, deep link, and error-state flows.
|
|
179
|
+
- [x] Add scenario style guide for resilient selectors and waits.
|
|
180
|
+
- [x] Add `zmr import flow-yaml` as a one-time migration helper into native
|
|
181
|
+
`.zmr/*.json` scenarios.
|
|
182
|
+
|
|
183
|
+
Acceptance:
|
|
184
|
+
- [x] Invalid scenarios fail before touching a device.
|
|
185
|
+
- Docs explain how to write robust agent-generated scenarios.
|
|
186
|
+
|
|
187
|
+
### Track G: Packaging, CI, And Distribution
|
|
188
|
+
|
|
189
|
+
Purpose: make releases boring and reproducible.
|
|
190
|
+
|
|
191
|
+
Ship criteria:
|
|
192
|
+
- CI builds, tests, packages, and publishes checksummed artifacts.
|
|
193
|
+
- Users can install on macOS and Linux without building from source.
|
|
194
|
+
|
|
195
|
+
Work:
|
|
196
|
+
- [x] Add GitHub release workflow smoke test against packaged binary.
|
|
197
|
+
- [x] Add Homebrew formula.
|
|
198
|
+
- [x] Add macOS archive signing helper for credentialed release maintainers.
|
|
199
|
+
- [x] Add macOS archive notarization helper for credentialed release maintainers.
|
|
200
|
+
- [x] Add Linux x86_64/aarch64 archives with checksum verification docs.
|
|
201
|
+
- [x] Add SBOM generation.
|
|
202
|
+
- [x] Add dependency/license report.
|
|
203
|
+
- [x] Add machine-readable release manifest with artifact digests and sizes.
|
|
204
|
+
- [x] Add tagged-release npm tarball generation and provenance publish path.
|
|
205
|
+
- [x] Add release checklist that includes demo, coverage, pilot E2E, benchmark, and docs review.
|
|
206
|
+
- [x] Add `scripts/release-gate.sh` so the local release gate is one command.
|
|
207
|
+
|
|
208
|
+
Acceptance:
|
|
209
|
+
- Fresh machine install path is documented and verified.
|
|
210
|
+
- `zmr version`, `zmr devices`, and `scripts/demo.sh` work from release artifact.
|
|
211
|
+
- Release notes explain platform support and known limitations.
|
|
212
|
+
|
|
213
|
+
### Track H: Security, Privacy, And Governance
|
|
214
|
+
|
|
215
|
+
Purpose: make the open-source product safe to use with real apps.
|
|
216
|
+
|
|
217
|
+
Ship criteria:
|
|
218
|
+
- Secret handling is explicit.
|
|
219
|
+
- Trace sharing rules are documented.
|
|
220
|
+
- Contribution rules are clear.
|
|
221
|
+
|
|
222
|
+
Work:
|
|
223
|
+
- [x] Add `SECURITY.md` with vulnerability reporting process.
|
|
224
|
+
- [x] Add `CONTRIBUTING.md` with test, formatting, and coverage expectations.
|
|
225
|
+
- [x] Add trace privacy guide.
|
|
226
|
+
- [x] Add screenshot/XML redaction strategy.
|
|
227
|
+
- [x] Add config controls for log capture and artifact capture.
|
|
228
|
+
- [x] Add denylist/allowlist for sensitive node ids and text.
|
|
229
|
+
- [x] Add governance note for protocol evolution.
|
|
230
|
+
|
|
231
|
+
Acceptance:
|
|
232
|
+
- No docs suggest sharing real traces without sanitization.
|
|
233
|
+
- Security-sensitive defaults are conservative.
|
|
234
|
+
- Contributors can run the full local gate.
|
|
235
|
+
|
|
236
|
+
## Suggested Release Milestones
|
|
237
|
+
|
|
238
|
+
### `v0.1.0`: Public Dev Preview
|
|
239
|
+
|
|
240
|
+
Scope:
|
|
241
|
+
- Current Android runner.
|
|
242
|
+
- iOS lifecycle/snapshot/selector support with `scripts/run-ios-pilot.sh`,
|
|
243
|
+
simulator-first capture, and physical-device lifecycle through `devicectl`.
|
|
244
|
+
- Fake demo.
|
|
245
|
+
- Public Android and iOS pilot wrappers.
|
|
246
|
+
- Release archives and docs.
|
|
247
|
+
|
|
248
|
+
Exit gate:
|
|
249
|
+
- Full acceptance gate passes.
|
|
250
|
+
- README and shipping docs match reality.
|
|
251
|
+
- Known limitations are explicit.
|
|
252
|
+
|
|
253
|
+
### `v0.2.0-alpha`: Android Reliability Alpha
|
|
254
|
+
|
|
255
|
+
Scope:
|
|
256
|
+
- `zmr doctor`, `zmr validate`, public schemas.
|
|
257
|
+
- Improved Android wait/diagnostics.
|
|
258
|
+
- Repeated Android pilot benchmark report.
|
|
259
|
+
- Public error taxonomy.
|
|
260
|
+
|
|
261
|
+
Exit gate:
|
|
262
|
+
- Android pilot flows pass repeated local runs.
|
|
263
|
+
- Trace failures are actionable.
|
|
264
|
+
- Coverage remains above 90%.
|
|
265
|
+
|
|
266
|
+
### `v0.3.0-alpha`: Native Android Shim
|
|
267
|
+
|
|
268
|
+
Scope:
|
|
269
|
+
- Android shim for hierarchy and interaction.
|
|
270
|
+
- ADB fallback retained.
|
|
271
|
+
- Repeated benchmark reports with trace diagnostics.
|
|
272
|
+
|
|
273
|
+
Exit gate:
|
|
274
|
+
- Shim path is faster or materially more reliable than shell-only path.
|
|
275
|
+
- Android shim has fake and emulator tests.
|
|
276
|
+
- Failure modes are cleanly surfaced through JSON-RPC.
|
|
277
|
+
|
|
278
|
+
### `v0.4.0-alpha`: iOS UI Alpha
|
|
279
|
+
|
|
280
|
+
Scope:
|
|
281
|
+
- XCTest/XCUIAutomation shim.
|
|
282
|
+
- Selector-grade iOS tap/type/swipe.
|
|
283
|
+
- Real iOS simulator smoke flow.
|
|
284
|
+
|
|
285
|
+
Exit gate:
|
|
286
|
+
- iOS selector flow passes repeated runs.
|
|
287
|
+
- Shared scenario semantics work across Android/iOS where possible.
|
|
288
|
+
- Unsupported platform differences are documented.
|
|
289
|
+
|
|
290
|
+
### `v0.5.0-beta`: Agent Integration Beta
|
|
291
|
+
|
|
292
|
+
Scope:
|
|
293
|
+
- TypeScript and Python reference clients.
|
|
294
|
+
- Live trace export.
|
|
295
|
+
- Streaming observation mode.
|
|
296
|
+
- Stable protocol fixtures.
|
|
297
|
+
|
|
298
|
+
Exit gate:
|
|
299
|
+
- External client can run a full scenario over stdio.
|
|
300
|
+
- Protocol compatibility tests gate CI.
|
|
301
|
+
- Trace bundles are inspectable without local code knowledge.
|
|
302
|
+
|
|
303
|
+
### `v1.0.0`: Stable Local Runner
|
|
304
|
+
|
|
305
|
+
Scope:
|
|
306
|
+
- Android production support.
|
|
307
|
+
- iOS simulator support with documented limitations.
|
|
308
|
+
- Trace viewer.
|
|
309
|
+
- Release packaging.
|
|
310
|
+
- Security/contribution docs.
|
|
311
|
+
|
|
312
|
+
Exit gate:
|
|
313
|
+
- Clean install experience.
|
|
314
|
+
- Stable protocol policy.
|
|
315
|
+
- Full release checklist passes.
|
|
316
|
+
- At least two real app pilot flows are documented with repeated-run results.
|
|
317
|
+
|
|
318
|
+
## Immediate Next Sprint
|
|
319
|
+
|
|
320
|
+
The next sprint should focus on making the current preview easier to trust and easier to install.
|
|
321
|
+
|
|
322
|
+
1. [x] Add `zmr doctor`.
|
|
323
|
+
2. [x] Add `zmr validate`.
|
|
324
|
+
3. [x] Add JSON schemas for scenarios and snapshots.
|
|
325
|
+
4. [x] Expand `docs/protocol.md` with exact request/response examples and error codes.
|
|
326
|
+
5. [x] Add release notes/changelog.
|
|
327
|
+
6. [x] Run the Android pilot flows repeatedly and record results in `docs/benchmarking.md`. Android V1 acceptance is 20/20 auth probe and 20/20 login smoke on 2026-04-29.
|
|
328
|
+
|
|
329
|
+
## Open Decisions
|
|
330
|
+
|
|
331
|
+
- Should the Android native shim be an APK, instrumentation test APK, or long-running local service?
|
|
332
|
+
- Should the trace viewer be bundled as static files or shipped as a separate package?
|
|
333
|
+
- Should public SDKs live in this repo or separate repos once protocol stabilizes?
|
|
334
|
+
- What exact compatibility promise should `protocolVersion` make before `v1.0.0`?
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# Scenario Authoring
|
|
2
|
+
|
|
3
|
+
ZMR scenarios are plain JSON so agents can generate and mutate them without a
|
|
4
|
+
second DSL. Keep scenarios explicit, short, and biased toward stable selectors.
|
|
5
|
+
|
|
6
|
+
## Selector Strategy
|
|
7
|
+
|
|
8
|
+
Prefer selectors in this order:
|
|
9
|
+
|
|
10
|
+
1. `id` or `resourceId` for app-owned controls.
|
|
11
|
+
2. `contentDesc` for intentional accessibility labels.
|
|
12
|
+
3. Exact `text` for stable product copy.
|
|
13
|
+
4. `textContains` only for headings, errors, or partial copy that is expected to
|
|
14
|
+
vary.
|
|
15
|
+
|
|
16
|
+
Avoid selecting by text that includes user data, timestamps, counts, prices, or
|
|
17
|
+
network-provided content. If a selector is hard to make stable, add an app-owned
|
|
18
|
+
test id or accessibility identifier instead of widening the selector until it
|
|
19
|
+
matches unrelated nodes.
|
|
20
|
+
|
|
21
|
+
## Waits And Assertions
|
|
22
|
+
|
|
23
|
+
Use a wait before actions that depend on navigation, network, or app state:
|
|
24
|
+
|
|
25
|
+
```json
|
|
26
|
+
{ "action": "waitVisible", "selector": { "id": "email-login-submit-button" }, "timeoutMs": 15000 }
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Use assertions for product expectations, not for synchronization that is already
|
|
30
|
+
covered by a wait. Prefer `waitAny` when either of two legitimate states can
|
|
31
|
+
appear, such as an already-authenticated dashboard or a sign-in prompt.
|
|
32
|
+
|
|
33
|
+
Add `assertHealthy` after launch, deep links, and major navigation steps to
|
|
34
|
+
fail on common mobile crash overlays and development-server error screens that
|
|
35
|
+
can coexist with otherwise valid UI:
|
|
36
|
+
|
|
37
|
+
```json
|
|
38
|
+
{ "action": "assertHealthy" }
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Use `assertNoneVisible` when a flow needs app-specific negative assertions that
|
|
42
|
+
are not part of ZMR's built-in health guard.
|
|
43
|
+
|
|
44
|
+
## Optional And Recovery Steps
|
|
45
|
+
|
|
46
|
+
Use `"optional": true` only for dismissals or recovery actions that are not part
|
|
47
|
+
of the required product behavior:
|
|
48
|
+
|
|
49
|
+
```json
|
|
50
|
+
{ "action": "tap", "selector": { "textContains": "Not now" }, "optional": true }
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
Optional steps still emit trace events, so failures remain inspectable without
|
|
54
|
+
making the whole flow flaky.
|
|
55
|
+
|
|
56
|
+
## Importing Existing Flows
|
|
57
|
+
|
|
58
|
+
Use the importer as a one-time migration helper when evaluating ZMR against an
|
|
59
|
+
existing mobile-flow YAML suite:
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
zmr import flow-yaml flows/login.yaml --out .zmr/login-smoke.json --json
|
|
63
|
+
zmr validate .zmr/login-smoke.json
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
The importer supports the common subset needed for smoke scenarios:
|
|
67
|
+
`launchApp`, `stopApp`, `clearState`, `tapOn`, `inputText`, `eraseText`,
|
|
68
|
+
`hideKeyboard`, `assertVisible`, `assertNotVisible`, `assertHealthy`,
|
|
69
|
+
`openLink`, `back`,
|
|
70
|
+
`scrollUntilVisible`, `takeScreenshot`, and simple wait commands. Review the
|
|
71
|
+
generated JSON before committing it; native `.zmr/*.json` scenarios remain the
|
|
72
|
+
runtime contract for agents and CI.
|
|
73
|
+
|
|
74
|
+
## Example Templates
|
|
75
|
+
|
|
76
|
+
The example directory includes templates for common app flows:
|
|
77
|
+
|
|
78
|
+
- `examples/android-app-auth-probe.json`
|
|
79
|
+
- `examples/android-app-login-smoke.json`
|
|
80
|
+
- `examples/android-app-onboarding.json`
|
|
81
|
+
- `examples/android-app-referral-deep-link.json`
|
|
82
|
+
- `examples/android-app-error-state.json`
|
|
83
|
+
- `examples/ios-dev-client-open-link.json`
|
|
84
|
+
- `examples/ios-dev-client-route-snapshot.json`
|
|
85
|
+
|
|
86
|
+
Run `zmr validate --json <scenario.json>` before touching a device. Invalid
|
|
87
|
+
scenarios report `fieldPath`, `line`, and `column` when ZMR can identify the
|
|
88
|
+
source location.
|
package/docs/shipping.md
ADDED
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
# V1 Dev Preview Shipping Checklist
|
|
2
|
+
|
|
3
|
+
This checklist defines the current shippable unit. It is intentionally narrower than a general mobile automation product.
|
|
4
|
+
|
|
5
|
+
For the maintainer-facing GitHub upload and npm publication sequence, see
|
|
6
|
+
[docs/publication.md](publication.md).
|
|
7
|
+
For the artifact-by-artifact proof checklist behind release and benchmark
|
|
8
|
+
claims, see [docs/release-evidence.md](release-evidence.md).
|
|
9
|
+
For the release-candidate command that writes `evidence.jsonl` and
|
|
10
|
+
`summary.md`, see [docs/release-candidate.md](release-candidate.md).
|
|
11
|
+
Use `zmr-release-readiness --evidence <evidence.jsonl> --target dev-preview`
|
|
12
|
+
to verify the generated evidence supports a dev-preview release. Use
|
|
13
|
+
`--target production` or `--target market-claim` only when the corresponding
|
|
14
|
+
real-device pilot and benchmark evidence exists. Pass `--evidence` more than
|
|
15
|
+
once when production evidence lives in a private app repository.
|
|
16
|
+
|
|
17
|
+
## Included
|
|
18
|
+
|
|
19
|
+
- Local CLI: `zmr devices`, `zmr run`, `zmr serve`.
|
|
20
|
+
- Local hardening CLI: `zmr doctor`, `zmr init`, `zmr validate`.
|
|
21
|
+
- Local migration CLI: `zmr import flow-yaml <flow.yaml> --out .zmr/<name>.json`.
|
|
22
|
+
- Local failure summary CLI: `zmr explain <trace-dir>`.
|
|
23
|
+
- JSON-RPC over stdio and localhost TCP.
|
|
24
|
+
- Live JSON-RPC trace capture with `zmr serve --trace-dir` and redacted
|
|
25
|
+
`trace.export` bundles.
|
|
26
|
+
- Live `trace.events` cursor polling for long-running agent sessions.
|
|
27
|
+
- Android ADB/UI Automator adapter.
|
|
28
|
+
- iOS simulator adapter through `xcrun simctl` for lifecycle, deep links,
|
|
29
|
+
screenshots, logs, and snapshots, with XCTest shim support for hierarchy and
|
|
30
|
+
selector-grade actions.
|
|
31
|
+
- Fake-device test harness for emulator-free protocol and runner tests.
|
|
32
|
+
- Fake Android/iOS demo shims and `scripts/demo.sh`.
|
|
33
|
+
- Real Android pilot wrapper with redacted trace export: `scripts/run-android-pilot.sh`.
|
|
34
|
+
- Real iOS simulator smoke wrapper with redacted trace export: `scripts/run-ios-pilot.sh`.
|
|
35
|
+
- Generic baseline command benchmark collection:
|
|
36
|
+
`scripts/benchmark-command.sh` / `zmr-benchmark-command`.
|
|
37
|
+
- Generic candidate-vs-baseline benchmark comparison:
|
|
38
|
+
`scripts/compare-benchmarks.py` / `zmr-compare-benchmarks`.
|
|
39
|
+
- Local device-matrix runner with `matrix.jsonl` and `summary.json` outputs:
|
|
40
|
+
`scripts/device-matrix.sh` / `zmr-device-matrix`.
|
|
41
|
+
- Deterministic trace event stream and snapshot artifacts.
|
|
42
|
+
- Static trace viewer with snapshot replay controls, timeline, screenshots,
|
|
43
|
+
UI tree inspection, selected node details, payloads, and artifact links.
|
|
44
|
+
- Redacted persisted JSON events/snapshots for common text secrets.
|
|
45
|
+
- Release archive script with checksums, SPDX SBOM, third-party notices,
|
|
46
|
+
generated Homebrew formula, and `RELEASE_MANIFEST.json`.
|
|
47
|
+
- npm package tarball generation with bundled prebuilt binaries.
|
|
48
|
+
- Maintainer macOS signing helper: `scripts/sign-macos-release.sh`.
|
|
49
|
+
- Maintainer macOS notarization helper: `scripts/notarize-macos-release.sh`.
|
|
50
|
+
- npm package wrapper, app initializer, setup wizard, and npm tarball builder.
|
|
51
|
+
- Standalone `zmr init --app` bootstrap for source or release-archive users who
|
|
52
|
+
need app-local `.zmr/config.json` plus Android/iOS smoke scenarios without npm.
|
|
53
|
+
- TypeScript reference client under `clients/typescript/`.
|
|
54
|
+
- Python reference client under `clients/python/`.
|
|
55
|
+
- Go reference client under `clients/go/`.
|
|
56
|
+
- Rust reference client under `clients/rust/`.
|
|
57
|
+
- npm-exposed Android shim installer for app-local instrumentation source,
|
|
58
|
+
optional module source copy, optional Gradle `testInstrumentationRunner` and
|
|
59
|
+
dependency patching, and command setup.
|
|
60
|
+
- npm-exposed iOS shim installer for app-local XCTest source, command setup,
|
|
61
|
+
and optional Xcode project/workspace UI test target and scheme wiring through
|
|
62
|
+
`xcodeproj`.
|
|
63
|
+
- iOS workspace project resolution when exactly one workspace project contains
|
|
64
|
+
the configured `--app-target`, or when `--bundle-id` disambiguates matching
|
|
65
|
+
app targets.
|
|
66
|
+
- CI workflows for tests, coverage, formatting, script syntax, and release artifacts.
|
|
67
|
+
- Tagged release workflow publishes GitHub artifact attestation for release
|
|
68
|
+
archives, npm tarballs, and metadata.
|
|
69
|
+
- Public JSON Schemas under `schemas/`.
|
|
70
|
+
- Machine-readable import output contract under `schemas/import-output.schema.json`.
|
|
71
|
+
- Protocol compatibility fixtures under `docs/protocol-fixtures/`.
|
|
72
|
+
- Machine-readable protocol compatibility policy in `runner.capabilities`.
|
|
73
|
+
- App-local `.zmr/config.json` schema and CLI default loading.
|
|
74
|
+
- Internal iOS/Android shim protocol scaffolds under `shims/`.
|
|
75
|
+
- Security, contribution, trace privacy, and protocol versioning docs.
|
|
76
|
+
- Release evidence checklist in `docs/release-evidence.md`.
|
|
77
|
+
- Release candidate evidence gate in `docs/release-candidate.md` and
|
|
78
|
+
`scripts/release-candidate.sh`.
|
|
79
|
+
- Feature catalog in `FEATURES.md`.
|
|
80
|
+
- Architecture decision records under `docs/adr/`.
|
|
81
|
+
- AI agent integration guide in `docs/ai-agents.md`.
|
|
82
|
+
- Reusable agent skill under `skills/zmr-mobile-testing/`.
|
|
83
|
+
- Changelog and release notes template.
|
|
84
|
+
- Android pilot scenarios:
|
|
85
|
+
- `examples/android-app-auth-probe.json`
|
|
86
|
+
- `examples/android-app-login-smoke.json`
|
|
87
|
+
- Scenario authoring templates:
|
|
88
|
+
- `examples/android-app-onboarding.json`
|
|
89
|
+
- `examples/android-app-referral-deep-link.json`
|
|
90
|
+
- `examples/android-app-error-state.json`
|
|
91
|
+
|
|
92
|
+
## Acceptance Gate
|
|
93
|
+
|
|
94
|
+
Before tagging a dev-preview release:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
./scripts/release-gate.sh
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Use `./scripts/release-gate.sh --dry-run` to inspect the exact local commands.
|
|
101
|
+
The script includes formatting, script syntax checks, focused shell tests, npm
|
|
102
|
+
package tests, viewer tests, Zig unit tests, fake-tool validation, the
|
|
103
|
+
no-emulator demo, coverage, release archive generation, checksum verification,
|
|
104
|
+
packaged binary smoke, and `npm pack --dry-run`.
|
|
105
|
+
The fake-tool validation uses `zmr doctor --strict` so warning or missing setup
|
|
106
|
+
checks fail the local release gate instead of requiring callers to parse JSON.
|
|
107
|
+
|
|
108
|
+
For Android and iOS simulator pilot validation, run the app-facing pilot gate
|
|
109
|
+
against a booted Android emulator, a booted iOS simulator, and the built
|
|
110
|
+
simulator `.app`:
|
|
111
|
+
|
|
112
|
+
```bash
|
|
113
|
+
zmr-pilot-gate --android --ios --android-app-root /path/to/mobile-app --android-app-id com.example.mobiletest --android-device emulator-5554 --ios-app-root /path/to/mobile-app --ios-app-path /path/to/mobile-app/build/Debug-iphonesimulator/Sample.app --ios-app-id com.example.mobiletest --ios-device booted --ios-shim /path/to/mobile-app/.zmr/ios-shim --runs 20 --min-pass-rate 100 --max-failures 0 --evidence-out /path/to/mobile-app/traces/zmr-pilots/evidence.jsonl
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
For physical iOS pilot validation, use a physical device identifier from
|
|
117
|
+
`zmr devices --json --platform ios --ios-device-type physical` and a signed
|
|
118
|
+
device artifact:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
zmr-pilot-gate --ios --ios-device-type physical --ios-device <physical-device-id> --ios-app-root /path/to/mobile-app --ios-app-path /path/to/mobile-app/build/Release-iphoneos/Sample.ipa --ios-app-id com.example.mobiletest --ios-shim /path/to/mobile-app/.zmr/ios-shim --runs 20 --min-pass-rate 100 --max-failures 0 --evidence-out /path/to/mobile-app/traces/zmr-pilots/evidence.jsonl
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Pass `--zmr-bin /path/to/zmr` when the pilot should use an explicit runner
|
|
125
|
+
binary; the wrapper forwards it to Android, iOS, and physical iOS readiness
|
|
126
|
+
checks.
|
|
127
|
+
|
|
128
|
+
Add `--evidence-out /path/to/mobile-app/traces/zmr-pilots/evidence.jsonl` to
|
|
129
|
+
write production-readiness rows that can be passed to
|
|
130
|
+
`zmr-release-readiness` alongside the public release-candidate evidence.
|
|
131
|
+
|
|
132
|
+
Run the pilot gates before publishing reliability or performance claims.
|
|
133
|
+
|
|
134
|
+
## Not Yet Included
|
|
135
|
+
|
|
136
|
+
- Automatic iOS workspace resolution when multiple workspace projects contain
|
|
137
|
+
the same requested app target and bundle id. In that case, callers must pass
|
|
138
|
+
`--project`.
|
|
139
|
+
- Full physical iOS artifact parity. Physical iOS lifecycle, screenshots, and
|
|
140
|
+
XCTest shim interaction are supported locally, but physical-device log capture
|
|
141
|
+
still needs a supported capture channel.
|
|
142
|
+
- Pixel-level screenshot or video masking. Redacted bundles at present replace
|
|
143
|
+
PNG screenshots with placeholder frames or omit screenshots entirely, and
|
|
144
|
+
omit screen recordings instead of attempting visual masking.
|
|
145
|
+
- Broad cloud device farm certification. Local matrix validation is included;
|
|
146
|
+
hosted farm adapters and OS/device matrix certification are not.
|
|
147
|
+
|
|
148
|
+
## Release Process
|
|
149
|
+
|
|
150
|
+
1. Run `./scripts/release-gate.sh`.
|
|
151
|
+
2. Run the Android and iOS pilot scenarios if emulator/simulator app builds are available.
|
|
152
|
+
3. If publishing signed macOS archives, run `./scripts/sign-macos-release.sh --identity "<Developer ID Application identity>"` after `./scripts/build-release.sh`.
|
|
153
|
+
4. If publishing notarized macOS archives, run `./scripts/notarize-macos-release.sh --keychain-profile "<notarytool profile>"`, then rerun `./scripts/verify-release-artifacts.sh`.
|
|
154
|
+
5. Create the tag `v0.1.0` for the current package version. Bump
|
|
155
|
+
`src/version.zig` and `package.json` before using a different tag.
|
|
156
|
+
6. Push the tag; `.github/workflows/release.yml` builds archives, smokes the host-compatible packaged binary, builds `dist/zig-mobile-runner-*.tgz`, publishes GitHub artifact attestation, and uploads checksums, SBOM, third-party notices, `RELEASE_MANIFEST.json`, the npm tarball, and the generated Homebrew formula.
|
|
157
|
+
7. If `NPM_TOKEN` is configured, the release workflow publishes the npm tarball with `npm publish dist/zig-mobile-runner-*.tgz --provenance --access public`. Without that secret, the workflow skips npm publish but still uploads the tarball for manual inspection.
|
|
158
|
+
|
|
159
|
+
`./scripts/verify-release-artifacts.sh` can also be run directly after
|
|
160
|
+
`./scripts/build-release.sh`. It verifies every `SHA256SUMS` entry, requires the
|
|
161
|
+
release archives plus SBOM/notices/Homebrew formula/manifest to be covered,
|
|
162
|
+
also requires any generated npm tarball to be covered, cross-checks manifest
|
|
163
|
+
sizes and digests, and fails on tampered or missing files before upload.
|
|
164
|
+
|
|
165
|
+
`./scripts/sign-macos-release.sh --dry-run --identity "<identity>"` lists the
|
|
166
|
+
macOS archives that would be signed without invoking `codesign`.
|
|
167
|
+
|
|
168
|
+
`./scripts/notarize-macos-release.sh --dry-run --keychain-profile "<profile>"`
|
|
169
|
+
lists the macOS archives that would be packaged and submitted to notarytool
|
|
170
|
+
without invoking `ditto` or `xcrun`.
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# Trace Privacy
|
|
2
|
+
|
|
3
|
+
ZMR traces are debugging artifacts. They can contain sensitive app state even
|
|
4
|
+
when scenario files are generic.
|
|
5
|
+
|
|
6
|
+
Raw trace directories may include:
|
|
7
|
+
|
|
8
|
+
- screenshots
|
|
9
|
+
- screen recordings
|
|
10
|
+
- UI hierarchy XML or JSON
|
|
11
|
+
- visible text and accessibility labels
|
|
12
|
+
- log windows
|
|
13
|
+
- app ids, package/activity names, and timing data
|
|
14
|
+
- action inputs
|
|
15
|
+
|
|
16
|
+
## Sharing Rules
|
|
17
|
+
|
|
18
|
+
- Disable unnecessary raw artifacts in `.zmr/config.json`:
|
|
19
|
+
|
|
20
|
+
```json
|
|
21
|
+
{
|
|
22
|
+
"schemaVersion": 1,
|
|
23
|
+
"artifacts": {
|
|
24
|
+
"screenshots": false,
|
|
25
|
+
"hierarchy": false,
|
|
26
|
+
"logs": false
|
|
27
|
+
},
|
|
28
|
+
"redaction": {
|
|
29
|
+
"denylistText": ["customer dob", "internal token"],
|
|
30
|
+
"denylistResourceIds": ["password-field", "ssn"],
|
|
31
|
+
"allowlistResourceIds": ["public-token-label"]
|
|
32
|
+
}
|
|
33
|
+
}
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
- Add app-specific text and resource-id denylist rules for customer data,
|
|
37
|
+
regulated identifiers, and internal identifiers that ZMR cannot infer.
|
|
38
|
+
- Share redacted `.zmrtrace` bundles, not raw trace directories.
|
|
39
|
+
- Use `zmr export <trace-dir> --out <bundle.zmrtrace> --redact`.
|
|
40
|
+
- Review redacted bundles before attaching them to public issues.
|
|
41
|
+
- Do not publish raw private app screenshots.
|
|
42
|
+
- Do not publish private app screen recordings.
|
|
43
|
+
- Do not publish logs containing credentials, tokens, emails, or customer data.
|
|
44
|
+
|
|
45
|
+
## Current Redaction Behavior
|
|
46
|
+
|
|
47
|
+
Persisted trace JSON scrubs obvious emails, bearer/JWT-like tokens, sensitive
|
|
48
|
+
JSON keys, app-configured denylisted text, and app-configured sensitive
|
|
49
|
+
resource ids. Resource-id denylist matches redact the id and force that node's
|
|
50
|
+
`text` and `contentDesc` to secret placeholders. App-specific allowlists only
|
|
51
|
+
skip app-specific denylist matches; built-in email/token scrubbing still
|
|
52
|
+
applies.
|
|
53
|
+
|
|
54
|
+
Redacted exports scrub common text secrets, replace PNG screenshots with safe
|
|
55
|
+
placeholder frames, and omit screen recordings. Add `--omit-screenshots` to
|
|
56
|
+
remove screenshot artifacts from the exported bundle entirely. Local trace
|
|
57
|
+
directories are not mutated by export.
|
|
58
|
+
|
|
59
|
+
Pixel-level screenshot masking is not implemented. Raw hierarchy XML can still
|
|
60
|
+
contain app text; disable hierarchy capture or review redacted bundles before
|
|
61
|
+
sharing them outside a trusted machine.
|
|
62
|
+
|
|
63
|
+
## Screenshot And XML Strategy
|
|
64
|
+
|
|
65
|
+
ZMR uses conservative artifact handling instead of partial visual masking:
|
|
66
|
+
|
|
67
|
+
- Redacted `.zmrtrace` exports replace PNG screenshot files with generated
|
|
68
|
+
placeholder PNGs at the same artifact paths. This keeps trace replay frames
|
|
69
|
+
stable without shipping rendered app pixels.
|
|
70
|
+
- `zmr export --redact --omit-screenshots` omits screenshot files entirely for
|
|
71
|
+
teams that cannot share visual artifacts outside trusted machines.
|
|
72
|
+
- Redacted `.zmrtrace` exports omit screen recording files entirely. This avoids
|
|
73
|
+
shipping unreliable video masking that could miss rendered secrets.
|
|
74
|
+
- Redacted exports scrub text-based artifacts, including JSON events, snapshot
|
|
75
|
+
JSON, logs, reports, and XML-like hierarchy files, for common emails, bearer
|
|
76
|
+
tokens, sensitive key names, and sensitive node attributes.
|
|
77
|
+
- App-specific `.zmr/config.json` redaction rules apply before trace files are
|
|
78
|
+
written. Use them for private resource ids and business-specific text that
|
|
79
|
+
generic scrubbing cannot identify.
|
|
80
|
+
- Set `artifacts.hierarchy: false` for apps whose raw accessibility trees are
|
|
81
|
+
sensitive by default. Selector matching still uses the live tree; this only
|
|
82
|
+
disables raw hierarchy persistence.
|
|
83
|
+
- Keep unredacted local trace directories on trusted developer machines only.
|
|
84
|
+
|
|
85
|
+
This strategy favors predictable placeholders, explicit visual omission, and
|
|
86
|
+
app-owned denylist rules over a best-effort screenshot masker. Pixel-level
|
|
87
|
+
masking can be added later as an opt-in exporter mode once it has visual
|
|
88
|
+
verification tests.
|