zeno-mobile-runner 0.1.8 → 0.2.1

package/docs/benchmarks/2026-06-09-ios-maestro-comparison.md

@@ -0,0 +1,128 @@
+# 2026-06-09 iOS Simulator ZMR vs Maestro Comparison
+
+This evidence pack compares ZMR with Maestro on the generated public iOS demo
+app. It is a command-level benchmark: each row starts the runner command,
+executes one smoke flow on the same booted simulator, and records the command
+duration.
+
+This is not a universal claim about either project. Treat it as reproducible
+evidence for this app, host, simulator, and flow shape.
+
+## Result
+
+| Tool | Runs | Pass rate | Failures | Mean duration | p95 duration |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| ZMR | 20 | 100.00% | 0 | 2236 ms | 2282 ms |
+| Maestro | 20 | 95.00% | 1 | 7578 ms | 7918 ms |
+
+For this run, ZMR was 3.39x faster on mean duration and 3.47x faster on p95
+duration. Maestro run 8 failed while launching the app with a driver connection
+error: `Failed to connect to /127.0.0.1:7001`.
+
+## Environment
+
+| Field | Value |
+| --- | --- |
+| ZMR runner | `0.1.8` |
+| ZMR protocol | `2026-04-28` |
+| Maestro | `1.40.3` |
+| Java | OpenJDK 21.0.6 |
+| Host OS | macOS 26.6, arm64 |
+| Xcode | 26.5, build 17F42 |
+| Simulator | Booted iOS 26.5 simulator, `zmr-iPhone` |
+| App id | `com.example.mobiletest` |
+| App build label | `generated-ios-demo-20260609` |
+| Demo app source | Generated by `scripts/create-ios-demo-app.sh` |
+
+## ZMR Command
+
+```bash
+ZMR_BIN="$PWD/zig-out/bin/zmr" scripts/benchmark.sh \
+  --zmr examples/ios-smoke.json \
+  --platform ios \
+  --ios-device-type simulator \
+  --device booted \
+  --app-id com.example.mobiletest \
+  --xcrun xcrun \
+  --app-build generated-ios-demo-20260609 \
+  --runs 20 \
+  --trace-root traces/public-benchmarks/20260609-ios-maestro-comparison/zmr \
+  --results traces/public-benchmarks/20260609-ios-maestro-comparison/results.jsonl \
+  --replace \
+  --min-pass-rate 100 \
+  --max-failures 0
+```
+
+The ZMR scenario launches the app, opens `exampleapp:///e2e-auth?probe=1`, and
+captures a snapshot:
+
+```json
+{
+  "name": "ZMR iOS simulator smoke demo",
+  "appId": "com.example.mobiletest",
+  "steps": [
+    { "action": "launch" },
+    { "action": "openLink", "url": "exampleapp:///e2e-auth?probe=1" },
+    { "action": "snapshot" }
+  ]
+}
+```
+
+## Maestro Command
+
+```bash
+scripts/benchmark-command.sh \
+  --tool maestro \
+  --platform ios \
+  --device booted \
+  --app-id com.example.mobiletest \
+  --scenario examples/ios-smoke.json \
+  --app-build generated-ios-demo-20260609 \
+  --runs 20 \
+  --trace-root traces/public-benchmarks/20260609-ios-maestro-comparison/maestro \
+  --results traces/public-benchmarks/20260609-ios-maestro-comparison/results.jsonl \
+  --min-pass-rate 100 \
+  --max-failures 0 \
+  -- maestro --platform ios --device F839B002-5388-4814-B64F-DBE5A113EBED test --no-ansi /tmp/zmr-maestro-benchmark-20260609/ios-smoke.yaml
+```
+
+The Maestro flow launches the same app, opens the same deep link, and asserts
+the resulting status text:
+
+```yaml
+appId: com.example.mobiletest
+---
+- launchApp
+- openLink: exampleapp:///e2e-auth?probe=1
+- assertVisible: Deep link opened
+```
+
+## Comparison Report
+
+```text
+# Benchmark Comparison
+
+| Tool | Runs | Pass rate | Failures | Mean ms | P95 ms |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| zmr | 20 | 100.00% | 0 | 2236 | 2282 |
+| maestro | 20 | 95.00% | 1 | 7578 | 7918 |
+
+- Mean speedup: 3.39x (-70.5% candidate vs baseline)
+- P95 speedup: 3.47x (-71.2% candidate vs baseline)
+- Same benchmark context: yes
+```
+
+The sanitized result rows are committed in
+[2026-06-09-ios-maestro-comparison.results.jsonl](2026-06-09-ios-maestro-comparison.results.jsonl).
+
+Raw local trace and runner logs are not committed because they can include local
+absolute paths. The failure detail above was taken from the failed Maestro
+run's local stdout log.
+
+## Scope
+
+This benchmark compares the local command surfaces under repeated execution. It
+does not compare cloud execution, long multi-screen flows, Detox, XCTest-only
+flows, or app-specific selectors. Detox requires a project-specific native test
+harness, so it should get its own evidence pack rather than being forced into
+this demo comparison.

package/docs/benchmarks/2026-06-09-ios-maestro-comparison.results.jsonl

@@ -0,0 +1,40 @@
+{"tool":"zmr","run":1,"status":"ok","durationMs":2347,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-1","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":2,"status":"ok","durationMs":2233,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-2","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":3,"status":"ok","durationMs":2282,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-3","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":4,"status":"ok","durationMs":2204,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-4","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":5,"status":"ok","durationMs":2231,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-5","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":6,"status":"ok","durationMs":2185,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-6","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":7,"status":"ok","durationMs":2208,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-7","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":8,"status":"ok","durationMs":2196,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-8","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":9,"status":"ok","durationMs":2189,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-9","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":10,"status":"ok","durationMs":2211,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-10","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":11,"status":"ok","durationMs":2268,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-11","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":12,"status":"ok","durationMs":2245,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-12","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":13,"status":"ok","durationMs":2244,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-13","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":14,"status":"ok","durationMs":2249,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-14","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":15,"status":"ok","durationMs":2224,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-15","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":16,"status":"ok","durationMs":2236,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-16","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":17,"status":"ok","durationMs":2236,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-17","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":18,"status":"ok","durationMs":2226,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-18","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":19,"status":"ok","durationMs":2262,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-19","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":20,"status":"ok","durationMs":2248,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/zmr/zmr-20","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"maestro","run":1,"status":"ok","durationMs":7762,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-1","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":2,"status":"ok","durationMs":7793,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-2","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":3,"status":"ok","durationMs":7771,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-3","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":4,"status":"ok","durationMs":7762,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-4","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":5,"status":"ok","durationMs":7838,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-5","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":6,"status":"ok","durationMs":7918,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-6","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":7,"status":"ok","durationMs":7975,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-7","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":8,"status":"failed","durationMs":5091,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-8","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":9,"status":"ok","durationMs":6764,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-9","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":10,"status":"ok","durationMs":7716,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-10","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":11,"status":"ok","durationMs":7738,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-11","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":12,"status":"ok","durationMs":7738,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-12","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":13,"status":"ok","durationMs":7808,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-13","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":14,"status":"ok","durationMs":7772,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-14","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":15,"status":"ok","durationMs":7681,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-15","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":16,"status":"ok","durationMs":7629,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-16","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":17,"status":"ok","durationMs":7702,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-17","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":18,"status":"ok","durationMs":7661,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-18","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":19,"status":"ok","durationMs":7787,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-19","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"maestro","run":20,"status":"ok","durationMs":7658,"traceDir":"traces/public-benchmarks/20260609-ios-maestro-comparison/maestro/maestro-20","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}

package/docs/benchmarks/2026-06-09-ios-workflow-comparison.md

@@ -0,0 +1,143 @@
+# 2026-06-09 iOS Simulator Workflow Comparison
+
+This evidence pack compares ZMR with Maestro on the generated public iOS demo
+app using a longer workflow than the smoke benchmark. The flow launches the
+app, fills a profile form, opens a catalog item, saves it, reviews the order,
+and asserts the final state.
+
+This is not a universal claim about either project. Treat it as reproducible
+evidence for this app, host, simulator, app build, and workflow shape.
+
+## Result
+
+| Tool | Runs | Pass rate | Failures | Mean duration | p95 duration |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| ZMR | 20 | 100.00% | 0 | 19539 ms | 22970 ms |
+| Maestro | 20 | 100.00% | 0 | 36240 ms | 47502 ms |
+
+For this run, ZMR was 1.85x faster on mean duration and 2.07x faster on p95
+duration.
+
+## Environment
+
+| Field | Value |
+| --- | --- |
+| ZMR runner | `0.1.8` |
+| ZMR protocol | `2026-04-28` |
+| Maestro | `1.40.3` |
+| Java | OpenJDK 21.0.6 |
+| Host OS | macOS 26.6, arm64 |
+| Xcode | 26.5, build 17F42 |
+| Simulator | Booted iOS 26.5 simulator, `zmr-iPhone` |
+| App id | `com.example.mobiletest` |
+| App build label | `generated-ios-workflow-demo-20260609` |
+| Demo app source | Generated by `scripts/create-ios-demo-app.sh` |
+
+## ZMR Command
+
+```bash
+ZMR_BIN="$PWD/zig-out/bin/zmr" scripts/benchmark.sh \
+  --zmr examples/ios-shim-workflow.json \
+  --platform ios \
+  --ios-device-type simulator \
+  --device booted \
+  --app-id com.example.mobiletest \
+  --xcrun xcrun \
+  --ios-shim /tmp/zmr-ios-workflow-demo-20260609/.zmr/ios-shim \
+  --app-build generated-ios-workflow-demo-20260609 \
+  --runs 20 \
+  --trace-root traces/public-benchmarks/20260609-ios-workflow-comparison/zmr \
+  --results traces/public-benchmarks/20260609-ios-workflow-comparison/results.jsonl \
+  --replace \
+  --min-pass-rate 100 \
+  --max-failures 0
+```
+
+The ZMR scenario is committed as
+[`examples/ios-shim-workflow.json`](../../examples/ios-shim-workflow.json).
+
+## Maestro Command
+
+```bash
+MAESTRO_BIN="$HOME/.maestro/bin/maestro"
+scripts/benchmark-command.sh \
+  --tool maestro \
+  --platform ios \
+  --device booted \
+  --app-id com.example.mobiletest \
+  --scenario examples/ios-shim-workflow.json \
+  --app-build generated-ios-workflow-demo-20260609 \
+  --runs 20 \
+  --trace-root traces/public-benchmarks/20260609-ios-workflow-comparison/maestro \
+  --results traces/public-benchmarks/20260609-ios-workflow-comparison/results.jsonl \
+  --min-pass-rate 90 \
+  --max-failures 2 \
+  -- "$MAESTRO_BIN" \
+    --platform ios \
+    --device F839B002-5388-4814-B64F-DBE5A113EBED \
+    test \
+    --no-ansi \
+    /tmp/zmr-maestro-workflow-20260609/ios-workflow.yaml
+```
+
+The measured Maestro flow used the same visible labels and app state:
+
+```yaml
+appId: com.example.mobiletest
+---
+- launchApp
+- assertVisible: ZMR iOS Demo
+- tapOn: Continue
+- assertVisible: Profile
+- tapOn: Name
+- inputText: Riley
+- tapOn: Email
+- inputText: riley@example.test
+- hideKeyboard
+- tapOn: Save profile
+- assertVisible: Catalog
+- tapOn: North Ridge Pack
+- assertVisible: North Ridge Pack
+- tapOn: Save item
+- assertVisible: Saved North Ridge Pack
+- tapOn: Review order
+- assertVisible: Workflow complete
+```
+
+## Comparison Report
+
+```text
+# Benchmark Comparison
+
+| Tool | Runs | Pass rate | Failures | Mean ms | P95 ms |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| zmr | 20 | 100.00% | 0 | 19539 | 22970 |
+| maestro | 20 | 100.00% | 0 | 36240 | 47502 |
+
+- Mean speedup: 1.85x (-46.1% candidate vs baseline)
+- P95 speedup: 2.07x (-51.6% candidate vs baseline)
+- Same benchmark context: yes
+```
+
+The sanitized result rows are committed in
+[2026-06-09-ios-workflow-comparison.results.jsonl](2026-06-09-ios-workflow-comparison.results.jsonl).
+
+Raw local trace and runner logs are not committed because they can include local
+absolute paths.
+
+## Other Baselines
+
+Appium 3.5.0 with XCUITest driver 11.10.0 was attempted against the same app
+and simulator. The smoke run failed twice while starting WebDriverAgent with
+`connect ECONNREFUSED 127.0.0.1:8100`, so no Appium timing rows are included in
+this evidence pack.
+
+Detox needs a project-specific native test harness and app build wiring, so it
+should get its own fixture rather than being forced into this generated SwiftUI
+demo.
+
+## Scope
+
+This benchmark compares local command surfaces under repeated execution. It
+does not compare cloud execution, Android, Flutter widget-tree automation, or
+framework-specific project harnesses.

package/docs/benchmarks/2026-06-09-ios-workflow-comparison.results.jsonl

@@ -0,0 +1,40 @@
+{"tool":"zmr","run":1,"status":"ok","durationMs":21874,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-1","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":2,"status":"ok","durationMs":17173,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-2","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":3,"status":"ok","durationMs":16724,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-3","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":4,"status":"ok","durationMs":16453,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-4","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":5,"status":"ok","durationMs":17385,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-5","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":6,"status":"ok","durationMs":23807,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-6","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":7,"status":"ok","durationMs":18335,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-7","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":8,"status":"ok","durationMs":18195,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-8","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":9,"status":"ok","durationMs":17859,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-9","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":10,"status":"ok","durationMs":21344,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-10","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":11,"status":"ok","durationMs":21375,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-11","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":12,"status":"ok","durationMs":20713,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-12","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":13,"status":"ok","durationMs":18959,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-13","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":14,"status":"ok","durationMs":20610,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-14","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":15,"status":"ok","durationMs":22176,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-15","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":16,"status":"ok","durationMs":18852,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-16","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":17,"status":"ok","durationMs":22970,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-17","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":18,"status":"ok","durationMs":18682,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-18","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":19,"status":"ok","durationMs":18347,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-19","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":20,"status":"ok","durationMs":18951,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/zmr-final/zmr-20","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609","traceStatus":"passed"}
+{"tool":"maestro","run":1,"status":"ok","durationMs":47502,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-1","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":2,"status":"ok","durationMs":29793,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-2","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":3,"status":"ok","durationMs":31898,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-3","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":4,"status":"ok","durationMs":27151,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-4","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":5,"status":"ok","durationMs":31825,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-5","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":6,"status":"ok","durationMs":32080,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-6","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":7,"status":"ok","durationMs":27544,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-7","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":8,"status":"ok","durationMs":30177,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-8","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":9,"status":"ok","durationMs":31528,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-9","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":10,"status":"ok","durationMs":30227,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-10","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":11,"status":"ok","durationMs":29175,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-11","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":12,"status":"ok","durationMs":30119,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-12","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":13,"status":"ok","durationMs":38762,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-13","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":14,"status":"ok","durationMs":36592,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-14","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":15,"status":"ok","durationMs":29813,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-15","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":16,"status":"ok","durationMs":34198,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-16","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":17,"status":"ok","durationMs":42114,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-17","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":18,"status":"ok","durationMs":46397,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-18","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":19,"status":"ok","durationMs":44429,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-19","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}
+{"tool":"maestro","run":20,"status":"ok","durationMs":73476,"traceDir":"traces/public-benchmarks/20260609-ios-workflow-comparison/maestro/maestro-20","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-shim-workflow.json","appBuild":"generated-ios-workflow-demo-20260609"}

package/docs/benchmarks/2026-06-09-ios-xctest-floor.md

@@ -0,0 +1,106 @@
+# 2026-06-09 iOS Simulator XCTest Shim Floor
+
+This evidence pack compares the optimized ZMR iOS shim-backed smoke flow with a
+direct XCTest-shim floor on the generated public iOS demo app.
+
+The floor is not a competing product. It bypasses ZMR scenario parsing, trace
+events, screenshots, result normalization, and reporting. It is useful because
+it shows how much room remains between ZMR and the warmed native shim path.
+
+## Result
+
+| Tool | Runs | Pass rate | Failures | Mean duration | p95 duration |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| ZMR with XCTest shim | 20 | 100.00% | 0 | 2007 ms | 2969 ms |
+| Direct XCTest shim floor | 20 | 100.00% | 0 | 1004 ms | 962 ms |
+
+Before this run, ZMR was changed to skip the slow best-effort iOS "Open"
+confirmation probe for custom URL schemes. Universal `http://` and `https://`
+links still keep that safety check.
+
+The result: ZMR is now roughly 2.0x the direct warmed shim mean for this flow,
+while still producing scenario semantics and trace artifacts.
+
+## Environment
+
+| Field | Value |
+| --- | --- |
+| ZMR runner | `0.1.8` |
+| ZMR protocol | `2026-04-28` |
+| Host OS | macOS 26.6, arm64 |
+| Xcode | 26.5, build 17F42 |
+| Simulator | Booted iOS 26.5 simulator, `zmr-iPhone` |
+| App id | `com.example.mobiletest` |
+| App build label | `generated-ios-demo-20260609` |
+| Demo app source | Generated by `scripts/create-ios-demo-app.sh` |
+
+## ZMR Command
+
+```bash
+ZMR_BIN="$PWD/zig-out/bin/zmr" scripts/benchmark.sh \
+  --zmr examples/ios-smoke.json \
+  --platform ios \
+  --ios-device-type simulator \
+  --device booted \
+  --app-id com.example.mobiletest \
+  --xcrun xcrun \
+  --ios-shim /tmp/zmr-public-benchmark-ios-demo-20260609-plain/.zmr/ios-shim \
+  --app-build generated-ios-demo-20260609 \
+  --runs 20 \
+  --trace-root traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr \
+  --results traces/public-benchmarks/20260609-ios-xctest-floor-optimized/results.jsonl \
+  --replace \
+  --min-pass-rate 100 \
+  --max-failures 0
+```
+
+## Direct XCTest Shim Floor Command
+
+The direct floor launches the already-installed app, opens the same custom
+scheme URL, sends one `snapshot` command to the warmed XCTest shim server, and
+asserts that the snapshot contains `Deep link opened`.
+
+It intentionally does not terminate the app before each run. Terminating the app
+under a persistent XCTest runner can invalidate XCTest's connection to the app
+and create failures unrelated to ZMR.
+
+```bash
+scripts/benchmark-command.sh \
+  --tool xctest-shim \
+  --platform ios \
+  --device booted \
+  --app-id com.example.mobiletest \
+  --scenario examples/ios-smoke.json \
+  --app-build generated-ios-demo-20260609 \
+  --runs 20 \
+  --trace-root traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim \
+  --results traces/public-benchmarks/20260609-ios-xctest-floor-optimized/results.jsonl \
+  --min-pass-rate 100 \
+  --max-failures 0 \
+  -- bash -lc '<simctl launch, simctl openurl, direct shim snapshot, grep status text>'
+```
+
+## Comparison Report
+
+```text
+# Benchmark Comparison
+
+| Tool | Runs | Pass rate | Failures | Mean ms | P95 ms |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| zmr | 20 | 100.00% | 0 | 2007 | 2969 |
+| xctest-shim | 20 | 100.00% | 0 | 1004 | 962 |
+
+- Mean speedup: 0.50x (+99.9% candidate vs baseline)
+- P95 speedup: 0.32x (+208.6% candidate vs baseline)
+- Same benchmark context: yes
+```
+
+The sanitized result rows are committed in
+[2026-06-09-ios-xctest-floor.results.jsonl](2026-06-09-ios-xctest-floor.results.jsonl).
+
+## Scope
+
+This floor is a native-path reference point, not a replacement for product
+benchmarks. It does not include trace writing, scenario validation, artifact
+capture, report generation, JSONL result normalization, or multi-step selector
+logic.

package/docs/benchmarks/2026-06-09-ios-xctest-floor.results.jsonl

@@ -0,0 +1,40 @@
+{"tool":"zmr","run":1,"status":"ok","durationMs":4500,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-1","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":2,"status":"ok","durationMs":1810,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-2","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":3,"status":"ok","durationMs":2969,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-3","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":4,"status":"ok","durationMs":1816,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-4","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":5,"status":"ok","durationMs":1778,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-5","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":6,"status":"ok","durationMs":1782,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-6","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":7,"status":"ok","durationMs":1936,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-7","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":8,"status":"ok","durationMs":1820,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-8","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":9,"status":"ok","durationMs":1797,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-9","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":10,"status":"ok","durationMs":1794,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-10","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":11,"status":"ok","durationMs":1791,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-11","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":12,"status":"ok","durationMs":1801,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-12","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":13,"status":"ok","durationMs":1901,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-13","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":14,"status":"ok","durationMs":1709,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-14","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":15,"status":"ok","durationMs":1847,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-15","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":16,"status":"ok","durationMs":1805,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-16","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":17,"status":"ok","durationMs":1810,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-17","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":18,"status":"ok","durationMs":1800,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-18","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":19,"status":"ok","durationMs":1843,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-19","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"zmr","run":20,"status":"ok","durationMs":1833,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/zmr/zmr-20","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609","traceStatus":"passed"}
+{"tool":"xctest-shim","run":1,"status":"ok","durationMs":3403,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-1","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":2,"status":"ok","durationMs":879,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-2","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":3,"status":"ok","durationMs":808,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-3","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":4,"status":"ok","durationMs":912,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-4","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":5,"status":"ok","durationMs":904,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-5","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":6,"status":"ok","durationMs":874,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-6","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":7,"status":"ok","durationMs":959,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-7","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":8,"status":"ok","durationMs":871,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-8","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":9,"status":"ok","durationMs":870,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-9","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":10,"status":"ok","durationMs":870,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-10","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":11,"status":"ok","durationMs":865,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-11","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":12,"status":"ok","durationMs":876,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-12","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":13,"status":"ok","durationMs":962,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-13","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":14,"status":"ok","durationMs":864,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-14","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":15,"status":"ok","durationMs":823,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-15","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":16,"status":"ok","durationMs":861,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-16","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":17,"status":"ok","durationMs":862,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-17","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":18,"status":"ok","durationMs":878,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-18","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":19,"status":"ok","durationMs":865,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-19","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}
+{"tool":"xctest-shim","run":20,"status":"ok","durationMs":865,"traceDir":"traces/public-benchmarks/20260609-ios-xctest-floor-optimized/xctest-shim/xctest-shim-20","platform":"ios","device":"booted","appId":"com.example.mobiletest","scenario":"examples/ios-smoke.json","appBuild":"generated-ios-demo-20260609"}

package/docs/benchmarks/README.md

@@ -0,0 +1,36 @@
+# Benchmark Evidence
+
+This directory contains public-safe benchmark evidence collected from
+reproducible ZMR demo apps.
+
+Evidence here is intentionally narrow:
+
+- It records the command, platform, runner version, run count, pass rate, and
+  duration summary.
+- It does not claim ZMR is faster than another tool unless an equivalent
+  baseline was collected on the same app build, device state, and scenario.
+- Raw local traces are not committed because generated reports and JUnit files
+  can include absolute local paths. Public rows are sanitized before commit.
+
+## Evidence Packs
+
+- [Benchmark Lab v1](benchmark-lab-v1.md): public fixture, timing-mode,
+  runner-adapter, and claim-rule plan for framework-level evidence.
+- [2026-06-09 iOS simulator demo](2026-06-09-ios-demo.md): 20 repeated runs of
+  the public iOS smoke scenario on a booted simulator.
+- [2026-06-09 iOS simulator ZMR vs Maestro comparison](2026-06-09-ios-maestro-comparison.md):
+  command-level comparison against Maestro on the same generated iOS demo app.
+- [2026-06-09 iOS simulator ZMR vs Appium comparison](2026-06-09-ios-appium-comparison.md):
+  command-level comparison against Appium on the same generated iOS demo app.
+- [2026-06-09 iOS simulator workflow comparison](2026-06-09-ios-workflow-comparison.md):
+  longer profile, catalog, save, and review workflow on the generated iOS demo
+  app.
+- [2026-06-09 Android emulator workflow](2026-06-09-android-workflow.md):
+  20 repeated ZMR runs of the generated Android workflow demo app.
+- React Native/Expo fixture: generated app and ZMR workflow scenarios are
+  available through `zmr-create-react-native-expo-demo-app`; public timing rows
+  are still pending.
+- [2026-06-09 iOS simulator XCTest shim floor](2026-06-09-ios-xctest-floor.md):
+  native-path floor for the optimized iOS shim-backed smoke flow.
+- [2026-06-09 framework baseline status](2026-06-09-framework-baseline-status.md):
+  status for Detox, Flutter, Espresso, and other framework-specific baselines.