zeno-mobile-runner 0.1.3 → 0.2.0

package/CHANGELOG.md

@@ -4,6 +4,196 @@ All notable changes to Zeno Mobile Runner are tracked here.
 
 ## Unreleased
 
+## 0.2.0 (2026-06-10)
+
+### Added
+
+- Added a public-safe iOS simulator benchmark evidence pack with 20 repeated
+  runs of the generated iOS smoke scenario.
+- Added a public-safe iOS simulator baseline runner benchmark comparison on the
+  same generated demo app.
+- Added a second public-safe iOS baseline comparison plus a native shim floor
+  evidence pack for the generated demo app.
+- Added a richer public-safe iOS workflow benchmark pack covering profile
+  entry, catalog selection, save, review, and final-state assertion on the
+  generated demo app.
+- Added the first Android workflow benchmark pack for the generated demo app,
+  covering 20 repeated UIAutomator-path ZMR runs.
+- Added a generated React Native/Expo benchmark fixture with stable `testID`
+  values, accessibility labels, deep-link setup, and Android/iOS ZMR workflow
+  scenarios.
+- The trace viewer loads a served bundle directly from
+  `viewer/index.html?bundle=<url>`, so CI artifact links and shared triage can
+  open a trace without manual file selection.
+- The iOS XCTest shim cold-build timeout is tunable with the
+  `ZMR_IOS_SHIM_TIMEOUT_MS` environment variable for slower CI hardware.
+- Added a nightly `device-smoke` GitHub Actions workflow that runs the public
+  demo apps on a real Android emulator and iOS simulator and uploads traces,
+  reports, and redacted bundles as evidence artifacts.
+- Added real captured screenshots under `docs/assets/` (trace viewer, device
+  screens, CLI failure-diagnosis loop, HTML report) plus
+  `scripts/capture-screenshots.sh` to regenerate them from fresh demo runs.
+  The assets ship in the repository only, not in the npm package.
+- Added Mermaid architecture, verification-loop, trace-lifecycle, and
+  trace-to-test diagrams to the README and core docs, and rewrote the README
+  around the AI-coding-agent verification workflow.
+
+### Fixed
+
+- `zmr validate`, `zmr report`, `zmr export`, and `zmr import` now accept
+  flags before positional arguments, matching the documented command forms,
+  and unknown-flag errors print a help hint.
+- Generated Android demo scenarios clear app state before launching so
+  repeated runs no longer fail on leftover screens from a previous session.
+
+- Fixed generated iOS shim one-shot log file creation on macOS by using a
+  portable `mktemp` template with `XXXXXX` at the end.
+- Skipped the slow iOS system-open confirmation probe for simulator custom URL
+  schemes while keeping it for universal web links.
+
+## 0.1.8 (2026-06-06)
+
+### Changed
+
+- `zmr-release-readiness --target production` now enforces an
+  `agent workflow smoke` gate, satisfied by the local release gate or
+  structured MCP/JSON-RPC, trace, discovery, validation, and redacted-export
+  evidence.
+
+## 0.1.7 (2026-06-06)
+
+### Added
+
+- Added MCP `assert_visible`, `assert_not_visible`, and `assert_healthy` tools
+  so MCP agents can use assertion-grade checks without dropping to JSON-RPC.
+- Added MCP `erase_text`, `hide_keyboard`, `wait_not_visible`, `wait_any`, and
+  `scroll_until_visible` tools so agents can cover common mobile flow control
+  without dropping to JSON-RPC.
+- Added MCP `install_app`, `launch_app`, `stop_app`, `clear_state`, and
+  `swipe` tools so MCP agents can run full app lifecycle and gesture flows.
+- Added MCP `open_link`, unscoped `type`, and `press_back` trace events so
+  trace-backed discovery sees full simple action sessions from MCP agents.
+- Added MCP `trace_events` cursor metadata so MCP agents get `afterSeq`,
+  `nextSeq`, and `latestSeq` parity with JSON-RPC trace polling.
+- Added JSON-RPC `trace.explain` and MCP `trace_explain` so live agents can
+  get the same failure summary, diagnostics, and next commands as
+  `zmr explain --json` without leaving the session.
+- Added TypeScript, Python, Go, Rust, Swift, and Kotlin trace explanation helpers
+  so reference clients can call JSON-RPC `trace.explain` directly.
+- Added `zmr explore --from-trace <trace-dir> --out <scenario.json> --goal
+  <goal> --include-actions --validate --json` as a review-first CLI
+  exploration handoff for agents. It reuses trace-backed discovery, carries the
+  goal in JSON, and returns explicit guardrails instead of claiming autonomous
+  crawling.
+- Added JSON-RPC `trace.explore` and MCP `trace_explore` so live agents can
+  generate the same goal-carrying, review-required scenario draft without
+  leaving the active traced session.
+- Added TypeScript `exploreTrace()`, Python `explore_trace()`, Go
+  `ExploreTrace()`, Rust `explore_trace()`, Swift `exploreTrace()`, and Kotlin
+  `exploreTrace()` helpers for the new JSON-RPC exploration method.
+- Added trace discovery auditing: `trace.discover` records a `trace.discover` event
+  for JSON-RPC and MCP agent sessions so generated scenario candidates are
+  visible in the trace.
+- Added `zmr draft --from-trace <trace-dir> --out <scenario.json> --json` for
+  trace-backed, review-first scenario drafting. The command reads the latest
+  semantic snapshot artifact and writes a conservative surface-smoke scenario
+  with `launch`, `snapshot`, and stable `assertVisible` checks only.
+- Added `zmr draft --include-actions` so agent sessions can turn supported
+  successful trace actions into reviewable replay drafts while unsupported
+  events are skipped with warnings instead of guessed.
+- Added `zmr discover --from-trace <trace-dir> --out <scenario.json>
+  --include-actions --validate --json` as the first-class trace-to-test handoff
+  for agents. It reuses the review-first draft engine, can validate the
+  generated scenario before returning, and reports `mode: "discover"` for
+  tooling.
+- Added JSON-RPC `trace.discover` and MCP `trace_discover` so live agents can
+  generate the same trace-backed scenario candidate without shelling out to the
+  CLI after a session.
+- Added JSON-RPC `scenario.validate` and MCP `scenario_validate` so agents can
+  validate generated or edited scenario files in-band before running them.
+- Added TypeScript `discoverTrace()` and Python `discover_trace()` helpers for
+  the new JSON-RPC discovery method.
+- Added TypeScript `validateScenario()` and Python `validate_scenario()`
+  helpers for in-band scenario validation.
+- Added Go `DiscoverTrace()` and `ValidateScenario()` helpers so Go agents can
+  use the same trace-to-test and validation loop without raw JSON-RPC calls.
+- Added Rust `discover_trace()` and `validate_scenario()` helpers so Rust
+  agents and host-side harnesses can use the same trace-to-test and validation
+  loop without raw JSON-RPC calls.
+- Added Swift `discoverTrace()` and `validateScenario()` helpers so macOS
+  host-side agents can use the same trace-to-test and validation loop without
+  raw JSON-RPC calls.
+- Added Kotlin `discoverTrace()` and `validateScenario()` helpers plus an
+  always-run source parity test so Kotlin host-side agents keep the same
+  trace-to-test and validation entry points even on machines without Gradle.
+- Added a trace-backed discovery handoff to traced `zmr run --json`
+  `nextCommands`, so agents can generate a reviewable replay scenario directly
+  from a run summary.
+- Added `zmr run --discover-out <scenario.json> --json` so traced runs can
+  write and validate the reviewable replay scenario before returning the run
+  summary.
+- Added replay coverage metadata to `zmr draft --json`, `zmr discover --json`,
+  and embedded run discovery output so agents can see how many trace actions
+  became replay steps and how many were skipped.
+- Added coordinate-complete `ui.swipe` trace replay so JSON-RPC sessions and
+  traced `zmr run` flows can carry swipes into generated replay scenarios
+  without guessing missing coordinates.
+- Added traced `pressBack` replay parity for `zmr run`, so generated replay
+  scenarios preserve back-navigation steps from ordinary scenario runs.
+- Added direction and timeout preserving `scrollUntilVisible` trace replay so
+  generated scenarios keep the original scroll intent from traced runs and
+  live agent sessions.
+- Added selector and timeout preserving wait replay so trace-backed discovery
+  keeps successful `waitVisible`, `waitNotVisible`, and matched `waitAny`
+  steps from ordinary scenario runs.
+- Added timeout context to native selector wait trace events and timeout
+  diagnostics, so real-device shim traces carry the same timing evidence as
+  snapshot-backed waits.
+- Added replay metadata for successful `assertNoneVisible` and timed
+  `assertHealthy` trace events, so generated replay scenarios preserve
+  assertion intent instead of dropping those checks.
+- Added selector and timeout preserving `assertVisible` and `assertNotVisible`
+  replay so trace-backed discovery keeps assertion intent distinct from waits.
+- Added `zmr report --junit <report.xml>` so trace directories and benchmark
+  result directories can produce CI-friendly JUnit XML alongside HTML reports.
+- Updated generated app report and reliability scripts to write `junit.xml`
+  beside `report.html` by default.
+- Updated Android and iOS pilot wrappers to emit `junit.xml` beside every
+  generated `report.html`.
+- Updated CI and tagged release workflows to retain generated run evidence as
+  GitHub Actions artifacts: CI keeps traces, coverage output, and the built
+  `zmr` binary for 14 days, while releases keep the generated `dist/` bundle
+  for 30 days.
+- Updated artifact upload workflow steps to a Node 24-compatible
+  `actions/upload-artifact` major so CI stays ahead of GitHub Actions runtime
+  deprecations.
+- Added `schemas/draft-output.schema.json` and included it in `zmr schemas
+  --json`.
+- Added `schemas/discover-output.schema.json` and included it in `zmr schemas
+  --json`.
+
+## 0.1.6 (2026-06-05)
+
+### Added
+
+- Added `zmr inspect --json` as a read-only app and agent handoff command. It
+  reports app-local config status, `.zmr/AGENTS.md` presence, configured
+  Android/iOS smoke scenarios, safe next commands, and explicit claim limits
+  without launching devices or writing tests.
+- Added `schemas/inspect-output.schema.json` and included it in `zmr schemas
+  --json` for generated clients and agent tooling.
+
+### Changed
+
+- Tagged release workflow now uses npm trusted publishing instead of a
+  long-lived `NPM_TOKEN` secret.
+- Fixed the tagged release publish step so npm receives the generated local
+  tarball instead of interpreting the path as a GitHub package spec.
+- Updated the tagged release workflow to Node 24 so npm trusted publishing can
+  use the GitHub Actions OIDC identity with a current npm CLI.
+- Added a public production-readiness checklist that ties release, framework,
+  reliability, trace privacy, and agent workflow claims to concrete evidence.
+
 ## 0.1.3 (2026-06-03)
 
 ### Fixed
@@ -329,8 +519,8 @@ All notable changes to Zeno Mobile Runner are tracked here.
 - Tagged release workflow now publishes GitHub artifact attestation for release
   archives and metadata.
 - Tagged release workflow now builds the npm tarball, attests it, uploads it
-  with the release assets, and publishes with npm provenance when `NPM_TOKEN`
-  is configured.
+  with the release assets, and can publish with npm provenance from supported
+  CI.
 - Release manifests and checksum verification now include generated npm
   tarballs when present.
 - Native selector wait timeouts now capture one final snapshot when possible,

package/FEATURES.md

@@ -22,7 +22,8 @@ state, and writes deterministic traces. It does not embed an LLM.
 
 - npm-first installation with `zeno-mobile-runner` as a dev dependency.
 - `npx zmr-wizard` scaffolds `.zmr/config.json`, Android and iOS smoke
-  scenarios, optional app package scripts, and `traces/` gitignore rules.
+  scenarios, optional app package scripts, HTML/JUnit report scripts, and
+  `traces/` gitignore rules.
 - `zmr init --app` provides the same app-local bootstrap for source and archive
   installs.
 - `.zmr/config.json` is schema validated, auto-discovered from app checkouts,
@@ -34,14 +35,44 @@ state, and writes deterministic traces. It does not embed an LLM.
 
 - JSON-RPC v1 over newline-delimited stdio or localhost TCP.
 - MCP stdio server with mobile-native tools for AI agents, including semantic
-  snapshots, selector actions, waits, trace polling, and trace export.
+  snapshots, selector actions, waits, assertions, trace polling, trace
+  exploration, trace discovery, and trace export.
 - `runner.capabilities`, `device.list`, `session.create`,
   `observe.snapshot`, `observe.semanticSnapshot`, UI actions, waits,
-  assertions, live trace events, and redacted trace export.
+  assertions, scenario validation, live trace events, trace-backed exploration,
+  trace-backed discovery, and redacted trace export.
 - TypeScript, Python, Go, Rust, Swift, and Kotlin reference clients.
 - Machine-readable CLI output for `zmr version --json`, `zmr schemas --json`,
-  `zmr doctor --json`, `zmr devices --json`, `zmr validate --json`,
+  `zmr inspect --json`, `zmr doctor --json`, `zmr devices --json`,
+  `zmr discover --json`, `zmr draft --json`, `zmr validate --json`,
   `zmr run --json`, and `zmr explain --json`.
+- Traced `zmr run --json` output includes executable follow-up commands for
+  reports, failure explanation, trace-backed scenario discovery, and redacted
+  export.
+- `zmr run --discover-out <scenario.json> --json` can generate and validate a
+  reviewable replay scenario from the trace immediately after the run.
+- `zmr explore --from-trace --goal ... --json` gives CLI agents a
+  goal-carrying, review-first exploration handoff with explicit guardrails,
+  replay coverage, validation, and deterministic next commands.
+- JSON-RPC `trace.explore` and MCP `trace_explore` expose the same
+  goal-carrying, review-required trace exploration from active traced sessions.
+- `zmr discover --from-trace` turns a traced agent session into a reviewable
+  scenario candidate and can validate the generated scenario before returning.
+- `zmr draft --from-trace` turns a traced semantic snapshot into a reviewable
+  surface-smoke scenario with conservative `assertVisible` checks.
+- `zmr draft --include-actions` can turn supported successful trace actions
+  into a replay draft while warning instead of guessing unsupported events.
+  Wait replay preserves selectors and timeouts when the trace records them.
+  Native selector wait traces include timeout context for successful waits and
+  timeout diagnostics.
+  Assertion replay preserves `assertVisible` and `assertNotVisible` selectors
+  and timeouts, `assertNoneVisible` selector arrays, and `assertHealthy`
+  timeouts when the trace records them.
+  Selector scroll replay preserves direction and timeout when the trace records
+  them.
+- Draft and discover JSON include `replay` coverage metadata so agents can
+  compare trace action events considered for replay, generated replay steps,
+  and skipped events.
 - Public JSON Schemas for scenarios, snapshots, semantic snapshots, action
   results, trace events, protocol messages, setup diagnostics, and release
   manifests.
@@ -58,6 +89,8 @@ state, and writes deterministic traces. It does not embed an LLM.
 - Wait and retry behavior around transient observation failures.
 - Import helper for a documented subset of common mobile-flow YAML commands
   into native `.zmr/*.json` scenarios.
+- Trace-backed draft helper for generating reviewable surface and replay
+  scenarios from observed UI state and successful supported trace actions.
 
 ## Traces And Diagnostics
 
@@ -68,6 +101,10 @@ state, and writes deterministic traces. It does not embed an LLM.
   selected node details, payloads, artifact links, and snapshot replay
   controls.
 - `zmr explain` summarizes failed traces for humans and agents.
+- `zmr report --junit <report.xml>` writes CI-friendly JUnit XML for trace
+  directories and benchmark result directories.
+- Android and iOS pilot wrappers emit `junit.xml` next to generated HTML
+  reports, so CI can collect the same artifacts used by app-local scripts.
 - Redacted `.zmrtrace` export can replace or omit screenshots, omit screen
   recordings, and redact common secrets plus app-configured denylist fields.
 
@@ -75,6 +112,8 @@ state, and writes deterministic traces. It does not embed an LLM.
 
 - `zmr-benchmark` repeats ZMR scenarios with pass-rate, failure-count, and p95
   duration gates.
+- Benchmark directories can be rendered as both HTML and JUnit XML artifacts
+  with `zmr report`.
 - `zmr-benchmark-command` records normalized rows for app-local baseline
   commands without hardcoding another tool.
 - `zmr-compare-benchmarks` compares candidate and baseline rows into generic
@@ -89,17 +128,21 @@ state, and writes deterministic traces. It does not embed an LLM.
 - Release archive builder with checksums, SPDX SBOM, third-party notices,
   generated Homebrew formula, and `RELEASE_MANIFEST.json`.
 - npm package tarball generation with bundled prebuilt binaries.
-- `zmr-release-readiness` checks repeated app/device evidence for teams that
-  want a machine-readable readiness summary.
+- `zmr-release-readiness` checks repeated app/device evidence and the
+  agent workflow smoke for teams that want a machine-readable readiness
+  summary.
 - Tagged release workflow with artifact attestation and optional npm
   provenance publishing.
+- CI workflow retains run traces, coverage output, and built runner artifacts
+  for 14 days; tagged release workflow retains the generated `dist/` bundle
+  for 30 days.
 - Security, contribution, trace privacy, troubleshooting, protocol versioning,
   app integration, benchmarking, and npm packaging docs.
 - Reusable agent skill under `skills/zmr-mobile-testing/`.
 
 ## Current Limitations
 
-- Current release status is `0.1.3`, a public developer preview rather than
+- Current release status is `0.2.0`, a public developer preview rather than
   a production-stable `1.0.0`.
 - Physical iOS log capture is still simulator-first. Physical iOS screenshots
   are available when the XCTest/XCUIAutomation shim is configured.