agent-scenario-loop 0.1.2 → 0.1.4

package/docs/authoring.md

@@ -80,12 +80,14 @@ Preferred fields:
 - `journey`: human-readable intent, actor, start state, and end state
 - `comparisonLane`: default historical baseline lane for runs of this scenario
 - `milestones`: named event checkpoints with phases and timeouts
-- `cycles`: iteration count and stop policy
+- `cycles`: iteration count, stop policy, and optional setup/body step ids
 - `budgets`: thresholds to evaluate only after truth-event health passes
 - `artifacts`: required and optional evidence outputs
 
 Use `comparisonLane` when a scenario should always compare within one stable proof mode, such as `feed-open-android-live`. Profile CLIs can also receive `--comparison-lane`; the CLI flag wins when one-off runs need a different lane.
 
+For repeated scenarios, separate setup from the measured body. Commands that clear state, navigate home, dismiss modals, or establish readiness should not be measured every iteration unless that cleanup is the journey under test. Use `cycles.setupStepIds` for leading setup commands that run once, or `cycles.bodyStepIds` to name the repeated command body. If neither is provided, ASL profile-session runners infer a conservative setup prefix from readiness waits and measured milestone budgets, but explicit ids are clearer for complex flows.
+
 ## Truth Events
 
 Treat truth events as app-owned facts, not runner observations. The app should emit them from the code path that actually represents the journey state.
@@ -105,6 +107,32 @@ Weak truth events:
 
 Timing is not trusted unless scenario health passes. If a required truth event is missing, the run can still write artifacts, but verdicts and comparisons must remain inconclusive.
 
+### Resume Scenarios
+
+`--lifecycle-phase resume` and related runner controls assert runner-owned lifecycle setup in `manifest.environment`; they do not create product truth events. If a scenario waits for `app_resumed`, `feed_restored_after_resume`, or another resumed-state milestone, the app must emit that event from the code path that proves resumed product readiness.
+
+## Budget Intervals
+
+Milestone budgets measure the interval the scenario names. A budget with only `toMilestone` measures elapsed time from the run or session clock origin to each matching milestone occurrence. That is correct for startup and first-usable-screen budgets, but it is cumulative for repeated interactions.
+
+For transition or gesture budgets, provide both ends of the interval:
+
+```json
+{
+  "name": "surface transition p95",
+  "source": "milestone",
+  "metric": "p95",
+  "unit": "ms",
+  "limit": 300,
+  "fromMilestone": "surfaceTransitionRequested",
+  "toMilestone": "surfaceSettled"
+}
+```
+
+Use app-owned truth events for both milestones. Do not use a command-delivered event as the start point unless that command delivery is the product fact being measured.
+
+When the start event is useful only as a timing anchor, keep it optional and keep scenario health tied to the completion truth. For repeated flows, set `metricEvents.milestone` or the completion-oriented cycle events to the truth that proves the iteration completed, then use the optional intent milestone as `fromMilestone` in the budget.
+
 ## Steps
 
 Use steps to describe intent and required adapter actions:
@@ -118,6 +146,8 @@ Use steps to describe intent and required adapter actions:
 
 Use `driverAction` only when the scenario truly requires a concrete operation such as `tap`, `scroll`, `assertVisible`, `screenshot`, `readLogs`, or `collectPerfSignals`. The planner fails early when no active runner or provider can satisfy a required driver action.
 
+For profile-session command transport, platform `waitMs` metadata is queue pacing. ASL preserves it in storage and deep-link command envelopes and waits before releasing the next queued command. App-owned milestones still provide the truth that a command produced the intended product state.
+
 Use `selector` to describe the intended app target without committing the scenario to one driver. Supported selector kinds are `testId`, `accessibilityId`, `accessibilityLabel`, `text`, `resourceId`, and `xpath`.
 
 ```json
@@ -159,12 +189,14 @@ asl-profile-android \
   --capture uiTree:artifacts/provider/ui-tree.json
 ```
 
-Signals are copied into `signals/js`, `signals/memory`, or `signals/network` and listed in `manifest.json`. Captures are copied into `captures`; screenshots are listed in `artifacts.captures.screenshots`, while video and UI tree captures replace the matching named capture path in the manifest. Every attached file is also listed in `artifacts.evidenceAttachments` with kind, run-relative path, source filename, byte size, and sha256 hash. Attached provider evidence is preserved as proof, but timing verdicts still come from app-owned truth events and budgets.
+Signals are copied into `signals/js`, `signals/memory`, or `signals/network` and listed in `manifest.json`. Captures are copied into `captures`; screenshots are listed in `artifacts.captures.screenshots`, while video and UI tree captures replace the matching named capture path in the manifest. Every attached file is also listed in `artifacts.evidenceAttachments` with kind, run-relative path, source filename, byte size, sha256 hash, completeness status, corruption status, redaction status, and transformation list. Attached provider evidence is preserved as proof, but timing verdicts still come from app-owned truth events and budgets.
 
-Provider manifests can also declare `providerCommands`. Profile runners execute those commands when passed with `--provider <manifest>`, but only when the provider manifest includes the selected platform. A provider with `platforms: ["ios"]` passed to an Android profile writes failed `health.json` with `provider_platform_unsupported` and does not run the command. Commands run without a shell, can use placeholders such as `{providerDir}`, `{runDir}`, `{runId}`, `{scenarioId}`, and `{platform}`, and must declare their output files. Provider-channel outputs are copied or preserved under `raw/providers/<provider-id>/` and inventoried in `artifacts.evidenceAttachments`; signal and capture outputs can still map into the standard `signals/*` or `captures/` folders. Command stdout, stderr, exit code, phase, and argv are preserved under `raw/provider-commands/`. When a provider command exits nonzero, the runner writes failed `health.json`, inconclusive `verdict.json`, and `agent-summary.md` with a next-action hint instead of making timing claims.
+Provider manifests can also declare `providerCommands`. Profile runners execute those commands when passed with `--provider <manifest>`, but only when the provider manifest includes the selected platform. Commands run after the platform evidence source has been collected or supplied, so heavy diagnostics can be attached in a post-loop rehydration run with `--adb-artifacts` or `--simctl-artifacts` instead of perturbing the measured command window. Use `phase: "afterCapture"` for capture-sidecar diagnostics and `phase: "postRun"` for post-profile enrichment; older `capture` phase values remain accepted for existing manifests. A provider with `platforms: ["ios"]` passed to an Android profile writes failed `health.json` with `provider_platform_unsupported` and does not run the command. Commands run without a shell, can use placeholders such as `{providerDir}`, `{runDir}`, `{runId}`, `{scenarioId}`, and `{platform}`, and must declare their output files. Provider-channel outputs are copied or preserved under `raw/providers/<provider-id>/` and inventoried in `artifacts.evidenceAttachments`; signal and capture outputs can still map into the standard `signals/*` or `captures/` folders. An output can set `required: true` when the provider treats that file as required evidence; matching entries in `manifest.artifacts.diagnostics` then remain marked required in addition to scenario-authored `artifacts.required` and `requiredCapabilities`. Command stdout, stderr, exit code, phase, and argv are preserved under `raw/provider-commands/`. When a provider command exits nonzero, the runner writes failed `health.json`, inconclusive `verdict.json`, and `agent-summary.md` with a next-action hint instead of making timing claims.
 
 The `examples/runners/script-*.json` manifests show package-neutral wrappers for accessibility, profiler, memory, and network evidence. They intentionally reference placeholder commands such as `capture-accessibility` or `capture-memory`; replace those with your project-local script, binary, or agent command. The contract that matters is the declared output path and evidence kind, not the specific tool used to create the file.
 
+For React Native profiling, prefer a provider that emits both the raw profiler export and a structured JSON summary. JSON outputs with `kind: "profiler"` are validated against ASL's profiler evidence schema, so include the provider id, platform, run id, scenario id, tool metadata, completeness status, and at least one content surface such as samples, metrics, events, traces, a profile object, summary, or attachment references. If profiler evidence depends on explicit start/stop commands, model it as lifecycle-owned evidence: declare `captureMode`, `profileKind`, `lifecycle`, `targetBinding`, and `comparability` so agents can distinguish passive existing reports from session captures, inline captures that may perturb budgets, and after-capture or rehydrated diagnostics. CPU summaries derived from a prior profiler session should not be attached as passive evidence unless the provider also preserves the session provenance and raw attachments. If your profiler only produces a native trace or flamegraph, attach it as preserved evidence and avoid making performance claims until a provider translates the relevant facts into structured metrics.
+
 ## Artifacts
 
 A completed profile run should leave the standard artifact set:
@@ -196,3 +228,7 @@ Run the release gate before publishing package changes:
 ```bash
 pnpm release:check
 ```
+
+## Read next
+
+- [Adapter Onboarding](adapters.md) for runner and provider integration

package/docs/concepts.md

@@ -104,33 +104,12 @@ The tooling may change. The runners may change. The agents may change. The scena
 
 That is a different philosophy from frameworks that primarily evaluate agents. Agent Scenario Loop is built to evaluate the evolution of software.
 
-## How it differs from testing frameworks
+## Boundary
 
-Agent Scenario Loop does not make existing testing frameworks obsolete.
+Agent Scenario Loop is not a replacement for testing frameworks, automation tools, mobile drivers, profilers, or agent evaluation systems. Those tools can still execute or observe work.
 
-Traditional frameworks usually optimize for:
-
-> Did the application behave correctly?
-
-Agent Scenario Loop optimizes for:
-
-> What did we learn from running this scenario?
-
-Both questions matter. Agent Scenario Loop focuses on the second question by preserving health, verdicts, metrics, logs, traces, comparisons, and other run evidence in a stable artifact shape.
-
-## How it differs from agent evaluation
-
-Agent Scenario Loop is not primarily evaluating agents.
-
-An agent may execute part of a run. A runner may drive a device. A profiler may collect signals. None of those is the center of the model.
-
-The scenario is.
-
-The feed, livestream, upload flow, checkout flow, or conversation thread is the thing being studied over time.
+The canonical boundary list lives in [What It Is Not](../README.md#what-it-is-not).
 
 ## Read next
 
 - [Principles](principles.md) for the project doctrine
-- [Contracts](contracts.md) for the current artifact and package surface
-- [Live Proofs](live-proofs.md) for fixture, Android, iOS, and comparison runs
-- [Runner docs](../runner/README.md) for the host execution boundary

package/docs/consumer-rehearsal.md

@@ -16,6 +16,32 @@ Package gates run child package-manager and CLI commands with a bounded timeout.
 ASL_PACKAGE_GATE_TIMEOUT_MS=300000 pnpm consumer:rehearse
 ```
 
+## Downstream Local-Package Gate
+
+Before publishing a release candidate, validate the packed local package inside at least one real downstream app when that app has already adopted durable ASL scenarios. This catches package, runner, schema, and helper regressions before npm distribution.
+
+From this repository, run the opt-in downstream gate with an explicit app root and explicit command arrays:
+
+```bash
+pnpm downstream:local-package -- \
+  --app-root /path/to/adopter-app \
+  --expected-branch chore/agent-scenario-loop-adoption \
+  --command-json '["pnpm","run","asl:validate"]'
+```
+
+The gate packs the current checkout, installs the tarball into the downstream app with `pnpm add`, verifies `node_modules/agent-scenario-loop/package.json` matches the local candidate version, runs the supplied commands, and restores `package.json` plus `pnpm-lock.yaml` unless `--keep-install` is passed. Generated downstream proof artifacts remain the consumer app's local ignored state.
+
+For live probes, pass direct package CLI commands as additional JSON arrays so the target scenario and artifact root are explicit:
+
+```bash
+pnpm downstream:local-package -- \
+  --app-root /path/to/adopter-app \
+  --command-json '["pnpm","run","asl:validate"]' \
+  --command-json '["node_modules/.bin/asl-profile-android","--config","asl.config.json","--scenario","scenarios/mobile/first-journey.json","--adb-capture","--profile-session","--android-profile-session-storage","--launch","--out","artifacts/asl/android","--run-id","first-journey-android-local-candidate"]'
+```
+
+Keep adopter-specific app ids, storage keys, dev-client URLs, simulator UDIDs, auth state, accounts, and scenarios in ignored local environment state or in the consuming app. ASL owns the package candidate and evidence contract; the downstream app owns product truth.
+
 ## 1. Initialize The Scaffold
 
 From the consuming app root:
@@ -81,7 +107,7 @@ asl-check-plan --scenario scenarios/mobile/first-journey.json --runner runner-ma
 asl-profile-ios --config asl.config.json --scenario scenarios/mobile/first-journey.json --simctl-capture --profile-session --profile-session-storage --launch --out artifacts/asl/ios --run-id first-journey-ios-live --comparison-lane first-journey-ios-live
 ```
 
-For Expo dev-client builds, set `ASL_ANDROID_DEV_CLIENT_URL` or `ASL_IOS_DEV_CLIENT_URL` to the app's dev-client URL in ignored local env state. Android opens it before profile-session deep links; iOS opens it before reading stored profile-session evidence. If Android bundle startup is slow, set `ASL_ANDROID_DEV_CLIENT_READY_PATTERN='Running "main"'` so profile-session links wait for app runtime readiness evidence.
+For Expo dev-client builds, set `ASL_ANDROID_DEV_CLIENT_URL` or `ASL_IOS_DEV_CLIENT_URL` to the app's dev-client URL in ignored local env state. Prefer the LAN URL advertised by Metro for physical-device validation. Use `127.0.0.1` only when the selected simulator/emulator resolves that address back to the host Metro process. Android opens the dev-client URL before profile-session control. When Android storage transport is enabled, ASL waits for `Running "main"` by default before writing profile-session storage; override `ASL_ANDROID_DEV_CLIENT_READY_PATTERN` only when the app has a better readiness marker. If startup readiness fails, ASL reports an unhealthy run and skips command delivery instead of writing into a stale native shell. iOS opens the dev-client URL before reading stored profile-session evidence.
 
 When Android deep-link delivery is unreliable in a dev-client shell, use `--android-profile-session-storage` so `asl-profile-android` seeds the app-owned AsyncStorage session through `run-as` before collecting evidence. The runner reads the selected device clock for the session start timestamp, which keeps app-emitted milestone durations meaningful. Keep custom storage key overrides local to the consuming app.
 
@@ -113,3 +139,7 @@ Before expanding beyond the first journey, confirm:
 - at least one platform has a passed live proof
 
 Only then add more scenarios, providers, or runner adapters.
+
+## Read next
+
+- [Live Proofs](live-proofs.md) for fixture, Android, iOS, comparison, and release-proof commands

package/docs/contracts.md

@@ -4,6 +4,8 @@ This package ships the scenario, runner, and artifact contracts that make Agent
 
 The package is intentionally contract-first: adopt the scenario and artifact shape once, then add or swap runner loops without rewriting your scenarios.
 
+See [Architecture](architecture.md) for the TypeScript-first implementation and language-neutral contract boundary.
+
 ## What ships today
 
 - [app/profile-session.ts](../app/profile-session.ts): thin React Native integration for session control, truth events, and signal attachments
@@ -64,15 +66,20 @@ Portable scenario manifests describe the durable app behavior before choosing a
 - `truthEvents`: app-owned milestone events keyed by stable milestone id
 - `milestones`: inspectable milestone list with event names, phases, timeouts, and descriptions
 - `expectedEvents`: event names the runner or log ingest should expect to observe
-- `cycles`: repeat count, warmup count, and failure policy for repeated journeys
+- `cycles`: repeat count, warmup count, failure policy, and optional setup/body step ids for repeated journeys
 - `budgets`: product thresholds evaluated only after scenario health passes
 - `steps`: runner-facing launch, command, wait, gesture, and capture actions
 - `selector`: optional app target on a step, such as a test id, accessibility id, label, text, resource id, or xpath
+- `uiContext`: optional UI ownership requirement on a step; UI driver actions default to `app`
 - `artifacts`: required and optional evidence outputs
 
 The scenario contract is intentionally runner-neutral. Runners can map steps to adb, XcodeBuildMCP, agent-device, accessibility tools, profilers, or custom scripts while preserving the same journey, milestones, budgets, and expected events.
 
-Runner capabilities describe ownership, such as launch, session control, command execution, log capture, artifact writing, or profiler support. Driver actions describe the concrete operations an adapter can perform inside a run. A runner may be able to own a scenario lifecycle without supporting every driver action; the planner fails only when a required step declares a `driverAction` that the selected runner or an active provider does not declare in `driverActions`.
+For repeated mobile command scenarios, `cycles.setupStepIds` names leading setup commands that run once before measured cycle work, while `cycles.bodyStepIds` names the first repeated body commands when inference would be ambiguous. Built-in profile-session runners also infer a setup prefix conservatively: leading readiness commands or leading commands before the first measured milestone command run once, and the remaining command body repeats for `cycles.iterations`. Wait gates remain strict; ASL does not synthesize missing app-owned truth events.
+
+Runner capabilities describe ownership, such as launch, session control, command execution, log capture, artifact writing, or profiler support. Driver actions describe the concrete operations an adapter can perform inside a run. UI contexts describe which surface the runner or provider can own: `app`, `systemDialog`, `notificationShade`, `externalBrowser`, `webView`, `shareSheet`, `picker`, or `otherApp`. UI and capture driver actions default to `app` when a step omits `uiContext`; a scenario must opt into system or external contexts explicitly. A runner may be able to own a scenario lifecycle without supporting every driver action or UI context; the planner fails when a required step declares a `driverAction` or `uiContext` that the selected runner or an active provider does not declare.
+
+Planner compatibility artifacts and planner-derived `health.json` include a `downgradePolicy` block with `mode: "no-silent-downgrade"`. Required capability, driver-action, UI-context, or artifact gaps are recorded as `unsupported`; optional gaps are recorded as warnings. `allowedSubstitutions` and `substitutions` are explicit arrays, so future semantic downgrades must be visible in artifacts instead of being inferred from a passed plan.
 
 `buildScenarioExecutionPlan()` turns the same scenario steps into a deterministic adapter-facing work list. Each normalized step records the scenario step id, original kind, required flag, optional driver action, and the runner port method that owns it: `launch`, `executeStep`, `waitForTruthEvent`, or `captureEvidence`.
 
@@ -80,11 +87,15 @@ Android adb capture routes normalized steps with `driverAction: "tap"`, `"scroll
 
 When Android adb `tap` or `scroll` steps provide a portable selector instead of coordinates, the runner captures `uiautomator dump` output, resolves supported selector kinds against node bounds, and derives adb input coordinates before executing the action. Built-in Android selector resolution supports `testId`, `resourceId`, `accessibilityId`, `accessibilityLabel`, and `text`; `xpath` stays available for external runners with native selector engines.
 
-I/O from iOS simctl capture routes through the simctl driver adapter. `readLogs` preserves bounded simulator logs under `raw/ios-simctl-log.txt`. A scenario step with `driverAction: "screenshot"` or `artifact: "screenshot"` requests a screenshot capture, defaulting to `captures/ios-screenshot.png`; when `--screenshot-type`, `--screenshot-display`, or `--screenshot-mask` are supplied to `asl-ios-simctl`, the command passes those supported `simctl io screenshot` options and records them in capture metadata. The profile manifest records the resulting capture path in `artifacts.captures.screenshots`.
+I/O from iOS simctl capture routes through the simctl driver adapter. `readLogs` preserves bounded simulator logs under `raw/ios-simctl-log.txt`. A scenario step with `driverAction: "screenshot"` or `artifact: "screenshot"` requests a screenshot capture, defaulting to `captures/ios-screenshot.png`. The profile manifest records the resulting capture path in `artifacts.captures.screenshots`, and capture metadata records any supported simulator screenshot options the runner used.
+
+Manifest artifact paths are evidence claims. Optional diagnostics such as `captures.video`, `captures.uiTree`, `raw.deviceLog`, JS/memory/network signals, accessibility exports, and profiler files appear as paths only when the file was produced or intentionally referenced as a sidecar dependency. Every profile manifest also includes `artifacts.diagnostics`, an inventory of common diagnostic surfaces with `kind`, `status`, `required`, optional `path`, and a `reason`/`nextAction` when evidence was unavailable or not requested.
 
 Planner compatibility also validates the adapter metadata that built-in runners require. Android adb `tap` steps need either `adapterOptions.androidAdb.x/y` or a portable selector; Android adb `scroll` steps need either `startX/startY/endX/endY` or a portable selector; iOS simctl command metadata needs non-empty command strings and positive integer waits/repeat counts. Argent `tap` steps need `adapterOptions.argent.x/y`, Argent `scroll` steps need `adapterOptions.argent.startX/startY/endX/endY`, and Argent `assertVisible` steps need a portable selector. These failures become `invalid_adapter_options` health checks before runtime execution starts.
 
-Adapter-target fixtures such as `agent-device-android`, `agent-device-ios`, `argent-ios`, `argent-android`, `argent-react-profiler-provider`, and `axe-accessibility-provider` describe where external tools can plug into the same contract. They are schema-checked and planner-tested capability manifests. The bundled `agent-device` capture runner implements the portable interaction subset for iOS and Android; broader agent-device surfaces such as React DevTools, traces, network, and performance still need explicit adapters or provider attachments before they become part of the stable artifact contract. The bundled Argent runner implements launch, coordinate-backed gestures, screenshot requests, and description-backed visibility proof for portable selector match modes while keeping React profiler output in a separate Android evidence-provider lane. Argent command-surface checks prove the configured tools exist; runtime health still owns whether the selected device backend produced screenshot evidence. Required screenshot failures fail health, and optional screenshot failures are preserved as warnings. Active evidence providers can satisfy required evidence artifacts and provider-owned driver actions such as `collectPerfSignals`; providers outside the selected platform do not contribute to the match. When those tools write files independently, profile CLIs can attach the files with `--signal <js|memory|network>:<path>` or `--capture <screenshot|video|uiTree>:<path>` so provider evidence lands in the stable manifest and artifact layout. The `script-accessibility-provider`, `script-profiler-provider`, `script-memory-provider`, and `script-network-provider` examples show provider-command wrappers for project-local tools without making those tools package dependencies.
+Adapter-target fixtures such as `agent-device-android`, `agent-device-ios`, `argent-ios`, `argent-android`, `argent-react-profiler-provider`, and `axe-accessibility-provider` describe where external tools can plug into the same contract. They are schema-checked and planner-tested capability manifests. The bundled `agent-device` capture runner implements the portable interaction subset for iOS and Android; broader agent-device surfaces such as React DevTools, traces, network, and performance still need explicit adapters or provider attachments before they become part of the stable artifact contract. The bundled Argent runner implements launch, coordinate-backed gestures, screenshot requests, and description-backed visibility proof for portable selector match modes while keeping React profiler output in a separate Android evidence-provider lane. Argent command-surface checks prove the configured tools exist; runtime health still owns whether the selected device backend produced screenshot evidence. Required screenshot failures fail health, and optional screenshot failures are preserved as warnings. Active evidence providers can satisfy required evidence artifacts and provider-owned driver actions such as `collectPerfSignals`; providers outside the selected platform do not contribute to the match. When those tools write files independently, attached provider evidence lands in the stable manifest and artifact layout. The `script-accessibility-provider`, `script-profiler-provider`, `script-memory-provider`, and `script-network-provider` examples show provider-command wrappers for project-local tools without making those tools package dependencies.
+
+Profiler evidence is a first-class artifact kind, but ASL does not pretend every profiler tool has the same native format. JSON profiler outputs should satisfy [schemas/profiler.schema.json](../schemas/profiler.schema.json), including provider, platform, run, scenario, tool/completeness metadata, and at least one useful content surface such as samples, metrics, events, traces, a profile object, summary, or referenced attachments. Lifecycle-backed profilers should also declare whether evidence came from passive report ingestion, an explicit session, inline capture, `afterCapture`, `postRun`, or rehydration; whether the target device/app binding was verified; whether capture perturbed timing; and whether the output is comparable or diagnostic-only. Native traces, CPU profiles, flamegraphs, React DevTools exports, and recordings can still be attached as profiler evidence through provider outputs, but agents should treat them as preserved evidence until a provider also emits structured metrics that ASL can compare or summarize.
 
 ## Public artifact layout
 
@@ -111,11 +122,23 @@ Profile runner artifacts:
 
 `manifest.json`, `metrics.json`, `budget-verdict.json`, and `causal-run.json` are schema-checked before the runner writes them. This keeps profile artifacts stable across fixture logs, adb-captured logs, and future runner adapters.
 
+`causal-run.json` preserves app-emitted timeline events through the public causal phase/status vocabulary. If an app emits richer phase or status values, ASL writes schema-valid top-level values and preserves the originals as timeline metadata. Timeline metadata also preserves scalar correlation fields such as `iteration`, `sequence`, `queueId`, `commandId`, `operationId`, `attemptId`, and `clockDomain` when the app emits them. Profile-session command acknowledgements are included as ASL-owned timeline entries with command status, result, source, sequence, queue, wait, and command ID metadata, so agents can inspect runtime ordering without treating command transport as product truth. Repeated runs include `iterationSummary` so agents can distinguish complete, partial, failed, and timeout iteration evidence without scraping raw logs. Scenarios without budget thresholds still produce schema-valid causal artifacts with an empty `budgets` object.
+
+`manifest.attempt` records the run attempt identity and terminal semantics independently of prose summaries. It includes an `attemptId`, `attemptNumber`, `maxAttempts`, optional retry lineage, terminal state, failure classification, cleanup outcome, and whether preserved partial artifacts are valid for diagnosis. Retry attempts must identify the prior attempt and retry reason. A failed attempt can therefore keep usable raw evidence without implying that product verdict, timing, or comparison claims are trustworthy.
+
+`manifest.provenance.cohort` records product-neutral compatibility inputs for comparing runs. Profile runners populate known fields such as `appId`, `platform`, `runnerName`, `runnerVersion`, `commandTransport`, and active provider IDs; richer callers can add app/build version, build mode, OS version, device class, feature flags, and seed identity. ASL derives `manifest.provenance.cohortHash` from the normalized cohort. Latest-trusted comparison requires the same cohort hash when the current run records one, so old artifacts remain comparable only when the current artifact has not opted into cohort-aware selection.
+
+`manifest.attempt.terminalState` uses a terminal vocabulary of `passed`, `failed`, `timeout`, `cancelled`, `aborted`, `inconclusive`, `unsupported`, `skipped`, and `unhealthy`. Attempt construction rejects misleading terminal combinations: passed attempts must end as `passed`, failed attempts must use a failure terminal state, timeout/cancelled/aborted attempts must preserve valid partial artifact paths, and cleanup statuses such as `passed`, `failed`, or `partial` must include a cleanup message. `manifest.environment` records product-neutral lifecycle and environment preconditions and postconditions. Each field is an assertion object with a `value` and `evidence` state. Generated profile artifacts default to `value: "unknown"` and `evidence: "not-asserted"` unless the runner can prove more. The dedicated `lifecyclePhase` assertion supports `cold-launch`, `warm-launch`, `hot-launch`, `resume`, `foreground`, `background`, `force-stop`, `process-death`, `scene-recreation`, `activity-recreation`, `os-reclaim`, `reboot`, and `relaunch`. This preserves what the runner did not prove instead of letting agents infer installed state, app data state, auth state, route, foreground state, permissions, locale, timezone, theme, font scale, orientation, network, animations, cleanup, data, or artifact completeness from surrounding logs.
+
+Profile `agent-summary.md` files include an `attempt` section when the run has a manifest attempt block, including terminal state, cleanup state, partial-artifact validity, and retry lineage. Latest-trusted baseline selection treats attempt-aware runs as baseline-trusted only when health and verdict passed, the attempt is a clean first attempt, cleanup did not fail or remain partial, and partial artifacts are not marked valid diagnostic fragments. Older artifacts without `manifest.attempt` remain legacy-trusted when health and verdict passed, but new attempt-aware runs cannot hide retry laundering behind a green final verdict.
+
+Profile runners assert only environment facts they own. Every completed profile manifest records ASL-controlled artifact completeness and cleanup postconditions. Live adb/simctl capture paths also assert runner-controlled foreground state, explicit lifecycle preconditions, and foreground postconditions. Use `--lifecycle-phase <phase>` when a runner can prove a non-cold precondition such as `warm-launch` or `resume`; log-ingest and preexisting artifact ingestion keep those fields `unknown/not-asserted`. Lifecycle assertions are not product milestones: a runner proving `lifecyclePhase: "resume"` does not synthesize `app_resumed` or any other app truth event. Resume readiness must still be emitted by the consuming app when a scenario waits for it.
+
 Aggregate live proof commands write `live-proof.json` and `agent-summary.md` under `_live-proof/<run-id>`. The live-proof artifact points to preflight evidence, every scenario run, optional interaction proofs from tools such as agent-device or Argent, optional skipped interaction proof declarations, and optional latest-trusted comparison outputs, giving agents one stable entrypoint after a proof run. Preflight, profile, and interaction pointers include health and verdict status from the linked run artifacts, so agents can see what passed before opening deeper evidence. Interaction proof pointers also include sidecar screenshot capture inventory when the sidecar produced screenshots, plus `warnings` when optional sidecar checks failed without invalidating the required proof. If profile health or verdict fails, requested sidecars are not executed; they are recorded in `skippedInteractionProofs` with a reason and next action so agent feedback stays explicit without mixing runner evidence into an untrusted timing run. The aggregate artifact records `status`, `comparisonStatus`, `comparisonCounts`, optional per-comparison `metricSummary` counts/highlights, and a `nextAction` hint so agents can distinguish failed proof gates, regressions, mixed metric movement, missing baselines, inconclusive comparisons, partial sidecar evidence, and clean summaries without scraping prose.
 
 Platform-set proof commands write `live-proof-set.json` and `agent-summary.md` under the caller-provided proof-set output directory. The proof-set artifact records required platforms, present platforms, missing platforms, each linked `live-proof.json`, failed proof reasons, regression-gate reasons, and a next action. This gives agents one stable Android-plus-iOS gate after the per-platform live proofs have written their own aggregate evidence.
 
-Provider or custom-script evidence attached with `--signal` or `--capture` is copied into stable run folders and inventoried in `manifest.artifacts.evidenceAttachments`. Each inventory entry records the evidence channel, kind, run-relative path, source filename, byte size, and sha256 hash; it does not preserve local absolute source paths.
+Provider or custom-script evidence attachments are copied into stable run folders and inventoried in `manifest.artifacts.evidenceAttachments`. Each inventory entry records the evidence channel, kind, run-relative path, source filename, byte size, sha256 hash, completeness status, corruption status, redaction status, and transformations; it does not preserve local absolute source paths.
 
 Evidence folders:
 
@@ -133,6 +156,14 @@ The current profile runner writes health, verdict, agent summary, metrics, causa
 
 Budgets are supported but optional for adoption.
 
+Milestone budget interval semantics are explicit:
+
+- `toMilestone` without `fromMilestone` measures elapsed time from the run or session clock origin to the matching milestone occurrence.
+- `fromMilestone` plus `toMilestone` measures the interval between the two app-owned truth events for each iteration.
+- repeated transition, gesture, open, close, scroll, or handoff budgets should use both milestones when the intended number is transition duration rather than cumulative elapsed time.
+
+This distinction is visible in `metrics.json`: elapsed milestone-only runs populate `durationsMs` with milestone timestamps, while interval runs populate `durationsMs` with `to - from` values. Timing still remains untrusted unless `health.json` passes.
+
 `buildRunIndex()` can scan an artifact root after runs complete. It indexes folders that contain both `health.json` and `verdict.json`, marks a run trusted only when health and verdict both passed, and lets agents find the latest trusted prior run for a scenario without relying on terminal history.
 
 ## Supported Runner Surface
@@ -164,104 +195,17 @@ Not yet shipped as supported public features:
 - Computer Use flows
 - product-specific scenarios
 
-## Preflight planning
-
-Use `check-plan` to validate a scenario, runner manifest, and optional evidence-provider manifests before execution:
-
-```bash
-pnpm check-plan -- --scenario examples/scenarios/mobile/app-startup.json --runner examples/runners/xcodebuildmcp-ios.json --platform ios --out artifacts/plan/app-startup
-```
-
-This validates the input manifests, writes schema-checked `health.json` and `verdict.json`, writes `agent-summary.md`, and includes the raw planner match in `planner-compatibility.json`.
-
-## Android adb readiness
-
-Use `android:preflight` to verify adb and connected-device readiness before adding live Android scenario execution:
-
-```bash
-pnpm android:preflight -- --package com.example.app --out artifacts/android-adb-preflight
-```
-
-The command writes:
-
-- `health.json`
-- `verdict.json`
-- `agent-summary.md`
-- `raw/adb-version.txt`
-- `raw/adb-devices.txt`
-- `raw/android-metadata.json`
-
-If adb, a connected online device, or an optional package check fails, health fails and the verdict remains `inconclusive`.
-
-Add `--capture-logcat --logcat-lines <count>` to write `raw/adb-logcat.txt` in the same artifact folder. Add `--react-native-debug-host <host:port>` with `--package <name>` for React Native development builds that need adb reverse plus the app `debug_http_host` preference before launch; the runner writes `raw/adb-react-native-reverse.txt` and `raw/adb-react-native-debug-host.txt`. Add `--clear-logcat --launch --wait-ms <ms>` with `--package <name>` to clear logs, launch the package, wait for a bounded capture window, and then collect logcat evidence. If requested capture-window setup or logcat capture fails, scenario health fails because timing and event evidence would be incomplete.
-
-Use that captured logcat evidence directly with Android profiling:
-
-```bash
-pnpm profile:android -- --config core/config-template.json --scenario examples/mobile-app/scenarios/android/app-startup.json --adb-artifacts artifacts/android-adb-preflight --run-id android-run-1
-```
-
-Or let Android profiling own the adb capture window before it writes profile artifacts:
-
-```bash
-pnpm profile:android -- --config core/config-template.json --scenario examples/mobile-app/scenarios/android/app-startup.json --adb-capture --react-native-debug-host localhost:8097 --clear-logcat --launch --run-id android-run-1
-```
-
-## iOS simulator capture
-
-Use `profile:ios --simctl-capture` when the example app or a consuming app is already installed on a booted simulator:
-
-```bash
-pnpm profile:ios -- --config core/config-template.json --scenario examples/mobile-app/scenarios/ios/app-startup.json --simctl-capture --profile-session --profile-session-storage --launch --run-id ios-run-1
-```
-
-The command writes a separate simctl capture folder under the selected output root, seeds the app-owned profile session into native AsyncStorage before launch, then collects stored app profile events after the capture window. Command scenarios seed the scenario command queue through the same storage contract before launch. When `raw/ios-profile-events.log` exists, the iOS profile runner ingests that stored truth-event log; otherwise it falls back to `raw/ios-simctl-log.txt`.
-
-For profile-session capture on Android or iOS, omitting `--wait-ms` lets ASL derive the final evidence window from scenario execution waits and cycle count. Explicit `--wait-ms` remains authoritative when a consuming app has a known startup or logging delay that the scenario cannot express.
-
-Scenario command targets live in `adapterOptions.iosSimctl.commands`, while the app handles them through `registerProfileCommandTargetHandler`. The iOS proof does not depend on unified logs carrying JavaScript console output; it depends on app-owned stored profile events.
-
-## Historical comparison
-
-Use `compare` to build `comparison.json` from two completed run folders:
-
-```bash
-pnpm compare -- --baseline artifacts/runs/app-startup/baseline --current artifacts/runs/app-startup/current --out artifacts/runs/app-startup/current --fail-on-regression
-```
-
-The comparison gate is intentionally strict. If either run failed scenario health, or if the scenario ids do not match, the comparison is `inconclusive`. Numeric budget checks are compared only after that health gate passes. `comparison.json` includes `comparisonBasis` with the baseline/current run ids and run directories, giving agents artifact-local provenance instead of forcing them to infer it from folder names.
-
-Use `compare:latest` when an artifact root contains run history and the agent should compare the current run against the newest trusted prior run for the same scenario:
-
-```bash
-pnpm compare:latest -- --root artifacts/runs --scenario app-startup --current artifacts/runs/app-startup/current --out artifacts/runs/app-startup/current --fail-on-regression
-```
-
-The latest-trusted command excludes the exact current run directory from baseline selection. Baseline trust requires passed health and passed verdict. Current runs must pass scenario health before the command will compare timing or budget evidence. If the current manifest declares `comparisonLane`, baseline selection is scoped to trusted prior runs with the same lane; if the current manifest has no lane, selection stays within unlabeled trusted prior runs. Profile manifests also include `scenarioHash`, a stable fingerprint of the normalized scenario contract. When the current run has that hash, latest-trusted selection only compares against trusted prior runs with the same hash; legacy runs without the hash remain comparable only to legacy current runs. This keeps proof modes such as plain live proof and live proof plus agent-device sidecar from comparing against each other, and it keeps migrated scenario definitions from poisoning before/after verdicts. Latest-trusted artifacts set `comparisonBasis.strategy` to `latest_trusted_prior` and record selection counts for inspected, trusted, trusted-prior, lane-comparable, and scenario-contract-comparable candidates.
-
-## Fixture loop
-
-Use `demo:loop` to run the current contract without a simulator:
-
-```bash
-pnpm demo:loop -- --out artifacts/demo-loop
-```
-
-The fixture loop writes:
+## Command guidance
 
-- `preflight/app-startup/health.json`
-- `preflight/app-startup/verdict.json`
-- `preflight/app-startup/agent-summary.md`
-- `profile-runs/app-startup/demo-baseline/*`
-- `profile-runs/app-startup/demo-current/*`
-- `profile-runs/app-startup/demo-current/comparison.json`
+Contracts defines the schemas, artifact fields, runner surfaces, and trust policy. Runnable walkthroughs live in [Live Proofs](live-proofs.md):
 
-This is not a replacement for live device proof. It is a stable contract check that keeps the evidence loop reproducible through trusted prior-run selection while iOS or Android runtime setup is unavailable.
+- [plan checks](live-proofs.md#plan-check)
+- [Android adb preflight and profile capture](live-proofs.md#platform-preflight-and-profile-capture)
+- [iOS simctl profile capture](live-proofs.md#platform-preflight-and-profile-capture)
+- [fixture loop](live-proofs.md#fixture-loop)
+- [explicit and latest-trusted comparison](live-proofs.md#comparison)
+- [generic Android and iOS live proof](live-proofs.md#generic-mobile-proof)
 
 ## Read next
 
-- [README](../README.md) for the shortest path through the project
-- [Concepts](concepts.md) for the broader product framing
-- [Adapter Onboarding](adapters.md) for adding runners and evidence providers
-- [Consumer App Rehearsal](consumer-rehearsal.md) for adopting the package in an existing app
-- [Runner docs](../runner/README.md) for current runner behavior and limits
+- [Scenario Authoring](authoring.md) for writing portable scenarios against these contracts

package/docs/external-adapter-protocol.md

@@ -0,0 +1,219 @@
+# External Adapter Protocol
+
+ASL core is TypeScript, but the adapter contract is language-neutral. An external adapter is an out-of-process executable that exchanges newline-delimited JSON messages over stdin and stdout. The executable can be written in any language and must not depend on ASL TypeScript internals.
+
+This document defines the minimal protocol surface for conformance fixtures and future adapter hosts. JSON Schema and this normative protocol document are the source of truth for portable behavior; built-in TypeScript runners remain implementations of the same contract, not the contract itself. The protocol message schema is published in `schemas/external-adapter-message.schema.json`.
+
+## Transport
+
+- The host starts the adapter as a child process without a shell.
+- Each message is one UTF-8 JSON object followed by `\n`.
+- stdout is reserved for protocol messages. Diagnostics must go to stderr.
+- Requests and responses are correlated by `operationId`.
+- `seq` is a monotonically increasing integer within each sender's stream.
+- Hosts and adapters maintain independent `seq` streams. A receiver must treat missing, repeated, or non-monotonic `seq` values as protocol health failures.
+- Timestamps use RFC 3339 strings. Timing-sensitive waits must declare their `clockDomain`.
+- Adapters must classify work received after its request `deadline` as a structured deadline failure instead of silently attempting stale work.
+- Paths in artifact references are run-relative unless `uri` is explicitly used.
+- Artifact and raw file references should include `sha256` and `sizeBytes` when the adapter can compute them.
+- Evidence bytes must not be embedded in protocol messages as raw data or base64.
+
+## Envelope
+
+Every message uses the same envelope:
+
+```json
+{
+  "protocolVersion": "1.0",
+  "seq": 1,
+  "operationId": "op-001",
+  "kind": "request",
+  "type": "hello",
+  "runId": "run-001",
+  "attemptId": "attempt-001",
+  "deadline": "2026-06-19T12:00:05.000Z",
+  "body": {}
+}
+```
+
+Fields:
+
+| Field | Required | Meaning |
+| --- | --- | --- |
+| `protocolVersion` | yes | Protocol major/minor string. This document defines `1.0`. |
+| `seq` | yes | Sender-local message sequence. |
+| `operationId` | yes | Correlates one request with one response. Cancellation targets this value. |
+| `kind` | yes | `request`, `response`, or `event`. |
+| `type` | yes | Operation or event name. |
+| `runId` | request after `hello` | Stable ASL run identifier. |
+| `attemptId` | request after `hello` | Stable retry/attempt identifier for the run. |
+| `deadline` | request operations | Absolute deadline for bounded work. |
+| `body` | yes | Operation-specific payload. |
+
+Responses must echo `protocolVersion`, `operationId`, `runId`, and `attemptId` when those fields were present on the request.
+
+## Hello And Capability Discovery
+
+The first host message must be `hello`. The adapter responds with its identity, supported protocol range, platforms, capabilities, driver actions, artifact outputs, and clock domains.
+
+Request body:
+
+```json
+{
+  "host": {
+    "name": "agent-scenario-loop",
+    "version": "0.1.x"
+  },
+  "platform": "android"
+}
+```
+
+Response body:
+
+```json
+{
+  "adapter": {
+    "name": "asl-python-conformance-fixture",
+    "version": "0.1.0"
+  },
+  "acceptedProtocolVersion": "1.0",
+  "platforms": ["android", "ios"],
+  "capabilities": ["prepare", "launch", "command", "truthEvent", "evidence", "cancel", "stop", "finalize"],
+  "driverActions": ["tap", "assertVisible"],
+  "artifactOutputs": ["logs", "screenshot", "truth-events"],
+  "clockDomains": ["host-monotonic", "device-log"]
+}
+```
+
+If the adapter cannot support the requested protocol or platform, it must return a structured failure response and then exit cleanly or wait for `finalize`.
+
+## Operations
+
+All operation responses use:
+
+```json
+{
+  "ok": true,
+  "result": {}
+}
+```
+
+or:
+
+```json
+{
+  "ok": false,
+  "failure": {
+    "category": "unsupported",
+    "code": "unsupported_action",
+    "message": "driverAction `pinch` is not supported",
+    "retryable": false,
+    "details": {
+      "driverAction": "pinch"
+    }
+  }
+}
+```
+
+`failure.category` is optional for older adapters but recommended for conformance. Use these product-neutral categories:
+
+| Category | Use |
+| --- | --- |
+| `adapter` | Adapter implementation failure that is not more specific. |
+| `cancelled` | Operation was cancelled before completion. |
+| `cleanup` | Stop/finalize/cleanup invariant failed. |
+| `deadline` | Request deadline expired before or during adapter work. |
+| `environment` | Host, device, simulator, permission, or tool environment prevented execution. |
+| `protocol` | Malformed message, invalid sequence, unsupported protocol, or decode failure. |
+| `runner` | Runner orchestration failed outside app product behavior. |
+| `unsupported` | Operation, platform, driver action, or evidence kind is unsupported. |
+
+### prepare
+
+Validates target configuration, creates or verifies run directories, and reports setup metadata. The request body should include `platform`, target identifiers, environment assumptions, and an optional `artifactsRoot`.
+
+The response should include normalized target metadata and any adapter-owned artifact directories.
+
+### launch
+
+Launches or verifies the app, device, browser, or other target. The request body should include `platform`, `target`, and optional launch arguments.
+
+The response should include launch status, a target reference when available, and artifact references for raw command output.
+
+### executeAction
+
+Executes one portable driver action. The request body must include `driverAction` and action-specific input. The adapter must reject unknown or unsupported actions with `ok: false`.
+
+### waitCondition
+
+Waits for a truth event, UI condition, log marker, or other bounded condition. The request body must include `condition`, `deadline`, and `clockDomain`.
+
+The response should include matched truth-event data when available. Timing values are not trustworthy verdict inputs unless scenario health passed.
+
+### captureEvidence
+
+Captures logs, screenshots, UI trees, videos, profiler output, or provider signals. The response must return artifact references:
+
+```json
+{
+  "artifacts": [
+    {
+      "kind": "screenshot",
+      "path": "captures/final-screen.png",
+      "contentType": "image/png",
+      "description": "Final screen after launch",
+      "sha256": "0000000000000000000000000000000000000000000000000000000000000000",
+      "sizeBytes": 0
+    }
+  ]
+}
+```
+
+### cancel
+
+Requests cancellation of an in-flight `operationId`. The body must include `targetOperationId` and a human-readable `reason`. Adapters should make cancellation best effort and respond to the original operation with `code: "cancelled"` if it was interrupted.
+
+### stop
+
+Stops the active app/session/target while preserving evidence produced so far. This is distinct from `finalize`; the adapter may still accept evidence capture or finalization work after stop.
+
+If there is no active launched target, `stop` must return a structured cleanup failure instead of pretending cleanup ran. Include `details.cleanupStatus` when the adapter can distinguish `not-required`, `partial`, `failed`, or `passed`.
+
+### finalize
+
+Flushes pending protocol output, closes adapter-owned resources, and reports final artifact inventory. After a successful `finalize` response the adapter should exit with code `0`.
+
+`finalize` is terminal for one adapter attempt. Repeated finalization must return a structured cleanup or protocol failure and must not rewrite the prior artifact inventory.
+
+## Events
+
+Adapters may emit `event` messages between request responses for truth events, progress, and evidence discovery:
+
+```json
+{
+  "protocolVersion": "1.0",
+  "seq": 4,
+  "kind": "event",
+  "type": "truthEvent",
+  "runId": "run-001",
+  "attemptId": "attempt-001",
+  "body": {
+    "name": "app.ready",
+    "clockDomain": "device-log",
+    "observedAt": "2026-06-19T12:00:02.000Z",
+    "payload": {
+      "screen": "Home"
+    }
+  }
+}
+```
+
+Events must not replace the response for an operation. The host should still receive one terminal response for every request except when the process exits unexpectedly.
+
+## Conformance Fixture
+
+The fixture under `runner/__tests__/fixtures/external-adapter/` is intentionally small and non-JavaScript. It proves that a conforming adapter can be an external process with no ASL TypeScript imports. Golden transcripts in the same directory define expected request/response behavior for the success path, unsupported action failure, expired deadline failure, cleanup/finalization failure, sequence monotonicity, and artifact references without embedded evidence bytes.
+
+## Read next
+
+- [Contracts](contracts.md) for the scenario, runner, artifact, health, verdict, comparison, and provenance shapes