openadapt-flow 0.1.0__tar.gz

openadapt_flow-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,40 @@
+name: CI
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install
+        run: pip install -e .[dev]
+
+      - name: Install Playwright browser
+        run: playwright install --with-deps chromium
+
+      # --basetemp pins pytest's tmp_path factory inside the workspace so
+      # run dirs (REPORT.md + step/heal PNGs) survive for artifact upload;
+      # by default they land in the system temp dir and are unrecoverable
+      # when a drift/heal test fails in CI.
+      - name: Test
+        run: mkdir -p runs && pytest -q --basetemp=runs/ci
+
+      - name: Upload run artifacts
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: runs
+          path: |
+            runs/**/REPORT.md
+            runs/**/BENCH.md
+            runs/**/report.json
+            runs/**/*.png
+          if-no-files-found: warn

openadapt_flow-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,47 @@
+name: Release and PyPI Publish
+
+on:
+  push:
+    tags:
+      - "v*"
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write # PyPI Trusted Publishing (OIDC)
+      contents: write # GitHub Release
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Verify tag matches package version
+        run: |
+          PKG_VERSION=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])")
+          TAG_VERSION="${GITHUB_REF_NAME#v}"
+          if [ "$PKG_VERSION" != "$TAG_VERSION" ]; then
+            echo "Tag $GITHUB_REF_NAME does not match pyproject version $PKG_VERSION" >&2
+            exit 1
+          fi
+
+      - name: Build package
+        run: |
+          python -m pip install --quiet build
+          python -m build
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+
+      - name: Publish GitHub Release
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          gh release create "$GITHUB_REF_NAME" dist/* \
+            --title "$GITHUB_REF_NAME" --generate-notes

openadapt_flow-0.1.0/.gitignore

@@ -0,0 +1,11 @@
+.venv/
+__pycache__/
+*.pyc
+*.egg-info/
+dist/
+build/
+.pytest_cache/
+.DS_Store
+/runs/
+/recordings/
+dist/

openadapt_flow-0.1.0/DESIGN.md

@@ -0,0 +1,246 @@
+# openadapt-flow — Design & Module Contracts (v0)
+
+**Product:** record a workflow demonstration once → compile it into a
+deterministic, vision-anchored script → replay it locally with a resolution
+ladder → when the UI drifts, heal the script (as a reviewable diff) instead of
+re-reasoning every run.
+
+The runtime is **vision-only**: it consumes PNG bytes and emits clicks/keys at
+pixel coordinates through the `Backend` protocol (`openadapt_flow/backend.py`).
+The reference backend is Playwright-driven (headless-capable, CI-friendly,
+permission-free); native OS / RDP backends are future adapters.
+
+## Frozen contracts (do not change without updating this doc)
+
+- `openadapt_flow/ir.py` — Workflow/Step/Anchor/Postcondition + runtime result
+  models. FROZEN for v0. Additive changes only, and only by the integrator.
+- `openadapt_flow/backend.py` — Backend protocol. FROZEN.
+
+## Bundle & recording formats
+
+**Workflow bundle** (compiler output, replayer input):
+```
+<bundle>/
+  workflow.json      # ir.Workflow
+  templates/*.png    # anchor crops (paths referenced from workflow.json)
+```
+
+**Recording** (recorder output, compiler input):
+```
+<recording>/
+  meta.json          # {"id", "created_at", "viewport": [w,h], "app_url",
+                     #  "params": {"<param_name>": "<value typed during demo>"}}
+  events.jsonl       # one JSON object per line, in order:
+                     # {"i":0,"kind":"click","x":123,"y":45,"t":1.20}
+                     # {"i":1,"kind":"type","text":"...","param":"note","t":2.03}
+                     # {"i":2,"kind":"key","key":"Enter","t":3.10}
+                     # "param" present iff the typed value is a parameter
+  frames/{i:04d}_before.png
+  frames/{i:04d}_after.png   # after the action settled
+```
+
+**Run directory** (replayer output):
+```
+<run>/
+  report.json        # ir.RunReport
+  steps/{step_id}_before.png / _after.png
+  heals/{step_id}/…  # heal crops + heal.json per heal event
+```
+
+## Module ownership (build phase — do not edit files outside your area)
+
+| Area | Owner | Files |
+|---|---|---|
+| Mock app + backend + recorder | Agent A | `openadapt_flow/mockmed/**`, `openadapt_flow/backends/playwright_backend.py`, `openadapt_flow/recorder.py`, `openadapt_flow/demo_driver.py`, `tests/test_mockmed.py`, `tests/test_recorder.py` |
+| Vision + compiler | Agent B | `openadapt_flow/vision/**`, `openadapt_flow/compiler/**`, `tests/test_vision.py`, `tests/test_compiler.py` |
+| Runtime (ladder/heal/verify) | Agent C | `openadapt_flow/runtime/**`, `tests/test_resolver.py`, `tests/test_replayer.py`, `tests/test_heal.py` |
+| Report, emit, CLI, bench, CI | Agent D | `openadapt_flow/report.py`, `openadapt_flow/bench.py`, `openadapt_flow/emit/**`, `openadapt_flow/__main__.py`, `.github/workflows/ci.yml`, `tests/test_report.py`, `tests/test_emit.py` |
+| E2E integration | Integrator | `tests/e2e/**`, cross-module fixes anywhere |
+
+## Vision API (Agent B implements; Agent C consumes via injection)
+
+`openadapt_flow/vision/__init__.py` re-exports:
+
+```python
+class Match(BaseModel):
+    point: Point          # click/center point, screen coords
+    region: Region        # matched region, screen coords
+    confidence: float
+
+# match.py
+def find_template(screen_png: bytes, template_png: bytes, *,
+                  search_region: Region | None = None,
+                  scales: tuple[float, ...] = (0.85, 1.0, 1.18),
+                  threshold: float = 0.82) -> Match | None
+
+# ocr.py  (rapidocr_onnxruntime; instantiate the engine once, module-level lazy)
+class OcrLine(BaseModel):
+    text: str; region: Region; confidence: float
+def ocr(screen_png: bytes, *, region: Region | None = None) -> list[OcrLine]
+def find_text(screen_png: bytes, text: str, *,
+              region: Region | None = None, min_ratio: float = 0.8) -> Match | None
+    # normalized fuzzy match (difflib ratio on lowercased/stripped text)
+
+# hashing.py
+def phash_png(png: bytes, region: Region | None = None) -> str
+def phash_distance(a: str, b: str) -> int
+
+# settle.py
+def wait_settled(backend, *, interval_s: float = 0.1,
+                 stable_frames: int = 2, timeout_s: float = 3.0) -> bytes
+    # poll screenshots until N consecutive identical (phash dist 0) or timeout;
+    # return the last PNG
+```
+
+## Compiler (Agent B)
+
+`openadapt_flow.compiler.compile_recording(recording_dir, out_bundle_dir, *, name) -> Workflow`
+
+Per click event (and `double_click`, compiled identically with action
+DOUBLE_CLICK): crop a template around the click point (target-sized crop,
+e.g. 160x64 clamped to frame, centered on click), OCR the crop for `ocr_text`,
+extract up to 2 landmarks (nearest OCR lines outside the crop, carrying both
+relation/distance and exact `dx_px`/`dy_px` offsets to the click point), set
+`click_point`, `region`. Per type event: TYPE step with `text` or `param` (from
+events.jsonl). Postconditions from the after-frame: pick the largest changed
+region between before/after (cv2.absdiff + threshold + bounding rect),
+REGION_STABLE with its phash; plus TEXT_PRESENT for the most distinctive new
+OCR text (text in after, not in before — compared whitespace-insensitively
+so OCR jitter cannot make permanently visible chrome look "new"; prefer
+longest). Parameterized typed values vary per run and are NEVER asserted in
+any step's postconditions — including downstream steps whose after-frames
+embed the typed value (e.g. a save-confirmation banner). Click target labels
+(any anchor's `ocr_text`) are likewise never asserted: they are mutable
+evidence the resolution ladder heals through under rename drift, not
+invariants. Intent: rule-based `"click '<ocr_text>'"` /
+`"type <param or text preview>"` (VLM annotation is a later enhancement —
+design for it, don't call any API).
+
+Also emit a readable Python *rendering* of the workflow (`workflow.py` in the
+bundle, generated, not parsed back) so humans can code-review the automation.
+
+## Runtime (Agent C)
+
+```python
+class Replayer:
+    def __init__(self, backend, *, vision=None, grounder=None): ...
+    def run(self, workflow: Workflow, *, params: dict[str, str] | None = None,
+            bundle_dir: Path, run_dir: Path,
+            save_healed_to: Path | None = None) -> RunReport
+```
+
+Parameters not supplied in `params` fall back to the recorded example/default
+values in `workflow.params`, so a bundle replays without any explicit params.
+
+Resolution ladder per step with an anchor (record rung + confidence + ms):
+1. `template` — find_template within anchor.region padded by search_pad
+2. `template_global` — find_template full frame
+3. `ocr` — find_text(anchor.ocr_text) full frame
+4. `geometry` — landmarks: locate landmark text, offset by the exact
+   `dx_px`/`dy_px` offsets when recorded, else by relation/distance
+5. `grounder` — optional injected `Grounder.locate(png, intent) -> Match|None`
+   (protocol in `runtime/grounder.py`; ship a `NullGrounder`; an Anthropic
+   implementation goes behind the `grounder` extra and is NOT used in tests)
+
+Click point = matched region origin + (anchor.click_point - anchor.region
+origin), scaled by match scale. After acting: `wait_settled`, then check
+postconditions (poll until each passes or times out). Postcondition failure →
+re-settle and re-check once → fail the step and abort the run (semantic drift
+halts; the report must name the step and embed before/after screenshots).
+
+**Heal:** when a step succeeds via any rung other than `template`, emit a
+HealEvent: new template crop at the resolved region from the live frame,
+updated region/click_point/ocr_text (re-OCR). Apply to the in-memory workflow;
+if `save_healed_to` is set, write the healed bundle (updated workflow.json +
+new crops). Record heal crops + heal.json under `run_dir/heals/<step_id>/`.
+Irreversible steps: if resolution needed a rung below `ocr`, do NOT act — fail
+with a clear "needs human confirmation" error (v0 policy).
+
+## MockMed (Agent A)
+
+Static single-page app (`openadapt_flow/mockmed/static/index.html` + app.js +
+styles.css, no external resources, no CSS transitions/animations, font-size
+≥ 14px). Served by `openadapt_flow.mockmed.server.serve(port=0) -> (url, stop)`
+(threaded http.server). Hash-routed screens:
+
+- `#login` — Username, Password fields; "Sign In" button
+- `#tasks` — "Referral Tasks" table, rows with fake patient names
+  (obviously fake: "Jane Sample", "Alex Testcase"), reason, priority, an
+  "Open" button per row
+- `#patient/<id>` — patient banner, "New Encounter" button, Encounters list
+- `#encounter` — Type chooser as segmented BUTTONS ("Triage" / "Consult" —
+  no native <select>), Note textarea, "Save Encounter" button
+- after save → back to patient screen with a banner `Encounter saved — <first
+  40 chars of note>` and the encounter listed
+
+Drift modes via query string `?drift=a,b` (applied before render):
+- `theme` — dark palette (breaks template matching for every anchor; OCR
+  still works for labeled targets, while unlabeled input-field anchors fall
+  through to the geometry rung)
+- `move` — "New Encounter" and "Save Encounter" buttons relocated to the
+  opposite side of their container (breaks local template search)
+- `rename` — "Save Encounter"→"Submit Encounter", "Open"→"View" (breaks
+  template + OCR; geometry rung must resolve — keep the button in the SAME
+  position as default so landmarks/geometry succeed)
+- `modal` — after clicking Save, a blocking "Survey" modal appears instead of
+  the banner (semantic drift: replay must FAIL gracefully)
+
+`demo_driver.py`: `record_triage_demo(url, out_dir, *, note_text, param_name
+= "note") -> Path` — drives the canonical demo via Playwright locators (record
+time may cheat with selectors; replay never does): login → tasks → Open first
+referral → New Encounter → click "Triage" → click Note field → type note
+(param) → click Save Encounter → done. It performs every action through
+`Recorder` so frames/events are captured (before frame, act, wait settle,
+after frame).
+
+`PlaywrightBackend(page)` implements `Backend` (chromium, fixed viewport
+1280x800, deviceScaleFactor=1). `Recorder(backend, out_dir)` wraps a backend
+with the same action methods plus `type_text(text, param=None)` and
+`finish() -> recording dir`.
+
+## Report / emit / CLI / bench / CI (Agent D)
+
+- `report.py`: `render_run_report(run_dir) -> Path` — REPORT.md in run_dir:
+  outcome, per-step table (intent, rung, confidence, ms, heal?), embedded
+  relative-path images for key steps and every heal (before/after side by
+  side), rung histogram, totals (ms, model calls, est cost).
+  `render_bench_report(bench.json, out) -> Path`.
+- `bench.py`: `run_bench(workflow_bundle, backend_factory, n) -> dict` —
+  replay N times; success rate, p50/p95 total ms, rung histogram, model calls
+  (0 in v0), cost 0; serialize bench.json.
+- `emit/skill.py`: workflow bundle → `SKILL.md` folder (Agent Skills format:
+  name, description, when-to-use, `openadapt-flow replay bundle --param k=v`
+  invocation). The bundle is copied into the skill folder (`bundle/`) so the
+  artifact is self-contained and portable. `emit/mcp_tool.py`: generate a
+  standalone `server.py` exposing the workflow as an MCP tool (string
+  template; must `ast.parse`; do not import mcp at generation time); the
+  bundle is copied next to `server.py` and referenced relative to
+  `__file__`, never by an emitting-machine absolute path.
+- `__main__.py` CLI: `demo-record`, `compile`, `replay`, `bench`, `emit-skill`,
+  `emit-mcp` (thin wrappers over the module APIs above). `replay --run-dir`
+  is optional and defaults to `runs/replay-<UTC timestamp>`.
+- CI (`.github/workflows/ci.yml`): ubuntu-latest, py3.12,
+  `pip install -e .[dev]`, `playwright install --with-deps chromium`,
+  `pytest -q --basetemp=runs/ci` (temp dirs pinned inside the workspace so
+  run artifacts survive), upload `runs/**/REPORT.md` + PNGs as artifacts.
+
+## Test policy
+
+- Unit tests per module, cross-module deps faked/injected (Agent C must not
+  import Agent B's vision in unit tests — inject fakes; the integrator wires
+  real parts in `tests/e2e/`).
+- Deterministic: no sleeps besides settle polling; server on ephemeral port.
+- No network beyond localhost. No API keys required anywhere in tests.
+- E2E matrix (integrator): baseline ×3 all-template zero-heal; theme / move /
+  rename each succeed WITH heals then replay-healed all-template; modal fails
+  gracefully naming the step; params substitution verified via banner OCR
+  with a note value DIFFERENT from the recorded one (the identity case
+  cannot distinguish substitution from replaying the baked-in literal); the
+  irreversible-step risk gate exercised end-to-end (step marked irreversible
+  + drift forcing a below-ocr rung must refuse to act).
+
+## Repo policies
+
+- Fake data only; no real patient/clinic/person names anywhere.
+- No references to customers or design partners in this repo.
+- Conventional commits; feature branches; never push to main.

openadapt_flow-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 OpenAdapt.AI (MLDSAI Inc.)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

openadapt_flow-0.1.0/PKG-INFO

@@ -0,0 +1,138 @@
+Metadata-Version: 2.4
+Name: openadapt-flow
+Version: 0.1.0
+Summary: Demonstration compiler: record a desktop workflow once, compile it into a deterministic, self-healing, locally-run script.
+Author: OpenAdapt.AI
+License: MIT
+License-File: LICENSE
+Requires-Python: >=3.10
+Requires-Dist: imagehash>=4.3
+Requires-Dist: numpy>=1.26
+Requires-Dist: opencv-python-headless>=4.9
+Requires-Dist: pillow>=10.0
+Requires-Dist: playwright>=1.44
+Requires-Dist: pydantic>=2.5
+Requires-Dist: rapidocr-onnxruntime>=1.3
+Provides-Extra: dev
+Requires-Dist: pytest-timeout>=2.2; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Provides-Extra: grounder
+Requires-Dist: anthropic>=0.40; extra == 'grounder'
+Description-Content-Type: text/markdown
+
+# openadapt-flow
+
+Record a GUI workflow once. Replay it deterministically, locally, for free.
+A model only touches the script to repair it.
+
+Computer-use agents re-reason on every run: screenshot, think, click, repeat.
+That's the right shape for a task nobody has automated before, and the wrong
+one for the 500th referral this month. openadapt-flow makes the other bet. It
+compiles a human demonstration into a plain, reviewable script with vision
+anchors and per-step assertions. A healthy script replays in milliseconds with
+zero model calls. When the UI shifts, a fallback ladder still finds the
+target, and the fix gets written back into the script as a diff you can
+review. The automation gets cheaper over time instead of billing you to
+re-think the same eleven clicks.
+
+## How it works
+
+A recording captures screenshots and input events while a person does the
+task once. The compiler turns that into a workflow bundle: for every click it
+stores a template crop, the OCR'd label, and offsets to nearby landmark text,
+then derives postconditions from what actually changed on screen after the
+action. Typed values can be tagged as parameters, so one demo yields a
+reusable, parameterized automation plus a generated `workflow.py` rendering a
+human can code-review.
+
+At replay time, each step walks a resolution ladder:
+
+1. template match near the recorded location (milliseconds, no model)
+2. template match across the whole screen
+3. OCR match on the label
+4. geometry from surrounding landmarks
+5. optionally, a grounding model (local or API)
+
+Resolving below the top rung triggers a heal: fresh crop, updated
+coordinates, re-read label, saved as a reviewable change to the bundle.
+Postconditions check that each step actually worked. If the screen stops
+matching expectations entirely (a surprise dialog, a changed process), the
+run halts with an illustrated report rather than guessing. Steps tagged
+irreversible refuse to act on low-confidence resolutions at all.
+
+The runtime is vision-only: PNG in, clicks and keys out, behind a small
+`Backend` protocol. The reference backend drives a headless browser, which is
+what lets the entire record→compile→replay→heal loop run in CI with no OS
+permissions. Native desktop and RDP backends are planned adapters behind the
+same protocol.
+
+## What it looks like
+
+One demonstration was recorded on the light UI (left). The compiled workflow
+then ran unattended against a dark theme it had never seen (right): all 11
+steps succeeded, every anchor re-resolved through the OCR or geometry rungs,
+and all 8 anchors healed themselves back onto the fast path. No human
+involved.
+
+| Recorded UI → baseline replay (all `template` rung, 0 model calls) | Theme-drifted UI → replay + self-heal (8/8 anchors healed) |
+| --- | --- |
+| ![Baseline replay final screen](docs/showcase/baseline-run/steps/step_010_after.png) | ![Theme-drift replay final screen](docs/showcase/theme-drift-run/steps/step_010_after.png) |
+
+Every replay generates an illustrated run report:
+[baseline](docs/showcase/baseline-run/REPORT.md) ·
+[theme drift with heals](docs/showcase/theme-drift-run/REPORT.md).
+
+## Quickstart
+
+```bash
+pip install -e '.[dev]'
+playwright install chromium
+
+# 1. Record a demonstration (drives the bundled MockMed demo app):
+openadapt-flow demo-record --out /tmp/rec
+
+# 2. Compile it into a workflow bundle:
+openadapt-flow compile /tmp/rec --out /tmp/bundle --name triage-demo
+
+# 3. Replay it — deterministic, local, zero model calls:
+openadapt-flow replay /tmp/bundle --run-dir /tmp/run
+open /tmp/run/REPORT.md
+
+# 4. Drift the UI (dark theme it has never seen) and watch it heal:
+openadapt-flow replay /tmp/bundle --drift theme \
+    --run-dir /tmp/run-drift --save-healed-to /tmp/healed
+open /tmp/run-drift/REPORT.md
+```
+
+Steps 3–4 serve the bundled MockMed app automatically. To replay against your
+own running app, pass `--url` (parameters default to the values recorded
+during the demo; `--param` overrides them):
+
+```bash
+openadapt-flow replay /tmp/bundle --url <APP_URL> \
+    --run-dir /tmp/run --param note="Booking 3 months"
+```
+
+`bench` replays a bundle N times and aggregates success rate, latency
+percentiles, and cost: `openadapt-flow bench /tmp/bundle --n 10 --run-root
+/tmp/bench`.
+
+The test suite includes the full drift matrix (theme, moved buttons, renamed
+buttons, surprise modal) end to end:
+
+```bash
+pytest -q
+```
+
+## Status
+
+v0. The record→compile→replay→heal loop runs end to end against MockMed, the
+bundled mock clinical app, under four kinds of deliberate UI drift. The suite
+(124 tests) runs headlessly in GitHub Actions and uploads run reports as
+build artifacts. Compiled workflows can also be emitted as Agent Skills
+(`SKILL.md`) and MCP servers, so other agents can invoke them.
+
+Solid for the reference browser backend; everything else should be treated as
+a design seam, not a finished feature. See `DESIGN.md` for module contracts
+and `docs/L1_INTEGRATION.md` for feeding layered clinical-data platforms
+(L1→L2 acquisition contracts). MIT license.

openadapt_flow-0.1.0/README.md

@@ -0,0 +1,116 @@
+# openadapt-flow
+
+Record a GUI workflow once. Replay it deterministically, locally, for free.
+A model only touches the script to repair it.
+
+Computer-use agents re-reason on every run: screenshot, think, click, repeat.
+That's the right shape for a task nobody has automated before, and the wrong
+one for the 500th referral this month. openadapt-flow makes the other bet. It
+compiles a human demonstration into a plain, reviewable script with vision
+anchors and per-step assertions. A healthy script replays in milliseconds with
+zero model calls. When the UI shifts, a fallback ladder still finds the
+target, and the fix gets written back into the script as a diff you can
+review. The automation gets cheaper over time instead of billing you to
+re-think the same eleven clicks.
+
+## How it works
+
+A recording captures screenshots and input events while a person does the
+task once. The compiler turns that into a workflow bundle: for every click it
+stores a template crop, the OCR'd label, and offsets to nearby landmark text,
+then derives postconditions from what actually changed on screen after the
+action. Typed values can be tagged as parameters, so one demo yields a
+reusable, parameterized automation plus a generated `workflow.py` rendering a
+human can code-review.
+
+At replay time, each step walks a resolution ladder:
+
+1. template match near the recorded location (milliseconds, no model)
+2. template match across the whole screen
+3. OCR match on the label
+4. geometry from surrounding landmarks
+5. optionally, a grounding model (local or API)
+
+Resolving below the top rung triggers a heal: fresh crop, updated
+coordinates, re-read label, saved as a reviewable change to the bundle.
+Postconditions check that each step actually worked. If the screen stops
+matching expectations entirely (a surprise dialog, a changed process), the
+run halts with an illustrated report rather than guessing. Steps tagged
+irreversible refuse to act on low-confidence resolutions at all.
+
+The runtime is vision-only: PNG in, clicks and keys out, behind a small
+`Backend` protocol. The reference backend drives a headless browser, which is
+what lets the entire record→compile→replay→heal loop run in CI with no OS
+permissions. Native desktop and RDP backends are planned adapters behind the
+same protocol.
+
+## What it looks like
+
+One demonstration was recorded on the light UI (left). The compiled workflow
+then ran unattended against a dark theme it had never seen (right): all 11
+steps succeeded, every anchor re-resolved through the OCR or geometry rungs,
+and all 8 anchors healed themselves back onto the fast path. No human
+involved.
+
+| Recorded UI → baseline replay (all `template` rung, 0 model calls) | Theme-drifted UI → replay + self-heal (8/8 anchors healed) |
+| --- | --- |
+| ![Baseline replay final screen](docs/showcase/baseline-run/steps/step_010_after.png) | ![Theme-drift replay final screen](docs/showcase/theme-drift-run/steps/step_010_after.png) |
+
+Every replay generates an illustrated run report:
+[baseline](docs/showcase/baseline-run/REPORT.md) ·
+[theme drift with heals](docs/showcase/theme-drift-run/REPORT.md).
+
+## Quickstart
+
+```bash
+pip install -e '.[dev]'
+playwright install chromium
+
+# 1. Record a demonstration (drives the bundled MockMed demo app):
+openadapt-flow demo-record --out /tmp/rec
+
+# 2. Compile it into a workflow bundle:
+openadapt-flow compile /tmp/rec --out /tmp/bundle --name triage-demo
+
+# 3. Replay it — deterministic, local, zero model calls:
+openadapt-flow replay /tmp/bundle --run-dir /tmp/run
+open /tmp/run/REPORT.md
+
+# 4. Drift the UI (dark theme it has never seen) and watch it heal:
+openadapt-flow replay /tmp/bundle --drift theme \
+    --run-dir /tmp/run-drift --save-healed-to /tmp/healed
+open /tmp/run-drift/REPORT.md
+```
+
+Steps 3–4 serve the bundled MockMed app automatically. To replay against your
+own running app, pass `--url` (parameters default to the values recorded
+during the demo; `--param` overrides them):
+
+```bash
+openadapt-flow replay /tmp/bundle --url <APP_URL> \
+    --run-dir /tmp/run --param note="Booking 3 months"
+```
+
+`bench` replays a bundle N times and aggregates success rate, latency
+percentiles, and cost: `openadapt-flow bench /tmp/bundle --n 10 --run-root
+/tmp/bench`.
+
+The test suite includes the full drift matrix (theme, moved buttons, renamed
+buttons, surprise modal) end to end:
+
+```bash
+pytest -q
+```
+
+## Status
+
+v0. The record→compile→replay→heal loop runs end to end against MockMed, the
+bundled mock clinical app, under four kinds of deliberate UI drift. The suite
+(124 tests) runs headlessly in GitHub Actions and uploads run reports as
+build artifacts. Compiled workflows can also be emitted as Agent Skills
+(`SKILL.md`) and MCP servers, so other agents can invoke them.
+
+Solid for the reference browser backend; everything else should be treated as
+a design seam, not a finished feature. See `DESIGN.md` for module contracts
+and `docs/L1_INTEGRATION.md` for feeding layered clinical-data platforms
+(L1→L2 acquisition contracts). MIT license.