kaloko 0.4.11 → 0.5.0

package/skills/kaloko/reference/apps.md

@@ -6,13 +6,19 @@ driver differs, and it is chosen by the scenario's `platform`.
 
 | platform | Driven by | UI tree | Interactions |
 |---|---|---|---|
-| `android` | `adb` (emulator or USB device) | uiautomator, in dp | deep links, tap by selector or point, type, back |
+| `android` | `adb`: emulator, USB phone or a phone over Wi-Fi | uiautomator, in dp | deep links, tap by selector or point, type, back |
 | `ios` | `xcrun simctl` (iOS Simulator) | `idb` when installed, otherwise text by OCR (macOS Vision) | deep links; taps and typing need `idb` |
+| `ios` + `real: true` | `xcrun devicectl` and `idevicescreenshot` or `pymobiledevice3` (a real iPhone) | text by OCR | install, launch, deep links; no taps |
 | `electron` | Playwright Electron | the full DOM: every web check applies | everything Playwright does (`page`) |
 | `desktop` | macOS window of a named app | macOS accessibility tree, otherwise OCR | none built in: use your tool, AppleScript or a person |
+| `windows` | the app's window through Windows PowerShell and UI Automation (nothing to install) | UI Automation tree | tap by selector or point, type; walk on Windows |
 
-Anything else, including a real iPhone, Windows apps, Flutter/React Native
-under Detox, Maestro flows or XCUITest, is captured by import:
+An installed web app (PWA) is a web scenario with `display_modes: [standalone]`; see
+[devices.md](devices.md). A WebView inside an Android app is captured as a web page; see
+[WebViews](#webviews-inside-apps).
+
+Anything else, including Flutter or React Native under Detox, Maestro flows,
+XCUITest or Appium runs, is captured by import:
 
 ```bash
 npx kaloko capture --step checkout --image shot.png --tree hierarchy.xml --device "Pixel 8" --density 2.625
@@ -24,6 +30,7 @@ npx kaloko capture --step checkout --image shot.png --tree hierarchy.xml --devic
 - Appium iOS XML (`XCUIElementType…`);
 - `idb ui describe-all --json`;
 - Maestro `hierarchy` JSON;
+- Windows UI Automation JSON from Kaloko's `windows.ps1 tree` (`--platform windows`);
 - Kaloko's own `qawalk.tree.v1`.
 
 `--ocr` reads the text from the image instead (macOS). `--density` or
@@ -46,6 +53,7 @@ environments:
     app:
       electron: { main: dist/main.js }          # or executable: path to the packaged app
       # desktop: { name: Notes }                # a macOS app by its window owner name
+      # windows: { name: contoso, path: dist/Contoso.exe }   # or aumid: a Store/MSIX app
 ```
 
 ```yaml
@@ -81,9 +89,50 @@ nodes:
 - The app language is set per locale: Android 13+ app locales, iOS
   `-AppleLanguages`, Electron `--lang`.
 
+## Devices, dark mode and motion
+
+`devices:` walks the same scenario on several emulators, simulators or phones; each device is its own capture key
+next to the colour schemes and motion of the run:
+
+```yaml
+# qa/flows/shop-android.yml
+platform: android
+devices: [pixel-8-api-34, pixel-7-api-33, galaxy]
+color_schemes: [light, dark]
+```
+
+```yaml
+# kaloko.config.yml
+environments:
+  staging:
+    app:
+      android:
+        package: com.acme.shop.staging
+        devices:
+          galaxy: { connect: 192.168.1.20:5555, title: Galaxy S24 }   # a phone over Wi-Fi
+      ios:
+        bundle: com.acme.shop
+        devices:
+          my-iphone: { udid: 00008110-000A…, real: true, title: Test iPhone }
+```
+
+- A name without an entry says what it is. Android: an adb serial (`emulator-5554`, `R58M…`, `host:port`) or an
+  AVD name (`pixel-8-api-34` finds `Pixel_8_API_34`; Kaloko starts it headless when it is not running). iOS: a
+  simulator by name and runtime (`iphone-15-ios-18` is "iPhone 15" on iOS 18; `iphone-se-3` is "iPhone SE (3rd
+  generation)").
+- `kaloko start --devices pixel-8-api-34` overrides the scenario; `kaloko walk --devices galaxy` walks one of them.
+- Real Android phones are woken up and unlocked (no PIN) and get a clean status bar (demo mode, 9:41, full battery).
+  `wake: false` and `status_bar: false` turn that off.
+- Dark mode and reduced motion are set on the device per capture: `cmd uimode night` and the animator scales on
+  Android (the app redraws; some apps recreate the screen), `simctl ui appearance` on the iOS Simulator, emulated
+  media in Electron. The iOS Simulator has no command for reduced motion, and real devices keep their own setting;
+  the capture notes it.
+- The canvas draws the device frame of the closest preset (`pixel-8-api-34` → Pixel 8) and offers the safe-area
+  overlay; app captures are the whole screen.
+
 **Step scripts**
 
-A script gets `{ app, page, locale, env, run, capture, link }`:
+A script gets `{ app, page, webview, device, locale, env, run, capture, link }`:
 
 ```js
 // steps/add-to-cart.mjs
@@ -97,9 +146,52 @@ export default async ({ app, link, capture }) => {
 ```
 
 - `app.tap`, `app.wait` and `app.type` take app selectors. `app.tap({ x, y })`
-  taps by coordinates, in dp or points.
+  taps by coordinates, in dp or points (logical pixels on Windows).
 - On Electron, selectors are CSS and `page` is the Playwright page, so
   any Playwright code works there.
+- `capture()` takes every key of the current device (light and dark);
+  `capture({ viewport: 'pixel-8-api-34.dark' })` one of them.
+
+## WebViews inside apps
+
+An Android WebView with debugging on (`WebView.setWebContentsDebuggingEnabled(true)` in debug builds) is a Chromium
+page. `await webview()` in a step script returns it as a Playwright page (Kaloko forwards its DevTools socket with
+adb), and `capture({ webview: true })`, or `webview: true` in `app.android`, captures it as a web page: the DOM, the
+web checks, the accessibility basics. The native screen around it is not in that capture; capture both when both
+matter.
+
+iOS WKWebViews (`isInspectable`) speak Safari's Web Inspector protocol, which Playwright cannot attach to. Capture
+those screens as app screens (tree or OCR), and test the web content itself as a web scenario in WebKit
+(`browsers: [webkit]`) with an iPhone preset.
+
+## Windows apps
+
+`platform: windows` with `app.windows: { name, path | aumid, args }` walks a desktop app on Windows through the
+PowerShell every Windows has: the window as a screenshot (at its display scale), the UI Automation tree (control
+types, AutomationId, names, values), clicks by selector and typing (SendKeys). It needs a signed-in desktop session
+(a person's machine, or a `windows-latest` CI runner); a locked screen or a service account cannot see windows.
+Elsewhere, take the screenshots on Windows and import them with
+`kaloko capture --image shot.png --tree uia.json --platform windows`, or run `kaloko import` on a folder of
+screenshots.
+
+## Real iPhones
+
+`real: true` with the `udid` uses Xcode's `devicectl` to install (`app:` a `.app` or `.ipa` signed for the device),
+launch and open deep links, and `idevicescreenshot` (`brew install libimobiledevice`) or `pymobiledevice3` for the
+screenshot. On iOS 17 and newer, `pymobiledevice3` needs its tunnel running (`sudo pymobiledevice3 remote tunneld`).
+There is no UI tree or tapping on a real device: text comes from OCR on macOS, and flows that need taps run in
+XCUITest, Maestro or Appium, with each screen imported by `kaloko capture --image`.
+
+## React Native and Flutter
+
+- **React Native** renders native views: `testID` becomes the Android resource-id and the iOS accessibility id, so
+  `[id$=checkout]` selectors work; `accessibilityLabel` becomes the name `app_labels` checks. Deep links through
+  React Navigation linking work as step paths.
+- **Flutter** draws its own widgets. The UI tree only has what the semantics tree exposes: wrap controls in
+  `Semantics(label: …)` or give them `Key`s that reach the accessibility layer, otherwise checks see one big view.
+  Flutter web builds are a web scenario. For taps, Maestro handles Flutter well; import its screens.
+- Both: keep a debug or staging flavour with a stable package name per environment, and deep links into every screen
+  a scenario needs. That makes steps a deep link instead of a script.
 
 ## Selectors and checks on the UI tree
 
@@ -127,8 +219,8 @@ dialog, webview and group.
 - **Pack `app` (AC7xx):** those four checks, plus a semantic question about
   the screen's purpose and next action.
 - **Web-only checks** (`no_overflow`, `http_status`, meta, `a11y_contrast`,
-  `perf_*`): these are "not evaluated" on app captures. Do not put them on
-  app steps.
+  `perf_*`, `url_*`, `style_*`, `js_equals`): these are "not evaluated" on
+  app captures. Do not put them on app steps.
 - **OCR trees:** these have text only, so role checks (labels, touch
   targets) are not evaluated there. Say so in the report and install `idb`
   when those checks matter.
@@ -150,7 +242,10 @@ questions about what is on the screen, not about the markup.
 
 - **Devices:** never install on a person's own phone or sign in with
   real accounts without asking. Emulators and simulators are yours to
-  start, reset and install builds on.
+  start, reset and install builds on. Dark mode and the animation scales
+  are restored after a walk, and so is the status bar of a real phone.
+- **Windows:** clicks and typing go to the real desktop. Walk on a machine
+  nobody is using at that moment, or a CI runner.
 - **Production apps:** production apps (store builds) are read-only in
   spirit. Walk them only with deep links and screens that change no data,
   unless the person agrees.

package/skills/kaloko/reference/ci.md

@@ -16,7 +16,9 @@ deployments; `--trigger pull_request` runs on every push instead and waits for
 the deployment):
 
 1. installs the project (`npm ci`, or pnpm, yarn, bun by the lockfile) and the
-   Chromium of kaloko's Playwright (`kaloko browser install --with-deps`);
+   Chromium of kaloko's Playwright (`kaloko browser install --with-deps`), plus
+   WebKit and Firefox when a scenario or `walk.browsers` names them
+   (`--browsers chromium,webkit`; see [devices.md](devices.md));
    `KALOKO_BROWSER_CHANNEL=chromium` makes the run use it, so no separate CI
    config is needed;
 2. `kaloko ci preview-url --wait 600 --github-env` finds the preview URL, the

package/skills/kaloko/reference/design.md

@@ -48,6 +48,45 @@ design/
   `kaloko design lint` checks the folder; `kaloko sync` runs it first and stops on errors (things that break on
   the live step), warnings are printed.
 
+## Devices, dark mode and app screens
+
+Design steps can target device presets and colour schemes like any run ([devices.md](devices.md)):
+
+```yaml
+# the design scenario (the same one development uses)
+viewports: [iphone-15, pixel-8, ipad-landscape, laptop]
+color_schemes: [light, dark]
+```
+
+- On a design environment a phone or tablet preset is the whole screen (393 × 852 on an iPhone 15), rendered with
+  the preset's touch, pixel ratio and user agent, so `(pointer: coarse)` and `(hover: none)` apply. The step detail
+  draws the device frame; *Safe areas* shows the notch, the home indicator and where the first screen ends.
+- Dark mode: write `@media (prefers-color-scheme: dark)` in the step's CSS, and the dark captures show it. A step
+  whose dark screen is a different page gets `<step>/dark/index.html` (or `<step>/<locale>/dark/index.html`); dark
+  captures render that page, and it travels with the revision.
+- Token modes follow the scheme: with a `theme` modifier (or `light`/`dark` groups) in the token set, Dev mode, the
+  step spec and the token check of a dark capture name the dark tokens, and drift compares dark with dark.
+- Browsers: design steps render in Chromium. Check the implementation in WebKit and Firefox instead.
+
+**Native app screens as portable HTML.** A mockup of an iOS or Android screen is a full-screen page:
+
+- Size it to the screen, not to a document: `html, body { height: 100% }` and the main area `min-height: 100dvh`.
+  Do not draw the status bar or the home indicator; the frame does.
+- Keep content out of the safe areas with
+  `padding-top: max(env(safe-area-inset-top), var(--kaloko-safe-top, 0px))` (and `-bottom`, `-left`, `-right`).
+  On the phone `env()` gives the inset; in Kaloko `--kaloko-safe-*` does, because desktop browsers report 0. Add
+  `viewport-fit=cover` to the viewport meta.
+- Use the roles and names the app will have: `<nav aria-label="Tabs">` with buttons for a tab bar, `<header>` with
+  an `<h1>` for the navigation bar, `<button>` and labelled inputs. The step spec and the checks match the
+  implementation by role and name, and React Native `accessibilityLabel` or Android `contentDescription` are those
+  names.
+- Touch targets of at least 44 × 44 pt (iOS) or 48 × 48 dp (Android); the app pack checks them on the real app.
+- One design serves both platforms unless they really differ. When they do, give the platform its own scenario
+  step or its own viewport set (`viewports: [iphone-15]` in one scenario, `[pixel-8]` in another), not user-agent
+  sniffing in the page.
+- System fonts are not portable (`-apple-system` renders differently on each machine): bundle the font the app
+  uses, as the portable HTML rules say.
+
 ## Sync and publish
 
 - `kaloko walk` renders only revisions whose captures are missing (same files, same capture).

package/skills/kaloko/reference/devices.md

@@ -0,0 +1,109 @@
+# Browsers, devices, colour schemes and motion
+
+A run captures every step in each locale and viewport. Five more dimensions can be added to that matrix: the
+browser engine, the colour scheme, reduced motion, forced colours (Windows high contrast) and the display mode of an
+installed web app. Viewports can name device presets instead of sizes.
+
+## Asking for them
+
+In the scenario, with defaults for the whole project in `walk:` of `kaloko.config.yml`, and per run on the
+command line (the command line wins, then the scenario, then the config):
+
+```yaml
+# qa/flows/checkout.yml
+viewports: [iphone-se, iphone-15, pixel-8, ipad-landscape, laptop]   # presets, or { desktop: [1280, 800] } as before
+browsers: [chromium, webkit, firefox]
+color_schemes: [light, dark]
+reduced_motion: [no-preference, reduce]      # optional
+forced_colors: [none, active]                # optional, Windows high contrast
+display_modes: [browser, standalone]         # optional, the page as an installed PWA
+```
+
+```bash
+npx kaloko start --scenario qa/flows/checkout.yml --env staging --browsers chromium,webkit --color-schemes light,dark
+npx kaloko start … --viewports iphone-se,fold-open,desktop-hd   # presets the scenario does not list
+npx kaloko walk                         # one pass per engine and device profile
+npx kaloko walk --browsers webkit       # only the WebKit captures of the run
+npx kaloko browser install --browsers webkit,firefox   # once per machine or CI job
+npx kaloko devices                      # the preset catalogue
+```
+
+Every combination is one capture: `checkout × 3 viewports × 2 browsers × 2 schemes` is 12 screenshots per step and
+locale. Keep the matrix to what the task is about. A Safari bug report needs `browsers: [chromium, webkit]` on the
+affected steps, not every dimension on every step. A run refuses more than 400 combinations.
+
+## Capture keys
+
+A capture is keyed by the viewport name plus the values that differ from the default, joined by dots:
+`mobile`, `mobile.webkit`, `iphone-15.webkit.dark`, `desktop.reduce`, `phone.standalone`. Chromium, light, no
+motion preference, no forced colours and the browser display mode are the defaults. Runs from before these
+dimensions have plain keys and read as Chromium. Use the keys wherever a viewport name goes: `--viewport
+iphone-15.dark`, `capture({ viewports: ['mobile'] })` (a viewport name takes all its keys in the current pass),
+`kaloko spec home --viewport desktop.dark`.
+
+## Device presets
+
+| Kind | Presets |
+|---|---|
+| Phones | `iphone-se` (small), `iphone-15`, `iphone-16`, `iphone-16-pro-max` (large), `pixel-8`, `pixel-8-pro`, `galaxy-s24` |
+| Foldables | `fold-folded`, `fold-open` |
+| Tablets | `ipad-portrait`, `ipad-landscape`, `ipad-pro-portrait`, `ipad-pro-landscape`, `android-tablet-portrait`, `android-tablet-landscape` |
+| Computers and TV | `laptop` (1440), `desktop-hd` (1920), `desktop-wide` (2560), `tv-1080p` |
+
+- `<preset>-landscape` turns any phone, foldable or tablet sideways (`iphone-15-landscape`).
+- A preset sets the CSS viewport, the pixel ratio, mobile mode, touch and the device's user agent. Web walks get
+  the screen minus the browser's bars (393 × 659 on an iPhone 15 in Safari); Design environments and installed
+  PWAs get the whole screen (393 × 852).
+- Overrides: `{ phone: { device: iphone-15, height: 700 } }`, or `full_screen: true` for a kiosk or an app mockup.
+- Screenshots stay in CSS pixels whatever the pixel ratio, so boxes, diffs and comments line up across devices.
+- The values are those of the device class, not of a lab. They catch layout bugs; a real device catches the rest.
+
+## How each dimension is produced
+
+| Dimension | How | Engines |
+|---|---|---|
+| Browser | Playwright's Chromium, WebKit (Safari's engine) and Firefox, each in its own pass | all |
+| Device preset | its own browser context with the pixel ratio, mobile mode, touch and user agent | all; Firefox has no mobile mode, so presets there set size, ratio, touch and agent |
+| Colour scheme, reduced motion, forced colours | `prefers-color-scheme`, `prefers-reduced-motion` and `forced-colors` emulated per capture | all |
+| Display mode | `display-mode` media rules of same-origin style sheets and `matchMedia` answered as installed; full-screen size | all |
+
+- Scripts that read `matchMedia('(display-mode: standalone)')` once at load still see the browser mode; CSS and
+  later calls see the installed mode. Check `navigator.standalone`-style logic on a real install.
+- The shared browser (Chromium over CDP) walks the default pass; WebKit, Firefox and device profiles always get
+  their own context, signed in through the scenario's sessions as usual.
+- E-mail steps are captured once, in the first pass.
+
+## Checks that only Chromium can run
+
+These are "not evaluated" on WebKit and Firefox captures, with the reason, and do not make a step partial when
+Chromium evaluated them:
+
+- `a11y_ax_names` (the accessibility tree is read over the DevTools protocol);
+- `a11y_focus_visible` (focus rings are compared on Chromium screenshots);
+- `perf_cls` (only Chromium reports layout shifts);
+- on WebKit also the Tab walk (`a11y_focus_order`, `a11y_keyboard_trap`, `a11y_skip_link`): WebKit moves Tab
+  between form controls only, like Safari by default, so the walk would report false gaps.
+
+Everything else (DOM checks, contrast, overflow, links, LCP, TTFB, weight, semantic and vision criteria) runs on
+every engine. Contrast is checked per colour scheme, so a dark-mode contrast bug fails on the `.dark` capture.
+
+Semantic questions read the page text: the same text in another engine or scheme is the same question, answered
+once and then from the cache. Vision questions (`vision: true`) look at each screenshot, so every combination is
+asked separately; keep `vision: true` to the steps where the look is the point.
+
+## On the canvas
+
+- Switchers for the browser, colour scheme, motion, forced colours and display mode sit next to the locale in the
+  top bar and in review mode. The map, the index, the step detail and review show the chosen combination.
+  With more than five values (20 locales, say) a switcher is a button that opens a searchable list with the
+  status of each value; `L` opens the locale list, `[` and `]` move to the previous and next locale.
+- The step detail of a device preset draws its frame: bezel, status bar, the browser's bars and the island, notch
+  or camera. *Safe areas* marks the insets and where the first screen ends. Both can be switched off.
+- *Beside* puts the same step in another browser or scheme next to it (WebKit beside Chromium, dark beside light).
+- Links keep the key (`#step=cart&vp=iphone-15.webkit.dark`) and switch the canvas to it.
+
+## Doctor and CI
+
+`kaloko doctor` lists the engines the scenarios and `walk.browsers` name and says how to install a missing one.
+In CI install them in the job: `npx kaloko browser install --browsers chromium,webkit --with-deps` (Linux needs
+the system libraries WebKit and Firefox bring).

package/skills/kaloko/reference/environments.md

@@ -18,6 +18,12 @@ environment whose URL comes from CI: `preview: { base_url: ${KALOKO_PREVIEW_URL}
 which `kaloko ci github|gitlab` adds and `kaloko ci preview-url` fills; see
 [ci.md](ci.md).
 
+An environment says where the product runs. In which browsers, on which devices and in which colour scheme it is
+walked is the run's matrix: `browsers`, `color_schemes`, `reduced_motion` and device presets in `viewports` of the
+scenario, defaults in `walk:` of the config, `--browsers` and friends on `kaloko start`; see
+[devices.md](devices.md). App devices (emulators, simulators, phones) are named per environment under
+`app.<platform>.devices`; see [apps.md](apps.md).
+
 ## Who does what
 
 Kaloko needs evidence in its format: captures of screens and e-mails in a run.

package/skills/kaloko/reference/execution.md

@@ -41,6 +41,12 @@ call `kaloko session reset` themselves when the plan starts as a new visitor.
 Prefer turning an agent step into a playwright script once it has worked
 twice the same way.
 
+A step script brings the page into the state the step accepts; it does not
+check it. Do not compute a result in the script and write a `data-qa-*`
+marker into the page for a criterion to find. The URL, computed styles and
+values in the page have their own checks (`url_matches`, `style_equals`,
+`js_equals`; see [acceptance.md](acceptance.md#url-computed-styles-and-values-in-the-page)).
+
 ## Driving the shared browser with your capabilities
 
 - Browser tool over MCP (e.g. Playwright MCP): connect to the CDP endpoint