kaloko 0.4.11 → 0.5.1

package/skills/kaloko/reference/apps.md

@@ -6,13 +6,19 @@ driver differs, and it is chosen by the scenario's `platform`.
 
 | platform | Driven by | UI tree | Interactions |
 |---|---|---|---|
-| `android` | `adb` (emulator or USB device) | uiautomator, in dp | deep links, tap by selector or point, type, back |
+| `android` | `adb`: emulator, USB phone or a phone over Wi-Fi | uiautomator, in dp | deep links, tap by selector or point, type, back |
 | `ios` | `xcrun simctl` (iOS Simulator) | `idb` when installed, otherwise text by OCR (macOS Vision) | deep links; taps and typing need `idb` |
+| `ios` + `real: true` | `xcrun devicectl` and `idevicescreenshot` or `pymobiledevice3` (a real iPhone) | text by OCR | install, launch, deep links; no taps |
 | `electron` | Playwright Electron | the full DOM: every web check applies | everything Playwright does (`page`) |
 | `desktop` | macOS window of a named app | macOS accessibility tree, otherwise OCR | none built in: use your tool, AppleScript or a person |
+| `windows` | the app's window through Windows PowerShell and UI Automation (nothing to install) | UI Automation tree | tap by selector or point, type; walk on Windows |
 
-Anything else, including a real iPhone, Windows apps, Flutter/React Native
-under Detox, Maestro flows or XCUITest, is captured by import:
+An installed web app (PWA) is a web scenario with `display_modes: [standalone]`; see
+[devices.md](devices.md). A WebView inside an Android app is captured as a web page; see
+[WebViews](#webviews-inside-apps).
+
+Anything else, including Flutter or React Native under Detox, Maestro flows,
+XCUITest or Appium runs, is captured by import:
 
 ```bash
 npx kaloko capture --step checkout --image shot.png --tree hierarchy.xml --device "Pixel 8" --density 2.625
@@ -24,6 +30,7 @@ npx kaloko capture --step checkout --image shot.png --tree hierarchy.xml --devic
 - Appium iOS XML (`XCUIElementType…`);
 - `idb ui describe-all --json`;
 - Maestro `hierarchy` JSON;
+- Windows UI Automation JSON from Kaloko's `windows.ps1 tree` (`--platform windows`);
 - Kaloko's own `qawalk.tree.v1`.
 
 `--ocr` reads the text from the image instead (macOS). `--density` or
@@ -46,6 +53,7 @@ environments:
     app:
       electron: { main: dist/main.js }          # or executable: path to the packaged app
       # desktop: { name: Notes }                # a macOS app by its window owner name
+      # windows: { name: contoso, path: dist/Contoso.exe }   # or aumid: a Store/MSIX app
 ```
 
 ```yaml
@@ -81,9 +89,50 @@ nodes:
 - The app language is set per locale: Android 13+ app locales, iOS
   `-AppleLanguages`, Electron `--lang`.
 
+## Devices, dark mode and motion
+
+`devices:` walks the same scenario on several emulators, simulators or phones; each device is its own capture key
+next to the colour schemes and motion of the run:
+
+```yaml
+# qa/flows/shop-android.yml
+platform: android
+devices: [pixel-8-api-34, pixel-7-api-33, galaxy]
+color_schemes: [light, dark]
+```
+
+```yaml
+# kaloko.config.yml
+environments:
+  staging:
+    app:
+      android:
+        package: com.acme.shop.staging
+        devices:
+          galaxy: { connect: 192.168.1.20:5555, title: Galaxy S24 }   # a phone over Wi-Fi
+      ios:
+        bundle: com.acme.shop
+        devices:
+          my-iphone: { udid: 00008110-000A…, real: true, title: Test iPhone }
+```
+
+- A name without an entry says what it is. Android: an adb serial (`emulator-5554`, `R58M…`, `host:port`) or an
+  AVD name (`pixel-8-api-34` finds `Pixel_8_API_34`; Kaloko starts it headless when it is not running). iOS: a
+  simulator by name and runtime (`iphone-15-ios-18` is "iPhone 15" on iOS 18; `iphone-se-3` is "iPhone SE (3rd
+  generation)").
+- `kaloko start --devices pixel-8-api-34` overrides the scenario; `kaloko walk --devices galaxy` walks one of them.
+- Real Android phones are woken up and unlocked (no PIN) and get a clean status bar (demo mode, 9:41, full battery).
+  `wake: false` and `status_bar: false` turn that off.
+- Dark mode and reduced motion are set on the device per capture: `cmd uimode night` and the animator scales on
+  Android (the app redraws; some apps recreate the screen), `simctl ui appearance` on the iOS Simulator, emulated
+  media in Electron. The iOS Simulator has no command for reduced motion, and real devices keep their own setting;
+  the capture notes it.
+- The canvas draws the device frame of the closest preset (`pixel-8-api-34` → Pixel 8) and offers the safe-area
+  overlay; app captures are the whole screen.
+
 **Step scripts**
 
-A script gets `{ app, page, locale, env, run, capture, link }`:
+A script gets `{ app, page, webview, device, locale, env, run, capture, link }`:
 
 ```js
 // steps/add-to-cart.mjs
@@ -97,9 +146,52 @@ export default async ({ app, link, capture }) => {
 ```
 
 - `app.tap`, `app.wait` and `app.type` take app selectors. `app.tap({ x, y })`
-  taps by coordinates, in dp or points.
+  taps by coordinates, in dp or points (logical pixels on Windows).
 - On Electron, selectors are CSS and `page` is the Playwright page, so
   any Playwright code works there.
+- `capture()` takes every key of the current device (light and dark);
+  `capture({ viewport: 'pixel-8-api-34.dark' })` one of them.
+
+## WebViews inside apps
+
+An Android WebView with debugging on (`WebView.setWebContentsDebuggingEnabled(true)` in debug builds) is a Chromium
+page. `await webview()` in a step script returns it as a Playwright page (Kaloko forwards its DevTools socket with
+adb), and `capture({ webview: true })`, or `webview: true` in `app.android`, captures it as a web page: the DOM, the
+web checks, the accessibility basics. The native screen around it is not in that capture; capture both when both
+matter.
+
+iOS WKWebViews (`isInspectable`) speak Safari's Web Inspector protocol, which Playwright cannot attach to. Capture
+those screens as app screens (tree or OCR), and test the web content itself as a web scenario in WebKit
+(`browsers: [webkit]`) with an iPhone preset.
+
+## Windows apps
+
+`platform: windows` with `app.windows: { name, path | aumid, args }` walks a desktop app on Windows through the
+PowerShell every Windows has: the window as a screenshot (at its display scale), the UI Automation tree (control
+types, AutomationId, names, values), clicks by selector and typing (SendKeys). It needs a signed-in desktop session
+(a person's machine, or a `windows-latest` CI runner); a locked screen or a service account cannot see windows.
+Elsewhere, take the screenshots on Windows and import them with
+`kaloko capture --image shot.png --tree uia.json --platform windows`, or run `kaloko import` on a folder of
+screenshots.
+
+## Real iPhones
+
+`real: true` with the `udid` uses Xcode's `devicectl` to install (`app:` a `.app` or `.ipa` signed for the device),
+launch and open deep links, and `idevicescreenshot` (`brew install libimobiledevice`) or `pymobiledevice3` for the
+screenshot. On iOS 17 and newer, `pymobiledevice3` needs its tunnel running (`sudo pymobiledevice3 remote tunneld`).
+There is no UI tree or tapping on a real device: text comes from OCR on macOS, and flows that need taps run in
+XCUITest, Maestro or Appium, with each screen imported by `kaloko capture --image`.
+
+## React Native and Flutter
+
+- **React Native** renders native views: `testID` becomes the Android resource-id and the iOS accessibility id, so
+  `[id$=checkout]` selectors work; `accessibilityLabel` becomes the name `app_labels` checks. Deep links through
+  React Navigation linking work as step paths.
+- **Flutter** draws its own widgets. The UI tree only has what the semantics tree exposes: wrap controls in
+  `Semantics(label: …)` or give them `Key`s that reach the accessibility layer, otherwise checks see one big view.
+  Flutter web builds are a web scenario. For taps, Maestro handles Flutter well; import its screens.
+- Both: keep a debug or staging flavour with a stable package name per environment, and deep links into every screen
+  a scenario needs. That makes steps a deep link instead of a script.
 
 ## Selectors and checks on the UI tree
 
@@ -127,8 +219,8 @@ dialog, webview and group.
 - **Pack `app` (AC7xx):** those four checks, plus a semantic question about
   the screen's purpose and next action.
 - **Web-only checks** (`no_overflow`, `http_status`, meta, `a11y_contrast`,
-  `perf_*`): these are "not evaluated" on app captures. Do not put them on
-  app steps.
+  `perf_*`, `url_*`, `style_*`, `js_equals`): these are "not evaluated" on
+  app captures. Do not put them on app steps.
 - **OCR trees:** these have text only, so role checks (labels, touch
   targets) are not evaluated there. Say so in the report and install `idb`
   when those checks matter.
@@ -150,7 +242,10 @@ questions about what is on the screen, not about the markup.
 
 - **Devices:** never install on a person's own phone or sign in with
   real accounts without asking. Emulators and simulators are yours to
-  start, reset and install builds on.
+  start, reset and install builds on. Dark mode and the animation scales
+  are restored after a walk, and so is the status bar of a real phone.
+- **Windows:** clicks and typing go to the real desktop. Walk on a machine
+  nobody is using at that moment, or a CI runner.
 - **Production apps:** production apps (store builds) are read-only in
   spirit. Walk them only with deep links and screens that change no data,
   unless the person agrees.

package/skills/kaloko/reference/ci.md

@@ -16,7 +16,9 @@ deployments; `--trigger pull_request` runs on every push instead and waits for
 the deployment):
 
 1. installs the project (`npm ci`, or pnpm, yarn, bun by the lockfile) and the
-   Chromium of kaloko's Playwright (`kaloko browser install --with-deps`);
+   Chromium of kaloko's Playwright (`kaloko browser install --with-deps`), plus
+   WebKit and Firefox when a scenario or `walk.browsers` names them
+   (`--browsers chromium,webkit`; see [devices.md](devices.md));
    `KALOKO_BROWSER_CHANNEL=chromium` makes the run use it, so no separate CI
    config is needed;
 2. `kaloko ci preview-url --wait 600 --github-env` finds the preview URL, the

package/skills/kaloko/reference/collaboration.md

@@ -72,8 +72,11 @@ comments; at the end they accept or return the run.
 Comments are threads in three places:
 
 - **A step:** optionally pinned to a position on the screenshot
-  (`pin: {x, y}` as fractions, per viewport) or to an element of a live
-  design step (`anchor`).
+  (`pin: {x, y}` as fractions, per viewport), drawn on it (a box, an arrow or
+  a line: `shape`, with `region` giving the area as fractions and in CSS
+  pixels of the page, and for an arrow the point it aims at), or attached to
+  an element of a live design step (`anchor`). Comments may carry images
+  (`attachments`, each with a download URL that needs the token).
 - **The whole run:** `node_id` is `''`.
 - **The whole flow:** these stay across runs, e.g. "add the empty-cart
   state".
@@ -85,8 +88,12 @@ said what.
 
 The loop, for every open thread:
 
-1. **Understand it.** Open the canvas link when a pin or an element
-   matters; the position is on the screenshot of that viewport and locale.
+1. **Understand it.** A `region` says exactly where to look: `px` is
+   `[x, y, width, height]` in CSS pixels of the captured page for that
+   viewport (`region.viewport`) and locale; find the element there in the
+   step's HTML or screenshot. Download attached images (a corrected copy, a
+   reference) before you change anything. Open the canvas link when a pin or
+   an element matters.
 2. **Fix it** in the code or the scenario. When the comment asks for a
    change of intent (criteria, scope), ask the person first.
 3. **Answer and close.** Use
@@ -181,14 +188,25 @@ e-mail works too). Use it when a step needs a decision from a specific person
   Put the relevant lines in the issue comment. On the canvas the same
   comparison is the **Compare** selector in the top bar (or `#cmp=<run>` in
   the URL): pixel differences per step, changed criteria, and a side-by-side
-  or diff view in the lightbox. Pixel diffs are computed in the browser;
-  nothing is uploaded.
+  or diff view in the lightbox, plus *Swipe* (a divider dragged across both
+  screenshots) and *Onion skin* (this run faded over the other). Pixel diffs
+  are computed in the browser; nothing is uploaded. **G** opens the grid: one
+  step in every locale and viewport at once, failing cells marked.
 - **Manual criteria** are decided on the canvas by a reviewer; treat a
   reviewer's `fail` like a failing deterministic check.
 - Admins may name **required approvers** per scenario (Settings). Such a
   run stays `pending` until every one of them approved; `kaloko feedback`
   prints `awaiting approval from: …`, so do not treat a single approval as
   acceptance there.
+- An accepted run has a **signed acceptance record** (Business and up): the
+  run, scenario, commit and environment, who approved what and when (people
+  apart from tokens and agents, approvals carried over unchanged steps with
+  where they came from), criteria results, capture hashes and the hash of
+  `run.json`, signed with the service's Ed25519 key. `kaloko signoff <run>`
+  downloads the JSON and the PDF and checks them; `kaloko signoff --verify
+  <file>` checks a record someone handed over (public key at
+  `/.well-known/kaloko-signoff.json`). Give it to the person when they ask
+  for proof of acceptance; never create one for a run that is not accepted.
 
 ## Cleaning up runs
 
@@ -245,6 +263,17 @@ Everything above is available over the API with an organization token:
   (the whole flow), and per comment `PATCH`, `DELETE`, `POST …/<cid>/resolve`
   (with `note`) and `…/<cid>/reopen`; `GET /api/v1/orgs/<org>/comments?state=open|all&scenario=&run=&mine=1`
   across the organization. Lists of runs and scenarios carry `open_comments`.
+  A comment body may carry `shape` (`{kind: box, box: [x, y, w, h]}`,
+  `{kind: arrow, from, to}` or `{kind: pen, points}` in fractions, with
+  `viewport` and `size`) and `attachments` (ids from
+  `POST …/comments/attachments?name=<file>` with the image as the body: PNG,
+  JPEG, WebP or GIF up to 5 MB, counted in the plan's storage).
+- `GET|POST /api/v1/orgs/<org>/runs/<id>/ignore-regions`,
+  `DELETE …/ignore-regions/<rid>`, `GET …/scenarios/<id>/ignore-regions[?state=all]`
+  and `POST …/scenarios/<id>/ignore-regions/applied` (`{ids}`): regions drawn
+  to leave out of pixel diffs (drawing needs Developer / Tester).
+- `GET /api/v1/orgs/<org>/runs/<id>/signoff` (JSON), `…/signoff.pdf`,
+  `…/signoff.html`: the signed acceptance record of an accepted run.
 - `GET /api/v1/orgs/<org>/notifications[?unread=1]`, `POST` with
   `{"read": "all"}` or `{"read": [ids]}`
 - `GET /api/v1/orgs/<org>/runs[?project=&scenario=&kind=&verdict=&author=&environment=&branch=&q=][&latest=1]`

package/skills/kaloko/reference/design.md

@@ -48,6 +48,45 @@ design/
   `kaloko design lint` checks the folder; `kaloko sync` runs it first and stops on errors (things that break on
   the live step), warnings are printed.
 
+## Devices, dark mode and app screens
+
+Design steps can target device presets and colour schemes like any run ([devices.md](devices.md)):
+
+```yaml
+# the design scenario (the same one development uses)
+viewports: [iphone-15, pixel-8, ipad-landscape, laptop]
+color_schemes: [light, dark]
+```
+
+- On a design environment a phone or tablet preset is the whole screen (393 × 852 on an iPhone 15), rendered with
+  the preset's touch, pixel ratio and user agent, so `(pointer: coarse)` and `(hover: none)` apply. The step detail
+  draws the device frame; *Safe areas* shows the notch, the home indicator and where the first screen ends.
+- Dark mode: write `@media (prefers-color-scheme: dark)` in the step's CSS, and the dark captures show it. A step
+  whose dark screen is a different page gets `<step>/dark/index.html` (or `<step>/<locale>/dark/index.html`); dark
+  captures render that page, and it travels with the revision.
+- Token modes follow the scheme: with a `theme` modifier (or `light`/`dark` groups) in the token set, Dev mode, the
+  step spec and the token check of a dark capture name the dark tokens, and drift compares dark with dark.
+- Browsers: design steps render in Chromium. Check the implementation in WebKit and Firefox instead.
+
+**Native app screens as portable HTML.** A mockup of an iOS or Android screen is a full-screen page:
+
+- Size it to the screen, not to a document: `html, body { height: 100% }` and the main area `min-height: 100dvh`.
+  Do not draw the status bar or the home indicator; the frame does.
+- Keep content out of the safe areas with
+  `padding-top: max(env(safe-area-inset-top), var(--kaloko-safe-top, 0px))` (and `-bottom`, `-left`, `-right`).
+  On the phone `env()` gives the inset; in Kaloko `--kaloko-safe-*` does, because desktop browsers report 0. Add
+  `viewport-fit=cover` to the viewport meta.
+- Use the roles and names the app will have: `<nav aria-label="Tabs">` with buttons for a tab bar, `<header>` with
+  an `<h1>` for the navigation bar, `<button>` and labelled inputs. The step spec and the checks match the
+  implementation by role and name, and React Native `accessibilityLabel` or Android `contentDescription` are those
+  names.
+- Touch targets of at least 44 × 44 pt (iOS) or 48 × 48 dp (Android); the app pack checks them on the real app.
+- One design serves both platforms unless they really differ. When they do, give the platform its own scenario
+  step or its own viewport set (`viewports: [iphone-15]` in one scenario, `[pixel-8]` in another), not user-agent
+  sniffing in the page.
+- System fonts are not portable (`-apple-system` renders differently on each machine): bundle the font the app
+  uses, as the portable HTML rules say.
+
 ## Sync and publish
 
 - `kaloko walk` renders only revisions whose captures are missing (same files, same capture).

package/skills/kaloko/reference/devices.md

@@ -0,0 +1,109 @@
+# Browsers, devices, colour schemes and motion
+
+A run captures every step in each locale and viewport. Five more dimensions can be added to that matrix: the
+browser engine, the colour scheme, reduced motion, forced colours (Windows high contrast) and the display mode of an
+installed web app. Viewports can name device presets instead of sizes.
+
+## Asking for them
+
+In the scenario, with defaults for the whole project in `walk:` of `kaloko.config.yml`, and per run on the
+command line (the command line wins, then the scenario, then the config):
+
+```yaml
+# qa/flows/checkout.yml
+viewports: [iphone-se, iphone-15, pixel-8, ipad-landscape, laptop]   # presets, or { desktop: [1280, 800] } as before
+browsers: [chromium, webkit, firefox]
+color_schemes: [light, dark]
+reduced_motion: [no-preference, reduce]      # optional
+forced_colors: [none, active]                # optional, Windows high contrast
+display_modes: [browser, standalone]         # optional, the page as an installed PWA
+```
+
+```bash
+npx kaloko start --scenario qa/flows/checkout.yml --env staging --browsers chromium,webkit --color-schemes light,dark
+npx kaloko start … --viewports iphone-se,fold-open,desktop-hd   # presets the scenario does not list
+npx kaloko walk                         # one pass per engine and device profile
+npx kaloko walk --browsers webkit       # only the WebKit captures of the run
+npx kaloko browser install --browsers webkit,firefox   # once per machine or CI job
+npx kaloko devices                      # the preset catalogue
+```
+
+Every combination is one capture: `checkout × 3 viewports × 2 browsers × 2 schemes` is 12 screenshots per step and
+locale. Keep the matrix to what the task is about. A Safari bug report needs `browsers: [chromium, webkit]` on the
+affected steps, not every dimension on every step. A run refuses more than 400 combinations.
+
+## Capture keys
+
+A capture is keyed by the viewport name plus the values that differ from the default, joined by dots:
+`mobile`, `mobile.webkit`, `iphone-15.webkit.dark`, `desktop.reduce`, `phone.standalone`. Chromium, light, no
+motion preference, no forced colours and the browser display mode are the defaults. Runs from before these
+dimensions have plain keys and read as Chromium. Use the keys wherever a viewport name goes: `--viewport
+iphone-15.dark`, `capture({ viewports: ['mobile'] })` (a viewport name takes all its keys in the current pass),
+`kaloko spec home --viewport desktop.dark`.
+
+## Device presets
+
+| Kind | Presets |
+|---|---|
+| Phones | `iphone-se` (small), `iphone-15`, `iphone-16`, `iphone-16-pro-max` (large), `pixel-8`, `pixel-8-pro`, `galaxy-s24` |
+| Foldables | `fold-folded`, `fold-open` |
+| Tablets | `ipad-portrait`, `ipad-landscape`, `ipad-pro-portrait`, `ipad-pro-landscape`, `android-tablet-portrait`, `android-tablet-landscape` |
+| Computers and TV | `laptop` (1440), `desktop-hd` (1920), `desktop-wide` (2560), `tv-1080p` |
+
+- `<preset>-landscape` turns any phone, foldable or tablet sideways (`iphone-15-landscape`).
+- A preset sets the CSS viewport, the pixel ratio, mobile mode, touch and the device's user agent. Web walks get
+  the screen minus the browser's bars (393 × 659 on an iPhone 15 in Safari); Design environments and installed
+  PWAs get the whole screen (393 × 852).
+- Overrides: `{ phone: { device: iphone-15, height: 700 } }`, or `full_screen: true` for a kiosk or an app mockup.
+- Screenshots stay in CSS pixels whatever the pixel ratio, so boxes, diffs and comments line up across devices.
+- The values are those of the device class, not of a lab. They catch layout bugs; a real device catches the rest.
+
+## How each dimension is produced
+
+| Dimension | How | Engines |
+|---|---|---|
+| Browser | Playwright's Chromium, WebKit (Safari's engine) and Firefox, each in its own pass | all |
+| Device preset | its own browser context with the pixel ratio, mobile mode, touch and user agent | all; Firefox has no mobile mode, so presets there set size, ratio, touch and agent |
+| Colour scheme, reduced motion, forced colours | `prefers-color-scheme`, `prefers-reduced-motion` and `forced-colors` emulated per capture | all |
+| Display mode | `display-mode` media rules of same-origin style sheets and `matchMedia` answered as installed; full-screen size | all |
+
+- Scripts that read `matchMedia('(display-mode: standalone)')` once at load still see the browser mode; CSS and
+  later calls see the installed mode. Check `navigator.standalone`-style logic on a real install.
+- The shared browser (Chromium over CDP) walks the default pass; WebKit, Firefox and device profiles always get
+  their own context, signed in through the scenario's sessions as usual.
+- E-mail steps are captured once, in the first pass.
+
+## Checks that only Chromium can run
+
+These are "not evaluated" on WebKit and Firefox captures, with the reason, and do not make a step partial when
+Chromium evaluated them:
+
+- `a11y_ax_names` (the accessibility tree is read over the DevTools protocol);
+- `a11y_focus_visible` (focus rings are compared on Chromium screenshots);
+- `perf_cls` (only Chromium reports layout shifts);
+- on WebKit also the Tab walk (`a11y_focus_order`, `a11y_keyboard_trap`, `a11y_skip_link`): WebKit moves Tab
+  between form controls only, like Safari by default, so the walk would report false gaps.
+
+Everything else (DOM checks, contrast, overflow, links, LCP, TTFB, weight, semantic and vision criteria) runs on
+every engine. Contrast is checked per colour scheme, so a dark-mode contrast bug fails on the `.dark` capture.
+
+Semantic questions read the page text: the same text in another engine or scheme is the same question, answered
+once and then from the cache. Vision questions (`vision: true`) look at each screenshot, so every combination is
+asked separately; keep `vision: true` to the steps where the look is the point.
+
+## On the canvas
+
+- Switchers for the browser, colour scheme, motion, forced colours and display mode sit next to the locale in the
+  top bar and in review mode. The map, the index, the step detail and review show the chosen combination.
+  With more than five values (20 locales, say) a switcher is a button that opens a searchable list with the
+  status of each value; `L` opens the locale list, `[` and `]` move to the previous and next locale.
+- The step detail of a device preset draws its frame: bezel, status bar, the browser's bars and the island, notch
+  or camera. *Safe areas* marks the insets and where the first screen ends. Both can be switched off.
+- *Beside* puts the same step in another browser or scheme next to it (WebKit beside Chromium, dark beside light).
+- Links keep the key (`#step=cart&vp=iphone-15.webkit.dark`) and switch the canvas to it.
+
+## Doctor and CI
+
+`kaloko doctor` lists the engines the scenarios and `walk.browsers` name and says how to install a missing one.
+In CI install them in the job: `npx kaloko browser install --browsers chromium,webkit --with-deps` (Linux needs
+the system libraries WebKit and Firefox bring).

package/skills/kaloko/reference/environments.md

@@ -18,6 +18,12 @@ environment whose URL comes from CI: `preview: { base_url: ${KALOKO_PREVIEW_URL}
 which `kaloko ci github|gitlab` adds and `kaloko ci preview-url` fills; see
 [ci.md](ci.md).
 
+An environment says where the product runs. In which browsers, on which devices and in which colour scheme it is
+walked is the run's matrix: `browsers`, `color_schemes`, `reduced_motion` and device presets in `viewports` of the
+scenario, defaults in `walk:` of the config, `--browsers` and friends on `kaloko start`; see
+[devices.md](devices.md). App devices (emulators, simulators, phones) are named per environment under
+`app.<platform>.devices`; see [apps.md](apps.md).
+
 ## Who does what
 
 Kaloko needs evidence in its format: captures of screens and e-mails in a run.

package/skills/kaloko/reference/execution.md

@@ -41,6 +41,12 @@ call `kaloko session reset` themselves when the plan starts as a new visitor.
 Prefer turning an agent step into a playwright script once it has worked
 twice the same way.
 
+A step script brings the page into the state the step accepts; it does not
+check it. Do not compute a result in the script and write a `data-qa-*`
+marker into the page for a criterion to find. The URL, computed styles and
+values in the page have their own checks (`url_matches`, `style_equals`,
+`js_equals`; see [acceptance.md](acceptance.md#url-computed-styles-and-values-in-the-page)).
+
 ## Driving the shared browser with your capabilities
 
 - Browser tool over MCP (e.g. Playwright MCP): connect to the CDP endpoint
@@ -82,6 +88,7 @@ When pixel diffs between runs keep flagging the same clock, ad or carousel:
 kaloko walk --stability 3        # 3 screenshots per capture, a moment apart; what moved is noted
 kaloko stability                 # this run against the 3 previous local runs of the scenario
 kaloko stability --apply         # adds the suggested ignore: to each step (block-style YAML only)
+kaloko stability --pull          # regions people drew on screenshots on kaloko.app go into ignore:
 ```
 
 `kaloko stability` writes `stability.json` and `stability.yml` into the run
@@ -92,6 +99,14 @@ changed is not noise. `--same-commit` compares only runs of the same commit,
 `--runs a,b` names them, `--last N` sets how many runs are compared. Capture
 the steps again after applying: the capture records the ignored regions.
 
+People mark noise by hand too: a Developer / Tester draws a rectangle on a
+screenshot (step detail → *Mark a changing region*). It is kept per flow, shows
+dashed on every run and the canvas pixel diff already skips it.
+`kaloko stability --pull` writes those regions into the scenario's `ignore:`
+(appended to an existing block list, never twice) and tells the service they
+are in the file; walk those steps again afterwards. MCP `list_ignore_regions`
+lists them, `add_ignore_region` suggests one.
+
 ## Only what changed
 
 After a fix, walk the steps the change reaches instead of the whole flow:

package/skills/kaloko/reference/mcp.md

@@ -42,8 +42,11 @@ file.
 
 | Tool | Use |
 |---|---|
-| `list_comments` | open threads (or `state: all`) across the organization, filtered by `scenario` or `run_id`, with replies, pins, anchors and who wrote them |
-| `add_comment` | comment on a step (`run_id` + `step`), the whole run (`run_id`) or the whole flow (`scenario`); `parent_id` replies; optional `pin` `{ x, y, viewport }` |
+| `list_comments` | open threads (or `state: all`) across the organization, filtered by `scenario` or `run_id`, with replies, pins, anchors, drawn regions (`region.px`: CSS pixels of the page in `region.viewport`), attached images with URLs, and who wrote them |
+| `add_comment` | comment on a step (`run_id` + `step`), the whole run (`run_id`) or the whole flow (`scenario`); `parent_id` replies; optional `pin` `{ x, y, viewport }` or `shape` (box or arrow in fractions, with `viewport` and `size`) |
+| `list_ignore_regions` | regions people drew to leave out of pixel diffs, per step, with the `ignore:` entry each becomes (`kaloko stability --pull` writes them) |
+| `add_ignore_region` | suggest such a region (`run_id`, `step`, `viewport`, `rect` in CSS px); Developer / Tester, scope upload or full |
+| `get_signoff_record` | the signed acceptance record of an accepted run (Business): approvals with provenance, criteria, capture hashes, signature |
 | `resolve_comment` | close a thread with a `note` that says what changed (it becomes the last reply) |
 | `reopen_comment`, `edit_comment`, `delete_comment` | reopen; edit or delete your own comment |