kaloko 0.4.11 → 0.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/README.md +45 -4
  2. package/package.json +1 -1
  3. package/schemas/config.v1.json +42 -2
  4. package/schemas/run.v1.json +33 -2
  5. package/schemas/scenario.v1.json +107 -16
  6. package/skills/kaloko/SKILL.md +12 -4
  7. package/skills/kaloko/reference/acceptance.md +74 -3
  8. package/skills/kaloko/reference/apps.md +103 -8
  9. package/skills/kaloko/reference/ci.md +3 -1
  10. package/skills/kaloko/reference/collaboration.md +35 -6
  11. package/skills/kaloko/reference/design.md +39 -0
  12. package/skills/kaloko/reference/devices.md +109 -0
  13. package/skills/kaloko/reference/environments.md +6 -0
  14. package/skills/kaloko/reference/execution.md +15 -0
  15. package/skills/kaloko/reference/mcp.md +5 -2
  16. package/src/cli.mjs +366 -192
  17. package/src/lib/access.mjs +5 -2
  18. package/src/lib/apps/devices.mjs +187 -0
  19. package/src/lib/apps/drivers.mjs +11 -3
  20. package/src/lib/apps/ios-device.mjs +78 -0
  21. package/src/lib/apps/tree.mjs +9 -1
  22. package/src/lib/apps/webview.mjs +54 -0
  23. package/src/lib/apps/windows.mjs +108 -0
  24. package/src/lib/apps/windows.ps1 +78 -0
  25. package/src/lib/assertions.mjs +300 -0
  26. package/src/lib/build.mjs +4 -1
  27. package/src/lib/capture.mjs +146 -119
  28. package/src/lib/ci.mjs +8 -4
  29. package/src/lib/design-cli.mjs +35 -6
  30. package/src/lib/design.mjs +3 -1
  31. package/src/lib/devices.mjs +85 -0
  32. package/src/lib/devspec.mjs +3 -2
  33. package/src/lib/dimensions.mjs +326 -0
  34. package/src/lib/doctor.mjs +31 -6
  35. package/src/lib/drift.mjs +8 -4
  36. package/src/lib/evaluate.mjs +25 -7
  37. package/src/lib/import.mjs +2 -1
  38. package/src/lib/jev.mjs +3 -1
  39. package/src/lib/manifest-core.mjs +12 -3
  40. package/src/lib/probe.mjs +60 -0
  41. package/src/lib/recheck.mjs +6 -1
  42. package/src/lib/scenario.mjs +3 -0
  43. package/src/lib/signoff-core.mjs +138 -0
  44. package/src/lib/signoff.mjs +83 -0
  45. package/src/lib/stability-core.mjs +4 -2
  46. package/src/lib/stability.mjs +82 -0
  47. package/src/lib/walk-dimensions.mjs +83 -0
  48. package/viewer/index.html +739 -48
package/README.md CHANGED
@@ -11,10 +11,11 @@ there, and the newest captures double as documentation.
11
11
  - Product and guides: <https://kaloko.app>
12
12
  - Service (organizations, shared canvases, reviews): <https://kaloko.app>
13
13
 
14
- Kaloko also walks **native apps**: Android over adb, the iOS Simulator,
15
- Electron and macOS apps. Screens taken by any other tool (Maestro, Appium,
16
- XCUITest) are imported with their UI tree. See the `apps` reference of the
17
- skill.
14
+ Pages are walked in Chromium, WebKit and Firefox, on phone, tablet and
15
+ desktop presets, in light and dark mode. Native apps too: Android, iOS,
16
+ Electron, macOS and Windows. Screens taken by any other tool (Maestro, Appium,
17
+ XCUITest) are imported with their UI tree. See
18
+ [What Kaloko can walk and design](#what-kaloko-can-walk-and-design).
18
19
 
19
20
  ## Install
20
21
 
@@ -71,8 +72,48 @@ once with `--concurrency 3`. When a step's selector no longer exists, the walk
71
72
  proposes one from the page and prints the patch; `--heal` applies it once the
72
73
  step passes.
73
74
 
75
+ Automatic criteria read the page itself: text, counts and attributes, the URL
76
+ after a step (`url_matches`), computed styles, also against a design token
77
+ (`style_equals` with `token: color.brand.primary`), and read-only values such
78
+ as `localStorage` or `dataLayer` (`js_equals`). A step script never has to
79
+ compute a result and leave a marker for a criterion to find. The acceptance
80
+ reference of the `kaloko` skill lists every check.
81
+
74
82
  `npx kaloko help` lists every command.
75
83
 
84
+ ## What Kaloko can walk and design
85
+
86
+ *Full*: Kaloko drives it, captures it and runs the checks. *Capture only*: Kaloko
87
+ takes the screens, your tool or a person moves between them. *Via import*: another
88
+ tool takes the screens and `kaloko capture --image` or `kaloko import` lays them out.
89
+
90
+ | Target | How | Checks | Design | Support |
91
+ |---|---|---|---|---|
92
+ | Web in Chromium | Playwright, the shared browser or an own one | all | live HTML steps | full |
93
+ | Web in WebKit (Safari's engine) and Firefox | Playwright, one pass per engine | all except the accessibility tree, focus rings and layout shifts (and the Tab walk on WebKit) | rendered in Chromium | full |
94
+ | Phones, foldables, tablets, laptop to 2560 px, TV | device presets: size, pixel ratio, touch, user agent | all | whole screen, device frame, safe areas | full |
95
+ | Light and dark, reduced motion, high contrast | emulated per capture | all; contrast per scheme | dark pages, token modes per scheme | full |
96
+ | Installed web app (PWA) | `display-mode: standalone` rules and `matchMedia` | all | yes | full for CSS; scripts that read the mode at load need a real install |
97
+ | Android: emulator, USB or Wi-Fi phone | adb: deep links, taps, typing, dark mode | UI tree, app pack | HTML mockups | full |
98
+ | Android WebView | DevTools over adb (debuggable builds) | web checks on the page | | full |
99
+ | iOS Simulator | simctl: deep links, dark mode; taps with idb | UI tree with idb, text by OCR without | HTML mockups | full with idb, capture only without |
100
+ | iPhone and iPad | devicectl, idevicescreenshot or pymobiledevice3 | text by OCR | HTML mockups | capture only |
101
+ | Electron | Playwright | all web checks | | full |
102
+ | macOS apps | window screenshot, accessibility tree | UI tree | | capture only |
103
+ | Windows apps | PowerShell and UI Automation: taps, typing | UI tree | | full on Windows, via import elsewhere |
104
+ | React Native, Flutter, iOS WKWebView, Maestro, Appium, XCUITest | their own runner | UI tree from a dump, or text | | via import |
105
+
106
+ ```bash
107
+ npx kaloko devices # the presets
108
+ npx kaloko browser install --browsers webkit,firefox
109
+ npx kaloko start --scenario qa/flows/checkout.yml --env staging \
110
+ --browsers chromium,webkit --color-schemes light,dark --viewports iphone-se,pixel-8,desktop-hd
111
+ ```
112
+
113
+ The canvas switches between browsers and schemes next to the locale, draws the device frame and shows the same
114
+ step in another browser beside it. `npx kaloko doctor` says which engines and app tools are missing on this
115
+ machine and how to install them. Details: `skills/kaloko/reference/devices.md` and `apps.md`.
116
+
76
117
  ## Pull request previews in CI
77
118
 
78
119
  ```bash
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "kaloko",
3
- "version": "0.4.11",
3
+ "version": "0.5.1",
4
4
  "description": "See what your AI agents built: every screen, language and device walked, assessed and approved on one canvas.",
5
5
  "keywords": [
6
6
  "qa",
@@ -78,7 +78,12 @@
78
78
  "type": "object",
79
79
  "additionalProperties": false,
80
80
  "properties": {
81
- "concurrency": { "type": "integer", "minimum": 1, "maximum": 16, "description": "Locales walked at the same time, each in its own browser context (kaloko walk --concurrency). Default: by CPU in an own browser, one at a time on the shared browser." }
81
+ "concurrency": { "type": "integer", "minimum": 1, "maximum": 16, "description": "Locales walked at the same time, each in its own browser context (kaloko walk --concurrency). Default: by CPU in an own browser, one at a time on the shared browser." },
82
+ "browsers": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["chromium", "webkit", "firefox"] }, "description": "Default browser engines for web runs when the scenario and kaloko start --browsers say nothing." },
83
+ "color_schemes": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["light", "dark"] }, "description": "Default colour schemes (prefers-color-scheme) for every run." },
84
+ "reduced_motion": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["no-preference", "reduce"] } },
85
+ "forced_colors": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["none", "active"] } },
86
+ "display_modes": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["browser", "standalone"] } }
82
87
  }
83
88
  },
84
89
  "capture": {
@@ -120,6 +125,18 @@
120
125
  }
121
126
  ]
122
127
  },
128
+ "appDevices": {
129
+ "type": "object",
130
+ "description": "Devices a scenario's devices: list names, each an emulator, simulator or real device. Without an entry the name is matched against adb serials and AVDs (pixel-8-api-34 → Pixel_8_API_34) or simulators (iphone-15-ios-18 → iPhone 15 on iOS 18).",
131
+ "additionalProperties": {
132
+ "type": "object", "additionalProperties": false,
133
+ "properties": {
134
+ "serial": { "type": "string" }, "avd": { "type": "string" }, "connect": { "type": "string" },
135
+ "udid": { "type": "string" }, "device": { "type": "string", "description": "Simulator name (iOS) or the canvas label (Android)." }, "runtime": { "type": "string" },
136
+ "real": { "type": "boolean" }, "title": { "type": "string", "description": "Label on the canvas." }
137
+ }
138
+ }
139
+ },
123
140
  "environment": {
124
141
  "type": "object",
125
142
  "additionalProperties": false,
@@ -143,7 +160,7 @@
143
160
  "app": {
144
161
  "type": "object",
145
162
  "additionalProperties": false,
146
- "description": "The app under test in this environment, per platform (scenario platform: android, ios, electron, desktop). Paths of app scenarios are deep links.",
163
+ "description": "The app under test in this environment, per platform (scenario platform: android, ios, electron, desktop, windows). Paths of app scenarios are deep links.",
147
164
  "properties": {
148
165
  "android": {
149
166
  "type": "object", "additionalProperties": false,
@@ -155,6 +172,12 @@
155
172
  "serial": { "type": "string", "description": "adb device serial; default ANDROID_SERIAL or the only device." },
156
173
  "device": { "type": "string", "description": "Label for the canvas; default the device model." },
157
174
  "reset": { "type": "boolean", "description": "Clear app data at the start of every locale (a new user)." },
175
+ "connect": { "type": "string", "description": "A real device over Wi-Fi: host:port for adb connect before the walk (pair it once with adb pair)." },
176
+ "avd": { "type": "string", "description": "Emulator (Android Virtual Device) to start when it is not running, e.g. Pixel_8_API_34." },
177
+ "emulator": { "type": "string", "description": "Path to the emulator binary (default: emulator on PATH or in ANDROID_HOME)." },
178
+ "wake": { "type": "boolean", "description": "Wake and unlock the screen of a real device before the walk (no PIN; default true)." },
179
+ "webview": { "type": "boolean", "description": "When the screen is a WebView with debugging on, capture its page (DOM, web checks) too; default false." },
180
+ "devices": { "$ref": "#/$defs/appDevices" },
158
181
  "adb": { "type": "string" }, "launch_wait": { "type": "integer" }, "settle": { "type": "integer" }
159
182
  }
160
183
  },
@@ -170,6 +193,9 @@
170
193
  "idb": { "type": "boolean", "description": "Use idb for the UI tree and taps when installed (default true)." },
171
194
  "ocr": { "type": "boolean", "description": "Without idb, read the screen text by OCR (macOS Vision; default true)." },
172
195
  "status_bar": { "type": "boolean", "description": "Clean status bar (9:41, full battery) for captures; default true." },
196
+ "runtime": { "type": "string", "description": "Simulator runtime, e.g. iOS 18 (the newest matching one)." },
197
+ "real": { "type": "boolean", "description": "A real iPhone or iPad (udid): installs and launches with xcrun devicectl, screenshots with idevicescreenshot (libimobiledevice); no taps or UI tree, OCR on macOS." },
198
+ "devices": { "$ref": "#/$defs/appDevices" },
173
199
  "scale": { "type": "integer" }, "launch_wait": { "type": "integer" }, "settle": { "type": "integer" }
174
200
  }
175
201
  },
@@ -192,6 +218,20 @@
192
218
  "launch": { "type": "boolean", "description": "Open the app at the start (default true)." },
193
219
  "reset": { "type": "boolean" }, "ocr": { "type": "boolean" }, "launch_wait": { "type": "integer" }, "settle": { "type": "integer" }
194
220
  }
221
+ },
222
+ "windows": {
223
+ "type": "object", "additionalProperties": false,
224
+ "description": "A Windows desktop app, driven through UI Automation with Windows PowerShell (no extra install).",
225
+ "properties": {
226
+ "name": { "type": "string", "description": "Process name or window title, e.g. notepad or Contoso Shop." },
227
+ "path": { "type": "string", "description": "Executable to start (relative to the repository or absolute)." },
228
+ "aumid": { "type": "string", "description": "AppUserModelID of a Store/MSIX app to start instead of path." },
229
+ "args": { "type": "array", "items": { "type": "string" } },
230
+ "launch": { "type": "boolean", "description": "Start the app at the start (default true)." },
231
+ "reset": { "type": "boolean", "description": "Close a running instance first." },
232
+ "powershell": { "type": "string", "description": "PowerShell binary (default powershell.exe; pwsh works on Windows too)." },
233
+ "launch_wait": { "type": "integer" }, "settle": { "type": "integer" }
234
+ }
195
235
  }
196
236
  }
197
237
  },
@@ -147,7 +147,32 @@
147
147
  }
148
148
  },
149
149
  "viewports": {
150
- "type": "object"
150
+ "type": "object",
151
+ "description": "Capture key -> [width, height]. A key is the viewport name plus the dimension values that differ from the default, joined by dots (mobile, mobile.webkit, iphone-15.webkit.dark.reduce)."
152
+ },
153
+ "variants": {
154
+ "type": "object",
155
+ "description": "Capture key -> { viewport, browser, color_scheme?, reduced_motion?, forced_colors?, display_mode?, device? }. Absent in runs without dimensions: every key is then a plain viewport in Chromium.",
156
+ "additionalProperties": {
157
+ "type": "object",
158
+ "properties": {
159
+ "viewport": { "type": "string" },
160
+ "browser": { "enum": ["chromium", "webkit", "firefox"] },
161
+ "color_scheme": { "enum": ["light", "dark"] },
162
+ "reduced_motion": { "enum": ["no-preference", "reduce"] },
163
+ "forced_colors": { "enum": ["none", "active"] },
164
+ "display_mode": { "enum": ["browser", "standalone"] },
165
+ "device": { "type": "string" }
166
+ }
167
+ }
168
+ },
169
+ "dimensions": {
170
+ "type": "object",
171
+ "description": "Values of each dimension the run walked (browsers, color_schemes, reduced_motion, forced_colors, display_modes) for the canvas switchers."
172
+ },
173
+ "devices": {
174
+ "type": "object",
175
+ "description": "Viewport name -> device preset (title, kind, screen, size, dpr, safe areas, frame) for device frames on the canvas."
151
176
  },
152
177
  "note": {
153
178
  "type": "string"
@@ -405,7 +430,13 @@
405
430
  "url": {
406
431
  "type": ["string", "null"]
407
432
  },
408
- "platform": { "type": "string", "description": "App captures: android, ios, electron or desktop." },
433
+ "platform": { "type": "string", "description": "App captures: android, ios, electron, desktop or windows." },
434
+ "browser": { "type": "string", "description": "Engine of a web capture when the run has dimensions; absent = chromium." },
435
+ "color_scheme": { "type": "string" },
436
+ "reduced_motion": { "type": "string" },
437
+ "forced_colors": { "type": "string" },
438
+ "display_mode": { "type": "string" },
439
+ "device_preset": { "type": "string", "description": "Viewport name of the device preset the capture emulated." },
409
440
  "device": { "type": ["string", "null"] },
410
441
  "tree": { "type": ["string", "null"], "description": "UI tree of an app capture (qawalk.tree.v1)." },
411
442
  "http_status": {
@@ -50,17 +50,25 @@
50
50
  "minItems": 1
51
51
  },
52
52
  "viewports": {
53
- "type": "object",
54
- "additionalProperties": {
55
- "type": "array",
56
- "items": {
57
- "type": "integer",
58
- "minimum": 200
53
+ "description": "Viewports by name: a size [width, height], a device preset (iphone-15, ipad-landscape, fold-open, laptop, tv-1080p, <preset>-landscape …) or { device, width?, height?, full_screen? }. A list of preset names works too: [iphone-se, pixel-8, desktop-hd].",
54
+ "oneOf": [
55
+ {
56
+ "type": "object",
57
+ "additionalProperties": { "$ref": "#/$defs/viewport" }
59
58
  },
60
- "minItems": 2,
61
- "maxItems": 2
62
- }
59
+ {
60
+ "type": "array",
61
+ "minItems": 1,
62
+ "items": { "oneOf": [{ "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*$" }, { "type": "object", "minProperties": 1, "maxProperties": 1, "additionalProperties": { "$ref": "#/$defs/viewport" } }] }
63
+ }
64
+ ]
63
65
  },
66
+ "browsers": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["chromium", "webkit", "firefox"] }, "description": "Browser engines to walk the web flow in (Playwright). Default: chromium. Captures of other engines are keyed <viewport>.<engine>." },
67
+ "color_schemes": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["light", "dark"] }, "description": "prefers-color-scheme values to capture every step in (keys <viewport>.dark). Not set: the browser default, nothing emulated." },
68
+ "reduced_motion": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["no-preference", "reduce"] }, "description": "prefers-reduced-motion values (keys <viewport>.reduce)." },
69
+ "forced_colors": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["none", "active"] }, "description": "forced-colors (Windows high contrast) values (keys <viewport>.forced)." },
70
+ "display_modes": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["browser", "standalone"] }, "description": "PWA display modes: standalone captures the page as an installed app (display-mode media rules and matchMedia; the whole screen of a device preset)." },
71
+ "devices": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "type": "string", "pattern": "^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$" }, "description": "App scenarios: devices to walk on, e.g. pixel-8-api-34, iphone-15-ios-18 (emulators, simulators or real devices named in the environment's app.<platform>.devices). One capture key per device." },
64
72
  "environments": {
65
73
  "type": "array",
66
74
  "items": {
@@ -72,9 +80,9 @@
72
80
  "default": false
73
81
  },
74
82
  "platform": {
75
- "enum": ["web", "android", "ios", "electron", "desktop"],
83
+ "enum": ["web", "android", "ios", "electron", "desktop", "windows"],
76
84
  "default": "web",
77
- "description": "What the flow runs on. web: pages in Chrome. android/ios/electron/desktop: an app, driven by the environment's app block (adb, iOS Simulator, Playwright Electron, macOS window) or captured with kaloko capture --image. Node paths are deep links for apps."
85
+ "description": "What the flow runs on. web: pages in a browser (an installed PWA too: display_modes). android/ios/electron/desktop/windows: an app, driven by the environment's app block (adb, iOS Simulator or device, Playwright Electron, macOS window, Windows UI Automation) or captured with kaloko capture --image. Node paths are deep links for apps."
78
86
  },
79
87
  "tags": {
80
88
  "type": "array",
@@ -182,6 +190,23 @@
182
190
  }
183
191
  ]
184
192
  },
193
+ "viewport": {
194
+ "oneOf": [
195
+ { "type": "array", "items": { "type": "integer", "minimum": 200 }, "minItems": 2, "maxItems": 2 },
196
+ { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*$", "description": "A device preset name." },
197
+ {
198
+ "type": "object",
199
+ "additionalProperties": false,
200
+ "required": ["device"],
201
+ "properties": {
202
+ "device": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*$" },
203
+ "width": { "type": "integer", "minimum": 200 },
204
+ "height": { "type": "integer", "minimum": 200 },
205
+ "full_screen": { "type": "boolean", "description": "The whole screen without browser bars (app mockups, kiosk). Design environments use it by default." }
206
+ }
207
+ }
208
+ ]
209
+ },
185
210
  "slug": {
186
211
  "type": "string",
187
212
  "pattern": "^[A-Za-z0-9][A-Za-z0-9._-]*$"
@@ -511,12 +536,19 @@
511
536
  "count",
512
537
  "text_matches",
513
538
  "text_absent",
539
+ "text_equals",
514
540
  "attr_equals",
541
+ "attr_matches",
515
542
  "no_overflow",
516
543
  "no_console_errors",
517
544
  "title_matches",
518
545
  "meta_matches",
519
546
  "link_matches",
547
+ "url_matches",
548
+ "url_equals",
549
+ "style_equals",
550
+ "style_matches",
551
+ "js_equals",
520
552
  "http_status",
521
553
  "link_home",
522
554
  "links_ok",
@@ -550,22 +582,73 @@
550
582
  },
551
583
  "selector": {
552
584
  "type": "string",
553
- "description": "CSS selector; links_ok: the links to check (default a[href])."
585
+ "description": "CSS selector; links_ok: the links to check (default a[href]); style_*: the elements whose computed style is read."
554
586
  },
555
587
  "pattern": {
556
588
  "type": "string",
557
- "description": "Regex; links_ok: link paths to skip."
589
+ "description": "Regex; links_ok: link paths to skip; url_matches: tested on the URL part; style_matches: on the computed value; attr_matches: on the attribute; js_equals: on the result."
558
590
  },
559
591
  "attr": {
560
592
  "type": "string"
561
593
  },
594
+ "property": {
595
+ "type": "string",
596
+ "minLength": 1,
597
+ "description": "style_equals, style_matches: the CSS property read with getComputedStyle (color, background-color, display, font-size, …). Use longhands, not shorthands like background."
598
+ },
599
+ "token": {
600
+ "type": "string",
601
+ "pattern": "^[^{}]+$",
602
+ "description": "style_equals: a DTCG token name (color.brand.primary) instead of value; resolved from the project's token set, or from the page's --color-brand-primary custom property."
603
+ },
604
+ "tolerance": {
605
+ "type": "number",
606
+ "minimum": 0,
607
+ "description": "style_equals: allowed difference for lengths, in px."
608
+ },
609
+ "op": {
610
+ "enum": [
611
+ "<",
612
+ "<=",
613
+ ">",
614
+ ">=",
615
+ "=="
616
+ ],
617
+ "description": "style_equals, js_equals: compare numerically with value instead of equality (font-size >= 16px)."
618
+ },
619
+ "all": {
620
+ "type": "boolean",
621
+ "description": "style_*, text_equals, attr_matches: every match must pass (default: at least one)."
622
+ },
623
+ "part": {
624
+ "enum": [
625
+ "href",
626
+ "path",
627
+ "query",
628
+ "hash",
629
+ "host",
630
+ "origin"
631
+ ],
632
+ "description": "url_matches: which part of the URL the pattern tests (default href, the whole URL)."
633
+ },
634
+ "ignore_query": {
635
+ "type": "boolean",
636
+ "description": "url_equals: compare without the query string and hash."
637
+ },
638
+ "expression": {
639
+ "type": "string",
640
+ "minLength": 1,
641
+ "maxLength": 500,
642
+ "description": "js_equals: one read-only JavaScript expression evaluated in the page (document.querySelectorAll('.item').length, localStorage.getItem('cart'), window.dataLayer?.length). No assignments, no ++/--, only reading calls; 2 s timeout."
643
+ },
562
644
  "value": {
563
645
  "type": [
564
646
  "string",
565
647
  "number",
566
- "boolean"
648
+ "boolean",
649
+ "null"
567
650
  ],
568
- "description": "attr_equals: the attribute value; http_status: the exact status (e.g. 404). Without value, http_status accepts min..max (default 200..299)."
651
+ "description": "attr_equals: the attribute value; text_equals: the whole text (whitespace collapsed); url_equals: a path (/thanks, query included unless ignore_query) or a whole URL; style_equals: the expected CSS value (colours in any CSS form, lengths in px or rem); js_equals: the expected result, or the number to compare with op; http_status: the exact status (e.g. 404). Without value, http_status accepts min..max (default 200..299)."
569
652
  },
570
653
  "min": {
571
654
  "type": "integer",
@@ -591,7 +674,15 @@
591
674
  "pattern": "^[imsu]{0,4}$",
592
675
  "description": "Regex flags for pattern: i (ignore case), m, s. Page text is innerText, so CSS text-transform (uppercase buttons) applies."
593
676
  }
594
- }
677
+ },
678
+ "allOf": [
679
+ {"if": {"properties": {"assert": {"enum": ["style_equals", "style_matches"]}}, "required": ["assert"]}, "then": {"required": ["selector", "property"]}},
680
+ {"if": {"properties": {"assert": {"const": "style_equals"}}, "required": ["assert"]}, "then": {"anyOf": [{"required": ["value"]}, {"required": ["token"]}]}},
681
+ {"if": {"properties": {"assert": {"enum": ["style_matches", "url_matches", "attr_matches"]}}, "required": ["assert"]}, "then": {"required": ["pattern"]}},
682
+ {"if": {"properties": {"assert": {"const": "attr_matches"}}, "required": ["assert"]}, "then": {"required": ["selector", "attr"]}},
683
+ {"if": {"properties": {"assert": {"enum": ["url_equals", "text_equals"]}}, "required": ["assert"]}, "then": {"required": ["value"]}},
684
+ {"if": {"properties": {"assert": {"const": "js_equals"}}, "required": ["assert"]}, "then": {"required": ["expression"]}}
685
+ ]
595
686
  },
596
687
  "semantic": {
597
688
  "type": "object",
@@ -5,7 +5,9 @@ description: >
5
5
  evidence: writing or updating ACCEPTANCE.md or qa/scenario.yml, running a
6
6
  scenario on local, review, staging or production (readonly), capturing
7
7
  screens and e-mails, evaluating criteria, or building and sharing the
8
- Kaloko canvas, for web pages as well as Android, iOS and desktop apps,
8
+ Kaloko canvas, for web pages in Chromium, WebKit and Firefox, on phone,
9
+ tablet and desktop presets, in light and dark mode, as well as Android, iOS,
10
+ macOS and Windows apps,
9
11
  including checking many pages of one template at once (magazine articles,
10
12
  product pages, an SEO page catalogue). Also when someone asks what a
11
13
  reviewer rejected on a canvas, what is waiting in the Kaloko inbox, or how
@@ -50,7 +52,8 @@ Start the shared browser, walk a scenario on a non-production environment,
50
52
  create test data there (users, roles, carts), capture screens and e-mails,
51
53
  evaluate, build the preview, and fix scenario mechanics (selectors, scripts)
52
54
  that do not change what is being accepted, including applying a selector
53
- proposal with `kaloko walk --heal` when it names the same element. Walk only
55
+ proposal with `kaloko walk --heal` when it names the same element, and pulling
56
+ the regions people drew to ignore into the scenario (`kaloko stability --pull`). Walk only
54
57
  the steps a change reaches (`--affected`) and import screenshots into a run. Record steps locally
55
58
  (`walk --record`) when a screenshot cannot show what a person has to judge, read traces, and
56
59
  export a walkthrough video into the run folder. On a design environment: change
@@ -71,7 +74,8 @@ a whole project (`capture.record` in the config: storage), a real clean-up
71
74
  anything the environment profile blocks. Never trash an accepted run, a
72
75
  baseline, a run with review comments, or a run linked from an open PR or issue
73
76
  comment. Never approve a
74
- step or accept a run on a person's behalf; report what waits for them
77
+ step or accept a run on a person's behalf, and never hand out a signed
78
+ acceptance record (`kaloko signoff`) for a run a person has not accepted; report what waits for them
75
79
  (`kaloko review`, MCP `get_review_queue`) instead. Ask the person
76
80
  for secret values (passwords, tokens, gate credentials) and for sign-ins only
77
81
  they can do; never write secrets into the config, a scenario or a script.
@@ -82,9 +86,13 @@ they can do; never write secrets into the config, a scenario or a script.
82
86
  the sample run, the first flow from a URL or a task, share, point the person
83
87
  to the canvas): [reference/first-run.md](reference/first-run.md)
84
88
  - Writing or updating the plan and scenario: [reference/acceptance.md](reference/acceptance.md)
85
- - Android, iOS (Simulator), Electron and macOS apps, or screens captured by
89
+ - Android, iOS (Simulator or a real iPhone), Electron, macOS and Windows apps,
90
+ several devices of one app, WebViews inside apps, or screens captured by
86
91
  another tool (Maestro, Appium, XCUITest) with their UI tree:
87
92
  [reference/apps.md](reference/apps.md)
93
+ - Walking in WebKit and Firefox, on device presets (phones, foldables,
94
+ tablets, TV), in dark mode, with reduced motion or as an installed PWA:
95
+ [reference/devices.md](reference/devices.md)
88
96
  - Many pages of one template in one step (articles, products, an SEO
89
97
  catalogue from a sitemap or a listing), with an index on the canvas:
90
98
  [reference/stacks.md](reference/stacks.md)
@@ -182,7 +182,8 @@ returning visitor). The plan is the source of truth; this is only the default.
182
182
  Each criterion gets exactly one check:
183
183
 
184
184
  - `deterministic` for anything countable or exact: presence, counts, text,
185
- prices, attributes, overflow, console errors, title/meta/canonical.
185
+ prices, attributes, the URL, computed styles, values in the page,
186
+ overflow, console errors, title/meta/canonical.
186
187
  - `semantic` for judgement: clarity, separation of consents, whether copy
187
188
  fits the purpose. Write the question in English, yes = met, one condition
188
189
  per question. Add `true`/`false` for edge cases and `scope` (CSS selector)
@@ -216,6 +217,74 @@ button styled uppercase reads `VYTVOŘIT`. Add `flags: i` to ignore case
216
217
  white-label check for `vyvolej` also matches the Czech verb "Vyvolej si";
217
218
  anchor it (`vyvolej\.to`).
218
219
 
220
+ `text_equals` compares the whole text of an element (whitespace collapsed,
221
+ `flags: i` ignores case) with `value`: a total, a status, a label.
222
+ `attr_matches` runs `pattern` on an attribute (`selector`, `attr`). Both pass
223
+ when one match passes; `all: true` asks every match.
224
+
225
+ ### URL, computed styles and values in the page
226
+
227
+ Never make a step script compute a check and leave a `data-qa-*` marker for
228
+ a criterion to find. The marker proves only that the script ran. Check the
229
+ URL, the style or the value directly:
230
+
231
+ ```yaml
232
+ # the URL after the step (part: href (default), path, query, hash, host, origin)
233
+ - { id: AC1, text: Lands on the thank-you page, check: { type: deterministic, assert: url_matches, part: path, pattern: '^/(cs|en)/thanks$' } }
234
+ # exact path; ignore_query drops ?… and #…; a value with https:// compares the whole URL
235
+ - { id: AC2, text: Stays on the cart, check: { type: deterministic, assert: url_equals, value: /cart, ignore_query: true } }
236
+ # computed CSS (getComputedStyle) of the matched elements
237
+ - { id: AC3, text: Primary button in brand colour, check: { type: deterministic, assert: style_equals, selector: button.primary, property: background-color, token: color.brand.primary, all: true } }
238
+ - { id: AC4, text: Body text at least 16 px, check: { type: deterministic, assert: style_equals, selector: main p, property: font-size, op: '>=', value: 16px, all: true } }
239
+ - { id: AC5, text: Error text red, check: { type: deterministic, assert: style_equals, selector: .field-error, property: color, value: '#d92d20' } }
240
+ - { id: AC6, text: Promo banner hidden, check: { type: deterministic, assert: style_matches, selector: .promo, property: display, pattern: '^none$' } }
241
+ # a read-only expression in the page
242
+ - { id: AC7, text: Three items in the cart, check: { type: deterministic, assert: js_equals, expression: "document.querySelectorAll('.cart-item').length", value: 3 } }
243
+ - { id: AC8, text: Cart kept for the next visit, check: { type: deterministic, assert: js_equals, expression: "JSON.parse(localStorage.getItem('cart') || '[]').length", op: '>=', value: 1 } }
244
+ - { id: AC9, text: Purchase sent to analytics, check: { type: deterministic, assert: js_equals, expression: "window.dataLayer?.some((e) => e.event === 'purchase')", value: true } }
245
+ ```
246
+
247
+ - `url_matches` and `url_equals` read the URL the browser shows after the
248
+ step, redirects included. The evidence names the URL.
249
+ - `style_equals` compares `property` (a longhand: `background-color`, not
250
+ `background`) with `value` or `token`. Colours compare in one form, so
251
+ `red`, `#ff0000` and `rgb(255, 0, 0)` are equal; transparency counts.
252
+ Lengths compare in px (`rem` uses the page's root size), with
253
+ `tolerance: 1` for rounding; `op: '<' | '<=' | '>' | '>=' | '=='` compares
254
+ numbers. Other values (`display`, `font-weight`, `font-family`) compare as
255
+ text without quotes or case. Visible matches count; when none is visible
256
+ (a check for `display: none`), every match does. `style_matches` runs
257
+ `pattern` on the computed value.
258
+ - `token: color.brand.primary` takes the expected value from the project's
259
+ DTCG token set (`kaloko tokens push`, or `tokens.json` of the design
260
+ folder) when `kaloko evaluate` runs, or from the page's
261
+ `--color-brand-primary` custom property during the walk. The evidence
262
+ reads `color is rgb(46, 107, 255) = color.brand.primary (#2e6bff)`.
263
+ - `:hover` and `:focus` states are not supported: the check reads the page
264
+ as the step left it. Hover or focus the element in the step script when
265
+ that state is what you accept, and say so in the criterion text.
266
+ - `js_equals` evaluates one `expression` in the page (2 s limit; promises are
267
+ awaited) and compares the result with `value` (a number value accepts
268
+ `"3"`; `null` matches a missing `localStorage` key), with `op` and a
269
+ number, or with `pattern` (objects as JSON). Without `value` and
270
+ `pattern` the result must be truthy. Only reading is allowed: no
271
+ assignment, no `++`/`--`, no `new` except `new URL(…)`, and calls only
272
+ from a list of reading methods (`querySelector(All)`, `getAttribute`,
273
+ `getItem`, `getComputedStyle`, `filter`, `some`, `JSON.parse`, …).
274
+ `kaloko validate` refuses anything else (`setItem`, `click`, `fetch`).
275
+ The guard is a convention, not a sandbox: a getter in the page can still
276
+ run code, so read data the page exposes (storage, `dataLayer`, the DOM)
277
+ and nothing that acts. Prefer `style_*`, `text_*` and `count` when they
278
+ can say the same.
279
+
280
+ On `kaloko evaluate --recheck` the URL checks run again from the stored URL.
281
+ The computed styles and expression results are stored with the capture, so a
282
+ recheck decides again from them when the criterion still reads the same
283
+ selector and property (or the same expression), for example after you fix
284
+ the expected value. A new selector, property or expression needs the live
285
+ page: the criterion is "not evaluated" with that reason until the step is
286
+ captured again.
287
+
219
288
  Two checks need the network and run next to the capture, once per page:
220
289
 
221
290
  - `links_ok`: every same-origin link (`selector`, default `a[href]`) answers
@@ -273,5 +342,7 @@ design pixel check (`AC801`) leave them out:
273
342
 
274
343
  Do not guess them: `kaloko walk --stability 3` notes what moves while the
275
344
  page sits still, and `kaloko stability` compares the step with earlier runs
276
- and writes suggestions (see [execution.md](execution.md)). Ignore only noise;
277
- a region that matters to a criterion stays in.
345
+ and writes suggestions (see [execution.md](execution.md)). Reviewers can also
346
+ draw a region on a screenshot on kaloko.app; `kaloko stability --pull` writes
347
+ those into `ignore:`. Ignore only noise; a region that matters to a criterion
348
+ stays in.