kaloko 0.4.10 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +64 -4
- package/package.json +1 -1
- package/schemas/config.v1.json +70 -2
- package/schemas/run.v1.json +33 -2
- package/schemas/scenario.v1.json +130 -17
- package/skills/kaloko/SKILL.md +18 -5
- package/skills/kaloko/reference/acceptance.md +70 -1
- package/skills/kaloko/reference/apps.md +103 -8
- package/skills/kaloko/reference/ci.md +3 -1
- package/skills/kaloko/reference/design.md +39 -0
- package/skills/kaloko/reference/devices.md +109 -0
- package/skills/kaloko/reference/environments.md +6 -0
- package/skills/kaloko/reference/execution.md +6 -0
- package/skills/kaloko/reference/mcp.md +1 -0
- package/skills/kaloko/reference/recording.md +95 -0
- package/src/cli.mjs +381 -167
- package/src/lib/access.mjs +5 -2
- package/src/lib/anim.mjs +125 -0
- package/src/lib/apps/devices.mjs +187 -0
- package/src/lib/apps/drivers.mjs +11 -3
- package/src/lib/apps/ios-device.mjs +78 -0
- package/src/lib/apps/recording-native.mjs +212 -0
- package/src/lib/apps/tree.mjs +9 -1
- package/src/lib/apps/webview.mjs +54 -0
- package/src/lib/apps/windows.mjs +108 -0
- package/src/lib/apps/windows.ps1 +78 -0
- package/src/lib/assertions.mjs +300 -0
- package/src/lib/build.mjs +63 -4
- package/src/lib/capture.mjs +146 -119
- package/src/lib/ci.mjs +8 -4
- package/src/lib/design-cli.mjs +35 -6
- package/src/lib/design.mjs +3 -1
- package/src/lib/devices.mjs +85 -0
- package/src/lib/devspec.mjs +3 -2
- package/src/lib/dimensions.mjs +326 -0
- package/src/lib/doctor.mjs +35 -6
- package/src/lib/drift.mjs +8 -4
- package/src/lib/evaluate.mjs +25 -7
- package/src/lib/import.mjs +3 -2
- package/src/lib/jev.mjs +3 -1
- package/src/lib/manifest-core.mjs +12 -3
- package/src/lib/probe.mjs +60 -0
- package/src/lib/recheck.mjs +6 -1
- package/src/lib/recording.mjs +373 -0
- package/src/lib/scenario.mjs +3 -0
- package/src/lib/stability-core.mjs +4 -2
- package/src/lib/trace.mjs +109 -0
- package/src/lib/video-cli.mjs +209 -0
- package/src/lib/video.mjs +427 -0
- package/src/lib/walk-dimensions.mjs +83 -0
- package/src/lib/webm.mjs +144 -0
- package/viewer/index.html +467 -25
package/README.md
CHANGED
|
@@ -11,10 +11,11 @@ there, and the newest captures double as documentation.
|
|
|
11
11
|
- Product and guides: <https://kaloko.app>
|
|
12
12
|
- Service (organizations, shared canvases, reviews): <https://kaloko.app>
|
|
13
13
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
14
|
+
Pages are walked in Chromium, WebKit and Firefox, on phone, tablet and
|
|
15
|
+
desktop presets, in light and dark mode. Native apps too: Android, iOS,
|
|
16
|
+
Electron, macOS and Windows. Screens taken by any other tool (Maestro, Appium,
|
|
17
|
+
XCUITest) are imported with their UI tree. See
|
|
18
|
+
[What Kaloko can walk and design](#what-kaloko-can-walk-and-design).
|
|
18
19
|
|
|
19
20
|
## Install
|
|
20
21
|
|
|
@@ -71,8 +72,48 @@ once with `--concurrency 3`. When a step's selector no longer exists, the walk
|
|
|
71
72
|
proposes one from the page and prints the patch; `--heal` applies it once the
|
|
72
73
|
step passes.
|
|
73
74
|
|
|
75
|
+
Automatic criteria read the page itself: text, counts and attributes, the URL
|
|
76
|
+
after a step (`url_matches`), computed styles, also against a design token
|
|
77
|
+
(`style_equals` with `token: color.brand.primary`), and read-only values such
|
|
78
|
+
as `localStorage` or `dataLayer` (`js_equals`). A step script never has to
|
|
79
|
+
compute a result and leave a marker for a criterion to find. The acceptance
|
|
80
|
+
reference of the `kaloko` skill lists every check.
|
|
81
|
+
|
|
74
82
|
`npx kaloko help` lists every command.
|
|
75
83
|
|
|
84
|
+
## What Kaloko can walk and design
|
|
85
|
+
|
|
86
|
+
*Full*: Kaloko drives it, captures it and runs the checks. *Capture only*: Kaloko
|
|
87
|
+
takes the screens, your tool or a person moves between them. *Via import*: another
|
|
88
|
+
tool takes the screens and `kaloko capture --image` or `kaloko import` lays them out.
|
|
89
|
+
|
|
90
|
+
| Target | How | Checks | Design | Support |
|
|
91
|
+
|---|---|---|---|---|
|
|
92
|
+
| Web in Chromium | Playwright, the shared browser or an own one | all | live HTML steps | full |
|
|
93
|
+
| Web in WebKit (Safari's engine) and Firefox | Playwright, one pass per engine | all except the accessibility tree, focus rings and layout shifts (and the Tab walk on WebKit) | rendered in Chromium | full |
|
|
94
|
+
| Phones, foldables, tablets, laptop to 2560 px, TV | device presets: size, pixel ratio, touch, user agent | all | whole screen, device frame, safe areas | full |
|
|
95
|
+
| Light and dark, reduced motion, high contrast | emulated per capture | all; contrast per scheme | dark pages, token modes per scheme | full |
|
|
96
|
+
| Installed web app (PWA) | `display-mode: standalone` rules and `matchMedia` | all | yes | full for CSS; scripts that read the mode at load need a real install |
|
|
97
|
+
| Android: emulator, USB or Wi-Fi phone | adb: deep links, taps, typing, dark mode | UI tree, app pack | HTML mockups | full |
|
|
98
|
+
| Android WebView | DevTools over adb (debuggable builds) | web checks on the page | | full |
|
|
99
|
+
| iOS Simulator | simctl: deep links, dark mode; taps with idb | UI tree with idb, text by OCR without | HTML mockups | full with idb, capture only without |
|
|
100
|
+
| iPhone and iPad | devicectl, idevicescreenshot or pymobiledevice3 | text by OCR | HTML mockups | capture only |
|
|
101
|
+
| Electron | Playwright | all web checks | | full |
|
|
102
|
+
| macOS apps | window screenshot, accessibility tree | UI tree | | capture only |
|
|
103
|
+
| Windows apps | PowerShell and UI Automation: taps, typing | UI tree | | full on Windows, via import elsewhere |
|
|
104
|
+
| React Native, Flutter, iOS WKWebView, Maestro, Appium, XCUITest | their own runner | UI tree from a dump, or text | | via import |
|
|
105
|
+
|
|
106
|
+
```bash
|
|
107
|
+
npx kaloko devices # the presets
|
|
108
|
+
npx kaloko browser install --browsers webkit,firefox
|
|
109
|
+
npx kaloko start --scenario qa/flows/checkout.yml --env staging \
|
|
110
|
+
--browsers chromium,webkit --color-schemes light,dark --viewports iphone-se,pixel-8,desktop-hd
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
The canvas switches between browsers and schemes next to the locale, draws the device frame and shows the same
|
|
114
|
+
step in another browser beside it. `npx kaloko doctor` says which engines and app tools are missing on this
|
|
115
|
+
machine and how to install them. Details: `skills/kaloko/reference/devices.md` and `apps.md`.
|
|
116
|
+
|
|
76
117
|
## Pull request previews in CI
|
|
77
118
|
|
|
78
119
|
```bash
|
|
@@ -113,6 +154,24 @@ jobs:
|
|
|
113
154
|
Uploads go by content hash: captures that did not change since an earlier run
|
|
114
155
|
are not sent again, and an interrupted `share` picks up where it stopped.
|
|
115
156
|
|
|
157
|
+
## When a screenshot is not enough
|
|
158
|
+
|
|
159
|
+
```bash
|
|
160
|
+
npx kaloko walk --record # each step on video, from the previous screen to its capture
|
|
161
|
+
npx kaloko walk --record failed # only the steps that fail (flaky ones, mostly)
|
|
162
|
+
npx kaloko walk --record flow # the whole walk per language, a chapter per step
|
|
163
|
+
npx kaloko video export --format mp4 # the run as one video, a title card and verdict before each step
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
The canvas plays a step's video next to its screenshot, and the Technical tab
|
|
167
|
+
shows its trace: clicks, typing, requests with status and timing, console
|
|
168
|
+
messages. Click a row and the video jumps there. Web recordings need nothing
|
|
169
|
+
but Chrome (WebM with VP9); with ffmpeg installed you also get MP4 for Safari,
|
|
170
|
+
app recordings (Android `screenrecord`, iOS Simulator, macOS) and the
|
|
171
|
+
walkthrough export. `npx kaloko video tools` says what this machine has.
|
|
172
|
+
Recordings stay in the local preview on Free and Team; Business and Enterprise
|
|
173
|
+
share them with the run.
|
|
174
|
+
|
|
116
175
|
## Screenshots you already have
|
|
117
176
|
|
|
118
177
|
```bash
|
|
@@ -193,6 +252,7 @@ Nothing in Kaloko is stored in a format only Kaloko can read. Every export opens
|
|
|
193
252
|
| Live files (handoff) | `npx kaloko export --live [--version n] --out design/` | the design folder as written |
|
|
194
253
|
| Shared run | `npx kaloko export --run <id>` | zip with `run.json` (`qawalk.run.v1`), screenshots and HTML snapshots, live HTML of design steps, comments and verdicts, and `index.html`: the canvas, offline |
|
|
195
254
|
| Documentation captures | `npx kaloko export --scenario <id>` | newest image and HTML of every step, `index.json` |
|
|
255
|
+
| Walkthrough video | `npx kaloko video export [--format mp4\|webm\|gif]` | one video of the run's recordings with title cards; step recordings are WebM (VP9) and MP4 (H.264), traces JSON (`qawalk.trace.v1`) |
|
|
196
256
|
|
|
197
257
|
The schemas of these formats ship with the package (`schemas/`).
|
|
198
258
|
|
package/package.json
CHANGED
package/schemas/config.v1.json
CHANGED
|
@@ -78,7 +78,21 @@
|
|
|
78
78
|
"type": "object",
|
|
79
79
|
"additionalProperties": false,
|
|
80
80
|
"properties": {
|
|
81
|
-
"concurrency": { "type": "integer", "minimum": 1, "maximum": 16, "description": "Locales walked at the same time, each in its own browser context (kaloko walk --concurrency). Default: by CPU in an own browser, one at a time on the shared browser." }
|
|
81
|
+
"concurrency": { "type": "integer", "minimum": 1, "maximum": 16, "description": "Locales walked at the same time, each in its own browser context (kaloko walk --concurrency). Default: by CPU in an own browser, one at a time on the shared browser." },
|
|
82
|
+
"browsers": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["chromium", "webkit", "firefox"] }, "description": "Default browser engines for web runs when the scenario and kaloko start --browsers say nothing." },
|
|
83
|
+
"color_schemes": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["light", "dark"] }, "description": "Default colour schemes (prefers-color-scheme) for every run." },
|
|
84
|
+
"reduced_motion": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["no-preference", "reduce"] } },
|
|
85
|
+
"forced_colors": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["none", "active"] } },
|
|
86
|
+
"display_modes": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["browser", "standalone"] } }
|
|
87
|
+
}
|
|
88
|
+
},
|
|
89
|
+
"capture": {
|
|
90
|
+
"type": "object",
|
|
91
|
+
"additionalProperties": false,
|
|
92
|
+
"description": "Defaults for every scenario of the project; a scenario's capture block wins.",
|
|
93
|
+
"properties": {
|
|
94
|
+
"record": { "$ref": "#/$defs/record" },
|
|
95
|
+
"trace": { "type": "boolean", "default": true, "description": "Keep a trace per walked step (actions, network requests, console) shown under Technical; false or walk --no-trace turns it off." }
|
|
82
96
|
}
|
|
83
97
|
},
|
|
84
98
|
"share": {
|
|
@@ -92,6 +106,37 @@
|
|
|
92
106
|
}
|
|
93
107
|
},
|
|
94
108
|
"$defs": {
|
|
109
|
+
"record": {
|
|
110
|
+
"description": "Recordings of walked steps (opt-in): true or step records each step from the previous screen to its capture; flow records the whole walk per locale with a chapter per step; failed keeps only steps that fail; false or off turns it off. Uploaded on Business and Enterprise.",
|
|
111
|
+
"oneOf": [
|
|
112
|
+
{ "type": "boolean" },
|
|
113
|
+
{ "enum": ["step", "flow", "failed", "off"] },
|
|
114
|
+
{
|
|
115
|
+
"type": "object",
|
|
116
|
+
"additionalProperties": false,
|
|
117
|
+
"properties": {
|
|
118
|
+
"mode": { "enum": ["step", "flow", "failed", "off"], "default": "step" },
|
|
119
|
+
"quality": { "enum": ["low", "medium", "high"], "default": "medium", "description": "low 8 fps up to 960 px, medium 12 fps up to 1280 px, high 24 fps up to 1920 px" },
|
|
120
|
+
"max_seconds": { "type": "integer", "minimum": 1, "maximum": 3600, "description": "Longest kept recording; a longer one keeps its end (default 60 for a step, 900 for a flow)" },
|
|
121
|
+
"codec": { "enum": ["auto", "vp9", "av1", "h264"], "default": "auto", "description": "auto: WebM VP9 (VP8 fallback) from the browser, plus MP4 H.264 when ffmpeg is installed; av1 needs ffmpeg" },
|
|
122
|
+
"mp4": { "oneOf": [{ "type": "boolean" }, { "const": "auto" }], "default": "auto", "description": "MP4 H.264 next to the WebM (Safari before iOS 17.4); needs ffmpeg" },
|
|
123
|
+
"cursor": { "type": "boolean", "default": true, "description": "Draw the pointer and clicks into web recordings" }
|
|
124
|
+
}
|
|
125
|
+
}
|
|
126
|
+
]
|
|
127
|
+
},
|
|
128
|
+
"appDevices": {
|
|
129
|
+
"type": "object",
|
|
130
|
+
"description": "Devices a scenario's devices: list names, each an emulator, simulator or real device. Without an entry the name is matched against adb serials and AVDs (pixel-8-api-34 → Pixel_8_API_34) or simulators (iphone-15-ios-18 → iPhone 15 on iOS 18).",
|
|
131
|
+
"additionalProperties": {
|
|
132
|
+
"type": "object", "additionalProperties": false,
|
|
133
|
+
"properties": {
|
|
134
|
+
"serial": { "type": "string" }, "avd": { "type": "string" }, "connect": { "type": "string" },
|
|
135
|
+
"udid": { "type": "string" }, "device": { "type": "string", "description": "Simulator name (iOS) or the canvas label (Android)." }, "runtime": { "type": "string" },
|
|
136
|
+
"real": { "type": "boolean" }, "title": { "type": "string", "description": "Label on the canvas." }
|
|
137
|
+
}
|
|
138
|
+
}
|
|
139
|
+
},
|
|
95
140
|
"environment": {
|
|
96
141
|
"type": "object",
|
|
97
142
|
"additionalProperties": false,
|
|
@@ -115,7 +160,7 @@
|
|
|
115
160
|
"app": {
|
|
116
161
|
"type": "object",
|
|
117
162
|
"additionalProperties": false,
|
|
118
|
-
"description": "The app under test in this environment, per platform (scenario platform: android, ios, electron, desktop). Paths of app scenarios are deep links.",
|
|
163
|
+
"description": "The app under test in this environment, per platform (scenario platform: android, ios, electron, desktop, windows). Paths of app scenarios are deep links.",
|
|
119
164
|
"properties": {
|
|
120
165
|
"android": {
|
|
121
166
|
"type": "object", "additionalProperties": false,
|
|
@@ -127,6 +172,12 @@
|
|
|
127
172
|
"serial": { "type": "string", "description": "adb device serial; default ANDROID_SERIAL or the only device." },
|
|
128
173
|
"device": { "type": "string", "description": "Label for the canvas; default the device model." },
|
|
129
174
|
"reset": { "type": "boolean", "description": "Clear app data at the start of every locale (a new user)." },
|
|
175
|
+
"connect": { "type": "string", "description": "A real device over Wi-Fi: host:port for adb connect before the walk (pair it once with adb pair)." },
|
|
176
|
+
"avd": { "type": "string", "description": "Emulator (Android Virtual Device) to start when it is not running, e.g. Pixel_8_API_34." },
|
|
177
|
+
"emulator": { "type": "string", "description": "Path to the emulator binary (default: emulator on PATH or in ANDROID_HOME)." },
|
|
178
|
+
"wake": { "type": "boolean", "description": "Wake and unlock the screen of a real device before the walk (no PIN; default true)." },
|
|
179
|
+
"webview": { "type": "boolean", "description": "When the screen is a WebView with debugging on, capture its page (DOM, web checks) too; default false." },
|
|
180
|
+
"devices": { "$ref": "#/$defs/appDevices" },
|
|
130
181
|
"adb": { "type": "string" }, "launch_wait": { "type": "integer" }, "settle": { "type": "integer" }
|
|
131
182
|
}
|
|
132
183
|
},
|
|
@@ -142,6 +193,9 @@
|
|
|
142
193
|
"idb": { "type": "boolean", "description": "Use idb for the UI tree and taps when installed (default true)." },
|
|
143
194
|
"ocr": { "type": "boolean", "description": "Without idb, read the screen text by OCR (macOS Vision; default true)." },
|
|
144
195
|
"status_bar": { "type": "boolean", "description": "Clean status bar (9:41, full battery) for captures; default true." },
|
|
196
|
+
"runtime": { "type": "string", "description": "Simulator runtime, e.g. iOS 18 (the newest matching one)." },
|
|
197
|
+
"real": { "type": "boolean", "description": "A real iPhone or iPad (udid): installs and launches with xcrun devicectl, screenshots with idevicescreenshot (libimobiledevice); no taps or UI tree, OCR on macOS." },
|
|
198
|
+
"devices": { "$ref": "#/$defs/appDevices" },
|
|
145
199
|
"scale": { "type": "integer" }, "launch_wait": { "type": "integer" }, "settle": { "type": "integer" }
|
|
146
200
|
}
|
|
147
201
|
},
|
|
@@ -164,6 +218,20 @@
|
|
|
164
218
|
"launch": { "type": "boolean", "description": "Open the app at the start (default true)." },
|
|
165
219
|
"reset": { "type": "boolean" }, "ocr": { "type": "boolean" }, "launch_wait": { "type": "integer" }, "settle": { "type": "integer" }
|
|
166
220
|
}
|
|
221
|
+
},
|
|
222
|
+
"windows": {
|
|
223
|
+
"type": "object", "additionalProperties": false,
|
|
224
|
+
"description": "A Windows desktop app, driven through UI Automation with Windows PowerShell (no extra install).",
|
|
225
|
+
"properties": {
|
|
226
|
+
"name": { "type": "string", "description": "Process name or window title, e.g. notepad or Contoso Shop." },
|
|
227
|
+
"path": { "type": "string", "description": "Executable to start (relative to the repository or absolute)." },
|
|
228
|
+
"aumid": { "type": "string", "description": "AppUserModelID of a Store/MSIX app to start instead of path." },
|
|
229
|
+
"args": { "type": "array", "items": { "type": "string" } },
|
|
230
|
+
"launch": { "type": "boolean", "description": "Start the app at the start (default true)." },
|
|
231
|
+
"reset": { "type": "boolean", "description": "Close a running instance first." },
|
|
232
|
+
"powershell": { "type": "string", "description": "PowerShell binary (default powershell.exe; pwsh works on Windows too)." },
|
|
233
|
+
"launch_wait": { "type": "integer" }, "settle": { "type": "integer" }
|
|
234
|
+
}
|
|
167
235
|
}
|
|
168
236
|
}
|
|
169
237
|
},
|
package/schemas/run.v1.json
CHANGED
|
@@ -147,7 +147,32 @@
|
|
|
147
147
|
}
|
|
148
148
|
},
|
|
149
149
|
"viewports": {
|
|
150
|
-
"type": "object"
|
|
150
|
+
"type": "object",
|
|
151
|
+
"description": "Capture key -> [width, height]. A key is the viewport name plus the dimension values that differ from the default, joined by dots (mobile, mobile.webkit, iphone-15.webkit.dark.reduce)."
|
|
152
|
+
},
|
|
153
|
+
"variants": {
|
|
154
|
+
"type": "object",
|
|
155
|
+
"description": "Capture key -> { viewport, browser, color_scheme?, reduced_motion?, forced_colors?, display_mode?, device? }. Absent in runs without dimensions: every key is then a plain viewport in Chromium.",
|
|
156
|
+
"additionalProperties": {
|
|
157
|
+
"type": "object",
|
|
158
|
+
"properties": {
|
|
159
|
+
"viewport": { "type": "string" },
|
|
160
|
+
"browser": { "enum": ["chromium", "webkit", "firefox"] },
|
|
161
|
+
"color_scheme": { "enum": ["light", "dark"] },
|
|
162
|
+
"reduced_motion": { "enum": ["no-preference", "reduce"] },
|
|
163
|
+
"forced_colors": { "enum": ["none", "active"] },
|
|
164
|
+
"display_mode": { "enum": ["browser", "standalone"] },
|
|
165
|
+
"device": { "type": "string" }
|
|
166
|
+
}
|
|
167
|
+
}
|
|
168
|
+
},
|
|
169
|
+
"dimensions": {
|
|
170
|
+
"type": "object",
|
|
171
|
+
"description": "Values of each dimension the run walked (browsers, color_schemes, reduced_motion, forced_colors, display_modes) for the canvas switchers."
|
|
172
|
+
},
|
|
173
|
+
"devices": {
|
|
174
|
+
"type": "object",
|
|
175
|
+
"description": "Viewport name -> device preset (title, kind, screen, size, dpr, safe areas, frame) for device frames on the canvas."
|
|
151
176
|
},
|
|
152
177
|
"note": {
|
|
153
178
|
"type": "string"
|
|
@@ -405,7 +430,13 @@
|
|
|
405
430
|
"url": {
|
|
406
431
|
"type": ["string", "null"]
|
|
407
432
|
},
|
|
408
|
-
"platform": { "type": "string", "description": "App captures: android, ios, electron or
|
|
433
|
+
"platform": { "type": "string", "description": "App captures: android, ios, electron, desktop or windows." },
|
|
434
|
+
"browser": { "type": "string", "description": "Engine of a web capture when the run has dimensions; absent = chromium." },
|
|
435
|
+
"color_scheme": { "type": "string" },
|
|
436
|
+
"reduced_motion": { "type": "string" },
|
|
437
|
+
"forced_colors": { "type": "string" },
|
|
438
|
+
"display_mode": { "type": "string" },
|
|
439
|
+
"device_preset": { "type": "string", "description": "Viewport name of the device preset the capture emulated." },
|
|
409
440
|
"device": { "type": ["string", "null"] },
|
|
410
441
|
"tree": { "type": ["string", "null"], "description": "UI tree of an app capture (qawalk.tree.v1)." },
|
|
411
442
|
"http_status": {
|
package/schemas/scenario.v1.json
CHANGED
|
@@ -50,17 +50,25 @@
|
|
|
50
50
|
"minItems": 1
|
|
51
51
|
},
|
|
52
52
|
"viewports": {
|
|
53
|
-
"
|
|
54
|
-
"
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
"
|
|
58
|
-
"minimum": 200
|
|
53
|
+
"description": "Viewports by name: a size [width, height], a device preset (iphone-15, ipad-landscape, fold-open, laptop, tv-1080p, <preset>-landscape …) or { device, width?, height?, full_screen? }. A list of preset names works too: [iphone-se, pixel-8, desktop-hd].",
|
|
54
|
+
"oneOf": [
|
|
55
|
+
{
|
|
56
|
+
"type": "object",
|
|
57
|
+
"additionalProperties": { "$ref": "#/$defs/viewport" }
|
|
59
58
|
},
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
59
|
+
{
|
|
60
|
+
"type": "array",
|
|
61
|
+
"minItems": 1,
|
|
62
|
+
"items": { "oneOf": [{ "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*$" }, { "type": "object", "minProperties": 1, "maxProperties": 1, "additionalProperties": { "$ref": "#/$defs/viewport" } }] }
|
|
63
|
+
}
|
|
64
|
+
]
|
|
63
65
|
},
|
|
66
|
+
"browsers": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["chromium", "webkit", "firefox"] }, "description": "Browser engines to walk the web flow in (Playwright). Default: chromium. Captures of other engines are keyed <viewport>.<engine>." },
|
|
67
|
+
"color_schemes": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["light", "dark"] }, "description": "prefers-color-scheme values to capture every step in (keys <viewport>.dark). Not set: the browser default, nothing emulated." },
|
|
68
|
+
"reduced_motion": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["no-preference", "reduce"] }, "description": "prefers-reduced-motion values (keys <viewport>.reduce)." },
|
|
69
|
+
"forced_colors": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["none", "active"] }, "description": "forced-colors (Windows high contrast) values (keys <viewport>.forced)." },
|
|
70
|
+
"display_modes": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["browser", "standalone"] }, "description": "PWA display modes: standalone captures the page as an installed app (display-mode media rules and matchMedia; the whole screen of a device preset)." },
|
|
71
|
+
"devices": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "type": "string", "pattern": "^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$" }, "description": "App scenarios: devices to walk on, e.g. pixel-8-api-34, iphone-15-ios-18 (emulators, simulators or real devices named in the environment's app.<platform>.devices). One capture key per device." },
|
|
64
72
|
"environments": {
|
|
65
73
|
"type": "array",
|
|
66
74
|
"items": {
|
|
@@ -72,9 +80,9 @@
|
|
|
72
80
|
"default": false
|
|
73
81
|
},
|
|
74
82
|
"platform": {
|
|
75
|
-
"enum": ["web", "android", "ios", "electron", "desktop"],
|
|
83
|
+
"enum": ["web", "android", "ios", "electron", "desktop", "windows"],
|
|
76
84
|
"default": "web",
|
|
77
|
-
"description": "What the flow runs on. web: pages in
|
|
85
|
+
"description": "What the flow runs on. web: pages in a browser (an installed PWA too: display_modes). android/ios/electron/desktop/windows: an app, driven by the environment's app block (adb, iOS Simulator or device, Playwright Electron, macOS window, Windows UI Automation) or captured with kaloko capture --image. Node paths are deep links for apps."
|
|
78
86
|
},
|
|
79
87
|
"tags": {
|
|
80
88
|
"type": "array",
|
|
@@ -163,6 +171,42 @@
|
|
|
163
171
|
}
|
|
164
172
|
},
|
|
165
173
|
"$defs": {
|
|
174
|
+
"record": {
|
|
175
|
+
"description": "Recordings of walked steps (opt-in): true or step records each step from the previous screen to its capture; flow records the whole walk per locale with a chapter per step; failed keeps only steps that fail; false or off turns it off. Uploaded on Business and Enterprise.",
|
|
176
|
+
"oneOf": [
|
|
177
|
+
{ "type": "boolean" },
|
|
178
|
+
{ "enum": ["step", "flow", "failed", "off"] },
|
|
179
|
+
{
|
|
180
|
+
"type": "object",
|
|
181
|
+
"additionalProperties": false,
|
|
182
|
+
"properties": {
|
|
183
|
+
"mode": { "enum": ["step", "flow", "failed", "off"], "default": "step" },
|
|
184
|
+
"quality": { "enum": ["low", "medium", "high"], "default": "medium", "description": "low 8 fps up to 960 px, medium 12 fps up to 1280 px, high 24 fps up to 1920 px" },
|
|
185
|
+
"max_seconds": { "type": "integer", "minimum": 1, "maximum": 3600, "description": "Longest kept recording; a longer one keeps its end (default 60 for a step, 900 for a flow)" },
|
|
186
|
+
"codec": { "enum": ["auto", "vp9", "av1", "h264"], "default": "auto", "description": "auto: WebM VP9 (VP8 fallback) from the browser, plus MP4 H.264 when ffmpeg is installed; av1 needs ffmpeg" },
|
|
187
|
+
"mp4": { "oneOf": [{ "type": "boolean" }, { "const": "auto" }], "default": "auto", "description": "MP4 H.264 next to the WebM (Safari before iOS 17.4); needs ffmpeg" },
|
|
188
|
+
"cursor": { "type": "boolean", "default": true, "description": "Draw the pointer and clicks into web recordings" }
|
|
189
|
+
}
|
|
190
|
+
}
|
|
191
|
+
]
|
|
192
|
+
},
|
|
193
|
+
"viewport": {
|
|
194
|
+
"oneOf": [
|
|
195
|
+
{ "type": "array", "items": { "type": "integer", "minimum": 200 }, "minItems": 2, "maxItems": 2 },
|
|
196
|
+
{ "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*$", "description": "A device preset name." },
|
|
197
|
+
{
|
|
198
|
+
"type": "object",
|
|
199
|
+
"additionalProperties": false,
|
|
200
|
+
"required": ["device"],
|
|
201
|
+
"properties": {
|
|
202
|
+
"device": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*$" },
|
|
203
|
+
"width": { "type": "integer", "minimum": 200 },
|
|
204
|
+
"height": { "type": "integer", "minimum": 200 },
|
|
205
|
+
"full_screen": { "type": "boolean", "description": "The whole screen without browser bars (app mockups, kiosk). Design environments use it by default." }
|
|
206
|
+
}
|
|
207
|
+
}
|
|
208
|
+
]
|
|
209
|
+
},
|
|
166
210
|
"slug": {
|
|
167
211
|
"type": "string",
|
|
168
212
|
"pattern": "^[A-Za-z0-9][A-Za-z0-9._-]*$"
|
|
@@ -269,6 +313,7 @@
|
|
|
269
313
|
"capture": {
|
|
270
314
|
"$ref": "#/$defs/capture"
|
|
271
315
|
},
|
|
316
|
+
"record": { "$ref": "#/$defs/record", "description": "Record this step even when the walk does not (true), or never (false)." },
|
|
272
317
|
"ignore": {
|
|
273
318
|
"type": "array",
|
|
274
319
|
"description": "Regions left out of pixel comparisons (clocks, ads, carousels): a selector, or a rectangle in page pixels of one viewport. kaloko stability suggests them.",
|
|
@@ -491,12 +536,19 @@
|
|
|
491
536
|
"count",
|
|
492
537
|
"text_matches",
|
|
493
538
|
"text_absent",
|
|
539
|
+
"text_equals",
|
|
494
540
|
"attr_equals",
|
|
541
|
+
"attr_matches",
|
|
495
542
|
"no_overflow",
|
|
496
543
|
"no_console_errors",
|
|
497
544
|
"title_matches",
|
|
498
545
|
"meta_matches",
|
|
499
546
|
"link_matches",
|
|
547
|
+
"url_matches",
|
|
548
|
+
"url_equals",
|
|
549
|
+
"style_equals",
|
|
550
|
+
"style_matches",
|
|
551
|
+
"js_equals",
|
|
500
552
|
"http_status",
|
|
501
553
|
"link_home",
|
|
502
554
|
"links_ok",
|
|
@@ -530,22 +582,73 @@
|
|
|
530
582
|
},
|
|
531
583
|
"selector": {
|
|
532
584
|
"type": "string",
|
|
533
|
-
"description": "CSS selector; links_ok: the links to check (default a[href])."
|
|
585
|
+
"description": "CSS selector; links_ok: the links to check (default a[href]); style_*: the elements whose computed style is read."
|
|
534
586
|
},
|
|
535
587
|
"pattern": {
|
|
536
588
|
"type": "string",
|
|
537
|
-
"description": "Regex; links_ok: link paths to skip."
|
|
589
|
+
"description": "Regex; links_ok: link paths to skip; url_matches: tested on the URL part; style_matches: on the computed value; attr_matches: on the attribute; js_equals: on the result."
|
|
538
590
|
},
|
|
539
591
|
"attr": {
|
|
540
592
|
"type": "string"
|
|
541
593
|
},
|
|
594
|
+
"property": {
|
|
595
|
+
"type": "string",
|
|
596
|
+
"minLength": 1,
|
|
597
|
+
"description": "style_equals, style_matches: the CSS property read with getComputedStyle (color, background-color, display, font-size, …). Use longhands, not shorthands like background."
|
|
598
|
+
},
|
|
599
|
+
"token": {
|
|
600
|
+
"type": "string",
|
|
601
|
+
"pattern": "^[^{}]+$",
|
|
602
|
+
"description": "style_equals: a DTCG token name (color.brand.primary) instead of value; resolved from the project's token set, or from the page's --color-brand-primary custom property."
|
|
603
|
+
},
|
|
604
|
+
"tolerance": {
|
|
605
|
+
"type": "number",
|
|
606
|
+
"minimum": 0,
|
|
607
|
+
"description": "style_equals: allowed difference for lengths, in px."
|
|
608
|
+
},
|
|
609
|
+
"op": {
|
|
610
|
+
"enum": [
|
|
611
|
+
"<",
|
|
612
|
+
"<=",
|
|
613
|
+
">",
|
|
614
|
+
">=",
|
|
615
|
+
"=="
|
|
616
|
+
],
|
|
617
|
+
"description": "style_equals, js_equals: compare numerically with value instead of equality (font-size >= 16px)."
|
|
618
|
+
},
|
|
619
|
+
"all": {
|
|
620
|
+
"type": "boolean",
|
|
621
|
+
"description": "style_*, text_equals, attr_matches: every match must pass (default: at least one)."
|
|
622
|
+
},
|
|
623
|
+
"part": {
|
|
624
|
+
"enum": [
|
|
625
|
+
"href",
|
|
626
|
+
"path",
|
|
627
|
+
"query",
|
|
628
|
+
"hash",
|
|
629
|
+
"host",
|
|
630
|
+
"origin"
|
|
631
|
+
],
|
|
632
|
+
"description": "url_matches: which part of the URL the pattern tests (default href, the whole URL)."
|
|
633
|
+
},
|
|
634
|
+
"ignore_query": {
|
|
635
|
+
"type": "boolean",
|
|
636
|
+
"description": "url_equals: compare without the query string and hash."
|
|
637
|
+
},
|
|
638
|
+
"expression": {
|
|
639
|
+
"type": "string",
|
|
640
|
+
"minLength": 1,
|
|
641
|
+
"maxLength": 500,
|
|
642
|
+
"description": "js_equals: one read-only JavaScript expression evaluated in the page (document.querySelectorAll('.item').length, localStorage.getItem('cart'), window.dataLayer?.length). No assignments, no ++/--, only reading calls; 2 s timeout."
|
|
643
|
+
},
|
|
542
644
|
"value": {
|
|
543
645
|
"type": [
|
|
544
646
|
"string",
|
|
545
647
|
"number",
|
|
546
|
-
"boolean"
|
|
648
|
+
"boolean",
|
|
649
|
+
"null"
|
|
547
650
|
],
|
|
548
|
-
"description": "attr_equals: the attribute value; http_status: the exact status (e.g. 404). Without value, http_status accepts min..max (default 200..299)."
|
|
651
|
+
"description": "attr_equals: the attribute value; text_equals: the whole text (whitespace collapsed); url_equals: a path (/thanks, query included unless ignore_query) or a whole URL; style_equals: the expected CSS value (colours in any CSS form, lengths in px or rem); js_equals: the expected result, or the number to compare with op; http_status: the exact status (e.g. 404). Without value, http_status accepts min..max (default 200..299)."
|
|
549
652
|
},
|
|
550
653
|
"min": {
|
|
551
654
|
"type": "integer",
|
|
@@ -571,7 +674,15 @@
|
|
|
571
674
|
"pattern": "^[imsu]{0,4}$",
|
|
572
675
|
"description": "Regex flags for pattern: i (ignore case), m, s. Page text is innerText, so CSS text-transform (uppercase buttons) applies."
|
|
573
676
|
}
|
|
574
|
-
}
|
|
677
|
+
},
|
|
678
|
+
"allOf": [
|
|
679
|
+
{"if": {"properties": {"assert": {"enum": ["style_equals", "style_matches"]}}, "required": ["assert"]}, "then": {"required": ["selector", "property"]}},
|
|
680
|
+
{"if": {"properties": {"assert": {"const": "style_equals"}}, "required": ["assert"]}, "then": {"anyOf": [{"required": ["value"]}, {"required": ["token"]}]}},
|
|
681
|
+
{"if": {"properties": {"assert": {"enum": ["style_matches", "url_matches", "attr_matches"]}}, "required": ["assert"]}, "then": {"required": ["pattern"]}},
|
|
682
|
+
{"if": {"properties": {"assert": {"const": "attr_matches"}}, "required": ["assert"]}, "then": {"required": ["selector", "attr"]}},
|
|
683
|
+
{"if": {"properties": {"assert": {"enum": ["url_equals", "text_equals"]}}, "required": ["assert"]}, "then": {"required": ["value"]}},
|
|
684
|
+
{"if": {"properties": {"assert": {"const": "js_equals"}}, "required": ["assert"]}, "then": {"required": ["expression"]}}
|
|
685
|
+
]
|
|
575
686
|
},
|
|
576
687
|
"semantic": {
|
|
577
688
|
"type": "object",
|
|
@@ -710,7 +821,9 @@
|
|
|
710
821
|
"full_page": {
|
|
711
822
|
"type": "boolean",
|
|
712
823
|
"description": "Screenshot of the whole page (default true; false for stack pages: the first screen only)"
|
|
713
|
-
}
|
|
824
|
+
},
|
|
825
|
+
"record": { "$ref": "#/$defs/record" },
|
|
826
|
+
"trace": { "type": "boolean", "default": true, "description": "Keep a trace per walked step (actions, network requests, console) shown under Technical; false or walk --no-trace turns it off." }
|
|
714
827
|
}
|
|
715
828
|
}
|
|
716
829
|
}
|
package/skills/kaloko/SKILL.md
CHANGED
|
@@ -5,7 +5,9 @@ description: >
|
|
|
5
5
|
evidence: writing or updating ACCEPTANCE.md or qa/scenario.yml, running a
|
|
6
6
|
scenario on local, review, staging or production (readonly), capturing
|
|
7
7
|
screens and e-mails, evaluating criteria, or building and sharing the
|
|
8
|
-
Kaloko canvas, for web pages
|
|
8
|
+
Kaloko canvas, for web pages in Chromium, WebKit and Firefox, on phone,
|
|
9
|
+
tablet and desktop presets, in light and dark mode, as well as Android, iOS,
|
|
10
|
+
macOS and Windows apps,
|
|
9
11
|
including checking many pages of one template at once (magazine articles,
|
|
10
12
|
product pages, an SEO page catalogue). Also when someone asks what a
|
|
11
13
|
reviewer rejected on a canvas, what is waiting in the Kaloko inbox, or how
|
|
@@ -13,7 +15,8 @@ description: >
|
|
|
13
15
|
design environment (live HTML steps, drafts, versions, the design
|
|
14
16
|
reference, checking implementation against it), and when implementing a
|
|
15
17
|
screen that has an approved design (read the step spec, build with its
|
|
16
|
-
tokens, check drift)
|
|
18
|
+
tokens, check drift), and when a step should be recorded on video or its
|
|
19
|
+
trace read (actions, requests, console).
|
|
17
20
|
---
|
|
18
21
|
|
|
19
22
|
# Kaloko
|
|
@@ -50,7 +53,9 @@ create test data there (users, roles, carts), capture screens and e-mails,
|
|
|
50
53
|
evaluate, build the preview, and fix scenario mechanics (selectors, scripts)
|
|
51
54
|
that do not change what is being accepted, including applying a selector
|
|
52
55
|
proposal with `kaloko walk --heal` when it names the same element. Walk only
|
|
53
|
-
the steps a change reaches (`--affected`) and import screenshots into a run.
|
|
56
|
+
the steps a change reaches (`--affected`) and import screenshots into a run. Record steps locally
|
|
57
|
+
(`walk --record`) when a screenshot cannot show what a person has to judge, read traces, and
|
|
58
|
+
export a walkthrough video into the run folder. On a design environment: change
|
|
54
59
|
the design folder, walk, and sync the draft (including taking others' steps
|
|
55
60
|
with `--theirs`). Move to the trash runs you uploaded in this session that
|
|
56
61
|
were superseded before anyone looked at them (debug rounds: no comments, no
|
|
@@ -62,7 +67,8 @@ Changing acceptance criteria or scope, any production run, writing or
|
|
|
62
67
|
replacing a CI workflow (`kaloko ci github|gitlab`), uploading with
|
|
63
68
|
`kaloko share` (on a design environment: publishing a version), setting a
|
|
64
69
|
design reference or baseline, merging a design branch, keeping your version of
|
|
65
|
-
a step in a sync conflict (`--mine`), turning on `html_view`,
|
|
70
|
+
a step in a sync conflict (`--mine`), turning on `html_view`, turning on recordings for
|
|
71
|
+
a whole project (`capture.record` in the config: storage), a real clean-up
|
|
66
72
|
(`kaloko prune --yes`), trashing someone else's run, archiving a scenario, and
|
|
67
73
|
anything the environment profile blocks. Never trash an accepted run, a
|
|
68
74
|
baseline, a run with review comments, or a run linked from an open PR or issue
|
|
@@ -78,9 +84,13 @@ they can do; never write secrets into the config, a scenario or a script.
|
|
|
78
84
|
the sample run, the first flow from a URL or a task, share, point the person
|
|
79
85
|
to the canvas): [reference/first-run.md](reference/first-run.md)
|
|
80
86
|
- Writing or updating the plan and scenario: [reference/acceptance.md](reference/acceptance.md)
|
|
81
|
-
- Android, iOS (Simulator), Electron and
|
|
87
|
+
- Android, iOS (Simulator or a real iPhone), Electron, macOS and Windows apps,
|
|
88
|
+
several devices of one app, WebViews inside apps, or screens captured by
|
|
82
89
|
another tool (Maestro, Appium, XCUITest) with their UI tree:
|
|
83
90
|
[reference/apps.md](reference/apps.md)
|
|
91
|
+
- Walking in WebKit and Firefox, on device presets (phones, foldables,
|
|
92
|
+
tablets, TV), in dark mode, with reduced motion or as an installed PWA:
|
|
93
|
+
[reference/devices.md](reference/devices.md)
|
|
84
94
|
- Many pages of one template in one step (articles, products, an SEO
|
|
85
95
|
catalogue from a sitemap or a listing), with an index on the canvas:
|
|
86
96
|
[reference/stacks.md](reference/stacks.md)
|
|
@@ -93,6 +103,9 @@ they can do; never write secrets into the config, a scenario or a script.
|
|
|
93
103
|
(`--heal`): [reference/execution.md](reference/execution.md)
|
|
94
104
|
- Screenshots, walkthroughs, Playwright reports and traces that an agent or a
|
|
95
105
|
tool already produced, laid out as a run: [reference/import.md](reference/import.md)
|
|
106
|
+
- Video of a step or a whole flow, traces (actions, requests, console) and the
|
|
107
|
+
walkthrough video; when recording is worth its storage:
|
|
108
|
+
[reference/recording.md](reference/recording.md)
|
|
96
109
|
- CI: a workflow that walks each pull request's preview deployment and
|
|
97
110
|
comments on it (`kaloko ci github|gitlab`): [reference/ci.md](reference/ci.md)
|
|
98
111
|
- Evaluation, JEV thresholds and reading results: [reference/evaluation.md](reference/evaluation.md)
|