kaloko 0.13.0 → 0.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -2
- package/bin/qawalk.mjs +2 -1
- package/package.json +1 -1
- package/schemas/changelog.v1.json +5 -0
- package/schemas/config.v1.json +2 -1
- package/schemas/evidence.v1.json +257 -0
- package/schemas/run.v1.json +67 -1
- package/schemas/scenario.v1.json +34 -1
- package/skills/kaloko/SKILL.md +24 -5
- package/skills/kaloko/reference/acceptance.md +37 -28
- package/skills/kaloko/reference/apps.md +1 -1
- package/skills/kaloko/reference/ci.md +37 -10
- package/skills/kaloko/reference/collaboration.md +8 -1
- package/skills/kaloko/reference/design.md +11 -9
- package/skills/kaloko/reference/docs.md +9 -3
- package/skills/kaloko/reference/documentation.md +2 -2
- package/skills/kaloko/reference/evaluation.md +42 -1
- package/skills/kaloko/reference/evidence.md +80 -0
- package/skills/kaloko/reference/integrations.md +23 -4
- package/skills/kaloko/reference/mcp.md +13 -5
- package/skills/kaloko/reference/portable-html.md +1 -1
- package/skills/kaloko/reference/stacks.md +1 -1
- package/src/cli.mjs +265 -70
- package/src/lib/build.mjs +10 -1
- package/src/lib/capture.mjs +7 -0
- package/src/lib/changelog-core.mjs +11 -8
- package/src/lib/ci.mjs +1 -1
- package/src/lib/cli-args.mjs +11 -5
- package/src/lib/config.mjs +25 -1
- package/src/lib/docs-export.mjs +44 -10
- package/src/lib/docs.mjs +3 -2
- package/src/lib/doctor.mjs +3 -1
- package/src/lib/draft.mjs +1 -1
- package/src/lib/evaluate.mjs +7 -2
- package/src/lib/evidence-core.mjs +300 -0
- package/src/lib/health-core.mjs +205 -0
- package/src/lib/health.mjs +219 -0
- package/src/lib/integrations-cli.mjs +14 -3
- package/src/lib/jev.mjs +4 -2
- package/src/lib/legacy.mjs +28 -0
- package/src/lib/login.mjs +1 -1
- package/src/lib/manifest-core.mjs +23 -10
- package/src/lib/pack-help.mjs +53 -51
- package/src/lib/packs.mjs +67 -55
- package/src/lib/pr.mjs +2 -0
- package/src/lib/run.mjs +13 -1
- package/src/lib/service.mjs +16 -4
- package/src/lib/signoff.mjs +86 -6
- package/src/lib/util.mjs +28 -2
- package/src/lib/video-cli.mjs +2 -2
- package/templates/scenario.yml +1 -1
- package/viewer/index.html +557 -51
|
@@ -39,28 +39,35 @@ sharpen each purpose and add the criteria the page must really meet.
|
|
|
39
39
|
## Check packs
|
|
40
40
|
|
|
41
41
|
`packs: [seo]` or `packs: [seo, usability]` on a node adds a fixed set of
|
|
42
|
-
|
|
42
|
+
health checks (ids `HC1xx` for `seo`, `HC2xx` for `usability`; acceptance
|
|
43
|
+
criteria are `AC`, health checks `HC`, and the number is the check's: the
|
|
44
|
+
same check kept in acceptance or adopted is `AC103` for `HC103`) at load time:
|
|
43
45
|
`seo` checks one H1, title and description length, canonical, hreflang for
|
|
44
46
|
every locale, Open Graph, indexing, viewport, `html lang`, overflow and
|
|
45
47
|
console; `usability` checks unlabeled inputs and link texts and asks the
|
|
46
|
-
evaluator questions that fit the step's page type.
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
for
|
|
48
|
+
evaluator questions that fit the step's page type. Pack checks are health,
|
|
49
|
+
not acceptance: they are evaluated on every walk but never decide a step's
|
|
50
|
+
verdict (see [evaluation.md#health](evaluation.md#health)). The `design` pack
|
|
51
|
+
always counts for acceptance; `health: { acceptance: [seo] }` on the scenario
|
|
52
|
+
keeps another pack in acceptance, which only the person decides. A node's own
|
|
53
|
+
criterion with the same id wins and counts for acceptance (that is what
|
|
54
|
+
`kaloko health adopt` writes). Pack checks do not count as drift in
|
|
55
|
+
`--coverage`; the plan lists only what the brief asks for. Use packs for
|
|
56
|
+
public pages and landings; task flows keep their own criteria.
|
|
50
57
|
|
|
51
58
|
Set `page` on every step (or once on the scenario) so the usability pack
|
|
52
59
|
asks the right questions:
|
|
53
60
|
|
|
54
61
|
| `page` | for | asks |
|
|
55
62
|
| --- | --- | --- |
|
|
56
|
-
| `landing` | public and marketing pages | purpose in the first screen (`
|
|
57
|
-
| `work` | app screens where people work: overview, canvas, detail, inbox item | the heading says what the screen is for (`
|
|
58
|
-
| `list` | lists and tables: runs, projects, inbox | `
|
|
59
|
-
| `settings` | settings, billing, tokens | `
|
|
60
|
-
| `form` | sign-up, sign-in, a form to fill in | `
|
|
61
|
-
| `doc` | documentation, guides, terms | `
|
|
62
|
-
|
|
63
|
-
Every type also gets `
|
|
63
|
+
| `landing` | public and marketing pages | purpose in the first screen (`HC201`), exactly one primary call to action (`HC202`) |
|
|
64
|
+
| `work` | app screens where people work: overview, canvas, detail, inbox item | the heading says what the screen is for (`HC205`) |
|
|
65
|
+
| `list` | lists and tables: runs, projects, inbox | `HC205`, each item says what it is and how to open it (`HC207`) |
|
|
66
|
+
| `settings` | settings, billing, tokens | `HC205`, each setting says what it changes and how it is saved (`HC208`) |
|
|
67
|
+
| `form` | sign-up, sign-in, a form to fill in | `HC205`, how to send it and what happens next (`HC206`) |
|
|
68
|
+
| `doc` | documentation, guides, terms | `HC201`, headings and short paragraphs (`HC209`) |
|
|
69
|
+
|
|
70
|
+
Every type also gets `HC203` and `HC204`. Without `page` a step with the
|
|
64
71
|
`seo` pack, or any step of a scenario allowed on production, is `landing`;
|
|
65
72
|
everything else is `work`. Do not put `landing` on signed-in screens: "one
|
|
66
73
|
primary call to action" is a marketing rule, and a work screen rightly offers
|
|
@@ -78,34 +85,35 @@ When a scenario walks several locales, a text check (`text_matches`,
|
|
|
78
85
|
reads in one language only (it looks for diacritics and common Czech or
|
|
79
86
|
English words; one plain word such as a brand name is left alone).
|
|
80
87
|
|
|
81
|
-
- `a11y` (`
|
|
88
|
+
- `a11y` (`HC3xx`): images with alt, links and buttons with a name, labelled
|
|
82
89
|
fields, heading levels not skipped, WCAG AA text contrast, a `main`
|
|
83
|
-
landmark, unique ids (`
|
|
84
|
-
screenshot the walk also presses Tab through the page: `
|
|
90
|
+
landmark, unique ids (`HC301`–`HC307`, read from the page). After the
|
|
91
|
+
screenshot the walk also presses Tab through the page: `HC308` tab order
|
|
85
92
|
(hidden focus, big jumps back, positive `tabindex`, controls left out or
|
|
86
|
-
mouse-only), `
|
|
87
|
-
screenshotted focused and unfocused), `
|
|
88
|
-
dialog may keep focus), `
|
|
89
|
-
stops, `
|
|
90
|
-
reader announces). `
|
|
91
|
-
`
|
|
93
|
+
mouse-only), `HC309` visible focus with at least 3 : 1 contrast (each stop
|
|
94
|
+
screenshotted focused and unfocused), `HC310` no keyboard trap (a modal
|
|
95
|
+
dialog may keep focus), `HC311` a skip link or main content within three
|
|
96
|
+
stops, `HC312` names from Chrome's accessibility tree (what a screen
|
|
97
|
+
reader announces). `HC313` landmarks (one `main`, labelled repeats) and
|
|
98
|
+
`HC314` ARIA (unknown roles and attributes, references to missing ids,
|
|
92
99
|
`aria-hidden` over focusable content) are read from the page. The first
|
|
93
100
|
five focus stops are kept as screenshots and the canvas can draw the tab
|
|
94
101
|
order on the capture. The walk only moves focus: nothing is clicked or
|
|
95
102
|
typed, and the page is left unfocused at the top. Stack steps walk the
|
|
96
103
|
keyboard on their first five pages. Still a subset of WCAG 2.2 AA; it
|
|
97
104
|
does not replace an audit with a screen reader.
|
|
98
|
-
- `perf` (`
|
|
105
|
+
- `perf` (`HC4xx`): LCP ≤ 2.5 s, CLS ≤ 0.1, TTFB ≤ 800 ms, page weight ≤ 3 MB,
|
|
99
106
|
measured in this walk's browser since the page load (lab values: the
|
|
100
107
|
machine and network of the walk matter; compare runs, not absolutes).
|
|
101
|
-
Override a limit with an own criterion
|
|
108
|
+
Override a limit with an own criterion with the check's number as an AC
|
|
109
|
+
id (`AC401` for `HC401`), e.g.
|
|
102
110
|
`{ id: AC401, text: LCP within 4 s, check: { type: deterministic, assert: perf_lcp, max: 4000 } }`.
|
|
103
|
-
- `console` (`
|
|
111
|
+
- `console` (`HC501`): the evaluator reads the step's console errors with its
|
|
104
112
|
purpose and decides whether they touch what the step accepts (pass: only
|
|
105
113
|
third-party noise; needs review: unsure; fail: relevant). Without errors it
|
|
106
114
|
passes without a call. Keep `no_console_errors` where any error must fail.
|
|
107
115
|
|
|
108
|
-
- `errors` (`
|
|
116
|
+
- `errors` (`HC6xx`) is for error pages. It checks:
|
|
109
117
|
- a real 4xx/5xx status (a "soft 404" answering 200 fails);
|
|
110
118
|
- a link to the home page and navigation or search;
|
|
111
119
|
- no stack trace, SQL error or framework debug page;
|
|
@@ -120,7 +128,7 @@ step with `packs: [errors]`:
|
|
|
120
128
|
- **404:** use `path: /no-such-page-{random}`. `{random}` becomes a new
|
|
121
129
|
token on every walk, so no cache or an accidentally created page makes it
|
|
122
130
|
pass. `kaloko draft` adds this step on its own.
|
|
123
|
-
- **Exact status:** override AC601, for example
|
|
131
|
+
- **Exact status:** override HC601 with your own AC601, for example
|
|
124
132
|
`{ id: AC601, text: Answers 410, check: { type: deterministic, assert: http_status, value: 410 } }`.
|
|
125
133
|
`http_status` also takes `min`/`max`; the default is 200–299.
|
|
126
134
|
- **Every step:** each capture records the document's HTTP status. The
|
|
@@ -192,7 +200,8 @@ When the work has a repository and an issue number, make them visible:
|
|
|
192
200
|
- scenario `id` carries the ticket (`TASK-1273-club-activation`, `WEB-12-checkout`); the CLI derives the issue number from it or from the branch (`feat/1273-…`) and links it to the repository's origin (`acme/shop#1273`);
|
|
193
201
|
- set `issue: { number, repo?, url? }` explicitly when the id or branch cannot carry it (for example a Plane item);
|
|
194
202
|
- the project is the repository name unless `project:` in `kaloko.config.yml` says otherwise; a single flow may sort itself elsewhere with `project: <slug>` in its scenario (optional). Projects are records of the organization: `kaloko projects` or MCP `list_projects` show the slugs to use; admins create them in Settings → Projects or assign a scenario in the overview, which wins over the scenario file;
|
|
195
|
-
- the canvas header, the organization overview and `kaloko feedback` show `repo#number · title`, so name the scenario `title` for humans, not for the ticket
|
|
203
|
+
- the canvas header, the organization overview and `kaloko feedback` show `repo#number · title`, so name the scenario `title` for humans, not for the ticket;
|
|
204
|
+
- requirement IDs from the team's tracker or QMS go into `refs:` on the scenario, a step or a criterion (`refs: [URS-12, FS-3.4]`, free text, never invented); they flow into the acceptance record and the trace matrix ([evidence.md](evidence.md)).
|
|
196
205
|
|
|
197
206
|
## Session and consent
|
|
198
207
|
|
|
@@ -216,7 +216,7 @@ dialog, webview and group.
|
|
|
216
216
|
- `app_no_truncation`: texts ending in "…";
|
|
217
217
|
- `app_no_crash`: FATAL EXCEPTION or ANR in logcat, a "keeps stopping"
|
|
218
218
|
dialog, or the iOS app no longer running.
|
|
219
|
-
- **Pack `app` (
|
|
219
|
+
- **Pack `app` (HC7xx):** those four checks, plus a semantic question about
|
|
220
220
|
the screen's purpose and next action.
|
|
221
221
|
- **Web-only checks** (`no_overflow`, `http_status`, meta, `a11y_contrast`,
|
|
222
222
|
`perf_*`, `url_*`, `style_*`, `js_equals`): these are "not evaluated" on
|
|
@@ -24,8 +24,9 @@ the deployment):
|
|
|
24
24
|
2. `kaloko ci preview-url --wait 600 --github-env` finds the preview URL, the
|
|
25
25
|
pull request and its base branch and puts them into `KALOKO_PREVIEW_URL`,
|
|
26
26
|
`KALOKO_PR`, `KALOKO_BASE`;
|
|
27
|
-
3. for each scenario: `kaloko affected --
|
|
28
|
-
not reach any of its steps
|
|
27
|
+
3. for each scenario: `kaloko affected --fail-if-none` skips it when the change
|
|
28
|
+
does not reach any of its steps (workflows written earlier say `--check`,
|
|
29
|
+
which works the same and prints a note), otherwise `start --env preview`,
|
|
29
30
|
`walk --ephemeral --affected` and `share --pr` (one comment per scenario,
|
|
30
31
|
updated on later pushes).
|
|
31
32
|
|
|
@@ -52,9 +53,10 @@ include `preview`, or `start` refuses them; the command says which.
|
|
|
52
53
|
`deployment_status` event; `VERCEL_BRANCH_URL`/`VERCEL_URL`,
|
|
53
54
|
`DEPLOY_PRIME_URL`/`DEPLOY_URL` (Netlify), `CF_PAGES_URL`,
|
|
54
55
|
`CI_ENVIRONMENT_URL` (GitLab), `RENDER_EXTERNAL_URL`; then the commit's GitHub
|
|
55
|
-
deployments, skipping production, polled every 10 s up to `--wait` seconds
|
|
56
|
-
Only the URL goes to stdout, so
|
|
57
|
-
works in any CI
|
|
56
|
+
deployments, skipping production, polled every 10 s up to `--wait` (seconds,
|
|
57
|
+
or with a unit: `--wait 10m`). Only the URL goes to stdout, so
|
|
58
|
+
`export KALOKO_PREVIEW_URL="$(kaloko ci preview-url)"` works in any CI;
|
|
59
|
+
`--json` prints the URL, its source and the pull request as one JSON object.
|
|
58
60
|
|
|
59
61
|
## Affected steps in CI
|
|
60
62
|
|
|
@@ -67,11 +69,36 @@ maps routes as well as a laptop with earlier runs. See
|
|
|
67
69
|
|
|
68
70
|
## Exit codes
|
|
69
71
|
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
72
|
+
| Code | Meaning |
|
|
73
|
+
|---|---|
|
|
74
|
+
| `0` | done |
|
|
75
|
+
| `1` | checks found problems, or the command failed (an HTTP error from the service other than 401 too) |
|
|
76
|
+
| `2` | usage error: unknown command or option, missing argument |
|
|
77
|
+
| `3` | the Kaloko service cannot be reached (no connection: offline, DNS, refused, cut off); retrying may help |
|
|
78
|
+
| `4` | no token, or an invalid, expired or revoked one: `kaloko login`, or a new `KALOKO_TOKEN` secret |
|
|
79
|
+
|
|
80
|
+
A CI step can tell a broken workflow line (2) or a missing secret (4) from a
|
|
81
|
+
failing check (1), and retry only on 3. Checks keep 1: `docs check`, `drift`,
|
|
82
|
+
`design lint`, `affected --fail-if-none`, `validate`. `kaloko help <command>`
|
|
83
|
+
or `kaloko <command> --help` shows the options of one command.
|
|
84
|
+
|
|
85
|
+
## Machine output
|
|
86
|
+
|
|
87
|
+
`--json` on a command that prints data makes stdout one JSON document and
|
|
88
|
+
nothing else: progress, notes and warnings go to stderr. An error is one line
|
|
89
|
+
on stderr, `{"error":{"message":"…","code":"…","next":"…"}}`, with the exit
|
|
90
|
+
code above (`code` is `usage`, `unreachable`, `no_token`, `unauthorized`, a
|
|
91
|
+
service reason such as `plan`, or `failed`). Commands with `--json` include
|
|
92
|
+
`start`, `status`, `evaluate`, `build`, `share` (run id and canvas URL),
|
|
93
|
+
`validate`, `scenarios`, `runs`, `review`, `feedback`, `auth list`,
|
|
94
|
+
`browser status`, `ci preview-url` and `affected`.
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
url=$(npx kaloko share --json | jq -r .url)
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
`kaloko runs --all` follows the service's pages to the last run; `--limit N`
|
|
101
|
+
stops after N and prints the cursor to go on with (`--cursor`) on stderr.
|
|
75
102
|
|
|
76
103
|
## Done means
|
|
77
104
|
|
|
@@ -197,7 +197,8 @@ e-mail works too). Use it when a step needs a decision from a specific person
|
|
|
197
197
|
- Admins may name **required approvers** per scenario (Settings). Such a
|
|
198
198
|
run stays `pending` until every one of them approved; `kaloko feedback`
|
|
199
199
|
prints `awaiting approval from: …`, so do not treat a single approval as
|
|
200
|
-
acceptance there.
|
|
200
|
+
acceptance there. Those approvers are asked on their own (mail, inbox,
|
|
201
|
+
Slack or Teams, one reminder); do not ping them again.
|
|
201
202
|
- An accepted run has a **signed acceptance record** (Business and up): the
|
|
202
203
|
run, scenario, commit and environment, who approved what and when (people
|
|
203
204
|
apart from tokens and agents, approvals carried over unchanged steps with
|
|
@@ -278,6 +279,12 @@ Everything above is available over the API with an organization token:
|
|
|
278
279
|
`{"read": "all"}` or `{"read": [ids]}`
|
|
279
280
|
- `GET /api/v1/orgs/<org>/runs[?project=&scenario=&kind=&verdict=&author=&environment=&branch=&q=][&latest=1]`
|
|
280
281
|
(`latest=1`: the newest run per scenario with `older`), `GET /api/v1/orgs/<org>/runs/<id>`
|
|
282
|
+
- Paging: runs, `comments`, `notifications`, `tokens` and `audit` take
|
|
283
|
+
`?limit=` and `?cursor=`. The body keeps its shape (an array stays an
|
|
284
|
+
array); the next page is named in the `X-Next-Cursor` header and in
|
|
285
|
+
`Link: <…?cursor=…>; rel="next"`, and object answers (`notifications`,
|
|
286
|
+
`audit`) also carry `next`. No header, no next page. `projects` and
|
|
287
|
+
`products` answer the complete list.
|
|
281
288
|
- Clean-up: `DELETE …/runs/<id>` (trash), `POST …/runs/<id>/restore`,
|
|
282
289
|
`POST …/runs/<id>/keep` with `{"keep": true|false}`,
|
|
283
290
|
`POST /api/v1/orgs/<org>/runs/prune` with `{ scenario?, project?, environment?, keep_latest: 3, older_than_days?, dry_run: true }`
|
|
@@ -83,7 +83,7 @@ design/credit/late/index.html
|
|
|
83
83
|
- `kaloko sync` sends the variants' captures with the step; the published version shows the stack index with every
|
|
84
84
|
variant and its verdict, and each variant opens live in the lightbox.
|
|
85
85
|
- `kaloko design lint` reports a variant folder without its `index.html` as an error.
|
|
86
|
-
- `kaloko export
|
|
86
|
+
- `kaloko export live` and `export design` write the variant folders; the design export adds
|
|
87
87
|
`screenshots/<step>/<variant>/<locale>/<viewport>.webp`.
|
|
88
88
|
- Not for steps whose variants are different screens of a flow: those stay separate steps.
|
|
89
89
|
|
|
@@ -126,7 +126,7 @@ and `_` works). Draw each one so it can be reviewed and built:
|
|
|
126
126
|
Token modes: when the token set has two or more modes (resolver contexts or mode groups), every step renders in each
|
|
127
127
|
mode. The walk sets the mode's tokens as custom properties on `:root` after the page's own styles and
|
|
128
128
|
`data-kaloko-mode="<mode>"` on `<html>`; a page that switches themes by class can read the attribute. Captures are
|
|
129
|
-
keyed `<viewport>.mode-<name>`; the environment's `
|
|
129
|
+
keyed `<viewport>.mode-<name>`; the environment's `default_token_mode` (older name `tokens_mode`; else the first mode) is the default with the plain
|
|
130
130
|
key, and the canvas switches modes like colour schemes. A set with only `light`/`dark` modes follows the colour
|
|
131
131
|
scheme instead (`color_schemes: [light, dark]`), unless `token_modes` names them. `token_modes: [brand-a, brand-b]` (environment, scenario or `kaloko start
|
|
132
132
|
--token-modes`) picks modes; `false` turns it off. Live view shows the default mode; other modes are screenshots.
|
|
@@ -279,7 +279,7 @@ link the screens you reuse in your notes.
|
|
|
279
279
|
|
|
280
280
|
Development builds from the design reference:
|
|
281
281
|
|
|
282
|
-
- `kaloko export
|
|
282
|
+
- `kaloko export live [--version n] --out design/` writes the live files into a folder.
|
|
283
283
|
- MCP `get_step` (scenario, step, `ref: design`) returns the file list, a signed entry URL, the token set, the
|
|
284
284
|
page structure and open comments; `list_step_changes` names the steps that changed since the last reference.
|
|
285
285
|
- MCP `get_step_spec` (or `kaloko spec <step>`) returns what to build, element by element, with tokens; see
|
|
@@ -321,10 +321,11 @@ Everything exports in open formats, readable without Kaloko:
|
|
|
321
321
|
|
|
322
322
|
| What | Command | Contents |
|
|
323
323
|
|---|---|---|
|
|
324
|
-
| a design version (or `--draft`, default the design reference) | `kaloko export
|
|
325
|
-
| the live files only (the handoff) | `kaloko export
|
|
326
|
-
| a shared run | `kaloko export --run <id>` | `run.json`, every capture, the live HTML of design steps (per revision and merged in `live/site/`), comments and verdicts, and `index.html`: the canvas, offline from `file://` |
|
|
324
|
+
| a design version (or `--draft`, default the design reference) | `kaloko export design [--version n] --out dir/` or `--out v4.zip` | the live files in the folder layout, `tokens.json` (DTCG), `scenario.yml` (`qawalk.scenario.v1`), `screenshots/<step>/<locale>/<viewport>.webp`, `kaloko.json` (what is where) and `index.html`, a start page that opens from disk |
|
|
325
|
+
| the live files only (the handoff) | `kaloko export live [--version n] --out design/` | the design folder as the designer wrote it |
|
|
326
|
+
| a shared run | `kaloko export run --run <id>` | `run.json`, every capture, the live HTML of design steps (per revision and merged in `live/site/`), comments and verdicts, and `index.html`: the canvas, offline from `file://` |
|
|
327
327
|
|
|
328
|
+
The flag forms from before (`kaloko export --design`, `--live`, `--run <id>`) still work.
|
|
328
329
|
When someone asks for the design or the evidence outside Kaloko, export; do not screenshot the canvas.
|
|
329
330
|
|
|
330
331
|
## Tokens
|
|
@@ -335,7 +336,7 @@ When someone asks for the design or the evidence outside Kaloko, export; do not
|
|
|
335
336
|
`{ "$ref": "#/json/pointer" }`, `$extends`, `$root` and `$deprecated`. New sets: prefer the object values.
|
|
336
337
|
- Modes (light, dark): a DTCG resolver `design/tokens.resolver.json` (sets plus a `theme` modifier whose
|
|
337
338
|
contexts point at files) wins over `tokens.json`; `push` inlines its files. Older sets with top-level
|
|
338
|
-
`light`/`dark` groups still work. `tokens_mode` in the environment picks the mode the checks use.
|
|
339
|
+
`light`/`dark` groups still work. `default_token_mode` (or `tokens_mode`) in the environment picks the mode the checks use.
|
|
339
340
|
- `kaloko tokens validate` reports names with `{`, `}` or `.`, references that point nowhere and values that
|
|
340
341
|
cannot be read; `push` refuses a set with problems, so does the service.
|
|
341
342
|
- `kaloko tokens push` stores the set as the project's next token set version; `pull` gets it.
|
|
@@ -352,7 +353,8 @@ Turn on `packs: [design]` (usually `[design, a11y]`) for the steps in the implem
|
|
|
352
353
|
| AC802 | structure: headings, landmarks, actions and form fields as in the reference |
|
|
353
354
|
| AC803 | tokens: authored colors, fonts, sizes, spacing and radii come from the token set |
|
|
354
355
|
|
|
355
|
-
Contrast and the other accessibility basics come from the `a11y` pack
|
|
356
|
+
The design checks count for acceptance. Contrast and the other accessibility basics come from the `a11y` pack
|
|
357
|
+
(`packs: [design, a11y]`) as health: advisory, reported apart (see [evaluation.md#health](evaluation.md#health)).
|
|
356
358
|
|
|
357
359
|
`kaloko evaluate` fetches the design reference (of the design branch linked to the git branch, if any) and
|
|
358
360
|
fills the checks. Findings name what differs and the token to use; fix the implementation, not the reference.
|
|
@@ -377,7 +379,7 @@ the step, then sync.
|
|
|
377
379
|
|
|
378
380
|
Experimental, Business plan. `html_view: true` on an app environment keeps a static, script-free copy of each
|
|
379
381
|
captured page (secrets and hidden values removed) that opens in the canvas like a live step, so the inspector
|
|
380
|
-
works on the real product. It is refused on production pages behind a sign-in. `kaloko export
|
|
382
|
+
works on the real product. It is refused on production pages behind a sign-in. `kaloko export live --views
|
|
381
383
|
--run <id> --out design/` turns those copies into design steps: a redesign that starts from the product.
|
|
382
384
|
|
|
383
385
|
## Bring your own design skill
|
|
@@ -13,13 +13,19 @@ A picture in Markdown names a step instead of a PNG file:
|
|
|
13
13
|
 the capture of release 2.4.0
|
|
14
14
|
 the release before the one this entry belongs to
|
|
15
15
|

|
|
16
|
+
 the step's recording plays on the picture
|
|
16
17
|
 a step of the product's default_scenario
|
|
17
18
|
```
|
|
18
19
|
|
|
19
20
|
`<scenario>` is the scenario id, `<step>` the node id. Without `locale` and `viewport` the reader's language and
|
|
20
21
|
the product's default screen are used. `state` picks a capture variant (`dark`, `webkit`, `reduce`: the values in
|
|
21
|
-
capture keys like `mobile.webkit.dark`).
|
|
22
|
-
|
|
22
|
+
capture keys like `mobile.webkit.dark`). When the accepted run has a recording of the step (`kaloko walk --record`),
|
|
23
|
+
docs, changelog and training mode show the picture with a play button and play the recording on a click; exports
|
|
24
|
+
carry it, the PDF and the "What's new" mail show the picture. `media=video` says the step needs its recording
|
|
25
|
+
(`kaloko docs check` warns under `no_recording` when the run has none: walk the step with `--record` and have the
|
|
26
|
+
run accepted); `media=image` never plays it. Use `media=video` for interactions a picture cannot show (drag and drop,
|
|
27
|
+
a dialog, a language switch), not for every step. Check a reference with MCP `resolve_step_image` or
|
|
28
|
+
`GET /api/v1/orgs/<org>/steps/resolve?ref=…`: it answers the signed image URL (and `video` with `mp4`, `webm` and `duration_ms` when it plays), the run and how the version was found
|
|
23
29
|
(`release`, `app_version`, `reference_env`, `accepted`; `exact: false` means a fallback).
|
|
24
30
|
|
|
25
31
|
Never invent a step id or a version. Use ids from the scenario files and versions from `CHANGELOG.md` or git tags.
|
|
@@ -176,7 +182,7 @@ stale sections and training steps).
|
|
|
176
182
|
|
|
177
183
|
### Exports
|
|
178
184
|
|
|
179
|
-
`kaloko export --format md|docusaurus|vitepress|mkdocs|html|pdf|json [--product p] [--version v] [--lang cs]`
|
|
185
|
+
`kaloko export docs --format md|docusaurus|vitepress|mkdocs|html|pdf|json [--product p] [--version v] [--lang cs]`
|
|
180
186
|
writes the docs and the changelog with the pictures as files: Markdown with an `assets/` folder (any static site),
|
|
181
187
|
a Docusaurus, VitePress or MkDocs folder with sidebars and front matter, offline HTML, a PDF (printed by the
|
|
182
188
|
browser), or the changelog as JSON (`schemas/changelog.v1.json`). `--local` exports the files in the repository
|
|
@@ -17,7 +17,7 @@ rebuilt from the newest accepted run instead of screenshotting by hand.
|
|
|
17
17
|
|
|
18
18
|
## Stable URLs
|
|
19
19
|
|
|
20
|
-
With an organization token (`Authorization: Bearer
|
|
20
|
+
With an organization token (`Authorization: Bearer kal_…`, a read-only
|
|
21
21
|
viewer token is enough):
|
|
22
22
|
|
|
23
23
|
```
|
|
@@ -58,7 +58,7 @@ seconds, and do the work asynchronously. Also over the API:
|
|
|
58
58
|
## Export into a folder
|
|
59
59
|
|
|
60
60
|
```bash
|
|
61
|
-
npx kaloko export --scenario checkout --accepted --out docs/assets/checkout
|
|
61
|
+
npx kaloko export steps --scenario checkout --accepted --out docs/assets/checkout
|
|
62
62
|
```
|
|
63
63
|
|
|
64
64
|
writes `steps/<step>/<locale>/<viewport>.<ext>` and `.html` plus
|
|
@@ -68,6 +68,46 @@ evaluators:
|
|
|
68
68
|
Report the reason with each `needs_review` or failing criterion; when the
|
|
69
69
|
evaluator saw the screenshot, say so.
|
|
70
70
|
|
|
71
|
+
## Health
|
|
72
|
+
|
|
73
|
+
The checks of packs (`seo`, `a11y`, `perf`, `usability`, `console`,
|
|
74
|
+
`errors`, `app`) are the run's **health** (CS *kondice*): advisory, stored in
|
|
75
|
+
`node.health` and `summary.health` of `run.json`, never in `criteria`. They
|
|
76
|
+
do not change a step's status, the run verdict, the review queue, carried
|
|
77
|
+
approvals, required approvers, the acceptance record or exit codes. The
|
|
78
|
+
`design` pack and packs listed in `health.acceptance` stay acceptance
|
|
79
|
+
criteria. Runs shared before the split keep their stored verdicts; their pack
|
|
80
|
+
criteria show as health.
|
|
81
|
+
|
|
82
|
+
- **Check it before you hand over.** `kaloko walk` and `kaloko evaluate`
|
|
83
|
+
print one health line; `kaloko health` lists the findings by pack with
|
|
84
|
+
severity (error: people or search engines are shut out; warning: the rest;
|
|
85
|
+
info: the evaluator was unsure), what is new or fixed since the previous run
|
|
86
|
+
of the scenario, and a fix for each check. `--json` for the structure,
|
|
87
|
+
`--remote` (or MCP `get_health`) for a shared run.
|
|
88
|
+
- **Fix what your change can fix**, walk the step again and check that the
|
|
89
|
+
finding is gone. Leave the rest and say so.
|
|
90
|
+
- **Never hide health.** Do not remove a pack, raise a limit, narrow a
|
|
91
|
+
selector or add `environments:` to make a finding disappear. A limit or a
|
|
92
|
+
check that does not fit the project is the person's call; name it in the
|
|
93
|
+
report.
|
|
94
|
+
- **Ids.** Health checks are `HC<n>` (`home:HC103`), acceptance criteria
|
|
95
|
+
`AC<n>`. Runs from before 7 Oct 2026 say `AC103` for the health check;
|
|
96
|
+
read it as `HC103`, and both ids work in commands and MCP.
|
|
97
|
+
- **Adopting is the team's decision.** `kaloko health adopt <step>:<check>
|
|
98
|
+
[--text "…"]` writes the check into the scenario as an acceptance
|
|
99
|
+
criterion of the step (`home:HC103` becomes `AC103` with `from: HC103`),
|
|
100
|
+
so it counts for acceptance from the next walk. Do it only when
|
|
101
|
+
the person asks. Checks people adopted on the canvas (or through
|
|
102
|
+
`adopt_health_check`) wait on the service until `kaloko health pull` writes
|
|
103
|
+
them into the scenario; pulling them is routine, then walk those steps.
|
|
104
|
+
- **Report it apart from the verdicts**: "Health: 2 errors (1 new), 5
|
|
105
|
+
warnings; fixed HC108 on home". Never present a health finding as a
|
|
106
|
+
failing criterion, or a clean health as acceptance.
|
|
107
|
+
- `kaloko health --fail-on error` exits 1 on errors; use it in CI only where
|
|
108
|
+
the team asked for that gate. Hosted runs and CI runs alert on new errors
|
|
109
|
+
by themselves (inbox, mail, channels, webhook `health.worse`).
|
|
110
|
+
|
|
71
111
|
## Triage findings (experimental)
|
|
72
112
|
|
|
73
113
|
When the organization switched the experiment on, `kaloko evaluate` also asks
|
|
@@ -111,7 +151,8 @@ turns them off; they do not count against the hosted allowance.
|
|
|
111
151
|
|
|
112
152
|
After `kaloko build`, report: the summary line, each failing or
|
|
113
153
|
`needs_review` criterion with step, locale, viewport and evidence, blocked
|
|
114
|
-
requests if any,
|
|
154
|
+
requests if any, the health counts (apart from the verdicts), and the preview
|
|
155
|
+
path. Do not claim a criterion passed when
|
|
115
156
|
it was not evaluated.
|
|
116
157
|
|
|
117
158
|
`needs_review` is not a failure: look at the screenshot and state your
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# Evidence for your own documentation (regulated work)
|
|
2
|
+
|
|
3
|
+
Use this when a team's own repository turns Kaloko runs into its documents: test protocols, reports, trace
|
|
4
|
+
matrices, release summaries for a quality system. Kaloko provides the evidence and a way to check it; the team's
|
|
5
|
+
own templates and skills write the documents. Kaloko never formats anything for a particular QMS, and you do not put
|
|
6
|
+
the team's SOP structure into Kaloko.
|
|
7
|
+
|
|
8
|
+
The background for people (what a record proves, the two signing paths, known gaps) is in Kaloko's help, page
|
|
9
|
+
*Kaloko in regulated work (GxP)*. Evidence is a Business feature.
|
|
10
|
+
|
|
11
|
+
## The contract
|
|
12
|
+
|
|
13
|
+
Everything below is `qawalk.evidence.v1` (`schemas/evidence.v1.json` in the `kaloko` package, every field
|
|
14
|
+
described). Within v1 fields are only added; read the fields you need and ignore the rest.
|
|
15
|
+
|
|
16
|
+
- `acceptance`: `none`, `accepted` or `returned`. Write your document from this field, never from the presence of
|
|
17
|
+
others.
|
|
18
|
+
- `signing`: `in_kaloko` when a signature made in Kaloko stands, `none` otherwise. With `none` the approval, if any,
|
|
19
|
+
is signed elsewhere; your document says so ("executed in Kaloko; approval recorded in <your eQMS>").
|
|
20
|
+
- `record.sha256` and `record.kind` (`acceptance` or `execution`): the signed record the evidence came from. Put the
|
|
21
|
+
hash into your document so a reader can tie it to the package.
|
|
22
|
+
- `scenario` (`version`, `file`, `sha256`, `refs`), `environment`, `git`, `tools` (CLI, Node.js, OS, browsers,
|
|
23
|
+
service version).
|
|
24
|
+
- `steps[].criteria[]`: the criterion text as walked, `refs`, and `results[]` per locale and viewport with
|
|
25
|
+
`assessed_by` (`person`, `deterministic`, `ai`, `none`) and `assessor` (an e-mail or a model id). `advisory: true`
|
|
26
|
+
means an AI result no person decided: never present it as a person's verdict.
|
|
27
|
+
- `steps[].decision`, `verdict`, `approvals`: people's decisions apart from tokens' and agents'
|
|
28
|
+
(`counts_as_person`).
|
|
29
|
+
- `signatures[]` (name, e-mail, `signed_at`, `time_zone`, `local_time`, `meaning`, `auth`, `withdrawn`) and
|
|
30
|
+
`decision_changes[]` with reasons.
|
|
31
|
+
- `audit[]`: the run's audit trail; `trace[]`: the trace matrix, one row per requirement, criterion, step, locale and
|
|
32
|
+
viewport (columns in `TRACE_COLUMNS` order, the same as `trace.csv`).
|
|
33
|
+
- A release (`kind: release`) has `runs[]` (each a run evidence), `missing[]` with reasons and one `trace[]`.
|
|
34
|
+
|
|
35
|
+
## Getting it
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
npx kaloko evidence <run-id> --json > evidence.json # one run, any shared run
|
|
39
|
+
npx kaloko evidence --release 2.4.0 --product lims --json > release.json
|
|
40
|
+
npx kaloko evidence <run-id> --csv > trace.csv # the trace matrix only
|
|
41
|
+
npx kaloko signoff <run-id> --package --out evidence/ # the signed zip, checked on download
|
|
42
|
+
npx kaloko signoff --verify evidence/kaloko-evidence-<run-id>.zip # check it again later, no token needed
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Over the API with a read token: `GET /api/v1/orgs/<org>/runs/<id>/evidence`, `…/evidence.zip`, `…/trace.csv`,
|
|
46
|
+
`…/audit.csv`, and `GET /api/v1/orgs/<org>/evidence?product=<p>&version=<v>`. Over MCP: `get_evidence` with `run_id`, or
|
|
47
|
+
`product` and `version`.
|
|
48
|
+
|
|
49
|
+
## When it is ready
|
|
50
|
+
|
|
51
|
+
An organization admin adds a webhook with the event `evidence.ready` (Settings → Integrations → Webhooks). It fires
|
|
52
|
+
when a run is shared, accepted or returned, and when a release is recorded or updated. It is signed like every Kaloko
|
|
53
|
+
webhook (`Kaloko-Signature: t=<unix>,v1=<hex HMAC-SHA256 of "<t>.<body>">`); check the signature and the age of `t`
|
|
54
|
+
before you trust it. The body:
|
|
55
|
+
|
|
56
|
+
```json
|
|
57
|
+
{ "id": "evt_…", "type": "evidence.ready", "created_at": "2026-10-07T10:00:00.000Z", "org": "acme",
|
|
58
|
+
"data": { "kind": "run", "reason": "accepted", "run": { "id": "2026-10-07-batch-x1y2z3", "scenario": "BATCH", "environment": "validation", "project": "lims", "verdict": "accepted" },
|
|
59
|
+
"acceptance": "accepted", "signing": "none", "schema": "qawalk.evidence.v1",
|
|
60
|
+
"evidence_url": "https://kaloko.app/api/v1/orgs/acme/runs/…/evidence", "package_url": "…/evidence.zip", "trace_csv_url": "…/trace.csv" } }
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
The URLs need a token (`Authorization: Bearer …`); the webhook never carries the evidence itself.
|
|
64
|
+
|
|
65
|
+
A neutral client-side job, for example a small service or a CI job triggered by the webhook:
|
|
66
|
+
|
|
67
|
+
1. Check `Kaloko-Signature` with the webhook's secret; drop events older than five minutes.
|
|
68
|
+
2. Download `package_url` with a read token into your own storage.
|
|
69
|
+
3. Run `npx kaloko signoff --verify <zip>`; stop and alert on any failure.
|
|
70
|
+
4. Read `evidence.json` from the zip and hand it to your generator with your templates.
|
|
71
|
+
5. Store the generated document next to the package and the record's SHA-256; your eQMS takes it from there.
|
|
72
|
+
|
|
73
|
+
## Rules for an agent
|
|
74
|
+
|
|
75
|
+
- Never accept, approve or sign for a person. Tokens and agents cannot sign; where an organization signs in Kaloko,
|
|
76
|
+
only a person does, in the app.
|
|
77
|
+
- Never present `advisory` results as decisions, and never drop the AI model name from a result.
|
|
78
|
+
- Do not turn the signature policy on or off; that is an admin's decision in Settings → Security.
|
|
79
|
+
- Add `refs:` to scenarios, steps and criteria when the team gives you requirement IDs; do not invent them.
|
|
80
|
+
- Keep the package as you got it. To change anything, generate a new document; the package stays the evidence.
|
|
@@ -33,10 +33,17 @@ Ask before filing issues in bulk; one issue per real problem, not per criterion.
|
|
|
33
33
|
## Channels per project
|
|
34
34
|
|
|
35
35
|
Slack and Teams channels are chosen per project by an admin, with the events
|
|
36
|
-
each wants (new run, returned, accepted, comment, mention, release recorded
|
|
37
|
-
quiet hours. Nothing to do from the CLI: sharing,
|
|
38
|
-
release reach the right channels. Restricted projects
|
|
39
|
-
name them.
|
|
36
|
+
each wants (new run, returned, accepted, comment, mention, release recorded,
|
|
37
|
+
waiting for approval, health got worse) and quiet hours. Nothing to do from the CLI: sharing,
|
|
38
|
+
returning and recording a release reach the right channels. Restricted projects
|
|
39
|
+
reach only channels that name them.
|
|
40
|
+
|
|
41
|
+
Required approvers are asked by name: a run that waits for them sends each one a
|
|
42
|
+
Slack direct message (when their e-mail is in the workspace) with a link into
|
|
43
|
+
review mode, names them in channels that take "waiting for approval" (the Teams
|
|
44
|
+
route), and reminds them once after 24 hours by default. Do not message
|
|
45
|
+
approvers yourself after sharing; tell the person the run is waiting and give
|
|
46
|
+
the review link.
|
|
40
47
|
|
|
41
48
|
## Hosted runs
|
|
42
49
|
|
|
@@ -49,8 +56,15 @@ kaloko schedule add --scenario qa/flows/account.yml --env staging --every week -
|
|
|
49
56
|
kaloko schedule list # next and last run, pages used this month
|
|
50
57
|
kaloko schedule run <id> # starts within 15 minutes
|
|
51
58
|
kaloko schedule pause|resume|remove <id>
|
|
59
|
+
kaloko schedule hook <id> # URL + secret that start the run after a deploy (secret shown once)
|
|
52
60
|
```
|
|
53
61
|
|
|
62
|
+
- Run after a deploy: `kaloko schedule hook <id>` prints a trigger URL and a
|
|
63
|
+
secret. A deploy (Vercel through a `deployment_status` workflow, Netlify's
|
|
64
|
+
outgoing webhook with the secret as JWS token) or any CI job calls
|
|
65
|
+
`curl -X POST <url> -H "Authorization: Bearer <secret>"`. Put the secret in the
|
|
66
|
+
CI or hosting secrets, never in a file you commit, and do not echo it into logs.
|
|
67
|
+
|
|
54
68
|
- Works for: public pages, pages behind HTTP Basic (the environment's
|
|
55
69
|
`access.basic` variables are sent once and stored encrypted), and signed-in
|
|
56
70
|
pages through a session saved with `kaloko auth save` (`--account`, cookies only).
|
|
@@ -58,6 +72,11 @@ kaloko schedule pause|resume|remove <id>
|
|
|
58
72
|
script that only calls `page.goto`). Steps that click or type are left out and
|
|
59
73
|
named in the answer. Criteria decided: `exists`, `absent`, `count`, `text_*`,
|
|
60
74
|
`attr_*`, `title_matches`, `meta_matches`; the rest shows "not evaluated".
|
|
75
|
+
- Health alerts: a hosted run (and a run shared from CI) compares its health
|
|
76
|
+
with the previous run of the scenario and alerts the followers, the channels
|
|
77
|
+
and the `health.worse` webhook on a new error-level finding. Each schedule
|
|
78
|
+
sets it to `error` (default), `warning` or `off` on kaloko.app or with
|
|
79
|
+
`kaloko schedule add --health-alert`; changing it is the person's call.
|
|
61
80
|
- Does not work for: native apps, local or private addresses (localhost, IPs,
|
|
62
81
|
`.local`), header credentials, base URLs per locale.
|
|
63
82
|
- For those, or for the full walk on a schedule, write a GitHub Actions
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
`https://kaloko.app/mcp` speaks MCP over Streamable HTTP (JSON-RPC 2.0,
|
|
4
4
|
POST only, one message per request). Authenticate with an organization API
|
|
5
|
-
token as `Authorization: Bearer
|
|
5
|
+
token as `Authorization: Bearer kal_…`. The server is part of the Team plan
|
|
6
6
|
and up. A read-only token is enough to read; tools that write need a token
|
|
7
7
|
with that scope, as noted below. Nothing binary crosses MCP: summaries carry the URLs of
|
|
8
8
|
screenshots and files, which the same token can fetch over HTTPS.
|
|
@@ -21,12 +21,16 @@ file.
|
|
|
21
21
|
|
|
22
22
|
## Tools
|
|
23
23
|
|
|
24
|
+
List tools (`list_runs`, `list_notifications`, `list_comments`) answer one page: when more match than `limit`, the
|
|
25
|
+
answer's `next` holds a cursor (also as a second text block). Call the tool again with `cursor: <next>` until `next`
|
|
26
|
+
is null. Projects and products come complete, without pages.
|
|
27
|
+
|
|
24
28
|
| Tool | Use it for |
|
|
25
29
|
|---|---|
|
|
26
|
-
| `list_runs` | published runs, newest first; filters `scenario`, `project`, `verdict`, `author`, `environment`, `branch`, `limit` |
|
|
30
|
+
| `list_runs` | published runs, newest first; filters `scenario`, `project`, `verdict`, `author`, `environment`, `branch`, `limit`; `cursor` for the next page |
|
|
27
31
|
| `get_run_summary` | one run: statuses per step and locale, criteria that need attention with evidence (semantic ones with score and the evaluator's one-sentence `reason` per locale and viewport), regions ignored in pixel diffs and regions seen moving (`ignored_regions`, `unstable_regions`), screenshot URLs, verdict, baseline, canvas URL, open feedback |
|
|
28
32
|
| `get_feedback` | run verdict with note, approved steps, unresolved rejections and comments, failed manual criteria (`run_id`, or `scenario` for its latest run) |
|
|
29
|
-
| `list_notifications` | the token owner's inbox (`unread_only`, `limit`) |
|
|
33
|
+
| `list_notifications` | the token owner's inbox (`unread_only`, `limit`, `cursor`) |
|
|
30
34
|
| `list_projects` | projects of the organization with counts; the slug is the `project` filter of `list_runs` and the value for `project:` in a scenario |
|
|
31
35
|
| `get_criteria_verdicts` | reviewer verdicts on manual or overridden criteria of a run |
|
|
32
36
|
| `get_stack_items` | the pages of a stack step (`run_id`, `step`; filters `status`, `criterion`, text): path, URL, title, status and failing criteria with evidence, screenshot URLs |
|
|
@@ -38,26 +42,30 @@ file.
|
|
|
38
42
|
| `list_products` | the products the caller reads: name, modules, the projects it maps that the caller sees, the newest release, published docs (versions, languages), who reads it (and the public link), whether the caller manages it, links to its page, changelog and docs |
|
|
39
43
|
| `get_changelog` | the visual changelog of a product (`product`, `version`, `resolve: true` for the pictures, `limit`): releases newest first with the entry as written and its groups; see [docs.md](docs.md) |
|
|
40
44
|
| `draft_changelog_entry` | what accepted runs since the last release (or `since`) show differently, as a Keep a Changelog draft with before/after step references (`product`, `since`) |
|
|
41
|
-
| `resolve_step_image` | one step reference (`ref` = `kaloko:<scenario>/<step>?…`, `product`, `version`, `locale`, `viewport`) as a signed image URL with its run, how the version was found and canvas links |
|
|
45
|
+
| `resolve_step_image` | one step reference (`ref` = `kaloko:<scenario>/<step>?…`, `product`, `version`, `locale`, `viewport`) as a signed image URL with its run, how the version was found and canvas links; `video` (mp4, webm, duration_ms) when the step's recording plays (`media=video` / `media=image` in the ref) |
|
|
42
46
|
| `get_docs` | a product's published docs (`product`; without `path` the versions, languages and sidebar; with `path` one page with its Markdown, sections, stale sections and, with `resolve: true`, its pictures) |
|
|
43
47
|
| `docs_check` | what needs attention in the published docs of a version: broken step references, references without an accepted picture, stale sections, missing or outdated translations, release items without a picture |
|
|
44
48
|
| `list_step_changes` | documentation layer: steps whose capture or status changed since the previous accepted run or `since` (`scenario`, `since`, `accepted`); see [documentation.md](documentation.md) |
|
|
45
49
|
| `get_started` | where the organization stands (`empty`, `sample`, `active`), its sample run, the next steps and a prompt; call it in a new project, see [first-run.md](first-run.md) |
|
|
46
50
|
| `add_sample_run` | adds the sample run (or returns the one there is); needs an upload or full token and the Developer / Tester role |
|
|
47
51
|
| `search_screens` | the screen library: newest screenshot of every step across the organization's flows (`q` words matched against titles, ids, paths, URLs, page title and H1; `project`, `product`, `module`, `scenario`, `locale`, `viewport`, `device`, `status`, `environment`; `source: design` or `all` for design versions; `group: none` for every locale and viewport; `cursor` from `next`), each with a signed image and the canvas link |
|
|
52
|
+
| `get_health` | the advisory health of a run (`run_id`), the newest run of a scenario (`scenario`) or the trend of a project (`project`): findings by severity, pack and step with evidence and a fix, and what is new or fixed since the previous run; never a verdict, see [evaluation.md#health](evaluation.md#health) |
|
|
53
|
+
| `adopt_health_check` | turns a health check into an acceptance criterion of the scenario (`run_id`, `step`, `check` such as `HC103`, `text` to reword it; it becomes `AC103` with `from: HC103`); Developer / Tester role. Only when the person asks; `kaloko health pull` then writes it into the scenario file |
|
|
48
54
|
| `get_affected_steps` | which steps of a scenario changed files reach, with the reason per step (`scenario`, `changed_files`, `run_id`, `upstream`); route files are matched against the step paths and URLs of the newest run. Imports and `affects:` globs need the repository: `kaloko affected` follows them |
|
|
49
55
|
|
|
50
56
|
## Comment tools
|
|
51
57
|
|
|
52
58
|
| Tool | Use |
|
|
53
59
|
|---|---|
|
|
54
|
-
| `list_comments` | open threads (or `state: all`) across the organization, filtered by `scenario` or `run_id`, with replies, pins, anchors, drawn regions (`region.px`: CSS pixels of the page in `region.viewport`), attached images with URLs, and who wrote them |
|
|
60
|
+
| `list_comments` | open threads (or `state: all`) across the organization, filtered by `scenario` or `run_id`, with replies, pins, anchors, drawn regions (`region.px`: CSS pixels of the page in `region.viewport`), attached images with URLs, and who wrote them; `limit`, `cursor` |
|
|
55
61
|
| `add_comment` | comment on a step (`run_id` + `step`), the whole run (`run_id`) or the whole flow (`scenario`); `parent_id` replies; optional `pin` `{ x, y, viewport }` or `shape` (box or arrow in fractions, with `viewport` and `size`) |
|
|
56
62
|
| `list_ignore_regions` | regions people drew to leave out of pixel diffs, per step, with the `ignore:` entry each becomes (`kaloko stability --pull` writes them) |
|
|
57
63
|
| `add_ignore_region` | suggest such a region (`run_id`, `step`, `viewport`, `rect` in CSS px); Developer / Tester, scope upload or full |
|
|
58
64
|
| `get_findings` | experimental Clef triage findings of a run (`run_id`): the kind of problem on each screen or in recording frames, with probability, severity and box, and the answers testers gave. Findings never change a verdict |
|
|
59
65
|
| `answer_finding` | answers a finding (`run_id`, `node`, `locale`, `viewport`, `finding`, `kind`, `answer`: right, wrong or comment with the `comment_id` from `add_comment`); scope comment |
|
|
66
|
+
| `get_triage_report` | the organization's Clef numbers of the last 90 days (read-only): findings shown and answered, precision and calibration per kind, findings turned into comments, median hours to the first review with and without findings, triage requests per useful finding. Report them as they are; the go/no-go is a person's |
|
|
60
67
|
| `get_signoff_record` | the signed acceptance record of an accepted run (Business): approvals with provenance, criteria, capture hashes, signature |
|
|
68
|
+
| `get_evidence` | the evidence of a run (any shared run) or a release (Business, `qawalk.evidence.v1`): acceptance and signing state, criteria as walked with `refs`, who assessed each result (AI advisory), screenshots with SHA-256, decisions, signatures, audit trail, trace matrix ([evidence.md](evidence.md)) |
|
|
61
69
|
| `resolve_comment` | close a thread with a `note` that says what changed (it becomes the last reply) |
|
|
62
70
|
| `reopen_comment`, `edit_comment`, `delete_comment` | reopen; edit or delete your own comment |
|
|
63
71
|
| `create_issue` | a Jira or Linear issue from a step (`run_id`, `step`, `to`: jira or linear, optional `criterion`, `title`, `note`, `locale`, `viewport`, `target`): screenshot, criterion, notes and a link back; the step's thread resolves when the issue is done. Developer / Tester, scope comment; see [integrations.md](integrations.md) |
|
|
@@ -5,7 +5,7 @@ Design steps are live pages. The same files have to work in three places:
|
|
|
5
5
|
- **on the live origin** of the canvas (`<revision>.kaloko-usercontent.com`): a sandboxed frame with an opaque origin and
|
|
6
6
|
`Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline'; connect-src 'self'; frame-src 'none';
|
|
7
7
|
worker-src 'none'; form-action 'none'` (no popups, no modals, no service workers);
|
|
8
|
-
- **from disk** (`file://`): the local preview, `kaloko export
|
|
8
|
+
- **from disk** (`file://`): the local preview, `kaloko export design`, and anyone who unzips an export;
|
|
9
9
|
- **in another tool**: the export is the handoff and the way out of Kaloko.
|
|
10
10
|
|
|
11
11
|
`kaloko design lint` checks a design folder against the rules below; `kaloko sync` runs it first and stops on errors
|
|
@@ -29,7 +29,7 @@ step script only to prepare each page, for example to dismiss a dialog
|
|
|
29
29
|
kind: stack
|
|
30
30
|
title: Magazine articles
|
|
31
31
|
purpose: Every article reads well and is findable.
|
|
32
|
-
packs: [seo] #
|
|
32
|
+
packs: [seo] # health checks of every page (advisory)
|
|
33
33
|
items:
|
|
34
34
|
sitemap: /sitemap.xml # or urls / file / crawl / script (exactly one)
|
|
35
35
|
match: '^/clanky/' # regex on the path with its query; exclude: '…' drops pages
|