kaloko 0.4.7 → 0.4.9

package/README.md

@@ -64,6 +64,32 @@ npx kaloko share --pr                 # upload to kaloko.app, comment on the PR
 `npx kaloko help` lists every command. When something does not work (Chrome,
 token, environment URL, evaluator key), `npx kaloko doctor` says what to fix.
 
+## Review only what changed
+
+After `kaloko share`, steps that look exactly as they did in the last accepted
+run (same screenshots, same criteria, automatic checks passing) keep the
+approval a person gave them, with who and when. The rest waits in review
+mode: one step at a time on the canvas, `J`/`K` to move, `A` to approve, `R`
+to return with a note. The share says it in one line:
+
+```
+12 steps unchanged, approvals carried over; 5 need review: https://kaloko.app/acme/runs/…/#review
+```
+
+`npx kaloko review` (or `--json`, or MCP `get_review_queue`) tells an agent
+what still waits for a person, so it can pass the link on. Agents never
+approve for people: their decisions are marked and do not count as a
+person's.
+
+Every AI verdict comes with one sentence saying why. Criteria marked
+`vision: true`, and answers the text evaluator is unsure about, get a second
+look at the screenshot from a vision model (`ANTHROPIC_API_KEY`, or through
+Kaloko); `kaloko build` lists what to act on with the score and the reason.
+The `a11y` pack walks the page with Tab (order, visible focus, traps, skip
+link) and reads names from the accessibility tree. Pixel diffs skip the
+regions a step ignores; `kaloko walk --stability 3` and `kaloko stability`
+find the clocks, ads and carousels worth ignoring.
+
 ## Design (Kaloko Design)
 
 Design the screens of a flow as live HTML with your own agent and skills, in the same scenario that development
@@ -81,6 +107,17 @@ npx kaloko share --name "Checkout v2"     # a version of the whole flow
 npx kaloko reference --design --version 2 # after the team accepted it
 ```
 
+Developers and their agents build from what was approved. *Dev mode* (`D` in the step detail) shows each
+element's box, sizes and distances, its colors, type and spacing with the token every value comes from (or a
+warning when it is not a token), contrast against WCAG AA, the `:hover`, `:focus`, `:active` and disabled states,
+and its images. The same data is the step spec an agent reads, and `kaloko drift` checks the implementation
+against it:
+
+```bash
+npx kaloko spec payment                   # elements, styles → tokens, states (also MCP get_step_spec)
+npx kaloko walk && npx kaloko drift       # expected token vs actual value per element; exit 1 while it differs
+```
+
 On the canvas, *Prototype* (or `P`) presents the steps full screen as live pages: click through links between
 steps, switch the device, walk with the arrow keys, and share a link that opens straight into a step. The rules
 for pages that work everywhere are in `skills/kaloko/reference/portable-html.md`.
@@ -121,10 +158,12 @@ captured HTML, page text and console output.
 - **E-mails**: built-in Mailpit client; any other mailbox through a small
   adapter module (`provider: script`, example in `templates/mail/`), or hand a
   message over with `npx kaloko mail --step <id> --file message.eml`.
-- **Service and evaluator**: `KALOKO_TOKEN` for `share`, `feedback`, `inbox` and clean-up (`runs`, `trash`, `prune`, `keep`, `archive`)
+- **Service and evaluator**: `KALOKO_TOKEN` for `share`, `feedback`, `review`, `inbox` and clean-up (`runs`, `trash`, `prune`, `keep`, `archive`)
   (`npx kaloko login` approves one in the browser and saves it to `~/.config/kaloko/.env`, or create it in your
   organization's settings); `TYPESAFE_API_KEY` for the
-  semantic evaluator (optional). Both names can be changed in the config.
+  semantic evaluator and `ANTHROPIC_API_KEY` for the vision evaluator (both
+  optional; without them Kaloko evaluates on your plan's allowance). The names
+  can be changed in the config.
 
 `npx kaloko doctor` checks that every variable the config names is set and that
 each environment lets you in.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "kaloko",
-  "version": "0.4.7",
+  "version": "0.4.9",
   "description": "See what your AI agents built: every screen, language and device walked, assessed and approved on one canvas.",
   "keywords": [
     "qa",

package/schemas/config.v1.json

@@ -45,6 +45,32 @@
             "signals": { "type": "boolean", "default": true, "description": "Quick signals on every screen (purpose met, expected language, looks broken); never change verdicts." },
             "concurrency": { "type": "integer", "minimum": 1, "maximum": 16, "default": 6, "description": "Evaluator requests in flight at once." }
           }
+        },
+        "vision": {
+          "description": "Second, vision-capable evaluator (#17): sees the screenshot (a crop for scoped criteria) and answers with a probability and a one-sentence reason. Asked only for criteria with vision: true and for text answers in the uncertain band; false turns it off.",
+          "oneOf": [
+            { "const": false },
+            {
+              "type": "object",
+              "additionalProperties": false,
+              "properties": {
+                "provider": { "enum": ["anthropic"], "default": "anthropic" },
+                "model": { "type": "string", "default": "claude-opus-5-5", "description": "Model id of the provider (Anthropic Messages API with image input)." },
+                "api_key_env": { "type": "string", "default": "ANTHROPIC_API_KEY" },
+                "base_url": { "type": "string", "default": "https://api.anthropic.com" },
+                "when": { "enum": ["uncertain", "flagged", "always", "off"], "default": "uncertain", "description": "flagged: only criteria with vision: true; uncertain: those plus text answers in the uncertain band; always: every semantic criterion (most expensive)." },
+                "uncertain": { "type": "array", "items": { "type": "number", "minimum": 0, "maximum": 1 }, "minItems": 2, "maxItems": 2, "default": [0.15, 0.85], "description": "Text scores strictly between these go to the vision evaluator." },
+                "max_calls": { "type": "integer", "minimum": 0, "default": 40, "description": "Vision requests per evaluation (uncached); the rest stay with the text answer." },
+                "max_width": { "type": "integer", "minimum": 320, "maximum": 1568, "default": 1000, "description": "Images are scaled down to this width (px)." },
+                "tile_height": { "type": "integer", "minimum": 400, "maximum": 1568, "default": 1400, "description": "A full page is cut into tiles of this height." },
+                "max_tiles": { "type": "integer", "minimum": 1, "maximum": 4, "default": 2, "description": "Tiles of a full page sent at most (from the top); scoped criteria get a crop instead." },
+                "effort": { "enum": ["low", "medium", "high"], "default": "low" },
+                "reason_language": { "type": "string", "default": "en", "description": "Language of the one-sentence reasons (en, cs, …)." },
+                "hosted": { "type": "boolean", "default": true, "description": "Without a local key, ask through the organization's service (counts against the monthly allowance; one vision question counts as 5)." },
+                "concurrency": { "type": "integer", "minimum": 1, "maximum": 8, "default": 3 }
+              }
+            }
+          ]
         }
       }
     },

package/schemas/scenario.v1.json

@@ -262,6 +262,11 @@
         "capture": {
           "$ref": "#/$defs/capture"
         },
+        "ignore": {
+          "type": "array",
+          "description": "Regions left out of pixel comparisons (clocks, ads, carousels): a selector, or a rectangle in page pixels of one viewport. kaloko stability suggests them.",
+          "items": { "$ref": "#/$defs/ignoreRegion" }
+        },
         "items": {
           "type": "object",
           "additionalProperties": false,
@@ -500,6 +505,13 @@
             "a11y_contrast",
             "a11y_main",
             "a11y_unique_ids",
+            "a11y_focus_order",
+            "a11y_focus_visible",
+            "a11y_keyboard_trap",
+            "a11y_skip_link",
+            "a11y_ax_names",
+            "a11y_landmarks",
+            "a11y_aria",
             "perf_lcp",
             "perf_cls",
             "perf_ttfb",
@@ -588,6 +600,10 @@
             "console"
           ],
           "description": "console: the evaluator reads the captured console errors with the step purpose (triage) instead of the page"
+        },
+        "vision": {
+          "type": "boolean",
+          "description": "Also show the evaluator the screenshot (a crop of scope when set): layout, imagery, brand. Needs a vision evaluator (evaluators.vision)."
         }
       }
     },
@@ -632,6 +648,22 @@
         }
       }
     },
+    "ignoreRegion": {
+      "type": "object",
+      "additionalProperties": false,
+      "oneOf": [
+        { "required": ["selector"] },
+        { "required": ["rect", "viewport"] }
+      ],
+      "properties": {
+        "selector": { "type": "string", "minLength": 1, "description": "CSS selector; every visible match is ignored." },
+        "rect": { "type": "array", "items": { "type": "number", "minimum": 0 }, "minItems": 4, "maxItems": 4, "description": "[x, y, width, height] in CSS pixels of the page." },
+        "viewport": { "type": "string", "description": "The viewport a rect belongs to." },
+        "viewports": { "type": "array", "items": { "type": "string" }, "description": "Only in these viewports (selector)." },
+        "locales": { "type": "array", "items": { "type": "string" }, "description": "Only in these locales." },
+        "note": { "type": "string" }
+      }
+    },
     "capture": {
       "type": "object",
       "additionalProperties": false,

package/schemas/spec.v1.json

@@ -0,0 +1,60 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://qawalk.com/schemas/spec.v1.json",
+  "title": "qawalk.spec.v1",
+  "description": "The step spec (#42 Dev mode): what to build for one step as it was approved. Written by the in-page engine src/lib/devspec.mjs at capture (steps/<step>/<locale>/<viewport>.spec.json, and the capture record of a design revision); the canvas Dev mode runs the same engine on the live page. Served by GET …/scenarios/<id>/steps/<step>/spec, MCP get_step_spec and kaloko spec. Elements are listed in page order; a style lists only what differs from what the element inherits (page, then the parent element).",
+  "type": "object",
+  "required": ["schema", "elements"],
+  "$defs": {
+    "value": {
+      "type": "object",
+      "description": "A computed value (colors as #rrggbb[aa], lengths in px) and the token it maps to. token: the DTCG path (several, space-separated, for a shorthand such as padding; 0 for a zero side); null when the set has tokens of that kind but none fits and none is near. nearest: the closest token when the value is not one. mode: the mode whose value it is (light, dark, …). hard: the stylesheet typed the raw value although the token exists. var: the custom property the author used when it names no token.",
+      "required": ["value"],
+      "properties": {
+        "value": { "type": "string" },
+        "token": { "type": ["string", "null"] },
+        "nearest": { "type": "string" },
+        "mode": { "type": "string" },
+        "hard": { "type": "boolean" },
+        "var": { "type": "string" }
+      },
+      "additionalProperties": false
+    },
+    "styles": {
+      "type": "object",
+      "description": "CSS property → value: color, background-color, border-color, font-family, font-size, font-weight, line-height, letter-spacing, padding, gap, row-gap, column-gap, border-width, border-radius, box-shadow, opacity, display and flex or grid properties.",
+      "additionalProperties": { "$ref": "#/$defs/value" }
+    }
+  },
+  "properties": {
+    "schema": { "const": "qawalk.spec.v1" },
+    "title": { "type": "string" },
+    "lang": { "type": ["string", "null"] },
+    "viewport": { "type": "array", "items": { "type": "number" }, "minItems": 2, "maxItems": 2 },
+    "size": { "type": "array", "items": { "type": "number" }, "minItems": 2, "maxItems": 2, "description": "Document width and height." },
+    "tokens": { "type": ["object", "null"], "description": "The token set the values were mapped with (the step's tokens.json); null without one.", "properties": { "count": { "type": "integer" }, "modes": { "type": "array", "items": { "type": "string" } } } },
+    "page": { "$ref": "#/$defs/styles", "description": "Typography and text color of the page (body), inherited by every element that does not say otherwise." },
+    "elements": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["i", "parent", "tag", "sel", "styles"],
+        "properties": {
+          "i": { "type": "integer" },
+          "parent": { "type": "integer", "description": "Index of the nearest listed ancestor, -1 at the top." },
+          "tag": { "type": "string" },
+          "role": { "type": ["string", "null"], "description": "ARIA role, explicit or implicit (button, link, heading, main, textbox, …)." },
+          "name": { "type": ["string", "null"], "description": "Accessible name or own text (at most 80 characters)." },
+          "sel": { "type": "string", "description": "A CSS selector of the element in the design page (ids, data-testid and classes first)." },
+          "box": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4, "description": "x, y, width, height in CSS px of the page." },
+          "styles": { "$ref": "#/$defs/styles" },
+          "contrast": { "type": "object", "properties": { "ratio": { "type": "number" }, "aa": { "type": "boolean" } } },
+          "states": { "type": "object", "description": "hover, focus, active, disabled → the styles that change in that state (the page's rules forced on the element).", "additionalProperties": { "$ref": "#/$defs/styles" } },
+          "asset": { "type": "object", "properties": { "kind": { "enum": ["img", "svg"] }, "src": { "type": "string" }, "size": { "type": "array" }, "alt": { "type": ["string", "null"] } } }
+        }
+      }
+    },
+    "used": { "type": "object", "description": "Token path → number of values that use it.", "additionalProperties": { "type": "integer" } },
+    "off": { "type": "array", "description": "Values outside the token set, most frequent first.", "items": { "type": "object", "properties": { "prop": { "type": "string" }, "value": { "type": "string" }, "count": { "type": "integer" }, "nearest": { "type": ["string", "null"] }, "sel": { "type": "string" } } } }
+  }
+}

package/skills/kaloko/SKILL.md

@@ -11,7 +11,9 @@ description: >
   reviewer rejected on a canvas, what is waiting in the Kaloko inbox, or how
   to report a run in an issue, and when designing screens of a flow in a
   design environment (live HTML steps, drafts, versions, the design
-  reference, checking implementation against it).
+  reference, checking implementation against it), and when implementing a
+  screen that has an approved design (read the step spec, build with its
+  tokens, check drift).
 ---
 
 # Kaloko
@@ -28,9 +30,14 @@ the canvas, storage and collaboration are the service.
   `kaloko capture` run `kaloko evaluate`) and `kaloko build` ran; you report the summary line,
   failing criteria with evidence, and the preview path (or shared URL).
 - When the run was shared, its URL and the summary are in the issue or PR
-  the work belongs to, and you read `kaloko feedback` and `kaloko inbox`
-  before continuing on that task.
+  the work belongs to, the person got the review-mode link with what waits
+  for them (`kaloko share` prints it; `kaloko review` repeats it), and you
+  read `kaloko feedback` and `kaloko inbox` before continuing on that task.
 - Changes to intent are written back to `ACCEPTANCE.md` (change log row).
+- When you implemented a step from an approved design: you read its spec
+  (`kaloko spec <step>`), walked the implementation with `packs: [design]`,
+  and `kaloko drift` is clean or the PR names each difference left for a
+  person to decide.
 - On a design environment: the folder follows the bundle rules and
   `kaloko design lint` shows no error, `kaloko walk` and `kaloko sync` ran,
   `kaloko status` shows no conflict, and you read the draft's feedback; a
@@ -56,12 +63,17 @@ a step in a sync conflict (`--mine`), turning on `html_view`, a real clean-up
 (`kaloko prune --yes`), trashing someone else's run, archiving a scenario, and
 anything the environment profile blocks. Never trash an accepted run, a
 baseline, a run with review comments, or a run linked from an open PR or issue
-comment. Ask the person
+comment. Never approve a
+step or accept a run on a person's behalf; report what waits for them
+(`kaloko review`, MCP `get_review_queue`) instead. Ask the person
 for secret values (passwords, tokens, gate credentials) and for sign-ins only
 they can do; never write secrets into the config, a scenario or a script.
 
 ## Where to look
 
+- A new project or an organization without runs of its own (install, sign in,
+  the sample run, the first flow from a URL or a task, share, point the person
+  to the canvas): [reference/first-run.md](reference/first-run.md)
 - Writing or updating the plan and scenario: [reference/acceptance.md](reference/acceptance.md)
 - Android, iOS (Simulator), Electron and macOS apps, or screens captured by
   another tool (Maestro, Appium, XCUITest) with their UI tree:
@@ -71,12 +83,14 @@ they can do; never write secrets into the config, a scenario or a script.
   [reference/stacks.md](reference/stacks.md)
 - Choosing playwright, agent or manual per step, and driving the shared
   browser with your capabilities: [reference/execution.md](reference/execution.md)
-- Evaluation, JEV thresholds and reading results: [reference/evaluation.md](reference/evaluation.md)
+- Evaluation, thresholds, the vision evaluator and its reasons, reading
+  results: [reference/evaluation.md](reference/evaluation.md)
 - Environments, access (gates, accounts, sign-in), e-mails, readonly and
   production rules, and what is yours to arrange for the project's stack:
   [reference/environments.md](reference/environments.md)
-- Sharing, reviewer feedback, mentions, inbox, verdicts and baselines, and
-  cleaning up runs (trash, prune, keep, archive):
+- Sharing, review mode and approvals carried over unchanged steps, reviewer
+  feedback, mentions, inbox, verdicts and baselines, and cleaning up runs
+  (trash, prune, keep, archive):
   [reference/collaboration.md](reference/collaboration.md)
 - Reading runs, feedback and the inbox over MCP instead of the CLI:
   [reference/mcp.md](reference/mcp.md)
@@ -87,6 +101,9 @@ they can do; never write secrets into the config, a scenario or a script.
   references, handoff, tokens, the design pack, canvas edits, branches, HTML
   view, presenting the prototype, exports, bringing your own design skill):
   [reference/design.md](reference/design.md)
+- Implementing a screen from the approved design: read the step spec, build
+  with its tokens, walk, check drift, fix:
+  [reference/design.md#implement-from-the-approved-design](reference/design.md#implement-from-the-approved-design)
 - Writing design pages that work on the live step, from disk and in other
   tools (relative URLs, no CDNs, state in the URL, deterministic content,
   links between steps; what `kaloko design lint` checks):

package/skills/kaloko/reference/acceptance.md

@@ -40,8 +40,21 @@ for public pages and landings; task flows keep their own criteria.
 
 - `a11y` (`AC3xx`): images with alt, links and buttons with a name, labelled
   fields, heading levels not skipped, WCAG AA text contrast, a `main`
-  landmark, unique ids. A subset of WCAG 2.2 AA checked in the page; it does
-  not replace an audit with a screen reader.
+  landmark, unique ids (`AC301`–`AC307`, read from the page). After the
+  screenshot the walk also presses Tab through the page: `AC308` tab order
+  (hidden focus, big jumps back, positive `tabindex`, controls left out or
+  mouse-only), `AC309` visible focus with at least 3 : 1 contrast (each stop
+  screenshotted focused and unfocused), `AC310` no keyboard trap (a modal
+  dialog may keep focus), `AC311` a skip link or main content within three
+  stops, `AC312` names from Chrome's accessibility tree (what a screen
+  reader announces). `AC313` landmarks (one `main`, labelled repeats) and
+  `AC314` ARIA (unknown roles and attributes, references to missing ids,
+  `aria-hidden` over focusable content) are read from the page. The first
+  five focus stops are kept as screenshots and the canvas can draw the tab
+  order on the capture. The walk only moves focus: nothing is clicked or
+  typed, and the page is left unfocused at the top. Stack steps walk the
+  keyboard on their first five pages. Still a subset of WCAG 2.2 AA; it
+  does not replace an audit with a screen reader.
 - `perf` (`AC4xx`): LCP ≤ 2.5 s, CLS ≤ 0.1, TTFB ≤ 800 ms, page weight ≤ 3 MB,
   measured in this walk's browser since the page load (lab values: the
   machine and network of the walk matter; compare runs, not absolutes).
@@ -227,3 +240,22 @@ skips a session's nodes there. The canvas shows a chip with the person on
 every step, groups the step list by person and draws hand-offs between
 people as dashed arrows. Sessions run one after another, never in parallel,
 and never share cookies.
+
+## Ignore regions
+
+Clocks, ads, carousels and "3 minutes ago" change on every capture and make
+pixel diffs noisy. List them on the step and both the canvas diff and the
+design pixel check (`AC801`) leave them out:
+
+```yaml
+- id: home
+  ignore:
+    - { selector: "#clock" }                       # every visible match
+    - { selector: ".ad-slot", viewports: [desktop] }
+    - { rect: [0, 620, 1280, 240], viewport: desktop } # page pixels of one viewport
+```
+
+Do not guess them: `kaloko walk --stability 3` notes what moves while the
+page sits still, and `kaloko stability` compares the step with earlier runs
+and writes suggestions (see [execution.md](execution.md)). Ignore only noise;
+a region that matters to a criterion stays in.

package/skills/kaloko/reference/collaboration.md

@@ -4,7 +4,10 @@ A run is only worth something once the people who decide have seen it. The
 loop teams use:
 
 1. **Share the run** (`kaloko share`, tester token, after asking). The
-   command prints the canvas URL.
+   command prints the canvas URL and one line on what waits for a person,
+   e.g. `12 steps unchanged, approvals carried over; 5 need review: <link>`.
+   Give the person that review-mode link (it ends in `#review`) with the
+   counts, not just the canvas.
 2. **Put the link where the work is tracked.** When the branch has a pull
    request and `gh` is signed in, `kaloko share --pr` posts one comment per
    scenario on it (summary line, canvas link, criteria that need attention
@@ -19,7 +22,10 @@ loop teams use:
    that waits for review and turns green or red with the verdict; do not
    report it as passed before a person accepted the run.
 3. **Read feedback before continuing.** `kaloko feedback` lists the run
-   verdict, open rejections and comments, and failed manual criteria.
+   verdict, returned steps first (with the reviewer's note), open comments,
+   failed manual criteria and how many steps still wait for a person.
+   `kaloko review` (or MCP `get_review_queue`) says exactly what a person
+   still has to decide; see *Review mode* below.
    `kaloko inbox` lists what needs your attention across runs: mentions,
    returned runs, rejected steps in your runs, new runs of scenarios you
    reviewed, and expiry warnings. Both take `--json`; `kaloko inbox --read`
@@ -31,6 +37,34 @@ loop teams use:
    reply in the tracker with the new link. The reviewer who returned it is
    told about the new run automatically.
 
+## Review mode
+
+Reviewers decide in **review mode** on the canvas (`Q`, the Review button,
+or a link ending in `#review`): one step at a time, the steps that need a
+person first (failed, returned, the AI was unsure or did not evaluate,
+changed since the last accepted run, open comments), with the accepted run
+beside it. `J`/`K` move, `A` approves, `R` returns with a note, `C`
+comments; at the end they accept or return the run.
+
+- **Approvals carry over unchanged steps.** When a new run is shared, a step
+  whose screenshots are byte-identical to the last accepted run, whose
+  criteria are the same and whose deterministic checks pass keeps the
+  approval a person gave it there, with its origin (who, when, which run).
+  Only changed or failing steps go to the queue. Anyone can revoke a carried
+  approval; the step then waits again. Organizations on Business can turn
+  this off (Settings → Approvals).
+- A carried approval never accepts the run and never counts for required
+  approvers: the run verdict is always a person's decision.
+- **Report what waits**, do not decide it: run `kaloko review` (add `--json`
+  for the data, `--run <id>` for another run) or MCP `get_review_queue` and
+  tell the person how many steps wait and give them `review_url`.
+- **Never approve or accept on a person's behalf.** Decisions by tokens and
+  agents (MCP `decide_step`, `add_comment` with `approved`) are recorded as
+  yours, shown apart, and do not count as a person's unless the organization
+  allows it. Return a step only when you found a real problem, with a note.
+- Returned steps are feedback: fix them, share a new run, and resolve the
+  thread with what changed (see below).
+
 ## Processing feedback
 
 Comments are threads in three places:
@@ -195,6 +229,15 @@ Settings → Storage (keep the newest N per scenario and environment).
 Everything above is available over the API with an organization token:
 
 - `GET /api/v1/orgs/<org>/feedback?run=<id|latest>[&scenario=<id>]`
+- `GET /api/v1/orgs/<org>/runs/<id>/review-queue` (or
+  `GET /api/v1/orgs/<org>/review-queue?run=<id|latest>&scenario=<id>`):
+  `items` in review order with `reasons`, `decision` (approved, returned or
+  carried, with who and how), `unconfirmed` (an agent's or token's decision
+  that does not count), `changed`, criteria with evidence, `screenshots` and
+  `ref_screenshots`; `carried` with provenance; `counts` (`queue`,
+  `decided`, `pending`, `carried`, `clear`); `reference` (the accepted run);
+  `review_url`. Decisions use the comment and verdict routes;
+  `POST …/runs/<id>/carried/<step>/revoke` revokes a carried approval.
 - `GET|POST /api/v1/orgs/<org>/runs/<id>/comments` (a step: `node_id`;
   the whole run: `node_id: ""`), `GET|POST …/scenarios/<id>/comments`
   (the whole flow), and per comment `PATCH`, `DELETE`, `POST …/<cid>/resolve`

package/skills/kaloko/reference/design.md

@@ -39,8 +39,10 @@ design/
   the bundle.
 - Links between steps are plain relative links to the page: `<a href="../payment/index.html">`. The canvas
   follows them; `data-kaloko-go="payment"` (or `data-qawalk-go`) moves to that step whatever the `href` says.
-- Reference token values as CSS custom properties (`var(--color-accent)`), so the inspector and the token
-  check see where values come from.
+- Reference token values as CSS custom properties named after the token path (`color.accent` →
+  `var(--color-accent)`), so Dev mode, the step spec and the token check see which token a value comes from.
+  Define interaction states in CSS (`:hover`, `:focus-visible`, `:active`, `:disabled`): Dev mode and the spec
+  read them from the stylesheet.
 - Write pages by the portable HTML rules ([portable-html.md](portable-html.md)): relative URLs only, bundled
   fonts, no CDNs, state in the URL instead of storage, deterministic content, viewport meta and `lang`.
   `kaloko design lint` checks the folder; `kaloko sync` runs it first and stops on errors (things that break on
@@ -94,8 +96,39 @@ Development builds from the design reference:
 - `kaloko export --live [--version n] --out design/` writes the live files into a folder.
 - MCP `get_step` (scenario, step, `ref: design`) returns the file list, a signed entry URL, the token set, the
   page structure and open comments; `list_step_changes` names the steps that changed since the last reference.
+- MCP `get_step_spec` (or `kaloko spec <step>`) returns what to build, element by element, with tokens; see
+  [Implement from the approved design](#implement-from-the-approved-design).
 - Translate to the project's stack with the project's own skills; keep the step ids and the token names.
 
+## Implement from the approved design
+
+Use this when you build or change a screen that has an approved design step (a design reference, or a version
+the person names). The step spec says what was approved in the terms you write code in: elements in page order
+with role, accessible name and selector, and their colors, typography, spacing, radius, shadow and layout, each
+value with the DTCG token it comes from. People see the same data in the canvas: *Dev mode* (`D`) in the step
+detail.
+
+1. **Read the spec** of each step you build: `kaloko spec <step>` (default `--ref design`; `--ref v4`, `draft` or a
+   run id), or MCP `get_step_spec`. `--json` gives `qawalk.spec.v1` (`schemas/spec.v1.json`).
+2. **Implement with the tokens.** A line such as `button "Pay" 120×44 bg #2e6bff=color.brand.primary · pad 12px
+   20px=space.3 space.5` means `background: var(--color-brand-primary); padding: var(--space-3) var(--space-5)`
+   (the custom property is the token path with dots as dashes) or the project's own token mechanism. `not a token;
+   nearest x` means the design itself used a raw value: ask the designer before you copy it, and say so in the PR.
+   `:hover`, `:focus`, `:active` and `:disabled` lines are states to build; `(light)` names the mode of a value.
+   Keep the roles and names: they are how the implementation is matched with the design.
+3. **Walk it** with `packs: [design]` on the implementation steps; the capture then stores the implementation's
+   spec next to the screenshot.
+4. **Check drift**: `kaloko drift [step]` compares the walked run with the design reference of the linked design
+   branch and exits 1 while something differs. A shared run: MCP `get_step_drift` (`run`: id or `latest`).
+   Findings name the element, the expected token and value, and what the page has; `typed raw` means the value
+   is right but should come from the token; `not found` means the design's element (by role and name) is missing.
+5. **Fix and repeat** until drift is clean or what is left is a decision for a person (a missing element on
+   purpose, a raw value the designer accepted); report those in the PR instead of hiding them.
+
+Specs exist for captures made with the current CLI: walk the design and sync again if `kaloko spec` says there is
+none. The pixel, structure and token checks of the design pack (`AC801`–`AC803`) still run in `kaloko evaluate`;
+drift is the per-element view an agent can act on.
+
 ## Your data is yours
 
 Everything exports in open formats, readable without Kaloko:

package/skills/kaloko/reference/evaluation.md

@@ -10,9 +10,11 @@ answer, so running it again is cheap.
 - deterministic results computed in the live page at capture time (with
   evidence: selector snippet, measured value);
 - the semantic evaluator for semantic criteria. It is a replaceable provider;
-  the default is JEV by TypeSafe (`TYPESAFE_API_KEY`), which returns only a
-  probability per question (`score`, 0–1). Verdicts: `pass` ≥ 0.85,
-  `fail` ≤ 0.15, otherwise `needs_review` (thresholds in config);
+  the default is JEV by TypeSafe (`TYPESAFE_API_KEY`), which reads the
+  reduced page state (no images) and returns only a probability per question
+  (`score`, 0–1). Verdicts: `pass` ≥ 0.85, `fail` ≤ 0.15, otherwise
+  `needs_review` (thresholds in config);
+- a vision evaluator for a second look with the screenshot (below);
 - `manual` criteria stay `not_evaluated` until a reviewer decides.
 
 Where the evaluator runs: with `TYPESAFE_API_KEY` set locally, your own key; without it, the organization's
@@ -25,6 +27,47 @@ report with the reason `kaloko evaluate` gives.
 The default evaluator is text-only and most accurate in English. It is weak
 at numbers, which is why those are checked deterministically.
 
+## Vision and reasons
+
+Every semantic verdict carries one sentence, `reason`, shown as "Why" next to
+the score on the canvas, in `kaloko build`, in the walk summary and in
+`get_run_summary`. With only the text evaluator the sentence says what it
+read and how sure it was (JEV gives no explanation). The vision evaluator
+gives a real one: it sees the screenshot, or a crop of the criterion's
+`scope` (or `highlight`), and answers with a probability and what in the
+image decided it. It is asked for:
+
+- criteria with `vision: true` on the check (layout, imagery, "looks like the
+  brand", anything the text cannot show);
+- text answers in the uncertain band (0.15–0.85 by default), which would
+  otherwise go to a person.
+
+Its answer replaces the score (the text score stays as `text_score` in the
+evaluation). Cost stays bounded: only those criteria, one request per
+capture, images scaled to 1000 px wide and at most two tiles of a full page,
+at most 40 uncached requests per evaluation, and answers cached by content
+like the text ones. The provider is Anthropic's Messages API
+(`ANTHROPIC_API_KEY`, model from `evaluators.vision.model`, default
+`claude-opus-5-5`). Without the key it goes through the organization's
+service, where one vision question counts as five of the monthly allowance;
+when the text evaluator runs with your own `TYPESAFE_API_KEY`, the vision
+evaluator also waits for your own key (`evaluators.vision.hosted: true`
+lets it use the service anyway).
+Configure or switch it off in `kaloko.config.yml`:
+
+```yaml
+evaluators:
+  vision:
+    when: uncertain      # flagged (only vision: true) | always | off
+    uncertain: [0.15, 0.85]
+    max_calls: 40
+    reason_language: en  # cs for Czech reviewers
+  # vision: false        # text evaluator only
+```
+
+Report the reason with each `needs_review` or failing criterion; when the
+evaluator saw the screenshot, say so.
+
 When a criterion comes back `needs_review`, read the reduced state the
 evaluator saw (`steps/<node>/<locale>/<viewport>.capture.json`, key `state`)
 and the screenshot. Either the page is genuinely unclear (report it), the
@@ -61,3 +104,8 @@ it was not evaluated.
 
 `needs_review` is not a failure: look at the screenshot and state your
 reading, or leave it to the reviewer.
+
+After a share, steps that did not change since the last accepted run keep a
+person's approval; only the rest waits in review mode. Report it as the
+share prints it ("12 steps unchanged, approvals carried over; 5 need review:
+<link>") and never call a carried or agent approval a person's acceptance.

package/skills/kaloko/reference/execution.md

@@ -73,3 +73,21 @@ The environment policy and gate apply to every context the CLI creates (the
 shared browser, `walk` sessions, one-off captures): blocked paths and, when
 readonly, non-GET requests outside `allowed_post_paths` fail and are listed as
 blocked requests in the capture; the gate's credentials go to its origins only.
+
+## Noisy regions (`--stability`, `kaloko stability`)
+
+When pixel diffs between runs keep flagging the same clock, ad or carousel:
+
+```bash
+kaloko walk --stability 3        # 3 screenshots per capture, a moment apart; what moved is noted
+kaloko stability                 # this run against the 3 previous local runs of the scenario
+kaloko stability --apply         # adds the suggested ignore: to each step (block-style YAML only)
+```
+
+`kaloko stability` writes `stability.json` and `stability.yml` into the run
+and prints the suggestions per step: a selector when an element of the stored
+page covers the region, otherwise a rectangle for one viewport. Look at the
+screenshots before applying; a region that changed because the page really
+changed is not noise. `--same-commit` compares only runs of the same commit,
+`--runs a,b` names them, `--last N` sets how many runs are compared. Capture
+the steps again after applying: the capture records the ignored regions.

package/skills/kaloko/reference/first-run.md

@@ -0,0 +1,51 @@
+# First run in a new project
+
+Use this when the project has no `kaloko.config.yml` yet, or the organization
+has no shared run of its own. The goal is one real flow on the canvas, shared,
+with a person who knows where to approve it.
+
+## Where the organization stands
+
+- `GET /api/v1/orgs/<org>/getting-started` (MCP tool `get_started`) answers
+  `stage` (`empty`, `sample`, `active`), the sample run's URL, how many real
+  runs exist, the next steps and a prompt to start from.
+- Every new organization has a **sample run**: a tea-shop checkout in Czech
+  and English on desktop and mobile, with one failing step per language, one
+  overflow on the phone and one criterion waiting for a person. It does not
+  count toward the plan and expires after 30 days. `<service>/<org>/sample`
+  opens it; it carries a short guide for the person (open a step, approve,
+  return with a note, share).
+- If the person has never reviewed a run, send them the sample link first:
+  they learn the canvas there, not on your first draft. When it is gone,
+  `POST /api/v1/orgs/<org>/sample` (MCP `add_sample_run`, needs an upload or
+  full token) adds it again. Do not delete it unless asked
+  (`DELETE /api/v1/orgs/<org>/sample`).
+
+## From zero to a first shared run
+
+1. `npm install --save-dev kaloko`, then
+   `npx kaloko init --agent claude --org <org>` (it prints the sample link
+   and these steps). Fill `environments` in `kaloko.config.yml` with the URL
+   the app runs on.
+2. `npx kaloko login`: the person approves a token in the browser. Ask them
+   to; never create or paste tokens yourself.
+3. `npx kaloko doctor` and fix what it marks ✕.
+4. Draft the first scenario:
+   - public pages: `npx kaloko draft <url> [url…]` writes a read-only
+     production scenario and a plan; sharpen purposes and criteria;
+   - a task or the main flow of the app: write `ACCEPTANCE.md` and
+     `qa/scenario.yml` as in [acceptance.md](acceptance.md). Pick the flow a
+     person would show a client (sign-up, checkout, the main form), not every
+     page.
+5. `npx kaloko validate qa/scenario.yml`, then
+   `npx kaloko start --scenario qa/scenario.yml --env local`, `kaloko walk`,
+   `kaloko build`. Fix scenario mechanics until every step has a capture.
+6. Ask before `kaloko share` (see *Ask first* in SKILL.md). After sharing,
+   give the person the canvas URL and say what to look at: the failing
+   criteria with their evidence and the steps that need a decision.
+7. Read `kaloko feedback` when they have looked; fix what they returned and
+   walk again.
+
+Done means: the scenario validates, the run is shared, its URL and summary
+line are in your reply (and in the issue or PR the work belongs to), and the
+person knows whether the sample or this run is what they should open.