kaloko 0.14.0 → 0.16.0

package/skills/kaloko/SKILL.md

@@ -22,8 +22,13 @@ description: >
   kaloko changelog draft, kaloko release) or a guide should show a step or
   be written, translated or checked (kaloko docs check), when screenshots,
   HTML or recordings must hide personal data (capture masks, pii: redact or
-  pseudonymize, kaloko share stopping on unmasked data), and when a run has
-  Clef triage findings (experimental) to read, mention or answer.
+  pseudonymize, kaloko share stopping on unmasked data), when a run has
+  Clef triage findings (experimental) to read, mention or answer, when
+  a run's health (the advisory checks of packs: SEO, accessibility,
+  performance, console) should be checked, fixed or adopted into acceptance,
+  and when a team's own repository turns runs into quality evidence for its
+  QMS or GxP (requirement refs, kaloko evidence, the evidence package, the
+  evidence.ready webhook).
 ---
 
 # Kaloko
@@ -39,6 +44,9 @@ the canvas, storage and collaboration are the service.
 - The captures are evaluated (`kaloko walk` ends with the verdicts; after
   `kaloko capture` run `kaloko evaluate`) and `kaloko build` ran; you report the summary line,
   failing criteria with evidence, and the preview path (or shared URL).
+- Health was checked before you hand over (`kaloko health`, MCP `get_health`):
+  you fixed what your change can fix, and you report the remaining counts and
+  what is new since the previous run, apart from the verdicts.
 - When the run was shared, its URL and the summary are in the issue or PR
   the work belongs to, the person got the review-mode link with what waits
   for them (`kaloko share` prints it; `kaloko review` repeats it), and you
@@ -81,19 +89,27 @@ verdict), and run `kaloko prune` without `--yes` to see what a clean-up would do
 Draft and edit the Unreleased section of the changelog (`kaloko changelog draft
 --apply`) and add step references to it and to guides. Put the products of the
 config on Kaloko (`kaloko products sync`: it records no release) and list them
-(`kaloko products list`). Write, translate and fix
+(`kaloko products list`). Create the project the config names
+(`kaloko projects sync` or `kaloko projects create` with the name the person
+gave). Write, translate and fix
 guides in the docs folder, run `kaloko docs check`, and export them for a docs
 site (`kaloko export docs --format …`) into a local folder.
 
 ## Ask first
 
-Changing acceptance criteria or scope, any production run, writing or
+Changing acceptance criteria or scope (adopting a health check into
+acceptance with `kaloko health adopt` is such a change; pulling checks a
+person already adopted with `kaloko health pull` is not), removing a pack or
+keeping one in acceptance (`health.acceptance`), any production run, writing or
 replacing a CI workflow (`kaloko ci github|gitlab`), uploading with
 `kaloko share` (on a design environment: publishing a version), setting a
 design reference or baseline, merging a design branch, keeping your version of
 a step in a sync conflict (`--mine`), turning on `html_view`, turning on recordings for
 a whole project (`capture.record` in the config: storage), a real clean-up
 (`kaloko prune --yes`), trashing someone else's run, archiving a scenario,
+renaming a project or restricting it (`kaloko projects rename`, a `kaloko
+projects sync` that would rename), picking one site of a repository that
+holds several (ask which one before `kaloko init` or a walk),
 recording a release (`kaloko release`) or asking to publish a changelog,
 publishing docs (`kaloko docs publish`, `--reviewed` above all: it says a
 person checked the text), writing a release workflow (`kaloko ci github
@@ -102,7 +118,8 @@ anything the environment profile blocks. Never trash an accepted run, a
 baseline, a run with review comments, or a run linked from an open PR or issue
 comment. Never approve a
 step or accept a run on a person's behalf, and never hand out a signed
-acceptance record (`kaloko signoff`) for a run a person has not accepted; report what waits for them
+acceptance record (`kaloko signoff`) for a run a person has not accepted, never present an AI verdict as a person's
+decision in generated documents, and never change the signature policy; report what waits for them
 (`kaloko review`, MCP `get_review_queue`) instead. Ask the person
 for secret values (passwords, tokens, gate credentials) and for sign-ins only
 they can do; never write secrets into the config, a scenario or a script. When
@@ -113,7 +130,11 @@ only when a person says that value may be uploaded.
 
 - A new project or an organization without runs of its own (install, sign in,
   the sample run, the first flow from a URL or a task, share, point the person
-  to the canvas): [reference/first-run.md](reference/first-run.md)
+  to the canvas), and naming the project on Kaloko (`project_name:`,
+  `kaloko projects create|rename|sync`): [reference/first-run.md](reference/first-run.md)
+- A repository with several sites or apps (monorepo, Rails engines, Folio
+  sites): ask which one, then `kaloko init --site … --base-url …`:
+  [reference/first-run.md#several-sites-in-one-repository](reference/first-run.md#several-sites-in-one-repository)
 - Writing or updating the plan and scenario: [reference/acceptance.md](reference/acceptance.md)
 - Android, iOS (Simulator or a real iPhone), Electron, macOS and Windows apps,
   several devices of one app, WebViews inside apps, or screens captured by
@@ -131,6 +152,9 @@ only when a person says that value may be uploaded.
   (`--heal`): [reference/execution.md](reference/execution.md)
 - Evaluation, JEV thresholds, the vision evaluator and its reasons, reading
   results: [reference/evaluation.md](reference/evaluation.md)
+- Health (CS Kondice): the advisory checks of packs, `kaloko health`,
+  severities, fixing before review, adopting a check into acceptance, alerts:
+  [reference/evaluation.md#health](reference/evaluation.md#health)
 - Clef triage findings (experimental): what they are, how to report them apart
   from the verdicts, `get_findings` and `answer_finding`:
   [reference/evaluation.md#triage-findings-experimental](reference/evaluation.md#triage-findings-experimental)
@@ -154,6 +178,11 @@ only when a person says that value may be uploaded.
   [reference/collaboration.md](reference/collaboration.md)
 - Reading runs, feedback and the inbox over MCP instead of the CLI:
   [reference/mcp.md](reference/mcp.md)
+- Quality evidence for a team's QMS (ISO 9001, ISO 13485, IEC 62304, ISO 27001;
+  GxP is the strictest case): requirement `refs:`, `kaloko evidence`, the
+  evidence package and its check, the `evidence.ready` webhook, feeding a
+  team's own document templates, signatures and what an agent must never do:
+  [reference/evidence.md](reference/evidence.md)
 - Filing a returned step or a failing criterion in Jira or Linear, issue keys in
   branches, channels per project, and hosted runs of read-only scenarios on a
   schedule (`kaloko issue`, `kaloko schedule`):

package/skills/kaloko/reference/acceptance.md

@@ -39,28 +39,35 @@ sharpen each purpose and add the criteria the page must really meet.
 ## Check packs
 
 `packs: [seo]` or `packs: [seo, usability]` on a node adds a fixed set of
-criteria (ids `AC1xx` for `seo`, `AC2xx` for `usability`) at load time:
+health checks (ids `HC1xx` for `seo`, `HC2xx` for `usability`; acceptance
+criteria are `AC`, health checks `HC`, and the number is the check's: the
+same check kept in acceptance or adopted is `AC103` for `HC103`) at load time:
 `seo` checks one H1, title and description length, canonical, hreflang for
 every locale, Open Graph, indexing, viewport, `html lang`, overflow and
 console; `usability` checks unlabeled inputs and link texts and asks the
-evaluator questions that fit the step's page type. A node's own criterion with the
-same id wins. Pack criteria are marked `pack` on the canvas and do not count
-as drift in `--coverage`; the plan may still mention them by id. Use packs
-for public pages and landings; task flows keep their own criteria.
+evaluator questions that fit the step's page type. Pack checks are health,
+not acceptance: they are evaluated on every walk but never decide a step's
+verdict (see [evaluation.md#health](evaluation.md#health)). The `design` pack
+always counts for acceptance; `health: { acceptance: [seo] }` on the scenario
+keeps another pack in acceptance, which only the person decides. A node's own
+criterion with the same id wins and counts for acceptance (that is what
+`kaloko health adopt` writes). Pack checks do not count as drift in
+`--coverage`; the plan lists only what the brief asks for. Use packs for
+public pages and landings; task flows keep their own criteria.
 
 Set `page` on every step (or once on the scenario) so the usability pack
 asks the right questions:
 
 | `page` | for | asks |
 | --- | --- | --- |
-| `landing` | public and marketing pages | purpose in the first screen (`AC201`), exactly one primary call to action (`AC202`) |
-| `work` | app screens where people work: overview, canvas, detail, inbox item | the heading says what the screen is for (`AC205`) |
-| `list` | lists and tables: runs, projects, inbox | `AC205`, each item says what it is and how to open it (`AC207`) |
-| `settings` | settings, billing, tokens | `AC205`, each setting says what it changes and how it is saved (`AC208`) |
-| `form` | sign-up, sign-in, a form to fill in | `AC205`, how to send it and what happens next (`AC206`) |
-| `doc` | documentation, guides, terms | `AC201`, headings and short paragraphs (`AC209`) |
-
-Every type also gets `AC203` and `AC204`. Without `page` a step with the
+| `landing` | public and marketing pages | purpose in the first screen (`HC201`), exactly one primary call to action (`HC202`) |
+| `work` | app screens where people work: overview, canvas, detail, inbox item | the heading says what the screen is for (`HC205`) |
+| `list` | lists and tables: runs, projects, inbox | `HC205`, each item says what it is and how to open it (`HC207`) |
+| `settings` | settings, billing, tokens | `HC205`, each setting says what it changes and how it is saved (`HC208`) |
+| `form` | sign-up, sign-in, a form to fill in | `HC205`, how to send it and what happens next (`HC206`) |
+| `doc` | documentation, guides, terms | `HC201`, headings and short paragraphs (`HC209`) |
+
+Every type also gets `HC203` and `HC204`. Without `page` a step with the
 `seo` pack, or any step of a scenario allowed on production, is `landing`;
 everything else is `work`. Do not put `landing` on signed-in screens: "one
 primary call to action" is a marketing rule, and a work screen rightly offers
@@ -78,34 +85,37 @@ When a scenario walks several locales, a text check (`text_matches`,
 reads in one language only (it looks for diacritics and common Czech or
 English words; one plain word such as a brand name is left alone).
 
-- `a11y` (`AC3xx`): images with alt, links and buttons with a name, labelled
+- `a11y` (`HC3xx`): images with alt, links and buttons with a name, labelled
   fields, heading levels not skipped, WCAG AA text contrast, a `main`
-  landmark, unique ids (`AC301`–`AC307`, read from the page). After the
-  screenshot the walk also presses Tab through the page: `AC308` tab order
+  landmark, unique ids (`HC301`–`HC307`, read from the page). After the
+  screenshot the walk also presses Tab through the page: `HC308` tab order
   (hidden focus, big jumps back, positive `tabindex`, controls left out or
-  mouse-only), `AC309` visible focus with at least 3 : 1 contrast (each stop
-  screenshotted focused and unfocused), `AC310` no keyboard trap (a modal
-  dialog may keep focus), `AC311` a skip link or main content within three
-  stops, `AC312` names from Chrome's accessibility tree (what a screen
-  reader announces). `AC313` landmarks (one `main`, labelled repeats) and
-  `AC314` ARIA (unknown roles and attributes, references to missing ids,
+  mouse-only), `HC309` visible focus with at least 3 : 1 contrast (each stop
+  screenshotted focused and unfocused), `HC310` no keyboard trap (a modal
+  dialog may keep focus), `HC311` a skip link or main content within three
+  stops, `HC312` names from Chrome's accessibility tree (what a screen
+  reader announces). `HC313` landmarks (one `main`, labelled repeats) and
+  `HC314` ARIA (unknown roles and attributes, references to missing ids,
   `aria-hidden` over focusable content) are read from the page. The first
-  five focus stops are kept as screenshots and the canvas can draw the tab
-  order on the capture. The walk only moves focus: nothing is clicked or
+  five focus stops are kept as screenshots (step attachments of kind
+  `focus`, "Keyboard focus · HC309" on the canvas, linked to the check; the
+  Health tab shows them under `HC309` and in the tab order) and the canvas
+  can draw the tab order on the capture. The walk only moves focus: nothing is clicked or
   typed, and the page is left unfocused at the top. Stack steps walk the
   keyboard on their first five pages. Still a subset of WCAG 2.2 AA; it
   does not replace an audit with a screen reader.
-- `perf` (`AC4xx`): LCP ≤ 2.5 s, CLS ≤ 0.1, TTFB ≤ 800 ms, page weight ≤ 3 MB,
+- `perf` (`HC4xx`): LCP ≤ 2.5 s, CLS ≤ 0.1, TTFB ≤ 800 ms, page weight ≤ 3 MB,
   measured in this walk's browser since the page load (lab values: the
   machine and network of the walk matter; compare runs, not absolutes).
-  Override a limit with an own criterion of the same id, e.g.
+  Override a limit with an own criterion with the check's number as an AC
+  id (`AC401` for `HC401`), e.g.
   `{ id: AC401, text: LCP within 4 s, check: { type: deterministic, assert: perf_lcp, max: 4000 } }`.
-- `console` (`AC501`): the evaluator reads the step's console errors with its
+- `console` (`HC501`): the evaluator reads the step's console errors with its
   purpose and decides whether they touch what the step accepts (pass: only
   third-party noise; needs review: unsure; fail: relevant). Without errors it
   passes without a call. Keep `no_console_errors` where any error must fail.
 
-- `errors` (`AC6xx`) is for error pages. It checks:
+- `errors` (`HC6xx`) is for error pages. It checks:
   - a real 4xx/5xx status (a "soft 404" answering 200 fails);
   - a link to the home page and navigation or search;
   - no stack trace, SQL error or framework debug page;
@@ -120,7 +130,7 @@ step with `packs: [errors]`:
 - **404:** use `path: /no-such-page-{random}`. `{random}` becomes a new
   token on every walk, so no cache or an accidentally created page makes it
   pass. `kaloko draft` adds this step on its own.
-- **Exact status:** override AC601, for example
+- **Exact status:** override HC601 with your own AC601, for example
   `{ id: AC601, text: Answers 410, check: { type: deterministic, assert: http_status, value: 410 } }`.
   `http_status` also takes `min`/`max`; the default is 200–299.
 - **Every step:** each capture records the document's HTTP status. The
@@ -192,7 +202,8 @@ When the work has a repository and an issue number, make them visible:
 - scenario `id` carries the ticket (`TASK-1273-club-activation`, `WEB-12-checkout`); the CLI derives the issue number from it or from the branch (`feat/1273-…`) and links it to the repository's origin (`acme/shop#1273`);
 - set `issue: { number, repo?, url? }` explicitly when the id or branch cannot carry it (for example a Plane item);
 - the project is the repository name unless `project:` in `kaloko.config.yml` says otherwise; a single flow may sort itself elsewhere with `project: <slug>` in its scenario (optional). Projects are records of the organization: `kaloko projects` or MCP `list_projects` show the slugs to use; admins create them in Settings → Projects or assign a scenario in the overview, which wins over the scenario file;
-- the canvas header, the organization overview and `kaloko feedback` show `repo#number · title`, so name the scenario `title` for humans, not for the ticket.
+- the canvas header, the organization overview and `kaloko feedback` show `repo#number · title`, so name the scenario `title` for humans, not for the ticket;
+- requirement IDs from the team's tracker or QMS go into `refs:` on the scenario, a step or a criterion (`refs: [URS-12, FS-3.4]`, free text, never invented); they flow into the acceptance record and the trace matrix ([evidence.md](evidence.md)).
 
 ## Session and consent
 

package/skills/kaloko/reference/apps.md

@@ -216,7 +216,7 @@ dialog, webview and group.
   - `app_no_truncation`: texts ending in "…";
   - `app_no_crash`: FATAL EXCEPTION or ANR in logcat, a "keeps stopping"
     dialog, or the iOS app no longer running.
-- **Pack `app` (AC7xx):** those four checks, plus a semantic question about
+- **Pack `app` (HC7xx):** those four checks, plus a semantic question about
   the screen's purpose and next action.
 - **Web-only checks** (`no_overflow`, `http_status`, meta, `a11y_contrast`,
   `perf_*`, `url_*`, `style_*`, `js_equals`): these are "not evaluated" on

package/skills/kaloko/reference/design.md

@@ -353,7 +353,8 @@ Turn on `packs: [design]` (usually `[design, a11y]`) for the steps in the implem
 | AC802 | structure: headings, landmarks, actions and form fields as in the reference |
 | AC803 | tokens: authored colors, fonts, sizes, spacing and radii come from the token set |
 
-Contrast and the other accessibility basics come from the `a11y` pack: `packs: [design, a11y]`.
+The design checks count for acceptance. Contrast and the other accessibility basics come from the `a11y` pack
+(`packs: [design, a11y]`) as health: advisory, reported apart (see [evaluation.md#health](evaluation.md#health)).
 
 `kaloko evaluate` fetches the design reference (of the design branch linked to the git branch, if any) and
 fills the checks. Findings name what differs and the token to use; fix the implementation, not the reference.

package/skills/kaloko/reference/docs.md

@@ -14,6 +14,7 @@ A picture in Markdown names a step instead of a PNG file:
 ![Payment](kaloko:checkout/payment?version=prev)          the release before the one this entry belongs to
 ![Payment](kaloko:checkout/payment?locale=cs&viewport=mobile&state=dark)
 ![Payment](kaloko:checkout/payment?media=video)           the step's recording plays on the picture
+![Pay](kaloko:checkout/payment#pay-button)                a named detail of the step (details: in the scenario)
 ![Payment](kaloko:payment)                                a step of the product's default_scenario
 ```
 
@@ -28,6 +29,29 @@ a dialog, a language switch), not for every step. Check a reference with MCP `re
 `GET /api/v1/orgs/<org>/steps/resolve?ref=…`: it answers the signed image URL (and `video` with `mp4`, `webm` and `duration_ms` when it plays), the run and how the version was found
 (`release`, `app_version`, `reference_env`, `accepted`; `exact: false` means a fallback).
 
+## Details for guides
+
+When a guide talks about one control, show that control in its part of the page instead of the whole screen. Declare
+it on the step in the scenario and refer to it with `#<id>`:
+
+```yaml
+details:
+  - id: pay-button                                   # letters, digits, - and _; used in references and file names
+    name: { en: Pay button, cs: Tlačítko Zaplatit }  # one text or one per locale
+    selector: "#pay"                                 # the first match is outlined
+    context: card                                    # optional: section | card | page | a CSS selector of an ancestor
+    padding: 24                                      # optional, px around the context (default 16)
+    states: [error]                                  # optional: only in these states of the step
+```
+
+Without `context` the picture is the nearest section, landmark or card around the element that is at least
+480 × 240 px (90 % of the width on a phone), at most one screen around the element, with a ring drawn around the
+element. Every walk takes each detail in every locale and viewport, masked like the screenshot, as an attachment of
+kind `detail` (canvas: Page tab → Step attachments → Details for guides, with the reference). Only screen steps take
+details. A reference to a detail no accepted run has yet shows the whole screen; `kaloko docs check` warns under
+`missing_details` (the scenario does not declare it) and `no_detail` (no accepted run has it: walk and accept).
+Prefer a detail over a crop when the text names one button, field or message; keep the whole screen for orientation.
+
 Never invent a step id or a version. Use ids from the scenario files and versions from `CHANGELOG.md` or git tags.
 
 ## After a user-visible change

package/skills/kaloko/reference/evaluation.md

@@ -68,6 +68,46 @@ evaluators:
 Report the reason with each `needs_review` or failing criterion; when the
 evaluator saw the screenshot, say so.
 
+## Health
+
+The checks of packs (`seo`, `a11y`, `perf`, `usability`, `console`,
+`errors`, `app`) are the run's **health** (CS *kondice*): advisory, stored in
+`node.health` and `summary.health` of `run.json`, never in `criteria`. They
+do not change a step's status, the run verdict, the review queue, carried
+approvals, required approvers, the acceptance record or exit codes. The
+`design` pack and packs listed in `health.acceptance` stay acceptance
+criteria. Runs shared before the split keep their stored verdicts; their pack
+criteria show as health.
+
+- **Check it before you hand over.** `kaloko walk` and `kaloko evaluate`
+  print one health line; `kaloko health` lists the findings by pack with
+  severity (error: people or search engines are shut out; warning: the rest;
+  info: the evaluator was unsure), what is new or fixed since the previous run
+  of the scenario, and a fix for each check. `--json` for the structure,
+  `--remote` (or MCP `get_health`) for a shared run.
+- **Fix what your change can fix**, walk the step again and check that the
+  finding is gone. Leave the rest and say so.
+- **Never hide health.** Do not remove a pack, raise a limit, narrow a
+  selector or add `environments:` to make a finding disappear. A limit or a
+  check that does not fit the project is the person's call; name it in the
+  report.
+- **Ids.** Health checks are `HC<n>` (`home:HC103`), acceptance criteria
+  `AC<n>`. Runs from before 7 Oct 2026 say `AC103` for the health check;
+  read it as `HC103`, and both ids work in commands and MCP.
+- **Adopting is the team's decision.** `kaloko health adopt <step>:<check>
+  [--text "…"]` writes the check into the scenario as an acceptance
+  criterion of the step (`home:HC103` becomes `AC103` with `from: HC103`),
+  so it counts for acceptance from the next walk. Do it only when
+  the person asks. Checks people adopted on the canvas (or through
+  `adopt_health_check`) wait on the service until `kaloko health pull` writes
+  them into the scenario; pulling them is routine, then walk those steps.
+- **Report it apart from the verdicts**: "Health: 2 errors (1 new), 5
+  warnings; fixed HC108 on home". Never present a health finding as a
+  failing criterion, or a clean health as acceptance.
+- `kaloko health --fail-on error` exits 1 on errors; use it in CI only where
+  the team asked for that gate. Hosted runs and CI runs alert on new errors
+  by themselves (inbox, mail, channels, webhook `health.worse`).
+
 ## Triage findings (experimental)
 
 When the organization switched the experiment on, `kaloko evaluate` also asks
@@ -111,7 +151,8 @@ turns them off; they do not count against the hosted allowance.
 
 After `kaloko build`, report: the summary line, each failing or
 `needs_review` criterion with step, locale, viewport and evidence, blocked
-requests if any, and the preview path. Do not claim a criterion passed when
+requests if any, the health counts (apart from the verdicts), and the preview
+path. Do not claim a criterion passed when
 it was not evaluated.
 
 `needs_review` is not a failure: look at the screenshot and state your

package/skills/kaloko/reference/evidence.md

@@ -0,0 +1,82 @@
+# Quality evidence for your own documentation
+
+Use this when a team's own repository turns Kaloko runs into its documents: test protocols, reports, trace
+matrices, release summaries for its quality management system (ISO 9001, ISO 13485, IEC 62304, ISO 27001, or GxP
+under 21 CFR Part 11 and EU GMP Annex 11, the strictest case). Kaloko provides the evidence and a way to check it;
+the team's own templates and skills write the documents. Kaloko never formats anything for a particular QMS, and you do not put
+the team's SOP structure into Kaloko.
+
+The background for people (what a record proves, the two signing paths, known gaps) is in Kaloko's help, page
+*Quality evidence for your QMS* (it maps the frameworks and has a GxP section). Evidence is a Business feature.
+Kaloko provides evidence; it never makes a team compliant or certified, so never write that it does.
+
+## The contract
+
+Everything below is `qawalk.evidence.v1` (`schemas/evidence.v1.json` in the `kaloko` package, every field
+described). Within v1 fields are only added; read the fields you need and ignore the rest.
+
+- `acceptance`: `none`, `accepted` or `returned`. Write your document from this field, never from the presence of
+  others.
+- `signing`: `in_kaloko` when a signature made in Kaloko stands, `none` otherwise. With `none` the approval, if any,
+  is signed elsewhere; your document says so ("executed in Kaloko; approval recorded in <your eQMS>").
+- `record.sha256` and `record.kind` (`acceptance` or `execution`): the signed record the evidence came from. Put the
+  hash into your document so a reader can tie it to the package.
+- `scenario` (`version`, `file`, `sha256`, `refs`), `environment`, `git`, `tools` (CLI, Node.js, OS, browsers,
+  service version).
+- `steps[].criteria[]`: the criterion text as walked, `refs`, and `results[]` per locale and viewport with
+  `assessed_by` (`person`, `deterministic`, `ai`, `none`) and `assessor` (an e-mail or a model id). `advisory: true`
+  means an AI result no person decided: never present it as a person's verdict.
+- `steps[].decision`, `verdict`, `approvals`: people's decisions apart from tokens' and agents'
+  (`counts_as_person`).
+- `signatures[]` (name, e-mail, `signed_at`, `time_zone`, `local_time`, `meaning`, `auth`, `withdrawn`) and
+  `decision_changes[]` with reasons.
+- `audit[]`: the run's audit trail; `trace[]`: the trace matrix, one row per requirement, criterion, step, locale and
+  viewport (columns in `TRACE_COLUMNS` order, the same as `trace.csv`).
+- A release (`kind: release`) has `runs[]` (each a run evidence), `missing[]` with reasons and one `trace[]`.
+
+## Getting it
+
+```bash
+npx kaloko evidence <run-id> --json > evidence.json            # one run, any shared run
+npx kaloko evidence --release 2.4.0 --product lims --json > release.json
+npx kaloko evidence <run-id> --csv > trace.csv                 # the trace matrix only
+npx kaloko signoff <run-id> --package --out evidence/          # the signed zip, checked on download
+npx kaloko signoff --verify evidence/kaloko-evidence-<run-id>.zip   # check it again later, no token needed
+```
+
+Over the API with a read token: `GET /api/v1/orgs/<org>/runs/<id>/evidence`, `…/evidence.zip`, `…/trace.csv`,
+`…/audit.csv`, and `GET /api/v1/orgs/<org>/evidence?product=<p>&version=<v>`. Over MCP: `get_evidence` with `run_id`, or
+`product` and `version`.
+
+## When it is ready
+
+An organization admin adds a webhook with the event `evidence.ready` (Settings → Integrations → Webhooks). It fires
+when a run is shared, accepted or returned, and when a release is recorded or updated. It is signed like every Kaloko
+webhook (`Kaloko-Signature: t=<unix>,v1=<hex HMAC-SHA256 of "<t>.<body>">`); check the signature and the age of `t`
+before you trust it. The body:
+
+```json
+{ "id": "evt_…", "type": "evidence.ready", "created_at": "2026-10-07T10:00:00.000Z", "org": "acme",
+  "data": { "kind": "run", "reason": "accepted", "run": { "id": "2026-10-07-batch-x1y2z3", "scenario": "BATCH", "environment": "validation", "project": "lims", "verdict": "accepted" },
+            "acceptance": "accepted", "signing": "none", "schema": "qawalk.evidence.v1",
+            "evidence_url": "https://kaloko.app/api/v1/orgs/acme/runs/…/evidence", "package_url": "…/evidence.zip", "trace_csv_url": "…/trace.csv" } }
+```
+
+The URLs need a token (`Authorization: Bearer …`); the webhook never carries the evidence itself.
+
+A neutral client-side job, for example a small service or a CI job triggered by the webhook:
+
+1. Check `Kaloko-Signature` with the webhook's secret; drop events older than five minutes.
+2. Download `package_url` with a read token into your own storage.
+3. Run `npx kaloko signoff --verify <zip>`; stop and alert on any failure.
+4. Read `evidence.json` from the zip and hand it to your generator with your templates.
+5. Store the generated document next to the package and the record's SHA-256; your eQMS takes it from there.
+
+## Rules for an agent
+
+- Never accept, approve or sign for a person. Tokens and agents cannot sign; where an organization signs in Kaloko,
+  only a person does, in the app.
+- Never present `advisory` results as decisions, and never drop the AI model name from a result.
+- Do not turn the signature policy on or off; that is an admin's decision in Settings → Security.
+- Add `refs:` to scenarios, steps and criteria when the team gives you requirement IDs; do not invent them.
+- Keep the package as you got it. To change anything, generate a new document; the package stays the evidence.

package/skills/kaloko/reference/first-run.md

@@ -24,11 +24,17 @@ with a person who knows where to approve it.
 ## From zero to a first shared run
 
 1. `npm install --save-dev kaloko`, then
-   `npx kaloko init --agent claude --org <org>` (it prints the sample link
-   and these steps). Fill `environments` in `kaloko.config.yml` with the URL
-   the app runs on.
+   `npx kaloko init --agent claude --org <org> --project <slug> --project-name "<name>"`
+   (it prints the sample link and these steps). Ask the person how the project
+   should be called on Kaloko when you do not know; the slug is lowercase
+   (`eco-podcasty`), the name is what people see (`ECO Podcasty`). Fill
+   `environments` in `kaloko.config.yml` with the URL the app runs on. When
+   the repository holds several sites, see [Several sites in one
+   repository](#several-sites-in-one-repository) first.
 2. `npx kaloko login`: the person approves a token in the browser. Ask them
-   to; never create or paste tokens yourself.
+   to; never create or paste tokens yourself. Then `npx kaloko projects sync`
+   puts the project of the config on Kaloko with its name (a first
+   `kaloko share` would create it too).
 3. `npx kaloko doctor` and fix what it marks ✕.
 4. Draft the first scenario:
    - public pages: `npx kaloko draft <url> [url…]` writes a read-only
@@ -49,3 +55,30 @@ with a person who knows where to approve it.
 Done means: the scenario validates, the run is shared, its URL and summary
 line are in your reply (and in the issue or PR the work belongs to), and the
 person knows whether the sample or this run is what they should open.
+
+## Several sites in one repository
+
+A monorepo (`apps/*` with their own dev server), a Rails app with engines or
+several Folio sites, or routes split by host: `kaloko init` lists the sites
+it found and drafts no steps, because routes of the whole repository mix the
+sites (favicons, ad endpoints and API routes of another site end up as
+"pages").
+
+1. Ask the person which site the work is about before `init` or a walk. Do
+   not pick one yourself.
+2. Run init for that site, best against the running site so the pages come
+   from its sitemap or its navigation:
+   - `npx kaloko init --site eco-podcasty --base-url http://eco-podcasty.localhost:3000`
+   - `npx kaloko init --site apps/web` (a workspace app: its own routes and dev
+     server; `--base-url` still reads the running site when it is up)
+   Only the host of `--base-url` is asked, with short time limits; offline the
+   code is read.
+3. Read the comment at the top of `qa/flows/main-pages.yml`: it says where the
+   pages came from and which paths were skipped and why. Sharpen the steps;
+   add pages the person names.
+4. When the site is not running and has no routes of its own (a Folio site),
+   `npx kaloko init --site <name> --skeleton` writes a scenario with a TODO;
+   fill it with the pages the person names.
+
+One config can serve one site. For a second site of the same repository use
+another config (`--config kaloko.<site>.config.yml`) with its own `project:`.

package/skills/kaloko/reference/integrations.md

@@ -34,7 +34,7 @@ Ask before filing issues in bulk; one issue per real problem, not per criterion.
 
 Slack and Teams channels are chosen per project by an admin, with the events
 each wants (new run, returned, accepted, comment, mention, release recorded,
-waiting for approval) and quiet hours. Nothing to do from the CLI: sharing,
+waiting for approval, health got worse) and quiet hours. Nothing to do from the CLI: sharing,
 returning and recording a release reach the right channels. Restricted projects
 reach only channels that name them.
 
@@ -72,6 +72,11 @@ kaloko schedule hook <id>       # URL + secret that start the run after a deploy
   script that only calls `page.goto`). Steps that click or type are left out and
   named in the answer. Criteria decided: `exists`, `absent`, `count`, `text_*`,
   `attr_*`, `title_matches`, `meta_matches`; the rest shows "not evaluated".
+- Health alerts: a hosted run (and a run shared from CI) compares its health
+  with the previous run of the scenario and alerts the followers, the channels
+  and the `health.worse` webhook on a new error-level finding. Each schedule
+  sets it to `error` (default), `warning` or `off` on kaloko.app or with
+  `kaloko schedule add --health-alert`; changing it is the person's call.
 - Does not work for: native apps, local or private addresses (localhost, IPs,
   `.local`), header credentials, base URLs per locale.
 - For those, or for the full walk on a schedule, write a GitHub Actions

package/skills/kaloko/reference/mcp.md

@@ -32,6 +32,7 @@ is null. Projects and products come complete, without pages.
 | `get_feedback` | run verdict with note, approved steps, unresolved rejections and comments, failed manual criteria (`run_id`, or `scenario` for its latest run) |
 | `list_notifications` | the token owner's inbox (`unread_only`, `limit`, `cursor`) |
 | `list_projects` | projects of the organization with counts; the slug is the `project` filter of `list_runs` and the value for `project:` in a scenario |
+| `upsert_project` | creates a project (`slug`, `name`, `visibility`, `product`; `create: true` refuses an existing one) or renames it; the same rules as `kaloko projects create\|rename\|sync` (a full token; renaming is for the project's admin). Ask before renaming |
 | `get_criteria_verdicts` | reviewer verdicts on manual or overridden criteria of a run |
 | `get_stack_items` | the pages of a stack step (`run_id`, `step`; filters `status`, `criterion`, text): path, URL, title, status and failing criteria with evidence, screenshot URLs |
 | `get_step_recording` | a step's recording and trace (`run_id`, `step`, `locale`, `viewport`, `trace: true` for the full trace): video URLs, duration, its chapter of a flow recording, failed attempts, the trace summary (failed requests, console errors, slowest request); see [recording.md](recording.md). `get_run_summary` lists trace summaries per step too |
@@ -42,13 +43,15 @@ is null. Projects and products come complete, without pages.
 | `list_products` | the products the caller reads: name, modules, the projects it maps that the caller sees, the newest release, published docs (versions, languages), who reads it (and the public link), whether the caller manages it, links to its page, changelog and docs |
 | `get_changelog` | the visual changelog of a product (`product`, `version`, `resolve: true` for the pictures, `limit`): releases newest first with the entry as written and its groups; see [docs.md](docs.md) |
 | `draft_changelog_entry` | what accepted runs since the last release (or `since`) show differently, as a Keep a Changelog draft with before/after step references (`product`, `since`) |
-| `resolve_step_image` | one step reference (`ref` = `kaloko:<scenario>/<step>?…`, `product`, `version`, `locale`, `viewport`) as a signed image URL with its run, how the version was found and canvas links; `video` (mp4, webm, duration_ms) when the step's recording plays (`media=video` / `media=image` in the ref) |
+| `resolve_step_image` | one step reference (`ref` = `kaloko:<scenario>/<step>?…[#<detail>]`, `product`, `version`, `locale`, `viewport`, `detail`) as a signed image URL with its run, how the version was found and canvas links; `video` (mp4, webm, duration_ms) when the step's recording plays (`media=video` / `media=image` in the ref); with a detail id the detail's picture and `detail` { id, name, box, size }, or the whole screen with `detail_missing` |
 | `get_docs` | a product's published docs (`product`; without `path` the versions, languages and sidebar; with `path` one page with its Markdown, sections, stale sections and, with `resolve: true`, its pictures) |
 | `docs_check` | what needs attention in the published docs of a version: broken step references, references without an accepted picture, stale sections, missing or outdated translations, release items without a picture |
 | `list_step_changes` | documentation layer: steps whose capture or status changed since the previous accepted run or `since` (`scenario`, `since`, `accepted`); see [documentation.md](documentation.md) |
 | `get_started` | where the organization stands (`empty`, `sample`, `active`), its sample run, the next steps and a prompt; call it in a new project, see [first-run.md](first-run.md) |
 | `add_sample_run` | adds the sample run (or returns the one there is); needs an upload or full token and the Developer / Tester role |
 | `search_screens` | the screen library: newest screenshot of every step across the organization's flows (`q` words matched against titles, ids, paths, URLs, page title and H1; `project`, `product`, `module`, `scenario`, `locale`, `viewport`, `device`, `status`, `environment`; `source: design` or `all` for design versions; `group: none` for every locale and viewport; `cursor` from `next`), each with a signed image and the canvas link |
+| `get_health` | the advisory health of a run (`run_id`), the newest run of a scenario (`scenario`) or the trend of a project (`project`): findings by severity, pack and step with evidence and a fix, and what is new or fixed since the previous run; never a verdict, see [evaluation.md#health](evaluation.md#health) |
+| `adopt_health_check` | turns a health check into an acceptance criterion of the scenario (`run_id`, `step`, `check` such as `HC103`, `text` to reword it; it becomes `AC103` with `from: HC103`); Developer / Tester role. Only when the person asks; `kaloko health pull` then writes it into the scenario file |
 | `get_affected_steps` | which steps of a scenario changed files reach, with the reason per step (`scenario`, `changed_files`, `run_id`, `upstream`); route files are matched against the step paths and URLs of the newest run. Imports and `affects:` globs need the repository: `kaloko affected` follows them |
 
 ## Comment tools
@@ -63,6 +66,7 @@ is null. Projects and products come complete, without pages.
 | `answer_finding` | answers a finding (`run_id`, `node`, `locale`, `viewport`, `finding`, `kind`, `answer`: right, wrong or comment with the `comment_id` from `add_comment`); scope comment |
 | `get_triage_report` | the organization's Clef numbers of the last 90 days (read-only): findings shown and answered, precision and calibration per kind, findings turned into comments, median hours to the first review with and without findings, triage requests per useful finding. Report them as they are; the go/no-go is a person's |
 | `get_signoff_record` | the signed acceptance record of an accepted run (Business): approvals with provenance, criteria, capture hashes, signature |
+| `get_evidence` | the evidence of a run (any shared run) or a release (Business, `qawalk.evidence.v1`): acceptance and signing state, criteria as walked with `refs`, who assessed each result (AI advisory), screenshots with SHA-256, decisions, signatures, audit trail, trace matrix ([evidence.md](evidence.md)) |
 | `resolve_comment` | close a thread with a `note` that says what changed (it becomes the last reply) |
 | `reopen_comment`, `edit_comment`, `delete_comment` | reopen; edit or delete your own comment |
 | `create_issue` | a Jira or Linear issue from a step (`run_id`, `step`, `to`: jira or linear, optional `criterion`, `title`, `note`, `locale`, `viewport`, `target`): screenshot, criterion, notes and a link back; the step's thread resolves when the issue is done. Developer / Tester, scope comment; see [integrations.md](integrations.md) |

package/skills/kaloko/reference/stacks.md

@@ -29,7 +29,7 @@ step script only to prepare each page, for example to dismiss a dialog
   kind: stack
   title: Magazine articles
   purpose: Every article reads well and is findable.
-  packs: [seo]                              # the pack criteria apply to every page
+  packs: [seo]                              # health checks of every page (advisory)
   items:
     sitemap: /sitemap.xml                   # or urls / file / crawl / script (exactly one)
     match: '^/clanky/'                      # regex on the path with its query; exclude: '…' drops pages