kaloko 0.3.0

package/README.md

@@ -0,0 +1,119 @@
+# Kaloko
+
+Acceptance QA walkthroughs rendered as a shareable flow canvas.
+
+Your coding agent (Claude Code, Codex or any LLM with a shell) runs QA from
+your acceptance plan with the `kaloko` skill. Kaloko captures every screen and
+e-mail of the flow, checks the criteria and builds a canvas that shows the
+whole flow (screens, branches, locales, evidence) for review, sign-off and
+living documentation.
+
+- Product and guides: <https://kaloko.app>
+- Service (organizations, shared canvases, reviews): <https://kaloko.app>
+
+Kaloko also walks **native apps**: Android over adb, the iOS Simulator,
+Electron and macOS apps. Screens taken by any other tool (Maestro, Appium,
+XCUITest) are imported with their UI tree. See the `apps` reference of the
+skill.
+
+## Install
+
+Requires Node 20 or newer.
+
+```bash
+npm install --save-dev kaloko
+npx kaloko init --agent claude --org <your-org>   # or --agent codex | all
+npx kaloko doctor
+```
+
+`init` creates `kaloko.config.yml` and copies the `kaloko` skill to
+`.claude/skills/kaloko` (Claude Code) or `skills/kaloko` (Codex). Run it again
+after `npm update kaloko` to refresh the skill.
+
+## Then ask your agent
+
+With the skill installed you do not type the commands below; the agent does.
+It writes the acceptance plan and the scenario from the task, walks the
+screens, evaluates, shares the canvas and later processes reviewers' feedback.
+For example:
+
+> Read the task in docs/tasks/TASK-123, write its acceptance plan and a Kaloko
+> scenario, walk it on local and share the result on the pull request.
+
+> Go through the open Kaloko feedback on TASK-123: fix what reviewers
+> rejected, run the affected steps again, share, and answer each comment with
+> what changed.
+
+> Draft a Kaloko scenario for https://example.com (home, pricing, contact)
+> with the seo, a11y and perf packs, run it on production read-only and
+> summarise what fails.
+
+## Run a scenario by hand (or in CI)
+
+```bash
+npx kaloko validate qa/flows/checkout.yml
+npx kaloko start --scenario qa/flows/checkout.yml --env local
+npx kaloko browser start
+npx kaloko walk                       # playwright steps
+npx kaloko capture --step <id>        # agent or manual steps
+npx kaloko evaluate                   # deterministic checks + semantic evaluator
+npx kaloko preview                    # local canvas
+npx kaloko share --pr                 # upload to kaloko.app, comment on the PR
+```
+
+`npx kaloko help` lists every command. When something does not work (Chrome,
+token, environment URL, evaluator key), `npx kaloko doctor` says what to fix.
+
+## Design (Kaloko Design)
+
+Design the screens of a flow as live HTML with your own agent and skills, in the same scenario development and
+QA use. A design environment (`kind: design`, a folder of steps) is walked and synced into a shared draft the
+team sees live; `kaloko share` there publishes a version of the flow (unchanged steps are stored once). An
+accepted version becomes the design reference, and `packs: [design]` checks every implementation run against
+it (pixels, structure, tokens, contrast). See `skills/kaloko/reference/design.md`, `docs/GLOSSARY.md` and
+`docs/SURFACES.md`.
+
+```bash
+npx kaloko start --scenario qa/flows/checkout.yml --env design
+npx kaloko walk && npx kaloko sync        # only changed steps render and travel
+npx kaloko share --name "Checkout v2"     # a version of the whole flow
+npx kaloko reference --design --version 2 # after the team accepted it
+```
+
+## Configuration and secrets
+
+Project configuration is `kaloko.config.yml` in the repository root:
+environments, readonly rules for production, browser, mailbox and evaluator.
+`npx kaloko init` writes a commented template.
+
+Secrets are never written in the config. Wherever a secret is needed, the
+config names an environment variable of your choice (`password_env:
+STAGING_PASSWORD`, `{ env: CF_ACCESS_CLIENT_SECRET }`). Values are read from the
+environment, then the project `.env` next to `kaloko.config.yml`, then
+`~/.config/kaloko/.env` (existing variables win), and are redacted from
+captured HTML, page text and console output.
+
+- **Protected environments** (staging, review apps): `access` sends HTTP Basic
+  credentials, service-token headers (Cloudflare Access, Vercel protection
+  bypass, your own) or a client certificate, to the environment's own origins
+  only. `base_url` may use `${VAR}` for review-app URLs from CI.
+- **Signed-in users**: `accounts` maps the scenario sessions' `account` keys to
+  credentials per environment (user, password, TOTP). For SSO or MFA a person
+  signs in once with `npx kaloko auth save --env <env> --account <key>`.
+- **E-mails**: built-in Mailpit client; any other mailbox through a small
+  adapter module (`provider: script`, example in `templates/mail/`), or hand a
+  message over with `npx kaloko mail --step <id> --file message.eml`.
+- **Service and evaluator**: `KALOKO_TOKEN` for `share`, `feedback`, `inbox` and clean-up (`runs`, `trash`, `prune`, `keep`, `archive`)
+  (create it in your organization's settings); `TYPESAFE_API_KEY` for the
+  semantic evaluator (optional). Both names can be changed in the config.
+
+`npx kaloko doctor` checks that every variable the config names is set and that
+each environment lets you in.
+
+Set `browser.channel: chrome` to use an installed Chrome, or run
+`npx playwright install chromium`.
+
+## License
+
+Proprietary. Use of Kaloko is subject to the terms at
+<https://kaloko.app/terms>.

package/bin/kaloko.mjs

@@ -0,0 +1,4 @@
+#!/usr/bin/env node
+import { main } from '../src/cli.mjs'
+
+await main(process.argv.slice(2))

package/bin/qawalk.mjs

@@ -0,0 +1,4 @@
+#!/usr/bin/env node
+import { main } from '../src/cli.mjs'
+
+await main(process.argv.slice(2))

package/package.json

@@ -0,0 +1,42 @@
+{
+  "name": "kaloko",
+  "version": "0.3.0",
+  "description": "See what your AI agents built: every screen, language and device walked, assessed and approved on one canvas.",
+  "keywords": [
+    "qa",
+    "acceptance-testing",
+    "playwright",
+    "claude-code",
+    "codex",
+    "agent-skills"
+  ],
+  "homepage": "https://kaloko.app",
+  "type": "module",
+  "license": "UNLICENSED",
+  "bin": {
+    "kaloko": "bin/kaloko.mjs",
+    "qawalk": "bin/qawalk.mjs"
+  },
+  "files": [
+    "bin",
+    "src",
+    "schemas",
+    "templates",
+    "viewer",
+    "skills/kaloko"
+  ],
+  "engines": {
+    "node": ">=20"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "test": "node --test --test-timeout=300000 test/*.test.mjs worker/test/*.test.mjs"
+  },
+  "dependencies": {
+    "ajv": "^8.20.0",
+    "js-yaml": "^4.1.0",
+    "playwright": "^1.55.0"
+  }
+}

package/schemas/bundle.v1.json

@@ -0,0 +1,19 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://qawalk.com/schemas/bundle.v1.json",
+  "title": "qawalk.bundle.v1",
+  "description": "The live files of one design step as the CLI and the service exchange them (#42). Paths are relative to the design folder: <step>/index.html (or <step>/<locale>/index.html), shared files in _shared/, the token set in tokens.json. Enforced by src/lib/design-core.mjs on both sides: at most 400 files, 10 MB per file, 40 MB per revision, 8 folders deep, 240 characters per path.",
+  "type": "object",
+  "required": ["entry", "files"],
+  "additionalProperties": false,
+  "properties": {
+    "entry": { "type": "string", "pattern": "\\.html$", "description": "The page of the step." },
+    "files": {
+      "type": "object",
+      "description": "Path → sha256 of the file content.",
+      "propertyNames": { "pattern": "^(?!/)(?!.*(?:^|/)\\.)[A-Za-z0-9_][A-Za-z0-9._-]*(?:/[A-Za-z0-9_][A-Za-z0-9._-]*)*\\.(html|css|js|mjs|json|svg|woff2|woff|png|jpg|jpeg|webp|avif|gif|mp4|webm|txt)$" },
+      "additionalProperties": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
+      "maxProperties": 400
+    }
+  }
+}

package/schemas/config.v1.json

@@ -0,0 +1,268 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://qawalk.com/schemas/config.v1.json",
+  "title": "Kaloko project configuration (kaloko.config.yml)",
+  "type": "object",
+  "required": ["schema", "org", "environments"],
+  "additionalProperties": false,
+  "properties": {
+    "schema": { "const": "qawalk.config.v1" },
+    "org": { "type": "string", "description": "Organization slug on kaloko.app." },
+    "project": { "type": "string" },
+    "scenarios": { "type": "array", "items": { "type": "string" }, "description": "Glob patterns of scenario files." },
+    "output": { "type": "string", "description": "Run folder; default tmp/kaloko (tmp/qawalk when only that one exists)" },
+    "browser": {
+      "type": "object",
+      "additionalProperties": false,
+      "properties": {
+        "channel": { "type": "string", "description": "Playwright channel, e.g. chrome. Empty = bundled Chromium." },
+        "port": { "type": "integer", "default": 9333 }
+      }
+    },
+    "environments": {
+      "type": "object",
+      "minProperties": 1,
+      "additionalProperties": { "$ref": "#/$defs/environment" }
+    },
+    "evaluators": {
+      "type": "object",
+      "additionalProperties": false,
+      "properties": {
+        "jev": {
+          "type": "object",
+          "additionalProperties": false,
+          "properties": {
+            "model": { "type": "string", "default": "jev-1.13.0" },
+            "api_key_env": { "type": "string", "default": "TYPESAFE_API_KEY" },
+            "base_url": { "type": "string", "default": "https://api.typesafe.ai" },
+            "thresholds": {
+              "type": "object",
+              "additionalProperties": false,
+              "properties": { "pass": { "type": "number" }, "fail": { "type": "number" } }
+            },
+            "max_state_chars": { "type": "integer", "default": 90000 },
+            "hosted": { "type": "boolean", "default": true, "description": "Without a local evaluator key, evaluate through the organization's service (plan allowance per month). false: only your own key." },
+            "signals": { "type": "boolean", "default": true, "description": "Quick signals on every screen (purpose met, expected language, looks broken); never change verdicts." },
+            "concurrency": { "type": "integer", "minimum": 1, "maximum": 16, "default": 6, "description": "Evaluator requests in flight at once." }
+          }
+        }
+      }
+    },
+    "share": {
+      "type": "object",
+      "additionalProperties": false,
+      "properties": {
+        "url": { "type": "string", "default": "https://kaloko.app" },
+        "token_env": { "type": "string", "default": "KALOKO_TOKEN" },
+        "expires_days": { "type": "integer", "default": 30 }
+      }
+    }
+  },
+  "$defs": {
+    "environment": {
+      "type": "object",
+      "additionalProperties": false,
+      "if": { "properties": { "kind": { "const": "design" } }, "required": ["kind"] },
+      "then": { "required": ["serve"] },
+      "else": { "anyOf": [{ "required": ["base_url"] }, { "required": ["app"] }] },
+      "properties": {
+        "kind": { "enum": ["app", "design"], "default": "app", "description": "app: the application under test (a web app at base_url, or a native app in the app block). design: a folder of live HTML steps served locally by the CLI (serve); its runs are published as design versions (#42)." },
+        "serve": { "type": "string", "description": "kind: design. Folder with one sub-folder per step (<step>/index.html, optional <step>/<locale>/index.html) and shared files in _shared/; relative to the config." },
+        "fixtures": { "type": "string", "description": "kind: design. Folder of JSON fixtures live steps may fetch instead of real APIs; relative to the config." },
+        "tokens_mode": { "type": "string", "description": "kind: design. Token mode (e.g. light, dark) steps are rendered in; default: the token set's first mode." },
+        "html_view": { "type": "boolean", "default": false, "description": "Experimental (#42): keep a static HTML view of every capture (assets rehosted, scripts removed, secrets scrubbed). Refused on production unless the environment is readonly." },
+        "base_url": {
+          "oneOf": [
+            { "type": "string" },
+            { "type": "object", "additionalProperties": { "type": "string" }, "description": "Per-locale base URL; key 'default' is the fallback." }
+          ],
+          "description": "Base URL, or per-locale base URLs (key \"default\" is the fallback). ${VAR} is replaced from the environment, e.g. https://${REVIEW_HOST} for review apps."
+        },
+        "readonly": { "type": "boolean", "default": false },
+        "app": {
+          "type": "object",
+          "additionalProperties": false,
+          "description": "The app under test in this environment, per platform (scenario platform: android, ios, electron, desktop). Paths of app scenarios are deep links.",
+          "properties": {
+            "android": {
+              "type": "object", "additionalProperties": false,
+              "properties": {
+                "package": { "type": "string", "description": "Application id, e.g. com.acme.shop.staging." },
+                "activity": { "type": "string", "description": "Launch activity (default: the launcher activity)." },
+                "apk": { "type": "string", "description": "APK to install before the walk (path relative to the repository)." },
+                "install": { "type": "boolean" },
+                "serial": { "type": "string", "description": "adb device serial; default ANDROID_SERIAL or the only device." },
+                "device": { "type": "string", "description": "Label for the canvas; default the device model." },
+                "reset": { "type": "boolean", "description": "Clear app data at the start of every locale (a new user)." },
+                "adb": { "type": "string" }, "launch_wait": { "type": "integer" }, "settle": { "type": "integer" }
+              }
+            },
+            "ios": {
+              "type": "object", "additionalProperties": false,
+              "properties": {
+                "bundle": { "type": "string", "description": "Bundle id, e.g. com.acme.shop." },
+                "app": { "type": "string", "description": ".app build for the simulator to install before the walk." },
+                "install": { "type": "boolean" },
+                "device": { "type": "string", "description": "Simulator name, e.g. iPhone 17; booted when needed." },
+                "udid": { "type": "string" },
+                "reset": { "type": "boolean" }, "args": { "type": "array", "items": { "type": "string" } },
+                "idb": { "type": "boolean", "description": "Use idb for the UI tree and taps when installed (default true)." },
+                "ocr": { "type": "boolean", "description": "Without idb, read the screen text by OCR (macOS Vision; default true)." },
+                "status_bar": { "type": "boolean", "description": "Clean status bar (9:41, full battery) for captures; default true." },
+                "scale": { "type": "integer" }, "launch_wait": { "type": "integer" }, "settle": { "type": "integer" }
+              }
+            },
+            "electron": {
+              "type": "object", "additionalProperties": false,
+              "properties": {
+                "main": { "type": "string", "description": "Main script of the app (e.g. dist/main.js), started with the project's electron." },
+                "executable": { "type": "string", "description": "Electron binary or the packaged app's executable." },
+                "args": { "type": "array", "items": { "type": "string" } },
+                "env": { "type": "object", "additionalProperties": { "type": "string" } },
+                "size": { "type": "array", "items": { "type": "integer" }, "minItems": 2, "maxItems": 2 }
+              }
+            },
+            "desktop": {
+              "type": "object", "additionalProperties": false,
+              "properties": {
+                "name": { "type": "string", "description": "The app as macOS shows it (window owner), e.g. Notes." },
+                "bundle": { "type": "string" }, "path": { "type": "string" },
+                "args": { "type": "array", "items": { "type": "string" } },
+                "launch": { "type": "boolean", "description": "Open the app at the start (default true)." },
+                "reset": { "type": "boolean" }, "ocr": { "type": "boolean" }, "launch_wait": { "type": "integer" }, "settle": { "type": "integer" }
+              }
+            }
+          }
+        },
+        "may_create_data": { "type": "boolean", "default": true },
+        "blocked_paths": { "type": "array", "items": { "type": "string" }, "description": "Path globs a readonly run must never open." },
+        "allowed_post_paths": { "type": "array", "items": { "type": "string" }, "description": "Path globs a readonly run may POST to (e.g. cart)." },
+        "http_auth": { "$ref": "#/$defs/basicAuth", "description": "Deprecated: use access.basic. Still read as access.basic." },
+        "mail": {
+          "type": "object",
+          "additionalProperties": false,
+          "properties": {
+            "provider": {
+              "enum": ["mailpit", "script", "none"],
+              "description": "mailpit: built-in REST client; script: a project adapter module (any other mailbox); none: capture e-mails with kaloko mail --file."
+            },
+            "url": { "type": "string" },
+            "user_env": { "type": "string", "pattern": "^[A-Za-z_][A-Za-z0-9_]*$", "description": "Deprecated: use auth.basic." },
+            "password_env": { "type": "string", "pattern": "^[A-Za-z_][A-Za-z0-9_]*$", "description": "Deprecated: use auth.basic." },
+            "address": { "type": "string", "description": "Recipient template, {run} is replaced by a short run id, e.g. qa+{run}@example.com." },
+            "script": {
+              "type": "string",
+              "description": "provider: script — module path relative to the config; default export async ({ to, subject, since, options, root, env }) => message | null."
+            },
+            "options": { "type": "object", "description": "Free-form options handed to the script adapter." },
+            "auth": {
+              "type": "object",
+              "additionalProperties": false,
+              "properties": { "basic": { "$ref": "#/$defs/basicAuth" }, "headers": { "$ref": "#/$defs/headers" } }
+            }
+          }
+        },
+        "access": {
+          "type": "object",
+          "additionalProperties": false,
+          "description": "Protection in front of the whole environment (staging, review apps). The CLI applies it to every browser context it creates, for the listed origins only.",
+          "properties": {
+            "origins": {
+              "type": "array",
+              "items": { "type": "string" },
+              "description": "Origins that receive the credentials; default: the base URL origins. https://*.example.com covers subdomains."
+            },
+            "basic": { "$ref": "#/$defs/basicAuth" },
+            "headers": {
+              "$ref": "#/$defs/headers",
+              "description": "Service-token headers, e.g. CF-Access-Client-Id / CF-Access-Client-Secret, x-vercel-protection-bypass."
+            },
+            "client_certificate": {
+              "type": "object",
+              "additionalProperties": false,
+              "description": "Mutual TLS. *_file is a path relative to the config; *_env holds a PEM or a path.",
+              "properties": {
+                "cert_file": { "type": "string" },
+                "key_file": { "type": "string" },
+                "cert_env": {
+                  "type": "string",
+                  "pattern": "^[A-Za-z_][A-Za-z0-9_]*$",
+                  "description": "Name of an environment variable (the value is never written in the config)."
+                },
+                "key_env": {
+                  "type": "string",
+                  "pattern": "^[A-Za-z_][A-Za-z0-9_]*$",
+                  "description": "Name of an environment variable (the value is never written in the config)."
+                },
+                "passphrase_env": {
+                  "type": "string",
+                  "pattern": "^[A-Za-z_][A-Za-z0-9_]*$",
+                  "description": "Name of an environment variable (the value is never written in the config)."
+                }
+              }
+            }
+          }
+        },
+        "accounts": {
+          "type": "object",
+          "description": "People who sign in to the application in this environment, keyed like the scenario sessions' `account`. Step scripts get them as session.credentials.",
+          "additionalProperties": {
+            "type": "object",
+            "additionalProperties": false,
+            "properties": {
+              "user": { "type": "string", "description": "Literal user name or e-mail (not secret)." },
+              "user_env": {
+                "type": "string",
+                "pattern": "^[A-Za-z_][A-Za-z0-9_]*$",
+                "description": "Name of an environment variable (the value is never written in the config)."
+              },
+              "password_env": {
+                "type": "string",
+                "pattern": "^[A-Za-z_][A-Za-z0-9_]*$",
+                "description": "Name of an environment variable (the value is never written in the config)."
+              },
+              "totp_env": { "type": "string", "pattern": "^[A-Za-z_][A-Za-z0-9_]*$", "description": "Base32 TOTP secret for MFA; scripts call session.credentials.totp()." },
+              "storage_state": { "type": "string", "description": "Saved sign-in (kaloko auth save); default <output>/.auth/<env>/<account>.json. Never commit it." }
+            }
+          }
+        }
+      }
+    },
+    "secretRef": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["env"],
+      "properties": {
+        "env": {
+          "type": "string",
+          "pattern": "^[A-Za-z_][A-Za-z0-9_]*$",
+          "description": "Name of an environment variable (the value is never written in the config)."
+        }
+      },
+      "description": "A value read from the environment variable `env`."
+    },
+    "basicAuth": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["user_env", "password_env"],
+      "properties": {
+        "user_env": {
+          "type": "string",
+          "pattern": "^[A-Za-z_][A-Za-z0-9_]*$",
+          "description": "Name of an environment variable (the value is never written in the config)."
+        },
+        "password_env": {
+          "type": "string",
+          "pattern": "^[A-Za-z_][A-Za-z0-9_]*$",
+          "description": "Name of an environment variable (the value is never written in the config)."
+        }
+      },
+      "description": "HTTP Basic credentials, both read from environment variables."
+    },
+    "headers": {
+      "type": "object",
+      "description": "Request headers; a string is a literal (non-secret) value, { env: NAME } reads a secret.",
+      "additionalProperties": { "oneOf": [{ "type": "string" }, { "$ref": "#/$defs/secretRef" }] }
+    }
+  }
+}

package/schemas/revision.v1.json

@@ -0,0 +1,36 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://qawalk.com/schemas/revision.v1.json",
+  "title": "qawalk.revision.v1",
+  "description": "A saved state of one step in one locale (and state) of a design draft (#42). The id is derived from the content (src/lib/design-core.mjs revisionId), so the same files make the same revision on every machine. Captures are renderings of a revision per viewport.",
+  "type": "object",
+  "required": ["id", "node", "locale", "entry", "files"],
+  "properties": {
+    "id": { "type": "string", "pattern": "^[0-9a-f]{32}$" },
+    "node": { "type": "string" },
+    "locale": { "type": "string" },
+    "state": { "type": "string", "default": "default", "description": "default, a design state (empty, error, …) or view:<viewport> for an HTML view." },
+    "kind": { "enum": ["design", "html_view"], "default": "design" },
+    "entry": { "type": "string" },
+    "files": { "$ref": "bundle.v1.json#/properties/files" },
+    "tokens": { "type": ["object", "null"], "properties": { "project": { "type": ["string", "null"] }, "version": { "type": ["integer", "null"] }, "hash": { "type": ["string", "null"] } } },
+    "based_on": { "type": ["string", "null"], "description": "The revision this one changed (a canvas edit, a sync on top of the draft)." },
+    "captures": {
+      "type": "object",
+      "description": "Viewport → rendered capture: blobs of the screenshot, thumbnail and DOM snapshot, the probe record and the evaluation.",
+      "additionalProperties": {
+        "type": "object",
+        "required": ["image", "html", "record"],
+        "properties": {
+          "image": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
+          "thumb": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
+          "html": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
+          "record": { "type": "object" },
+          "evaluation": { "type": ["object", "null"] },
+          "renderer": { "type": "string" },
+          "mode": { "type": "string" }
+        }
+      }
+    }
+  }
+}