cortad 0.2.0 → 0.2.2

package/README.md

@@ -1,24 +1,16 @@
 # cortad
 
-Connects the AI app on your machine to Cortad, so Cortad can run test conversations against it, and gives your coding agent the tools to run them.
+Runs Cortad's test conversations against the AI app on your machine, and gives your coding agent the tools to run them.
 
 ```
 npx cortad <code>
 ```
 
-Run it in your app's folder with the code from cortad.com. Nothing is installed in your project. Ctrl-C disconnects.
+Run it in your app's folder with the code from cortad.com. It starts your app, connects it to Cortad, and adds Cortad to Claude Code, Codex and Cursor on this machine. The first run starts on its own. Ctrl-C disconnects.
 
-## What it does
+## Your coding agent
 
-- Starts your app with its dev script, or uses it if it is already running.
-- Sends Cortad's test conversations to your app on localhost and returns the replies.
-- Signs in as a test account when your app needs one: an account made for the session, never an existing user and never an admin.
-- Makes Cortad known to the coding agents on this machine (Claude Code, Codex, Cursor): an MCP entry and a skill in each one's own home folder, so `npx cortad mcp` answers them from then on. Nothing is written into your repository.
-- Never writes your files. The agent that edits your code is your own.
-
-## For your coding agent
-
-After the first connect, your agent has eight tools: `status`, `run`, `run_status`, `findings`, `verify`, `dispute`, `field_connect`, `field`. The same eight work as shell commands:
+After that, your agent has eight tools: `status`, `run`, `run_status`, `findings`, `verify`, `dispute`, `field_connect`, `field`. The same eight work as commands:
 
 ```
 npx cortad status
@@ -27,26 +19,7 @@ npx cortad findings
 npx cortad verify <findingId>
 ```
 
-A run needs your app up. When nothing on this machine is holding it, the command starts it with the key the first connect left in `~/.cortad`, and leaves when no run has needed it for ten minutes.
-
-## What Cortad receives
-
-- The files git would commit, once. Nothing git ignores, and no env, key or data files.
-- Your app's replies to the test conversations, with values from your env files hidden and cookies removed.
-
-## What stays on your machine
-
-- Your env files and their values.
-- Tokens and cookies. A signed-in test request gets its token added here.
-- The key in `~/.cortad/<project>/token` (readable by you only). It starts runs and reads findings for this one repository and nothing else; revoke it from your account page.
-
-## The test shell
-
-Commands a run needs on this machine (your own test suite, a WebSocket door) are confined by the operating system (Seatbelt on macOS, bubblewrap on Linux). They can read your project and your toolchains, write only to temp and build folders, and reach only localhost.
-
-## Loaded into your app
-
-When cortad starts your app it preloads one short file, `lib/trace.cjs` for Node or `lib/pyhook/sitecustomize.py` for Python. It notes which request called a model, so Cortad finds your chat route. It writes only to cortad's temp folder.
+A run needs your app up. If nothing on this machine is holding it, the command starts it with the key in `~/.cortad` and stops it ten minutes after the last run.
 
 ## Flags
 
@@ -63,4 +36,4 @@ When cortad starts your app it preloads one short file, `lib/trace.cjs` for Node
 - `~/.cortad/identity.key`   the seed for the session's test accounts
 - `$TMPDIR/cortad-<pid>/`    removed on exit
 
-macOS and Linux, Node 20+. No dependencies, no install scripts.
+macOS and Linux, Node 20 or later. No dependencies. What is sent and what stays on your machine: `npx cortad --explain`, and cortad.com/privacy.

package/lib/cli.mjs

@@ -22,11 +22,11 @@ export async function main(argv, { root = process.cwd(), env = process.env, stdo
   if (!trusted) { stderr.write(`cortad  refusing: ${origin.host} is not Cortad.\n`); return 1; }
   const api = `${origin.origin}/api`;
   const project = projectOf(root);
-  const token = readToken(project);
   const [command, ...rest] = argv;
   const mcp = command === "mcp";
   const children = [];
-  const verbs = makeVerbs({ api, token, ensureRunner: () => ensureRunner({ root, project, env, keep: mcp, children, say: (line) => stderr.write(`cortad  ${line}\n`) }) });
+  // The key is read at every call: the MCP process outlives a connect that writes it after the process started.
+  const verbs = makeVerbs({ api, token: () => readToken(project), ensureRunner: () => ensureRunner({ root, project, env, keep: mcp, children, say: (line) => stderr.write(`cortad  ${line}\n`) }) });
 
   if (mcp) {
     const code = await serveMcp({ verbs, version: VERSION, log: (line) => stderr.write(`${line}\n`) });

package/lib/identities.mjs

@@ -1,4 +1,4 @@
-// Copied from brainsless-backend src/harness/identities.mjs. Keep the two in step: the recipes the
+// The same file lives on the Cortad side. Keep the two in step: the recipes the
 // cloud sends are gated here again, on the machine that holds the secrets.
 // Signing in as the customer's own callers, from inside the box, through their own doors.
 //

package/lib/mcp.mjs

@@ -5,7 +5,7 @@
 export const PROTOCOL = "2025-06-18";
 
 export const INSTRUCTIONS = [
-  "Cortad is an independent behavior test lab for the AI app in this repository. It runs simulated users through the app on this machine and grades every reply with about 100 checks. It is already connected; never run `npx cortad <code>` again.",
+  "Cortad is an independent behavior test lab for the AI app in this repository. It runs simulated users through the app on this machine and grades every reply with about 100 checks. It is already connected; never run `npx cortad <code>` again. The first run starts by itself after a connect; the skill says what to tell the person then.",
   "The loop: status, then run only when the person asks (the first run is free; after that run returns a checkout link, show it in one sentence and wait), poll run_status every 30 seconds and stay quiet unless the count moved, findings, fix one finding at a time in the repo, verify <id>, read the move.",
   "Rules: a move inside the interval is not a fix. Never change or remove a case to move a number; never make the app recognise test traffic; if a check reads wrong, call dispute with why. Cases, checks, seeds and the holdout are not yours to edit, and Cortad refuses it anyway.",
 ].join("\n\n");

package/lib/mint.mjs

@@ -137,7 +137,7 @@ export function makeIdentities({ root, work, envFiles, sourceFiles = () => [], s
     // nothing and says so; port the program to this file if that turns out to be common.
     // A Firebase caller with no emulator on this machine is signed in at their real project, below.
     // Handed to the program it waited its whole deadline on an emulator that was never there, and
-    // planless's sign-in arrived after the world had given up waiting for it.
+    // one app's sign-in arrived after the world had given up waiting for it.
     const merged = Object.assign({}, ...[...envs().all].reverse());
     const emulated = Boolean(merged.FIREBASE_AUTH_EMULATOR_HOST);
     const refused = new Map();

package/lib/replay.mjs

@@ -19,7 +19,7 @@ export function makeCapture({ work, keepSecret, onDoor }) {
   let held = null;
   let alive = false;
   // The ports a hooked process listens on. Loading is not listening: turbo's own Node launcher loads
-  // the hook, and databuddy was told we would see its messages while its API on Bun carried none.
+  // the hook, and one app was told we would see its messages while its API on Bun carried none.
   const ports = new Set();
   // Bun reads BUN_OPTIONS and splits it on spaces, quotes included, and only the `--preload=` form
   // leaves `bun run <script>` working. A hook path with a space in it is copied to one without.

package/lib/sample.mjs

@@ -20,7 +20,7 @@ const PER_STORE = 6;
 const PER_STORE_MAX = 60;
 // A few rows from MANY collections, never many rows from few: a store whose collection names are
 // the product's own grid (jordan_grade-10_sem-2_math) is only read when every cell is read. Forty
-// eight of them, twenty two rows apiece, filled the whole answer with seven of ulaim's cells and
+// eight of them, twenty two rows apiece, filled the whole answer with seven of one app's cells and
 // left grade 10 semester 2 maths unread, so the depth follows the breadth: how many rows a
 // collection gets is the answer's room divided by how many collections there are.
 const COLLECTIONS = 200;
@@ -122,7 +122,7 @@ async function qdrant(plan, names, env, per, until) {
   const each = Math.max(PER_MIN, Math.min(per, Math.floor(BUDGET / ROW_BYTES / Math.max(1, asking.length))));
   const rows = [];
   let read = 0;
-  // Where each collection that had rows left off. On ulaim a third of the 177 collections are
+  // Where each collection that had rows left off. On one app a third of the 177 collections are
   // empty, so the even split left half the answer unspent at three pages a cell.
   const more = new Map();
   const scroll = async (name, limit, offset) => {

package/lib/service.mjs

@@ -1,4 +1,4 @@
-// Some apps keep their sign-in on a different service than the one that serves the AI: databuddy
+// Some apps keep their sign-in on a different service than the one that serves the AI: one app
 // serves chat from apps/api and mounts better-auth in apps/dashboard, and the command starts only
 // apps/api. A recipe that names such a service (src/setup/road-identities.ts) is minted against
 // that service instead, so nobody is asked to send a message by hand. Cookies set on localhost
@@ -62,7 +62,7 @@ export async function waitForPort(port, deadlineMs = 90_000) {
 }
 
 // The page a sign-in believes it is being asked from. Nearly every auth library refuses a request
-// whose Origin it does not trust: better-auth answered databuddy's own sign-up with 403
+// whose Origin it does not trust: better-auth answered one app's own sign-up with 403
 // INVALID_ORIGIN because the ask went to 127.0.0.1 with no page behind it. Their own settings name
 // the page; where they do not, it is that port on localhost, which is what their own browser sends.
 export function originFor(port, origins = []) {

package/lib/trace.cjs

@@ -76,7 +76,7 @@ if (FILE) {
       return als.run(ctx, () => emit.call(this, type, req, ...rest));
     };
 
-    // Bun serves through Bun.serve, never node:http, so the patch above saw no request at all: databuddy's
+    // Bun serves through Bun.serve, never node:http, so the patch above saw no request at all: an
     // Elysia API on Bun took the customer's messages and nothing was written. Bun.serve is wrapped
     // instead, both the fetch handler and the per-route handlers Bun can dispatch to directly. The body
     // is read from a clone, so the app's own reader is untouched.
@@ -176,7 +176,7 @@ if (FILE) {
 
     // Outbound to a service their own settings name (a vector store, a database API, a search
     // host): which setting, which host and whether it answered. No body, no headers, no query. A
-    // run that tested ulaim while every curriculum lookup failed inside Node never knew; this is
+    // run that tested a tutoring app while every lookup failed inside Node never knew; this is
     // the witness that lets the run say retrieval was not tested instead of grading without it.
     let depHosts = null, depSeen = -1;
     const depOf = (input) => {

package/lib/verbs.mjs

@@ -6,12 +6,13 @@ const NOT_CONNECTED = "This project is not connected to Cortad yet. Ask the pers
 
 export function makeVerbs({ api, token, fetchImpl = fetch, ensureRunner = async () => ({ ok: true }) }) {
   const call = async (method, path, body) => {
-    if (!token) return { status: 0, ok: false, data: { error: NOT_CONNECTED } };
+    const key = typeof token === "function" ? token() : token;
+    if (!key) return { status: 0, ok: false, data: { error: NOT_CONNECTED } };
     let res;
     try {
       res = await fetchImpl(`${api}${path}`, {
         method,
-        headers: { authorization: `Bearer ${token}`, ...(body === undefined ? {} : { "content-type": "application/json" }) },
+        headers: { authorization: `Bearer ${key}`, ...(body === undefined ? {} : { "content-type": "application/json" }) },
         body: body === undefined ? undefined : JSON.stringify(body),
         signal: AbortSignal.timeout(60_000),
       });
@@ -108,6 +109,8 @@ export function runText(d) {
   const tail = [];
   if (d.finished && typeof d.score === "number") tail.push(`Score ${d.score} of 100.`);
   if (d.finished && typeof d.findings === "number") tail.push(d.findings ? `${plural(d.findings, "finding")}; call findings.` : "No findings stood out.");
+  if (d.stopped) tail.push(`Stopped after ${d.stopped.after} of ${d.of}: ${d.stopped.why}${d.stopped.side === "theirs" ? " (their side)" : d.stopped.side === "ours" ? " (Cortad's side)" : ""}. ${d.stopped.fix}`);
+  for (const f of d.faults ?? []) tail.push(`${f.what} ${f.fix}`);
   if (d.error) tail.push(`Error: ${d.error}`);
   if (d.verify) tail.push(verifyText(d.verify));
   if (!d.finished) tail.push("Poll again in 30 seconds.");

package/local.mjs

@@ -440,7 +440,7 @@ const onPath = (bin) => (process.env.PATH ?? "").split(":").some((dir) => dir &&
 let appDir = root;
 
 // ---- a sign-in mounted in another workspace
-// databuddy serves its AI from apps/api and mounts sign-in in apps/dashboard: a test account can
+// One app served its AI from apps/api and mounted sign-in in apps/dashboard: a test account can
 // only be made where the sign-in is, and posting a sign-up at the AI's own port is a 404.
 const PORT_IN_SCRIPT = /(?:^|\s)(?:PORT=|-p[ =]|--port[ =])(\d{2,5})\b/;
 // The port a framework serves on when nobody names one. sveltekit and astro before vite: both bring
@@ -684,7 +684,7 @@ async function start(waitMs) {
     const tail = () => seen.replace(/\x1b\[[0-9;]*m/g, "").split("\n").filter(Boolean).slice(-25).join("\n");
     if (exited !== null) return { port: null, exited, tail: tail() };
     // nodemon and its kind outlive the app they watch: the app is gone, the process is not, and
-    // the wait ran its whole three minutes on planless with the reason sitting in the output.
+    // the wait ran its whole three minutes on one app with the reason sitting in the output.
     // Also: the port is taken, under a watcher that does not exit when its app cannot listen. The app
     // that holds the port is theirs and already running, which startApp turns into attaching to it.
     if (/app crashed - waiting for file changes|waiting for (?:file )?changes before restart|Failed running|EADDRINUSE|address already in use/i.test(seen)) {
@@ -737,7 +737,7 @@ async function listening(pid) {
   } catch { return []; }
 }
 // Your app and everything it started, stopped for certain. A process group is not enough: nodemon
-// gives the real server a group of its own, and planless's server outlived this command as an
+// gives the real server a group of its own, and one app's server outlived this command as an
 // orphan still holding its port and its production connections. Asked first, then made to.
 async function stopApp(pid) {
   const family = await familyOf(pid).catch(() => [pid]);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cortad",
-  "version": "0.2.0",
+  "version": "0.2.2",
   "description": "Connects the AI app on your machine to Cortad for test conversations, and gives your coding agent the MCP and skill to run them. No dependencies.",
   "bin": {
     "cortad": "local.mjs"

package/skill/SKILL.md

@@ -6,9 +6,17 @@ Cortad is connected to this repository. Do not run `npx cortad <code>` again.
 
 If the `cortad` MCP tools are in your tool list, use them. Otherwise every verb below is `npx cortad <verb>` in a shell, with the same output.
 
+## Right after connect
+The first run starts by itself: never call `run` for it. As soon as the command says "Go back to the browser", tell the person, in your own words, all of this:
+- Cortad has started their app on this machine and is reading their code to write realistic users of it, with situations that move.
+- It will play those users through the app and grade every reply against about 100 checks: their own rules, and what a good reply is.
+- The report lands in the browser, with each finding at a file and line, and you can fix and verify from here.
+- They should go back to the browser now and finish the two onboarding questions while it plays; you will report when it is done.
+Then `status` every 30 seconds, quietly, until the latest run is finished. Report the score, the number of findings and the link.
+
 ## The loop
 1. `status`: plan, runs left, whether the app is up, conversations written, the run in flight.
-2. `run`: only when the person asks. The first run is free. After that it returns a checkout link; show it in one sentence and wait for the person.
+2. `run`: only when the person asks. The first run started by itself and was free. After that `run` returns a checkout link; show it in one sentence and wait for the person.
 3. `run_status <jobId>` every 30 seconds; say nothing unless the count moved. When it finishes, say the score, how many findings, and the link.
 4. `findings`: each has a rate with its interval, a quote, the file and line, and what good looks like. Start from the worst rate.
 5. Fix ONE finding: the smallest change in the file it names. Then `verify <findingId>`.

package/skill/references/results.md

@@ -5,7 +5,7 @@ One result per verb, as the tool returns it. The numbers are from a walk of a tu
 ## status
 
 ```
-Cortad · ulaim · Free: 0 of 1 run left this month, 60 of 60 verify trials.
+Cortad · your-app · Free: 0 of 1 run left this month, 60 of 60 verify trials.
 App: Your app answered during startup on port 3100. This is the last recorded state, not a new health check.
 Conversations written: 51. A run can start.
 Latest run 8f2a1c4e-... succeeded: played 51 of 51. Score 71 of 100. 7 findings; call findings. https://cortad.com/lab