cortad 0.3.11 → 0.4.0

package/README.md

@@ -6,11 +6,16 @@ Runs Cortad's test conversations against the AI app on your machine, and gives y
 npx cortad <code>
 ```
 
-Run it in your app's folder with the code from cortad.com. It starts your app, connects it to Cortad, and adds Cortad to Claude Code, Codex, Cursor and Copilot on this machine. Cortad sends each AI endpoint it finds a test request, and the first run starts by itself once they have answered; later runs start when you press Run in the browser or ask your agent for one. Ctrl-C disconnects.
+Run it in your app's folder with the code from cortad.com and your own start command after `--`: Cortad runs that command as you typed it, with its hook inside and nothing else added, connects the app, and adds Cortad to Claude Code, Codex, Cursor and Copilot on this machine. Then send your AI one message the way a user would; that one request is what Cortad learns the endpoint from. Runs start when you press Run in the browser or ask your agent for one. Ctrl-C disconnects.
+
+```
+npx cortad <code> -- npm run dev           your command, as you start it, from the terminal that holds its settings
+npx cortad <code> --port 3000              an app already running: its replies are seen, its model calls are not
+```
 
 ## Your coding agent
 
-After that, your agent has ten tools: `status`, `reach`, `run`, `run_status`, `findings`, `verify`, `dispute`, `feedback`, `field_connect`, `field`. The same ten work as commands:
+After that, your agent has the tools `status`, `reach`, `run`, `run_status`, `findings`, `verify`, `dispute`, `feedback`, `field_connect`, `field`, `world`, `world_status`, `test`, `connect`, `intelligence`, `incidents`, `outcomes`, `subscribe` and `query`. The same work as commands:
 
 ```
 npx cortad status
@@ -22,7 +27,7 @@ npx cortad verify <findingId>
 
 `npx cortad wait` returns when the next run ends, with what it found; an agent starts it in the background so the run's end wakes it.
 
-A run needs your app up. If nothing on this machine is holding it, the command starts it with the key in `~/.cortad` and stops it ten minutes after the last run.
+A run needs your app up. If nothing on this machine is holding it, the command starts it with the key and the start command kept in `~/.cortad`, and stops it ten minutes after the last run.
 
 To make checking with Cortad part of the repository:
 
@@ -36,19 +41,18 @@ The hooks run after an edit and on your next message: the first names the prompt
 ## Flags
 
 ```
+-- <command>         how your app starts; everything after -- is run as typed (--start "cmd" says the same)
+--port 3000          use an app that is already running, as it is
+--port 3000 -- <command>   start a container and watch the port it publishes; the model base URL to set inside it is printed
 --explain            show what would be sent and started, then exit
---port 3000          use an app that is already running
---start "make dev"   how your app starts
---data .env.staging  copy the databases this settings file names, not the ones your app's settings name
 --verbose            print your app's output
 ```
 
-Your app runs on copies of its databases, made when it starts and deleted when the command ends; the originals are only read. A database it uses that cannot be copied holds Run until you say yes to writing into it.
+Nothing about your environment is guessed or changed. Your app gets this terminal's environment and its own request limits raised for the session, by name; its env files are read only to hide their values in what leaves this machine. Nothing is copied or started beside it: test conversations write where your app writes, after your yes in the browser.
 
 ## Files it creates
 
-- `~/.cortad/<project>/`     the key, the last tree digest, which process holds your app up, your app's output in `app.log`, the run it asked for last, in `news.txt` the line a finished run left for your agent, removed once the agent has it, and in `heard.txt` the ids of the runs your agent has read
-- `~/.cortad/identity.key`   the seed for the session's test accounts
+- `~/.cortad/<project>/`     the key, the start command, the last tree digest, which process holds your app up, your app's output in `app.log`, the sign-in your own request carried, the run it asked for last, in `news.txt` the line a finished run left for your agent, removed once the agent has it, and in `heard.txt` the ids of the runs your agent has read
 - `$TMPDIR/cortad-<pid>/`    removed on exit
 
 macOS and Linux, Node 20 or later. No dependencies. What is sent and what stays on your machine: `npx cortad --explain`, and cortad.com/privacy.

package/lib/auto-reach.mjs

@@ -1,8 +1,9 @@
-// The connect command tests the endpoints the read lists by itself, as soon as status lists them:
-// no agent, no headers from anyone. Each endpoint is sent once per connection, and again only when
-// the file it is written in changes or the agent asks with `reach`. The agent's ask reaches this
-// process through two files beside the runner record, since the identities this machine signed in
-// live here and nowhere else.
+// The agent's `reach`, sent by the connect command: one request to each endpoint the ask names, or to
+// every endpoint status lists as not reached, from this machine to the app on its port. The ask
+// reaches this process through two files beside the runner record, since the sign-in the person's own
+// request carried lives here and nowhere else. A round with no ask sends each due endpoint once and
+// again only when its file changes; the connect command runs none, so nothing reaches the app on
+// Cortad's own account.
 import { existsSync, readdirSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs";
 import { join } from "node:path";
 import { notUp, picked, reachText, REACH_MS, sendOurs, testedLine } from "./reach.mjs";

package/lib/contact-text.mjs

@@ -125,11 +125,10 @@ function unreachedLine(m) {
   return `  ${m.method} ${m.path}${m.file ? `  ${at(m.file, m.line)}` : ""}${said ? `  ${said}` : ""}`;
 }
 
-// Cortad sends the rest itself, said only where the server has a request ready for one of them.
+// The agent sends each one message, or asks Cortad to send the request it read where there is one.
 function reachLine(missing) {
-  if (!missing.some((m) => m.request && !m.unsure)) return null;
-  const except = [missing.some((m) => m.unsure || m.byHand) && "marked by hand", missing.some((m) => m.excluded) && "left out of runs"].filter(Boolean);
-  return `Cortad sends each of them a test request${except.length ? `, except any ${except.join(" or ")}` : ""}.`;
+  const ready = missing.filter((m) => m.request && !m.unsure && !m.excluded);
+  return `Send each one message as a user would, signed in where your app needs it${ready.length ? `, or call reach to send the request Cortad read for ${ready.length === missing.length ? "them" : `${num(ready.length)} of them`}` : ""}.`;
 }
 
 // `data`: what the command did with the app's stores and services, said right under the run it is

package/lib/env-files.mjs

@@ -1,7 +1,3 @@
-import { readFileSync } from "node:fs";
-import { basename, dirname, join, sep } from "node:path";
-import { listingUrl } from "./listing.mjs";
-
 // An env file read the way dotenv and python-dotenv read one: comments and `export` dropped, a quoted
 // value kept whole across lines, an unquoted one cut at " #", ${NAME} filled from the names above it
 // and then from `outer`. A bare $ is kept as written, as dotenv keeps it: "pa$$word" and a bcrypt hash
@@ -41,84 +37,3 @@ function closes(body, quote) {
   }
   return 0;
 }
-
-const read = (file, outer) => { try { return parseEnv(readFileSync(file, "utf8"), outer); } catch { return {}; } };
-const PLACEHOLDERS = /\.(?:example|sample)$/;
-
-// The folders from the app's own up to the repository root, nearest first.
-const upFrom = (appDir, root) => {
-  const dirs = [];
-  for (let d = appDir; ; d = dirname(d)) {
-    dirs.push(d);
-    if (d === root || !d.startsWith(root + sep) || dirname(d) === d) return dirs;
-  }
-};
-
-// What a start by hand gives the app from its env files: the `.env` nearest the app, as `source .env`
-// does for an app with no loader of its own. A name another env file there sets to another value is
-// left to the app's own loader, which reads its files by its own mode and order: put in the process,
-// it outranks that loader (the process wins in dotenv, python-dotenv, pydantic-settings, Next and
-// Vite). Nitham's app reads .env.test then .env.local; the test file's key, handed over this way,
-// was refused on every call it made under Cortad and on none started by hand.
-export function appEnv(envFiles, appDir, root, outer = {}) {
-  const base = upFrom(appDir, root).map((d) => join(d, ".env")).find((f) => envFiles.includes(f));
-  if (!base) return {};
-  const values = read(base, outer);
-  const beside = envFiles.filter((f) => f !== base && !PLACEHOLDERS.test(f) && (dirname(f) === dirname(base) || dirname(f) === appDir));
-  for (const file of beside) {
-    for (const [name, value] of Object.entries(read(file, outer))) if (name in values && values[name] !== value) delete values[name];
-  }
-  return values;
-}
-
-// The real env files in the app's folders, lowest first, the way dotenv-flow, Next and Vite rank them
-// in development: .env, then the development file, then .env.local, and the file of the mode the shell
-// names above all. A test, staging or stage file of another mode goes under them all and still counts:
-// Nitham's app reads .env.test in its local mode, and a store only it names is a store the app writes
-// into. Placeholders are left out.
-const RANK = { ".env": 1, ".env.development": 2, ".env.dev": 2, ".env.local": 3, ".env.development.local": 4 };
-const MODE_FILE = /^\.env\.(test|staging|stage)$/;
-export function rankedFiles(envFiles, appDir, root, mode = "") {
-  const dirs = upFrom(appDir, root).reverse();
-  const want = mode.toLowerCase() === "staging" ? "stage" : mode.toLowerCase();
-  const rank = (f) => {
-    const name = basename(f);
-    const m = MODE_FILE.exec(name);
-    if (m) return (m[1] === "staging" ? "stage" : m[1]) === want ? 5 : 0;
-    return RANK[name] ?? -1;
-  };
-  return envFiles.filter((f) => dirs.includes(dirname(f)) && !PLACEHOLDERS.test(f) && rank(f) >= 0)
-    .sort((a, b) => dirs.indexOf(dirname(a)) - dirs.indexOf(dirname(b)) || rank(a) - rank(b));
-}
-
-// The values the app's settings could settle on, ranked as above: for reading which stores and ports
-// the app names, and for a shell that cannot read the files itself. `outer` names the mode and fills ${NAME}.
-export function localEnv(envFiles, appDir, root, outer = {}) {
-  const mode = outer.ENV ?? outer.APP_ENV ?? outer.NODE_ENV ?? outer.ENVIRONMENT ?? "";
-  return Object.assign({}, ...rankedFiles(envFiles, appDir, root, mode).map((f) => read(f, outer)));
-}
-
-// Each real env file in the app's folders, read whole: a key is only ever sent with the address the
-// same file gives it, or one the app's development files give every key.
-export function fileValues(envFiles, appDir, root, outer = {}) {
-  const dirs = upFrom(appDir, root);
-  return envFiles.filter((f) => dirs.includes(dirname(f)) && !PLACEHOLDERS.test(f)).map((f) => read(f, outer));
-}
-
-// The values a development start reads whatever its mode: .env, the development files and .env.local,
-// never a test, staging or stage file.
-export function devEnv(envFiles, appDir, root, outer = {}) {
-  return Object.assign({}, ...rankedFiles(envFiles, appDir, root).filter((f) => !MODE_FILE.test(basename(f))).map((f) => read(f, outer)));
-}
-
-// The key and address pairs a provider's model listing is asked with: the shell's key, else each
-// distinct key a file holds. A key goes only to the address the shell or its own file names, else the
-// one the development files name, else the provider's own; where only a test or staging file names
-// an address, which the app may not use, the key is not sent at all.
-export function keyTries(name, canonical, { shell, files, dev }) {
-  const prefix = name.replace(/_API_KEY$/, "");
-  const named = (v) => Boolean(v[`${prefix}_BASE_URL`] || v[`${prefix}_API_BASE`]);
-  const addressFor = (own) => (named(shell) || named(own) || named(dev) || !files.some(named) ? listingUrl(name, canonical, { ...dev, ...own, ...shell }) : null);
-  return [...new Map((shell[name] ? [shell] : files.filter((v) => v[name]))
-    .map((v) => ({ key: v[name], url: addressFor(v) })).filter((t) => t.url).map((t) => [`${t.url} ${t.key}`, t])).values()];
-}

package/lib/files.mjs

@@ -0,0 +1,5 @@
+// Which of the repository's files are code, and which are tests: where a problem is placed at its
+// line (lib/replay.mjs), and which files count as the app a run plays (lib/fresh.mjs).
+export const CODE = /\.(?:[cm]?[jt]sx?|py|rb|go|php)$/;
+// A test opens its own scratch file; that is not where the app keeps its data.
+export const TEST = /(?:^|\/)(?:tests?|__tests__|spec|e2e|fixtures?)\/|[._-](?:test|spec)\.[^/]+$|(?:^|\/)(?:test_[^/]+|conftest)\.py$/;

package/lib/fresh.mjs

@@ -7,7 +7,7 @@
 import { createHash } from "node:crypto";
 import { readFileSync, statSync } from "node:fs";
 import { join } from "node:path";
-import { TEST } from "./data.mjs";
+import { TEST } from "./files.mjs";
 import { homeOf, readApp } from "./home.mjs";
 import { readRunner } from "./runner.mjs";
 import { writesDirOf, writtenPaths } from "./writes.mjs";

package/lib/mcp.mjs

@@ -10,7 +10,7 @@ export const PROTOCOL = "2025-06-18";
 // Codex keeps only the first 512 characters self-contained and Claude Code shares about 4 KB across
 // every server, so the first paragraph stands alone and the whole stays under 1,500 characters.
 export const instructions = (spec = "cortad") => [
-  "Cortad tests the AI in this repository: simulated users written from the app's code talk to the app running on this machine, and every reply is checked. Cortad sends each AI endpoint a test request itself. status says whether the app is up, what Cortad's audit of the AI code found, and what a run would play and cost. A run starts when the person presses Run in the browser or asks you.",
+  "Cortad tests the AI in this repository: simulated users written from the app's code talk to the app running on this machine, and every reply is checked. You connect the app with its own start command after `--`, then send each AI endpoint one message as a user would; Cortad learns the endpoint from that request. status says whether the app is up, what Cortad's audit of the AI code found, and what a run would play and cost. A run starts when the person presses Run in the browser or asks you.",
   `Tell the person what status found in three or four short, plain lines, leaving out file paths, endpoints, status codes and setting names unless they ask. Before you stop, start \`npx -y ${spec} wait\` as a background command: it returns when the next run or verify ends, with what it found. After a run, call findings and tell the person what their users ran into and what you would change, worst first, three problems at most; the person then says which fix to make, and nothing is edited before their go. On their go, one fix at a time: verify replays the failing conversations and says whether the failure is gone, and a further round plays only when verify is called again. findings has each failure whole and findings numbers every rate. connect gives the steps to connect the live app. intelligence says what is happening to it and incidents what went wrong and why; outcomes, subscribe and query are the layer's other tools. dispute records a check that reads this app wrong.`,
 ].join("\n\n");
 export const INSTRUCTIONS = instructions();
@@ -25,8 +25,8 @@ export const TOOLS = [
   { name: "status", title: "Cortad status", annotations: readOnly,
     description: "What Cortad found in this repository: whether the app is up and on which port, its audit of the prompts and of the code around the model, each endpoint's answer to Cortad's test request or the file and line it failed at, the plan with runs left, and the latest run. Applies at the start of a session, right after a connect, and before a commit that changes prompts, tools, models or retrieval. With show, one section in full, in pages.",
     inputSchema: { type: "object", properties: { show: { type: "string", enum: SHOW, description: "A section to list in full: rules, standards, journeys, endpoints, trials, records or reviews; or machine, what Cortad does on this machine." }, page: { type: "integer", minimum: 1, description: "The page of that list. Default 1." } }, additionalProperties: false } },
-  { name: "reach", title: "Send Cortad's test requests again",
-    description: "Sends Cortad's test request again to the endpoints status lists as failed or not yet answered, or to the paths named, from this machine to the app on its port. Each endpoint comes back with its HTTP status, its time and, when it is not 2xx, the start of what the app answered.",
+  { name: "reach", title: "Send a request Cortad read to an endpoint",
+    description: "Sends the request Cortad read for an endpoint, to the ones status lists as failed or not yet answered, or to the paths named, from this machine to the app on its port, signed in as your own request was where one was seen. Each endpoint comes back with its HTTP status, its time and, when it is not 2xx, the start of what the app answered. A message you send the app yourself does the same.",
     inputSchema: { type: "object", properties: {
       paths: { type: "array", items: { type: "string" }, description: "Only these endpoints, by path. Default: every endpoint status lists as failed or not yet answered." },
     }, additionalProperties: false } },

package/lib/proc.mjs

@@ -58,7 +58,9 @@ export async function listenerOn(port) {
 // already in use", Go's "0.0.0.0:8080: bind: address already in use", Node's "EADDRINUSE: address
 // already in use :::3000" or "... 127.0.0.1:3000", Rails' "... for "127.0.0.1" port 3000". 0 when
 // it names none.
-export const portInError = (said) => Number(/[:\s(,](\d{4,5})\)?:?[^\n\d]{0,12}address already in use/i.exec(said)?.[1] ?? /(?:EADDRINUSE|address already in use)[^\n]{0,60}?[:\s](\d{4,5})\b/i.exec(said)?.[1] ?? 0);
+// The port in the app's own words for a taken one, however its runtime spells the address: Node's
+// "EADDRINUSE: address already in use :::3100", uvicorn's "('127.0.0.1', 58889): [errno 48] address already in use".
+export const portInError = (said) => Number(/[:\s(,](\d{4,5})\)?:?[^\n]{0,40}?address already in use/i.exec(said)?.[1] ?? /(?:EADDRINUSE|address already in use)[^\n]{0,60}?[:\s](\d{4,5})\b/i.exec(said)?.[1] ?? 0);
 
 // Whoever holds a port: its pid, the program and its first argument (never the rest of its command
 // line, which can carry a key), and the folder it runs in.

package/lib/proxy.mjs

@@ -34,12 +34,23 @@ const asked = (raw) => (/^[^=&\s]+=[^&]*(?:&[^=&\s]+=[^&]*)*$/.test(raw) ? [...n
 // ponytail: a worker slower than this gets a request said unseen; the wait per app if one is seen.
 const UNSEEN_MS = 3000;
 
-const listen = async (server) => { server.listen(0, "127.0.0.1"); await once(server, "listening"); return server.address().port; };
+// On the port asked for when it is free, else on any: a container's settings name the model port once.
+const listen = async (server, port = 0) => {
+  const at = await new Promise((resolve) => {
+    server.once("error", () => resolve(null));
+    server.listen(port, "127.0.0.1", () => resolve(server.address().port));
+  });
+  if (at !== null) return at;
+  server.removeAllListeners("error");
+  server.listen(0, "127.0.0.1");
+  await once(server, "listening");
+  return server.address().port;
+};
 
 // `file`: the trace file the command reads. `target()`: where the app answers now, { host, port };
 // it moves when the app is started again on another port. `onSeen`: the first model call came through.
 // `onUnseen(door)`: a request the app answered in words with no model call begun, said once, as "POST /chat".
-export async function openProxy({ file, rulesFile, target, onSeen = () => {}, onUnseen = () => {} }) {
+export async function openProxy({ file, rulesFile, target, modelPort: wantedModelPort = 0, onSeen = () => {}, onUnseen = () => {} }) {
   const row = (value) => { try { appendFileSync(file, JSON.stringify(scrubbed(value)) + "\n", { mode: 0o600 }); } catch { /* the command is gone */ } };
   const rec = makeRecorder({ row, rulesFile });
   // The requests in flight through the front that a model call can be made for.
@@ -156,7 +167,7 @@ export async function openProxy({ file, rulesFile, target, onSeen = () => {}, on
   const model = createServer((req, res) => { relay(req, res).catch(() => res.destroy()); });
 
   const frontPort = await listen(front);
-  const modelPort = await listen(model);
+  const modelPort = await listen(model, wantedModelPort);
   row({ hello: "proxy", pid: process.pid });
   const via = (upstream) => {
     if (!slots.includes(upstream)) slots.push(upstream);