cortad 0.3.11 → 0.4.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cortad",
-  "version": "0.3.11",
+  "version": "0.4.0",
   "description": "Connects the AI app on your machine to Cortad for test conversations, and gives your coding agent the MCP and skill to run them. No dependencies.",
   "bin": {
     "cortad": "local.mjs"
@@ -10,54 +10,39 @@
     "README.md",
     "lib/app-port.mjs",
     "lib/auto-reach.mjs",
-    "lib/backing.mjs",
     "lib/calls.mjs",
     "lib/cli.mjs",
     "lib/commits.mjs",
-    "lib/container.mjs",
     "lib/contact-text.mjs",
-    "lib/data.mjs",
     "lib/dissection-text.mjs",
     "lib/document.mjs",
-    "lib/elastic.mjs",
     "lib/failed-text.mjs",
+    "lib/files.mjs",
     "lib/fresh.mjs",
     "lib/health.mjs",
     "lib/held.mjs",
     "lib/home.mjs",
-    "lib/hosted.mjs",
     "lib/env-files.mjs",
     "lib/env-secrets.mjs",
-    "lib/identities.mjs",
-    "lib/keeping.mjs",
     "lib/keys.mjs",
-    "lib/ledger.mjs",
-    "lib/listing.mjs",
     "lib/lock.mjs",
     "lib/mcp.mjs",
-    "lib/mint.mjs",
-    "lib/mongo.mjs",
     "lib/numbers-text.mjs",
-    "lib/pg.mjs",
     "lib/proc.mjs",
     "lib/proof.mjs",
     "lib/proxy.mjs",
-    "lib/qdrant.mjs",
     "lib/pyhook/sitecustomize.py",
     "lib/pyhook/cortad_tied.py",
     "lib/pyhook/cortad_sites.py",
     "lib/reach.mjs",
     "lib/read-text.mjs",
-    "lib/redis.mjs",
     "lib/register.mjs",
     "lib/replay.mjs",
     "lib/runner.mjs",
     "lib/sample.mjs",
-    "lib/service.mjs",
     "lib/spec.mjs",
     "lib/start.mjs",
     "lib/stick.mjs",
-    "lib/stores.mjs",
     "lib/switches.mjs",
     "lib/tether.mjs",
     "lib/text.mjs",

package/skill/SKILL.md

@@ -30,24 +30,26 @@ The person owns the product. They may not be reading the code with you, so every
 
 ## Connecting
 
-`npx {{cortad}}` in the repository's folder connects it, with nothing to copy. Run it as a background command (in Claude Code: Bash with run_in_background) and leave it running: it starts the app and keeps it reachable from the browser until it is stopped.
-- A folder connected before reconnects at once with the key it kept.
-- A folder connected for the first time prints a link and four characters, good for ten minutes. Give the person both as printed; they approve the link in their browser, signed in to Cortad, and the command goes on by itself.
-- When a result says no connect command is running, run `npx {{cortad}}` again in this folder. No code is needed from the person.
-- The app runs on copies of the databases its settings name, deleted when the command ends. When the person wants a run on other data, such as their staging databases instead of the test ones, stop the command and run `npx {{cortad}} --data <settings file>`: the databases that file names are copied instead, under the names the app reads them by, and the app keeps the rest of its own settings. The choice holds for later runs until the next connect, and every setting the app needs stays in place.
+Cortad does not guess how the app runs: you tell it, in one line, from the repository's folder, run as a background command (in Claude Code: Bash with run_in_background) and left running.
 
-The connect command prints "Go back to the browser" once it is connected. Cortad then finds the AI endpoints in the app's code and sends each one a test request itself, from this machine to the app on its port.
+- `npx {{cortad}} -- <how you start your app>`: your start command, as you or the person would type it, from a terminal that holds the app's settings. Cortad runs it as written with its hook inside and nothing else added: no env file is read into the app, no database is copied, no service is started. Use the environment the person wants tested (their staging settings, their existing databases): whatever the command gives the app is what the run plays against.
+- `npx {{cortad}} --port <port>`: an app that is already running and should stay as it is; Cortad sees its replies and not its model calls, and says so. With `-- <how you start your app>` beside it, an app in a container: Cortad starts it, watches the port you name, and prints the model setting to give the container so its model calls are seen too.
+- Run with neither and Cortad prints the one start command the repository makes obvious and stops; run again with it after `--`, or with the right one.
+- A folder connected before reconnects with the key and the start command it kept. A first connect prints a link and four characters, good for ten minutes: give the person both as printed; they approve the link in their browser, signed in to Cortad.
+- When a result says no connect command is running, run the same line again in this folder.
+
+Once it prints "Go back to the browser", Cortad lists the AI endpoints it found in the code. Send each one message the way a user would: through the app's own page, or a request from this machine, signed in as a real test account where the app needs one (ask the person which account; Cortad makes none). That one request is what Cortad learns the endpoint from: its route, its body, its sign-in and where the answer comes back. Cortad sends nothing to the app before that, and nothing of its own before the person's yes on the card.
 
 1. Call `status` and tell the person, in three or four short lines:
-   - that their app is connected, or the one line that says why it did not start;
+   - that their app is connected, or the one line that says why it did not start (what the app printed is the app's own output);
    - what Cortad's audit of their AI code found: how many problems, and the one that matters most, in plain words. `status` with show `reviews` holds up to 45 seconds a call while the code is still being read;
    - the line on what a run plays and costs, as printed;
    - what happens next: they press Run on the card in the browser, or tell you to start it.
 
    The endpoints, the files and the audit's rows are yours to work from. They go to the person when they ask, or when you propose a fix.
-2. When `status` lists an endpoint as `Needs a signed-in user`, Cortad could not make a sign-in for it on this machine. Tell the person; when they ask for it, sign in a test account through the app's own sign-in on this machine, send that endpoint one request in that session, then call `status` again. That is the only sign-in this needs, and only for the endpoints listed that way.
-3. A failure on the app's side that every conversation would repeat, such as a model provider refusing, a retrieval with no answer or a tool returning an error: say what you would change, change it when the person agrees, then call `reach` to send Cortad's test request to that endpoint again.
-4. A question for the person on the card in the browser is said to them once, in one line. The first run starts when the person presses Run on the card, or asks you, and then you call `run`. Before you stop, start `npx -y {{cortad}} wait` as a background command (in Claude Code: Bash with run_in_background). It returns when the next run or verify ends, with what it found, and its ending wakes you. Then follow "After a run".
+2. An endpoint `status` lists as not answered yet is waiting for your one message to it. One it lists as `Needs a signed-in user` was refused: send it again signed in as a test account the person names, and call `status` again.
+3. A failure on the app's side that every conversation would repeat, such as a model provider refusing, a retrieval with no answer or a tool returning an error: say what you would change, change it when the person agrees, then send that endpoint one message again, or call `reach` to have Cortad send the request it read for it.
+4. The one question on the card in the browser is where test conversations will write: into the databases the app wrote into for your message, named there, with nothing copied. Say it to the person once, in one line. The first run starts when the person presses Run on the card, or asks you, and then you call `run`. Before you stop, start `npx -y {{cortad}} wait` as a background command (in Claude Code: Bash with run_in_background). It returns when the next run or verify ends, with what it found, and its ending wakes you. Then follow "After a run".
 
 `status` with `show` lists one section in full: `rules`, `standards`, `journeys`, `endpoints`, `trials`, `records` or `reviews` (in a shell, `npx {{cortad}} status rules`). `status` with show `machine` says what Cortad does on this machine; hand the person its lines when they ask.
 

package/skill/references/results.md

@@ -18,7 +18,7 @@ Free: 1 of 1 run left this month; 60 verify conversations left. Pro $99 a month:
 App: Your app is starting on this machine.
 ```
 
-Once each endpoint that reaches the model has answered a test request, two messages on the chat, and before any run:
+Once a request of your own has reached the model through an endpoint, two messages on the chat, and before any run:
 
 ```
 Repository: tutor-app.
@@ -34,15 +34,15 @@ Endpoints that answered (1):
 Endpoints found in your code that have not answered yet (2):
   POST /api/homework/explain  apps/api/src/routes/homework.ts:18  Cortad's test request was answered with HTTP 500.
   POST /api/progress/summary  apps/api/src/routes/progress.ts:9  Needs a signed-in user.
-Cortad sends each of them a test request.
+Send each one message as a user would, signed in where your app needs it, or call reach to send the request Cortad read for them.
 A run plays 36 simulated users in about 6 minutes. About $0.38 on your OpenAI key.
 A run starts when the person presses Run on the card in the browser or asks you for one; if they say they pressed it, call run rather than ask them again.
 ```
 
-When the app keeps its data in a hosted store no copy was made of, the decision is the person's, said once as the last line before the link:
+Before the person has said yes on the card, the last line before the link is the one question asked of them:
 
 ```
-DATABASE_URL points at the Postgres database shop that is not on this machine, and no migrations were found to build a copy on this machine from, so test conversations would save into it. Point DATABASE_URL at a database on this machine, then run the connect command again. Or the person allows it on the card in the browser: one press lets test conversations save into it.
+Test conversations use your app as it is: they write where it wrote during your own request (127.0.0.1:5432, 127.0.0.1:27017) and reach what it reached (api.openai.com). Nothing is copied. The person allows it on the card in the browser, with one press.
 The person sees this at https://cortad.com/lab
 ```
 
@@ -95,7 +95,7 @@ Rules in the code: 16, in 5 files.
 
 ## reach
 
-Sends Cortad's test request again, from the connect command on this machine, after a fix:
+Sends the request Cortad read from your code to an endpoint no request of yours has reached, from the connect command on this machine, with the sign-in you pass it:
 
 ```
 Sent 1 request to your app on port 3100, up to four at a time:

package/lib/backing.mjs

@@ -1,230 +0,0 @@
-// The services your app reaches on this machine, brought up when nothing answers there and the
-// repository says how: a store its compose file runs from an image, with Docker, or a second
-// service of its own, with that service's own start. Only a port the app is known to reach is
-// brought up: one its settings name when it could not start without it, or one the hook saw it
-// reach for. A compose file often lists a store the app is set up not to use, and starting it
-// would be a download of gigabytes nobody asked for. Each is stopped when this command ends, and
-// one that could not be brought up, or that nothing here brings up, is said before a run, with whose
-// side it is on.
-import { execFile } from "node:child_process";
-import { existsSync, readdirSync, readFileSync } from "node:fs";
-import { connect } from "node:net";
-import { basename, dirname, join, relative } from "node:path";
-import { ledger } from "./ledger.mjs";
-import { servicePort } from "./service.mjs";
-import { startPlan, workspaces } from "./start.mjs";
-
-// Something accepts a connection on the port: a database answers no HTTP, so this is the test.
-export function tcpOpen(host, port, ms = 1500) {
-  return new Promise((done) => {
-    const s = connect({ host: host && !host.startsWith("/") && host !== "0.0.0.0" ? host.replace(/^\[|\]$/g, "") : "127.0.0.1", port });
-    const end = (ok) => { s.destroy(); done(ok); };
-    s.setTimeout(ms, () => end(false));
-    s.once("connect", () => end(true));
-    s.once("error", () => end(false));
-  });
-}
-async function waitTcp(port, ms, gone = () => false) {
-  for (const until = Date.now() + ms; Date.now() < until && !gone(); await new Promise((r) => setTimeout(r, 1000))) if (await tcpOpen("127.0.0.1", port)) return true;
-  return false;
-}
-
-const COMPOSE_FILE = /^(?:docker-)?compose(?:\.[\w-]+)?\.ya?ml$/;
-// The files compose reads by default come first: a second file beside them is a variant.
-const DEFAULT_FIRST = (a, b) => Number(/^(?:docker-)?compose\.ya?ml$/.test(basename(b))) - Number(/^(?:docker-)?compose\.ya?ml$/.test(basename(a))) || a.localeCompare(b);
-export function composeFiles(dirs) {
-  const out = new Set();
-  for (const dir of dirs) { try { for (const f of readdirSync(dir)) if (COMPOSE_FILE.test(f)) out.add(join(dir, f)); } catch { /* no folder */ } }
-  return [...out].sort(DEFAULT_FIRST);
-}
-
-const unquote = (v) => String(v ?? "").trim().replace(/^(['"])(.*)\1$/, "$2");
-// ${PG_PORT:-5432} and ${PG_PORT}, from the settings the app starts with.
-const interpolate = (v, values) => v.replace(/\$\{(\w+)(?::?-([^}]*))?\}/g, (_, name, dflt) => values[name] || dflt || "");
-// The host port of one port mapping: "5432:5432", "127.0.0.1:5432:5432/tcp". None for a container
-// port alone (compose picks the host's) or a range.
-function published(entry, values) {
-  const parts = interpolate(unquote(entry), values).replace(/\/(?:tcp|udp)$/, "").split(":");
-  const host = parts.length >= 2 ? parts[parts.length - 2] : "";
-  return /^\d+$/.test(host) ? Number(host) : null;
-}
-
-// The services a compose file declares: name, whether it runs an image or builds this repository's
-// code, and the host ports it publishes. A reader of the shape compose files are written in, not of
-// YAML at large.
-// ponytail: no anchors, extends, include or profiles; add them when a repository needs one to be read.
-export function composeServices(text, values = {}) {
-  const out = [];
-  let inServices = false, nameIndent = -1, keyIndent = -1, svc = null, portsIndent = -1;
-  for (const raw of String(text).split("\n")) {
-    const line = raw.replace(/\t/g, "  ").replace(/\s+#.*$/, "");
-    const t = line.trim();
-    if (!t || t.startsWith("#")) continue;
-    const indent = line.length - line.trimStart().length;
-    if (indent === 0) { inServices = /^services:\s*$/.test(t); svc = null; continue; }
-    if (!inServices) continue;
-    if (nameIndent < 0) nameIndent = indent;
-    if (indent <= nameIndent) {
-      const m = /^["']?([\w.-]+)["']?:\s*$/.exec(t);
-      svc = indent === nameIndent && m ? { name: m[1], image: null, build: false, ports: [] } : null;
-      if (svc) out.push(svc);
-      keyIndent = -1; portsIndent = -1;
-      continue;
-    }
-    if (!svc) continue;
-    if (portsIndent >= 0 && indent > portsIndent) {
-      const item = /^-\s*(.*)$/.exec(t);
-      const port = item && !/^[A-Za-z_]\w*:(?:\s|$)/.test(item[1]) ? published(item[1], values) : Number(unquote(/^(?:-\s*)?published:\s*(.+)$/.exec(t)?.[1] ?? "")) || null;
-      if (port) svc.ports.push(port);
-      continue;
-    }
-    portsIndent = -1;
-    if (keyIndent < 0) keyIndent = indent;
-    if (indent !== keyIndent) continue;
-    const kv = /^([\w-]+):\s*(.*)$/.exec(t);
-    if (!kv) continue;
-    if (kv[1] === "image") svc.image = unquote(kv[2]);
-    else if (kv[1] === "build") svc.build = true;
-    else if (kv[1] === "ports") {
-      if (kv[2].startsWith("[")) for (const e of kv[2].replace(/^\[|\]$/g, "").split(",")) { const p = published(e, values); if (p) svc.ports.push(p); }
-      else portsIndent = indent;
-    }
-  }
-  return out;
-}
-
-// A setting that is an address a page is opened from, not a service the app calls.
-// The app's own address and sign-in redirects are pages, not services: ai-appointment-setter's
-// NEXT_PUBLIC_APP_URL serves only its OAuth redirect, and a run was held on it.
-const PAGE = /(?:^|_)(?:ORIGINS?|FRONTEND|CLIENT|SITE|WEB|CALLBACK|REDIRECT|CORS|ALLOWED|HOMEPAGE|APP|BASE|NEXTAUTH)(?:_|$)/;
-// Addresses on this machine the app's settings point at: http://localhost:4000/api under API_URL.
-export function localAddresses(values) {
-  const byPort = new Map();
-  for (const [name, raw] of Object.entries(values)) {
-    if (PAGE.test(name)) continue;
-    let u;
-    try { u = new URL(String(raw ?? "").trim()); } catch { continue; }
-    if (!/^https?:$/.test(u.protocol) || !u.port || !/^(?:localhost|127(?:\.\d+){3}|\[::1\]|0\.0\.0\.0)$/i.test(u.hostname)) continue;
-    const port = Number(u.port);
-    byPort.set(port, [...(byPort.get(port) ?? []), name]);
-  }
-  return [...byPort].map(([port, names]) => ({ port, names }));
-}
-
-const run = (bin, args, opts = {}) => new Promise((done) => {
-  execFile(bin, args, { timeout: 180_000, maxBuffer: 4 * 1024 * 1024, ...opts }, (err, stdout, stderr) => done({ ok: !err, out: String(stdout ?? ""), err: String(stderr ?? err?.message ?? "") }));
-});
-// A service's own last words go into a sentence that leaves this machine: an address's password never does.
-const lastLine = (text) => String(text).replace(/\x1b\[[0-9;]*[A-Za-z]/g, "").replace(/:\/\/[^\s/@:]+:[^\s/@]+@/g, "://[masked]@").split("\n").map((l) => l.trim()).filter(Boolean).at(-1)?.slice(0, 160) ?? "";
-
-// How compose is run here: the docker plugin, the older standalone binary, or not at all and why.
-async function composeCommand(onPath) {
-  if (!onPath("docker") && !onPath("docker-compose")) return { why: "Docker is not installed on this machine" };
-  if (!(await run("docker", ["info"], { timeout: 15_000 })).ok) return { why: "Docker is not running on this machine" };
-  if ((await run("docker", ["compose", "version"], { timeout: 15_000 })).ok) return { bin: "docker", pre: ["compose"] };
-  return onPath("docker-compose") ? { bin: "docker-compose", pre: [] } : { why: "Docker here has no compose command" };
-}
-
-const upper = (s) => s.charAt(0).toUpperCase() + s.slice(1);
-// Why a port is wanted: a setting names it, or the app was seen reaching for it.
-const pointer = (w) => (!w.names?.length ? "Your app reached for"
-  : w.names.length === 1 ? `${w.names[0]} points at` : `${w.names[0]} and ${w.names.length - 1} more setting${w.names.length > 2 ? "s" : ""} point at`);
-const said = {
-  up: (what, how, w) => `Started ${what} ${how}: ${pointer(w)} port ${w.port} on this machine and nothing answered there. It is stopped when this command ends.`,
-  theirs: (what, why, w) => `${pointer(w)} port ${w.port} on this machine and nothing answers there. ${upper(what)} would start it, but ${why}: start it yourself before a run. This is on your side.`,
-  exited: (what, tail, w) => `${pointer(w)} port ${w.port} on this machine, which ${what} serves, and it stopped before it answered: ${tail}. This is on your side.`,
-  none: (w) => `${pointer(w)} port ${w.port} on this machine and nothing answers there, and nothing in this repository starts it: start it yourself before a run. This is on your side.`,
-  slow: (what, secs, w) => `${pointer(w)} port ${w.port} on this machine; ${what} was started for it, but nothing answered there within ${secs} seconds. This is on our side.`,
-};
-
-// One session's services. up(wants) brings up each wanted port it can, where nothing answers: a
-// compose store first, else a workspace of this repository that serves it. `launch` starts a
-// workspace the way the app is started and hands back its process. Answers what it said and the
-// ports that now answer.
-export function makeBacking({ root, appDir, values, onPath, ledgerFile, launch, adopt = true, storeWaitMs = 120_000, serviceWaitMs = 90_000 }) {
-  const book = ledger(ledgerFile);
-  const started = [];
-  const tried = new Set();
-  const lastSaid = new Map();
-  let compose = null;
-  const files = composeFiles([...new Set([appDir, root])]).map((file) => ({ file, services: (() => { try { return composeServices(readFileSync(file, "utf8"), values); } catch { return []; } })() }));
-  // Services a session that died left running are stopped by the next session that starts the app,
-  // never removed: the person may have used them since, and a removed container takes what they put
-  // in it. A session attached to an app already running leaves them to that app.
-  for (const row of adopt ? book.orphans("service") : []) {
-    const [file, service] = row.key.split("#");
-    if (!existsSync(file)) continue;
-    const kept = { ...row, made: "stop" };
-    book.remove(row);
-    book.add(kept);
-    started.push({ file, service, row: kept });
-  }
-
-  async function fromCompose(found, w) {
-    const where = `the ${found.service} service in ${relative(root, found.file) || basename(found.file)}`;
-    compose ??= await composeCommand(onPath);
-    if (compose.why) return said.theirs(where, compose.why, w);
-    const opts = { cwd: dirname(found.file), env: { ...process.env } };
-    const args = (...a) => [...compose.pre, "-f", found.file, ...a];
-    const existed = Boolean((await run(compose.bin, args("ps", "-a", "-q", found.service), opts)).out.trim());
-    const up = await run(compose.bin, args("up", "-d", found.service), opts);
-    if (!up.ok) return said.theirs(where, `Docker could not start it: ${lastLine(up.err)}`, w);
-    const row = { kind: "service", key: `${found.file}#${found.service}`, made: existed ? "stop" : "remove" };
-    book.add(row);
-    started.push({ file: found.file, service: found.service, row });
-    if (!(await waitTcp(w.port, storeWaitMs))) return said.slow(where, Math.round(storeWaitMs / 1000), w);
-    return said.up(where, "with Docker", w);
-  }
-
-  async function fromWorkspace(dir, w) {
-    const plan = startPlan({ root: dir, onPath });
-    if (!plan?.cmd) return null;
-    const what = relative(root, dir) || basename(dir);
-    const kid = launch(plan);
-    let exited = null;
-    let tail = "";
-    kid.on("exit", (code) => { exited = code ?? 1; });
-    const keep = (d) => { tail = (tail + d).slice(-2000); };
-    kid.stdout?.on("data", keep);
-    kid.stderr?.on("data", keep);
-    if (await waitTcp(w.port, serviceWaitMs, () => exited !== null)) return said.up(what, `with ${plan.cmd}`, w);
-    if (exited !== null) return said.exited(what, lastLine(tail) || `it exited ${exited}`, w);
-    return said.slow(what, Math.round(serviceWaitMs / 1000), w);
-  }
-
-  return {
-    // wants: [{ port, names, store }], each a port on this machine the app reaches or its settings
-    // name; `store` lets a compose image serve it. Answers what was said and the ports brought up.
-    // `down`: the ports a setting names that still answer nothing after this, each with what was
-    // said of it: a run without them is a run against an app missing a part, and holds.
-    async up(wants) {
-      const lines = [];
-      const ports = [];
-      const down = [];
-      const still = (w, why) => { if (w.names?.length && why) down.push({ port: w.port, names: w.names, why }); };
-      for (const w of wants) {
-        if (await tcpOpen("127.0.0.1", w.port)) continue;
-        if (tried.has(w.port)) { still(w, lastSaid.get(w.port)); continue; }
-        const image = w.store ? files.flatMap(({ file, services }) => services.filter((x) => x.image && !x.build && x.ports.includes(w.port)).map((x) => ({ file, service: x.name })))[0] : null;
-        const dir = image ? null : workspaces(root).find((d) => d !== appDir && servicePort(d) === w.port);
-        // A port only seen reached, and not a store's, may be a passing connection of the app's own.
-        if (!image && !dir && !w.names?.length && !w.store) continue;
-        tried.add(w.port);
-        const line = !image && !dir ? said.none(w) : image ? await fromCompose(image, w) : await fromWorkspace(dir, w);
-        if (line) { lines.push(line); lastSaid.set(w.port, line); }
-        if (await tcpOpen("127.0.0.1", w.port)) ports.push(w.port);
-        else still(w, line);
-      }
-      return { lines, ports, down };
-    },
-    // Compose services only: a workspace's process is the caller's, stopped with the app.
-    async stop() {
-      compose ??= started.length ? await composeCommand(onPath) : null;
-      if (!compose || compose.why) return;
-      for (const { file, service, row } of started.splice(0)) {
-        const how = row.made === "stop" ? ["stop", service] : ["rm", "-s", "-f", service];
-        if ((await run(compose.bin, [...compose.pre, "-f", file, ...how], { cwd: dirname(file), env: { ...process.env } })).ok) book.remove(row);
-      }
-    },
-  };
-}

package/lib/container.mjs

@@ -1,60 +0,0 @@
-// A store's whole server copied into a Docker container of ours for one session, for an app that
-// names its databases or indexes in its code, so no setting can point it at a copy on the same
-// server. Ours runs the image the original runs where the original is a container, else the
-// engine's own image at the original's version. It listens on one port of this machine's loopback,
-// the same inside the container and out, so an address the server gives of itself (a replica set's
-// member, a search node's) is one the app reaches. It is removed with its volumes at the end.
-import { execFile } from "node:child_process";
-import { randomBytes } from "node:crypto";
-import { createServer } from "node:net";
-
-// `env` reaches the docker command only: `-e NAME` with no value hands a container that value
-// without it ever sitting on a command line another process can read.
-export const docker = (args, { env = {}, timeout = 120_000 } = {}) => new Promise((done) => {
-  execFile("docker", args, { timeout, maxBuffer: 16 * 1024 * 1024, env: { ...process.env, ...env } }, (err, out, errText) => done({ ok: !err, code: err?.code === "ENOENT" ? "missing" : err?.code ?? 0, out: String(out ?? ""), err: String(errText ?? err?.message ?? "") }));
-});
-
-let up = null;
-export const dockerUp = () => (up ??= docker(["info", "--format", "{{.ServerVersion}}"], { timeout: 15_000 }).then((r) => r.ok));
-
-export const freePort = () => new Promise((ok, no) => {
-  const s = createServer().once("error", no).listen(0, "127.0.0.1", () => { const { port } = s.address(); s.close(() => ok(port)); });
-});
-
-// The container publishing this port of the machine, with the network it sits on and its address
-// there, which ours joins to read from it: a port published only on loopback is out of reach of
-// another container on Linux.
-export async function servedBy(port) {
-  const ps = await docker(["ps", "--format", "{{.ID}}\t{{.Image}}\t{{.Ports}}"], { timeout: 15_000 });
-  for (const line of ps.out.split("\n")) {
-    const [id, image, ports = ""] = line.split("\t");
-    const inner = new RegExp(`(?:^|,\\s*)\\S*:${port}->(\\d+)/tcp`).exec(ports)?.[1];
-    if (!inner) continue;
-    const nets = await docker(["inspect", "-f", "{{range $n, $e := .NetworkSettings.Networks}}{{$n}} {{$e.IPAddress}}\n{{end}}", id], { timeout: 15_000 });
-    const [net, ip] = nets.out.trim().split("\n")[0]?.split(" ") ?? [];
-    return { image, inner: Number(inner), ...(net && ip ? { net, ip } : {}) };
-  }
-  return null;
-}
-
-export const copyName = (engine) => `cortad-${engine}-${process.pid}-${randomBytes(3).toString("hex")}`;
-
-// Started detached; a first pull of the image is part of the start. Answers true when it runs.
-// `secrets` reach the container by name only, their values through the docker command's environment.
-export async function runCopy({ name, image, port, env = {}, secrets = {}, entrypoint, args = [], network }) {
-  const placed = network ? ["--network", network] : ["--add-host", "host.docker.internal:host-gateway"];
-  const settings = [...Object.entries(env).flatMap(([k, v]) => ["-e", `${k}=${v}`]), ...Object.keys(secrets).flatMap((k) => ["-e", k])];
-  return (await docker(["run", "-d", "--name", name, "--label", "cortad.copy=1", ...placed, "-p", `127.0.0.1:${port}:${port}`, ...settings, ...(entrypoint ? ["--entrypoint", entrypoint] : []), image, ...args], { env: secrets, timeout: 900_000 })).ok;
-}
-
-export const removeCopy = async (name) => {
-  const r = await docker(["rm", "-f", "-v", name], { timeout: 60_000 });
-  return r.ok || /No such container/i.test(r.err);
-};
-
-export async function until(ok, ms) {
-  for (const end = Date.now() + ms; Date.now() < end; await new Promise((r) => setTimeout(r, 1000))) {
-    try { if (await ok()) return true; } catch { /* not yet */ }
-  }
-  return false;
-}

package/lib/data.mjs

@@ -1,143 +0,0 @@
-// Your app's own database, kept out of the run's way. A run creates people, conversations and
-// records by the hundred; written into the file your app keeps its real data in, they stay there
-// after the run. So an app started here runs against a copy of its database file, made in this
-// command's own temp folder and pointed at through the same variable, and the original is never
-// opened. What cannot be copied (a file your code names with no variable in front of it, an app that
-// was already running) holds Run until the person says yes to writing into it. A database server is
-// copied on its own server (lib/stores.mjs).
-import { createHash } from "node:crypto";
-import { copyFileSync, cpSync, existsSync, mkdirSync, readdirSync, readFileSync, statSync } from "node:fs";
-import { basename, dirname, isAbsolute, join, relative, resolve } from "node:path";
-import { storesOf } from "./stores.mjs";
-
-const FILE_DB = /\.(?:db|sqlite3?|duckdb)$/i;
-// sqlite:///x.db (SQLAlchemy, relative), sqlite:////abs/x.db, sqlite+aiosqlite:///x.db, file:./dev.db (Prisma, libsql).
-const FILE_URL = /^(sqlite(?:\+\w+)?:\/\/\/|file:(?:\/\/(?=\/))?)([^?]*)(\?.*)?$/i;
-
-// What one value names: a database file, with how to write the value again for another path, or nothing.
-export function databaseOf(value) {
-  const v = String(value ?? "").trim();
-  if (!v) return null;
-  const url = FILE_URL.exec(v);
-  if (url) {
-    const path = decodeURIComponent(url[2]);
-    if (!path || path === ":memory:" || path.startsWith(":memory")) return null;
-    return { path, write: (to) => `${url[1]}${to}${url[3] ?? ""}`, prisma: /^file:/i.test(url[1]) };
-  }
-  if (!v.includes("://") && FILE_DB.test(v)) return { path: v, write: (to) => to };
-  return null;
-}
-
-// A folder the app keeps its data in, named by a variable (DATA_DIR=./data): a database file sits
-// directly inside it. The whole folder is the database then, uploads and backups beside it.
-const FOLDER_NAME = /DATA|DB|DATABASE|STORAGE|STORE|PERSIST|SQLITE|DUCK/i;
-function dataFolder(name, value, dirs, appDir, root) {
-  const v = String(value ?? "").trim();
-  if (!FOLDER_NAME.test(name) || !v || v.includes("://") || FILE_DB.test(v) || !/^[.~/\w]/.test(v)) return null;
-  for (const place of [appDir, ...dirs].map((d) => resolve(d, v))) {
-    if (place === root || place === appDir || !place.startsWith(root + "/")) continue;
-    let names;
-    try { if (!statSync(place).isDirectory()) continue; names = readdirSync(place); } catch { continue; }
-    if (names.some((n) => FILE_DB.test(n))) return place;
-  }
-  return null;
-}
-const sizeOf = (dir, cap) => {
-  let total = 0;
-  const walk = (d) => { for (const e of readdirSync(d, { withFileTypes: true })) { if (total > cap) return; const p = join(d, e.name); if (e.isDirectory()) walk(p); else if (e.isFile()) total += statSync(p).size; } };
-  try { walk(dir); } catch { /* counted what it could */ }
-  return total;
-};
-
-// A name the code reads with a default written beside it: os.getenv("DATABASE_URL", "sqlite:///./app.db"),
-// process.env.DB_PATH ?? "./data.db". With nothing in the environment, the default is the database.
-const DEFAULTED = /(?:os\.(?:environ\.get|getenv)|\bgetenv)\(\s*['"]([A-Za-z_]\w*)['"]\s*,\s*['"]([^'"\n]+)['"]|process\.env(?:\.([A-Za-z_]\w*)|\[\s*['"]([A-Za-z_]\w*)['"]\s*\])\s*(?:\?\?|\|\|)\s*['"]([^'"\n]+)['"]/g;
-// A database file opened by a path written in the code, with no variable in front of it.
-const OPENED = /(?:sqlite3\.connect|aiosqlite\.connect|duckdb\.connect|new\s+(?:sqlite3\.)?Database|create_(?:async_)?engine)\(\s*['"]([^'"\n]+)['"]/g;
-export const CODE = /\.(?:[cm]?[jt]sx?|py|rb|go|php)$/;
-// A test opens its own scratch file; that is not where the app keeps its data.
-export const TEST = /(?:^|\/)(?:tests?|__tests__|spec|e2e|fixtures?)\/|[._-](?:test|spec)\.[^/]+$|(?:^|\/)(?:test_[^/]+|conftest)\.py$/;
-// A folder bigger than this is not copied: said instead, so the person decides.
-const FOLDER_CAP = 1024 ** 3;
-
-function scan(sources) {
-  const defaults = {};
-  const opened = [];
-  for (const file of sources) {
-    if (!CODE.test(file) || TEST.test(file)) continue;
-    let text = "";
-    try { if (statSync(file).size > 512_000) continue; text = readFileSync(file, "utf8"); } catch { continue; }
-    for (const m of text.matchAll(DEFAULTED)) {
-      const name = m[1] ?? m[3] ?? m[4];
-      const value = m[2] ?? m[5];
-      if (name && !(name in defaults) && (databaseOf(value) || storesOf({ [name]: value }).length)) defaults[name] = value;
-    }
-    for (const m of text.matchAll(OPENED)) { const db = databaseOf(m[1]); if (db?.path) opened.push({ file, path: db.path }); }
-  }
-  return { defaults, opened };
-}
-
-// Where a relative path lands. The app's own folder first, which is where it runs; then the folder
-// of the env file that named it, which some settings loaders resolve against; for Prisma, the folder
-// its schema lives in. The first place the file exists is the one; none, and it is the app's folder.
-function located(db, dirs, appDir) {
-  if (isAbsolute(db.path)) return db.path;
-  const prisma = db.prisma ? [join(appDir, "prisma")] : [];
-  const places = [...prisma, appDir, ...dirs].map((d) => resolve(d, db.path));
-  return places.find((p) => existsSync(p)) ?? places[0];
-}
-
-// A file the run would write into as it is, as a row that holds Run: named to the API by its setting,
-// or by the file's own name where the code opens it by path, and never by the path itself.
-const heldFile = (path, name, why) => ({ id: createHash("sha256").update(`file:${path}`).digest("hex").slice(0, 16), names: [name.replace(/\W+/g, "_").replace(/^(?=\d)/, "_").slice(0, 100)], why });
-
-// values: the environment the app will start with, names to values. from: the env file each name
-// was read from, when there is one. Returns the variables to start the app with, what to say, what
-// holds Run, and the settings with the code's own defaults beside them, which the stores are read from.
-export function keepData({ values, from = {}, appDir, root = appDir, work, sources = [], started = true }) {
-  const { defaults, opened } = scan(sources);
-  const all = { ...defaults, ...values };
-  const env = {};
-  const copied = [];
-  const said = [];
-  const held = [];
-  const moved = new Set();
-  const rel = (p) => relative(root, p) || basename(p);
-  for (const [name, value] of Object.entries(all)) {
-    const db = databaseOf(value);
-    const folder = db ? null : dataFolder(name, value, from[name] ? [dirname(from[name])] : [], appDir, root);
-    if (folder) {
-      moved.add(folder);
-      if (!started) { held.push(heldFile(folder, name, `${name} points at your data folder ${rel(folder)}/, and your app was already running when this command started, so test conversations would save into it. Stop your app, then run the connect command again.`)); continue; }
-      if (sizeOf(folder, FOLDER_CAP) > FOLDER_CAP) { held.push(heldFile(folder, name, `${name} points at your data folder ${rel(folder)}/, which is over the 1 GB a run copies, so test conversations would save into it. Point ${name} at a smaller copy, then run the connect command again.`)); continue; }
-      const to = join(work, "data", String(copied.length), basename(folder));
-      cpSync(folder, to, { recursive: true });
-      env[name] = to;
-      copied.push({ name, from: folder, to });
-      said.push(`Your app runs against a copy of its data folder ${rel(folder)}/, through ${name}; the original is not touched, and the copy is deleted when this command ends and made again from the original at the next connect.`);
-      continue;
-    }
-    if (!db) continue;
-    const original = located(db, from[name] ? [dirname(from[name])] : [], appDir);
-    moved.add(original);
-    if (!started) {
-      held.push(heldFile(original, name, `${name} points at your database ${rel(original)}, and your app was already running when this command started, so test conversations would save into it. Stop your app, then run the connect command again.`));
-      continue;
-    }
-    const to = join(work, "data", String(copied.length), basename(original));
-    mkdirSync(dirname(to), { recursive: true });
-    // The file and the journal beside it: SQLite keeps recent writes in -wal until a checkpoint.
-    if (existsSync(original)) for (const tail of ["", "-wal", "-shm", "-journal"]) if (existsSync(original + tail)) copyFileSync(original + tail, to + tail);
-    env[name] = db.write(to);
-    copied.push({ name, from: original, to });
-    said.push(`Your app runs against a copy of ${rel(original)}, through ${name}; the original is not touched, and the copy is deleted when this command ends and made again from the original at the next connect.`);
-  }
-  for (const o of opened) {
-    const path = resolve(dirname(o.file), o.path);
-    const cwdPath = resolve(appDir, o.path);
-    if ([path, cwdPath].some((p) => moved.has(p) || [...moved].some((m) => p.startsWith(m + "/")))) continue;
-    moved.add(cwdPath);
-    held.push(heldFile(cwdPath, basename(o.path), `Your app opens ${o.path} by a path written in ${rel(o.file)}, with no setting to point it at a copy, so test conversations would save into that file. Read its path from a setting, then run the connect command again.`));
-  }
-  return { env, copied, said, held, values: all };
-}