cortad 0.3.10 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "cortad",
3
- "version": "0.3.10",
3
+ "version": "0.4.0",
4
4
  "description": "Connects the AI app on your machine to Cortad for test conversations, and gives your coding agent the MCP and skill to run them. No dependencies.",
5
5
  "bin": {
6
6
  "cortad": "local.mjs"
@@ -10,50 +10,39 @@
10
10
  "README.md",
11
11
  "lib/app-port.mjs",
12
12
  "lib/auto-reach.mjs",
13
- "lib/backing.mjs",
14
13
  "lib/calls.mjs",
15
14
  "lib/cli.mjs",
15
+ "lib/commits.mjs",
16
16
  "lib/contact-text.mjs",
17
- "lib/data.mjs",
18
17
  "lib/dissection-text.mjs",
19
18
  "lib/document.mjs",
20
19
  "lib/failed-text.mjs",
20
+ "lib/files.mjs",
21
21
  "lib/fresh.mjs",
22
22
  "lib/health.mjs",
23
23
  "lib/held.mjs",
24
24
  "lib/home.mjs",
25
- "lib/hosted.mjs",
25
+ "lib/env-files.mjs",
26
26
  "lib/env-secrets.mjs",
27
- "lib/identities.mjs",
28
- "lib/keeping.mjs",
29
27
  "lib/keys.mjs",
30
- "lib/ledger.mjs",
31
- "lib/listing.mjs",
32
28
  "lib/lock.mjs",
33
29
  "lib/mcp.mjs",
34
- "lib/mint.mjs",
35
- "lib/mongo.mjs",
36
30
  "lib/numbers-text.mjs",
37
- "lib/pg.mjs",
38
31
  "lib/proc.mjs",
39
32
  "lib/proof.mjs",
40
33
  "lib/proxy.mjs",
41
- "lib/qdrant.mjs",
42
34
  "lib/pyhook/sitecustomize.py",
43
35
  "lib/pyhook/cortad_tied.py",
44
36
  "lib/pyhook/cortad_sites.py",
45
37
  "lib/reach.mjs",
46
38
  "lib/read-text.mjs",
47
- "lib/redis.mjs",
48
39
  "lib/register.mjs",
49
40
  "lib/replay.mjs",
50
41
  "lib/runner.mjs",
51
42
  "lib/sample.mjs",
52
- "lib/service.mjs",
53
43
  "lib/spec.mjs",
54
44
  "lib/start.mjs",
55
45
  "lib/stick.mjs",
56
- "lib/stores.mjs",
57
46
  "lib/switches.mjs",
58
47
  "lib/tether.mjs",
59
48
  "lib/text.mjs",
@@ -61,6 +50,7 @@
61
50
  "lib/trace.cjs",
62
51
  "lib/wire.cjs",
63
52
  "lib/verbs.mjs",
53
+ "lib/intel-text.mjs",
64
54
  "lib/writes.mjs",
65
55
  "lib/words.mjs",
66
56
  "local.mjs",
package/skill/SKILL.md CHANGED
@@ -30,23 +30,26 @@ The person owns the product. They may not be reading the code with you, so every
30
30
 
31
31
  ## Connecting
32
32
 
33
- `npx {{cortad}}` in the repository's folder connects it, with nothing to copy. Run it as a background command (in Claude Code: Bash with run_in_background) and leave it running: it starts the app and keeps it reachable from the browser until it is stopped.
34
- - A folder connected before reconnects at once with the key it kept.
35
- - A folder connected for the first time prints a link and four characters, good for ten minutes. Give the person both as printed; they approve the link in their browser, signed in to Cortad, and the command goes on by itself.
36
- - When a result says no connect command is running, run `npx {{cortad}}` again in this folder. No code is needed from the person.
33
+ Cortad does not guess how the app runs: you tell it, in one line, from the repository's folder, run as a background command (in Claude Code: Bash with run_in_background) and left running.
37
34
 
38
- The connect command prints "Go back to the browser" once it is connected. Cortad then finds the AI endpoints in the app's code and sends each one a test request itself, from this machine to the app on its port.
35
+ - `npx {{cortad}} -- <how you start your app>`: your start command, as you or the person would type it, from a terminal that holds the app's settings. Cortad runs it as written with its hook inside and nothing else added: no env file is read into the app, no database is copied, no service is started. Use the environment the person wants tested (their staging settings, their existing databases): whatever the command gives the app is what the run plays against.
36
+ - `npx {{cortad}} --port <port>`: an app that is already running and should stay as it is; Cortad sees its replies and not its model calls, and says so. With `-- <how you start your app>` beside it, an app in a container: Cortad starts it, watches the port you name, and prints the model setting to give the container so its model calls are seen too.
37
+ - Run with neither and Cortad prints the one start command the repository makes obvious and stops; run again with it after `--`, or with the right one.
38
+ - A folder connected before reconnects with the key and the start command it kept. A first connect prints a link and four characters, good for ten minutes: give the person both as printed; they approve the link in their browser, signed in to Cortad.
39
+ - When a result says no connect command is running, run the same line again in this folder.
40
+
41
+ Once it prints "Go back to the browser", Cortad lists the AI endpoints it found in the code. Send each one message the way a user would: through the app's own page, or a request from this machine, signed in as a real test account where the app needs one (ask the person which account; Cortad makes none). That one request is what Cortad learns the endpoint from: its route, its body, its sign-in and where the answer comes back. Cortad sends nothing to the app before that, and nothing of its own before the person's yes on the card.
39
42
 
40
43
  1. Call `status` and tell the person, in three or four short lines:
41
- - that their app is connected, or the one line that says why it did not start;
44
+ - that their app is connected, or the one line that says why it did not start (what the app printed is the app's own output);
42
45
  - what Cortad's audit of their AI code found: how many problems, and the one that matters most, in plain words. `status` with show `reviews` holds up to 45 seconds a call while the code is still being read;
43
46
  - the line on what a run plays and costs, as printed;
44
47
  - what happens next: they press Run on the card in the browser, or tell you to start it.
45
48
 
46
49
  The endpoints, the files and the audit's rows are yours to work from. They go to the person when they ask, or when you propose a fix.
47
- 2. When `status` lists an endpoint as `Needs a signed-in user`, Cortad could not make a sign-in for it on this machine. Tell the person; when they ask for it, sign in a test account through the app's own sign-in on this machine, send that endpoint one request in that session, then call `status` again. That is the only sign-in this needs, and only for the endpoints listed that way.
48
- 3. A failure on the app's side that every conversation would repeat, such as a model provider refusing, a retrieval with no answer or a tool returning an error: say what you would change, change it when the person agrees, then call `reach` to send Cortad's test request to that endpoint again.
49
- 4. A question for the person on the card in the browser is said to them once, in one line. The first run starts when the person presses Run on the card, or asks you, and then you call `run`. Before you stop, start `npx -y {{cortad}} wait` as a background command (in Claude Code: Bash with run_in_background). It returns when the next run or verify ends, with what it found, and its ending wakes you. Then follow "After a run".
50
+ 2. An endpoint `status` lists as not answered yet is waiting for your one message to it. One it lists as `Needs a signed-in user` was refused: send it again signed in as a test account the person names, and call `status` again.
51
+ 3. A failure on the app's side that every conversation would repeat, such as a model provider refusing, a retrieval with no answer or a tool returning an error: say what you would change, change it when the person agrees, then send that endpoint one message again, or call `reach` to have Cortad send the request it read for it.
52
+ 4. The one question on the card in the browser is where test conversations will write: into the databases the app wrote into for your message, named there, with nothing copied. Say it to the person once, in one line. The first run starts when the person presses Run on the card, or asks you, and then you call `run`. Before you stop, start `npx -y {{cortad}} wait` as a background command (in Claude Code: Bash with run_in_background). It returns when the next run or verify ends, with what it found, and its ending wakes you. Then follow "After a run".
50
53
 
51
54
  `status` with `show` lists one section in full: `rules`, `standards`, `journeys`, `endpoints`, `trials`, `records` or `reviews` (in a shell, `npx {{cortad}} status rules`). `status` with show `machine` says what Cortad does on this machine; hand the person its lines when they ask.
52
55
 
@@ -96,6 +99,10 @@ After the fixes, `run` again. It plays what your changes since the last full run
96
99
 
97
100
  Once a fix is verified gone and production is not connected, the result offers production: connect it to see what real users are doing as it happens and where the AI lets them down. `field_connect` gives the steps; the owner creates the key in the browser. `field` gives the numbers; message text stays out.
98
101
 
102
+ ## After a change: the one call, and a world
103
+
104
+ `test <what you built or want tested>` is the call after a change. Cortad decides what kind of test that is: a world, where this app has one and the sentence names something it can plant, or a run of the app's chat surfaces; the answer says which started and the call that follows it. A world is for an AI that is not a chat, a worker that reads data and writes a verdict: Cortad builds a copy of the product's data on this machine from the app's own seed, plants problems whose answers are known, runs the app's own detection and investigation on each, and scores what it raised and published: found, missed, false alarms where nothing was planted, flips between repeats. `world <ask>` asks for one outright, `world_status <id>` follows it, and the person sees it on the World page. A repository no world is written for yet says so: send `feedback` naming the AI to test, and the Cortad team writes one.
105
+
99
106
  ## Making it stick
100
107
 
101
108
  `npx {{cortad}} stick` makes this part of the repository: one line in AGENTS.md, CLAUDE.md, the Cursor rules and the Copilot instructions; a hook after each edit that names the changed prompt and tool files; and a hook on the person's prompt that hands you a run or verify that ended, for clients that a background command cannot wake. Run it only when the person asks for it. `npx {{cortad}} unstick` takes them out.
@@ -18,7 +18,7 @@ Free: 1 of 1 run left this month; 60 verify conversations left. Pro $99 a month:
18
18
  App: Your app is starting on this machine.
19
19
  ```
20
20
 
21
- Once each endpoint that reaches the model has answered a test request, two messages on the chat, and before any run:
21
+ Once a request of your own has reached the model through an endpoint, two messages on the chat, and before any run:
22
22
 
23
23
  ```
24
24
  Repository: tutor-app.
@@ -34,15 +34,15 @@ Endpoints that answered (1):
34
34
  Endpoints found in your code that have not answered yet (2):
35
35
  POST /api/homework/explain apps/api/src/routes/homework.ts:18 Cortad's test request was answered with HTTP 500.
36
36
  POST /api/progress/summary apps/api/src/routes/progress.ts:9 Needs a signed-in user.
37
- Cortad sends each of them a test request.
37
+ Send each one message as a user would, signed in where your app needs it, or call reach to send the request Cortad read for them.
38
38
  A run plays 36 simulated users in about 6 minutes. About $0.38 on your OpenAI key.
39
39
  A run starts when the person presses Run on the card in the browser or asks you for one; if they say they pressed it, call run rather than ask them again.
40
40
  ```
41
41
 
42
- When the app keeps its data in a hosted store no copy was made of, the decision is the person's, said once as the last line before the link:
42
+ Before the person has said yes on the card, the last line before the link is the one question asked of them:
43
43
 
44
44
  ```
45
- DATABASE_URL points at the Postgres database shop that is not on this machine, and no migrations were found to build a copy on this machine from, so test conversations would save into it. Point DATABASE_URL at a database on this machine, then run the connect command again. Or the person allows it on the card in the browser: one press lets test conversations save into it.
45
+ Test conversations use your app as it is: they write where it wrote during your own request (127.0.0.1:5432, 127.0.0.1:27017) and reach what it reached (api.openai.com). Nothing is copied. The person allows it on the card in the browser, with one press.
46
46
  The person sees this at https://cortad.com/lab
47
47
  ```
48
48
 
@@ -95,7 +95,7 @@ Rules in the code: 16, in 5 files.
95
95
 
96
96
  ## reach
97
97
 
98
- Sends Cortad's test request again, from the connect command on this machine, after a fix:
98
+ Sends the request Cortad read from your code to an endpoint no request of yours has reached, from the connect command on this machine, with the sign-in you pass it:
99
99
 
100
100
  ```
101
101
  Sent 1 request to your app on port 3100, up to four at a time:
package/lib/backing.mjs DELETED
@@ -1,230 +0,0 @@
1
- // The services your app reaches on this machine, brought up when nothing answers there and the
2
- // repository says how: a store its compose file runs from an image, with Docker, or a second
3
- // service of its own, with that service's own start. Only a port the app is known to reach is
4
- // brought up: one its settings name when it could not start without it, or one the hook saw it
5
- // reach for. A compose file often lists a store the app is set up not to use, and starting it
6
- // would be a download of gigabytes nobody asked for. Each is stopped when this command ends, and
7
- // one that could not be brought up, or that nothing here brings up, is said before a run, with whose
8
- // side it is on.
9
- import { execFile } from "node:child_process";
10
- import { existsSync, readdirSync, readFileSync } from "node:fs";
11
- import { connect } from "node:net";
12
- import { basename, dirname, join, relative } from "node:path";
13
- import { ledger } from "./ledger.mjs";
14
- import { servicePort } from "./service.mjs";
15
- import { startPlan, workspaces } from "./start.mjs";
16
-
17
- // Something accepts a connection on the port: a database answers no HTTP, so this is the test.
18
- export function tcpOpen(host, port, ms = 1500) {
19
- return new Promise((done) => {
20
- const s = connect({ host: host && !host.startsWith("/") && host !== "0.0.0.0" ? host.replace(/^\[|\]$/g, "") : "127.0.0.1", port });
21
- const end = (ok) => { s.destroy(); done(ok); };
22
- s.setTimeout(ms, () => end(false));
23
- s.once("connect", () => end(true));
24
- s.once("error", () => end(false));
25
- });
26
- }
27
- async function waitTcp(port, ms, gone = () => false) {
28
- for (const until = Date.now() + ms; Date.now() < until && !gone(); await new Promise((r) => setTimeout(r, 1000))) if (await tcpOpen("127.0.0.1", port)) return true;
29
- return false;
30
- }
31
-
32
- const COMPOSE_FILE = /^(?:docker-)?compose(?:\.[\w-]+)?\.ya?ml$/;
33
- // The files compose reads by default come first: a second file beside them is a variant.
34
- const DEFAULT_FIRST = (a, b) => Number(/^(?:docker-)?compose\.ya?ml$/.test(basename(b))) - Number(/^(?:docker-)?compose\.ya?ml$/.test(basename(a))) || a.localeCompare(b);
35
- export function composeFiles(dirs) {
36
- const out = new Set();
37
- for (const dir of dirs) { try { for (const f of readdirSync(dir)) if (COMPOSE_FILE.test(f)) out.add(join(dir, f)); } catch { /* no folder */ } }
38
- return [...out].sort(DEFAULT_FIRST);
39
- }
40
-
41
- const unquote = (v) => String(v ?? "").trim().replace(/^(['"])(.*)\1$/, "$2");
42
- // ${PG_PORT:-5432} and ${PG_PORT}, from the settings the app starts with.
43
- const interpolate = (v, values) => v.replace(/\$\{(\w+)(?::?-([^}]*))?\}/g, (_, name, dflt) => values[name] || dflt || "");
44
- // The host port of one port mapping: "5432:5432", "127.0.0.1:5432:5432/tcp". None for a container
45
- // port alone (compose picks the host's) or a range.
46
- function published(entry, values) {
47
- const parts = interpolate(unquote(entry), values).replace(/\/(?:tcp|udp)$/, "").split(":");
48
- const host = parts.length >= 2 ? parts[parts.length - 2] : "";
49
- return /^\d+$/.test(host) ? Number(host) : null;
50
- }
51
-
52
- // The services a compose file declares: name, whether it runs an image or builds this repository's
53
- // code, and the host ports it publishes. A reader of the shape compose files are written in, not of
54
- // YAML at large.
55
- // ponytail: no anchors, extends, include or profiles; add them when a repository needs one to be read.
56
- export function composeServices(text, values = {}) {
57
- const out = [];
58
- let inServices = false, nameIndent = -1, keyIndent = -1, svc = null, portsIndent = -1;
59
- for (const raw of String(text).split("\n")) {
60
- const line = raw.replace(/\t/g, " ").replace(/\s+#.*$/, "");
61
- const t = line.trim();
62
- if (!t || t.startsWith("#")) continue;
63
- const indent = line.length - line.trimStart().length;
64
- if (indent === 0) { inServices = /^services:\s*$/.test(t); svc = null; continue; }
65
- if (!inServices) continue;
66
- if (nameIndent < 0) nameIndent = indent;
67
- if (indent <= nameIndent) {
68
- const m = /^["']?([\w.-]+)["']?:\s*$/.exec(t);
69
- svc = indent === nameIndent && m ? { name: m[1], image: null, build: false, ports: [] } : null;
70
- if (svc) out.push(svc);
71
- keyIndent = -1; portsIndent = -1;
72
- continue;
73
- }
74
- if (!svc) continue;
75
- if (portsIndent >= 0 && indent > portsIndent) {
76
- const item = /^-\s*(.*)$/.exec(t);
77
- const port = item && !/^[A-Za-z_]\w*:(?:\s|$)/.test(item[1]) ? published(item[1], values) : Number(unquote(/^(?:-\s*)?published:\s*(.+)$/.exec(t)?.[1] ?? "")) || null;
78
- if (port) svc.ports.push(port);
79
- continue;
80
- }
81
- portsIndent = -1;
82
- if (keyIndent < 0) keyIndent = indent;
83
- if (indent !== keyIndent) continue;
84
- const kv = /^([\w-]+):\s*(.*)$/.exec(t);
85
- if (!kv) continue;
86
- if (kv[1] === "image") svc.image = unquote(kv[2]);
87
- else if (kv[1] === "build") svc.build = true;
88
- else if (kv[1] === "ports") {
89
- if (kv[2].startsWith("[")) for (const e of kv[2].replace(/^\[|\]$/g, "").split(",")) { const p = published(e, values); if (p) svc.ports.push(p); }
90
- else portsIndent = indent;
91
- }
92
- }
93
- return out;
94
- }
95
-
96
- // A setting that is an address a page is opened from, not a service the app calls.
97
- // The app's own address and sign-in redirects are pages, not services: ai-appointment-setter's
98
- // NEXT_PUBLIC_APP_URL serves only its OAuth redirect, and a run was held on it.
99
- const PAGE = /(?:^|_)(?:ORIGINS?|FRONTEND|CLIENT|SITE|WEB|CALLBACK|REDIRECT|CORS|ALLOWED|HOMEPAGE|APP|BASE|NEXTAUTH)(?:_|$)/;
100
- // Addresses on this machine the app's settings point at: http://localhost:4000/api under API_URL.
101
- export function localAddresses(values) {
102
- const byPort = new Map();
103
- for (const [name, raw] of Object.entries(values)) {
104
- if (PAGE.test(name)) continue;
105
- let u;
106
- try { u = new URL(String(raw ?? "").trim()); } catch { continue; }
107
- if (!/^https?:$/.test(u.protocol) || !u.port || !/^(?:localhost|127(?:\.\d+){3}|\[::1\]|0\.0\.0\.0)$/i.test(u.hostname)) continue;
108
- const port = Number(u.port);
109
- byPort.set(port, [...(byPort.get(port) ?? []), name]);
110
- }
111
- return [...byPort].map(([port, names]) => ({ port, names }));
112
- }
113
-
114
- const run = (bin, args, opts = {}) => new Promise((done) => {
115
- execFile(bin, args, { timeout: 180_000, maxBuffer: 4 * 1024 * 1024, ...opts }, (err, stdout, stderr) => done({ ok: !err, out: String(stdout ?? ""), err: String(stderr ?? err?.message ?? "") }));
116
- });
117
- // A service's own last words go into a sentence that leaves this machine: an address's password never does.
118
- const lastLine = (text) => String(text).replace(/\x1b\[[0-9;]*[A-Za-z]/g, "").replace(/:\/\/[^\s/@:]+:[^\s/@]+@/g, "://[masked]@").split("\n").map((l) => l.trim()).filter(Boolean).at(-1)?.slice(0, 160) ?? "";
119
-
120
- // How compose is run here: the docker plugin, the older standalone binary, or not at all and why.
121
- async function composeCommand(onPath) {
122
- if (!onPath("docker") && !onPath("docker-compose")) return { why: "Docker is not installed on this machine" };
123
- if (!(await run("docker", ["info"], { timeout: 15_000 })).ok) return { why: "Docker is not running on this machine" };
124
- if ((await run("docker", ["compose", "version"], { timeout: 15_000 })).ok) return { bin: "docker", pre: ["compose"] };
125
- return onPath("docker-compose") ? { bin: "docker-compose", pre: [] } : { why: "Docker here has no compose command" };
126
- }
127
-
128
- const upper = (s) => s.charAt(0).toUpperCase() + s.slice(1);
129
- // Why a port is wanted: a setting names it, or the app was seen reaching for it.
130
- const pointer = (w) => (!w.names?.length ? "Your app reached for"
131
- : w.names.length === 1 ? `${w.names[0]} points at` : `${w.names[0]} and ${w.names.length - 1} more setting${w.names.length > 2 ? "s" : ""} point at`);
132
- const said = {
133
- up: (what, how, w) => `Started ${what} ${how}: ${pointer(w)} port ${w.port} on this machine and nothing answered there. It is stopped when this command ends.`,
134
- theirs: (what, why, w) => `${pointer(w)} port ${w.port} on this machine and nothing answers there. ${upper(what)} would start it, but ${why}: start it yourself before a run. This is on your side.`,
135
- exited: (what, tail, w) => `${pointer(w)} port ${w.port} on this machine, which ${what} serves, and it stopped before it answered: ${tail}. This is on your side.`,
136
- none: (w) => `${pointer(w)} port ${w.port} on this machine and nothing answers there, and nothing in this repository starts it: start it yourself before a run. This is on your side.`,
137
- slow: (what, secs, w) => `${pointer(w)} port ${w.port} on this machine; ${what} was started for it, but nothing answered there within ${secs} seconds. This is on our side.`,
138
- };
139
-
140
- // One session's services. up(wants) brings up each wanted port it can, where nothing answers: a
141
- // compose store first, else a workspace of this repository that serves it. `launch` starts a
142
- // workspace the way the app is started and hands back its process. Answers what it said and the
143
- // ports that now answer.
144
- export function makeBacking({ root, appDir, values, onPath, ledgerFile, launch, adopt = true, storeWaitMs = 120_000, serviceWaitMs = 90_000 }) {
145
- const book = ledger(ledgerFile);
146
- const started = [];
147
- const tried = new Set();
148
- const lastSaid = new Map();
149
- let compose = null;
150
- const files = composeFiles([...new Set([appDir, root])]).map((file) => ({ file, services: (() => { try { return composeServices(readFileSync(file, "utf8"), values); } catch { return []; } })() }));
151
- // Services a session that died left running are stopped by the next session that starts the app,
152
- // never removed: the person may have used them since, and a removed container takes what they put
153
- // in it. A session attached to an app already running leaves them to that app.
154
- for (const row of adopt ? book.orphans("service") : []) {
155
- const [file, service] = row.key.split("#");
156
- if (!existsSync(file)) continue;
157
- const kept = { ...row, made: "stop" };
158
- book.remove(row);
159
- book.add(kept);
160
- started.push({ file, service, row: kept });
161
- }
162
-
163
- async function fromCompose(found, w) {
164
- const where = `the ${found.service} service in ${relative(root, found.file) || basename(found.file)}`;
165
- compose ??= await composeCommand(onPath);
166
- if (compose.why) return said.theirs(where, compose.why, w);
167
- const opts = { cwd: dirname(found.file), env: { ...process.env } };
168
- const args = (...a) => [...compose.pre, "-f", found.file, ...a];
169
- const existed = Boolean((await run(compose.bin, args("ps", "-a", "-q", found.service), opts)).out.trim());
170
- const up = await run(compose.bin, args("up", "-d", found.service), opts);
171
- if (!up.ok) return said.theirs(where, `Docker could not start it: ${lastLine(up.err)}`, w);
172
- const row = { kind: "service", key: `${found.file}#${found.service}`, made: existed ? "stop" : "remove" };
173
- book.add(row);
174
- started.push({ file: found.file, service: found.service, row });
175
- if (!(await waitTcp(w.port, storeWaitMs))) return said.slow(where, Math.round(storeWaitMs / 1000), w);
176
- return said.up(where, "with Docker", w);
177
- }
178
-
179
- async function fromWorkspace(dir, w) {
180
- const plan = startPlan({ root: dir, onPath });
181
- if (!plan?.cmd) return null;
182
- const what = relative(root, dir) || basename(dir);
183
- const kid = launch(plan);
184
- let exited = null;
185
- let tail = "";
186
- kid.on("exit", (code) => { exited = code ?? 1; });
187
- const keep = (d) => { tail = (tail + d).slice(-2000); };
188
- kid.stdout?.on("data", keep);
189
- kid.stderr?.on("data", keep);
190
- if (await waitTcp(w.port, serviceWaitMs, () => exited !== null)) return said.up(what, `with ${plan.cmd}`, w);
191
- if (exited !== null) return said.exited(what, lastLine(tail) || `it exited ${exited}`, w);
192
- return said.slow(what, Math.round(serviceWaitMs / 1000), w);
193
- }
194
-
195
- return {
196
- // wants: [{ port, names, store }], each a port on this machine the app reaches or its settings
197
- // name; `store` lets a compose image serve it. Answers what was said and the ports brought up.
198
- // `down`: the ports a setting names that still answer nothing after this, each with what was
199
- // said of it: a run without them is a run against an app missing a part, and holds.
200
- async up(wants) {
201
- const lines = [];
202
- const ports = [];
203
- const down = [];
204
- const still = (w, why) => { if (w.names?.length && why) down.push({ port: w.port, names: w.names, why }); };
205
- for (const w of wants) {
206
- if (await tcpOpen("127.0.0.1", w.port)) continue;
207
- if (tried.has(w.port)) { still(w, lastSaid.get(w.port)); continue; }
208
- const image = w.store ? files.flatMap(({ file, services }) => services.filter((x) => x.image && !x.build && x.ports.includes(w.port)).map((x) => ({ file, service: x.name })))[0] : null;
209
- const dir = image ? null : workspaces(root).find((d) => d !== appDir && servicePort(d) === w.port);
210
- // A port only seen reached, and not a store's, may be a passing connection of the app's own.
211
- if (!image && !dir && !w.names?.length && !w.store) continue;
212
- tried.add(w.port);
213
- const line = !image && !dir ? said.none(w) : image ? await fromCompose(image, w) : await fromWorkspace(dir, w);
214
- if (line) { lines.push(line); lastSaid.set(w.port, line); }
215
- if (await tcpOpen("127.0.0.1", w.port)) ports.push(w.port);
216
- else still(w, line);
217
- }
218
- return { lines, ports, down };
219
- },
220
- // Compose services only: a workspace's process is the caller's, stopped with the app.
221
- async stop() {
222
- compose ??= started.length ? await composeCommand(onPath) : null;
223
- if (!compose || compose.why) return;
224
- for (const { file, service, row } of started.splice(0)) {
225
- const how = row.made === "stop" ? ["stop", service] : ["rm", "-s", "-f", service];
226
- if ((await run(compose.bin, [...compose.pre, "-f", file, ...how], { cwd: dirname(file), env: { ...process.env } })).ok) book.remove(row);
227
- }
228
- },
229
- };
230
- }
package/lib/data.mjs DELETED
@@ -1,137 +0,0 @@
1
- // Your app's own database, kept out of the run's way. A run creates people, conversations and
2
- // records by the hundred; written into the file your app keeps its real data in, they stay there
3
- // after the run. So an app started here runs against a copy of its database file, made in this
4
- // command's own temp folder and pointed at through the same variable, and the original is never
5
- // opened. What cannot be copied (a file your code names with no variable in front of it, an app that
6
- // was already running) is said before the first run, in one sentence. A database server is copied
7
- // on its own server (lib/stores.mjs).
8
- import { copyFileSync, cpSync, existsSync, mkdirSync, readdirSync, readFileSync, statSync } from "node:fs";
9
- import { basename, dirname, isAbsolute, join, relative, resolve } from "node:path";
10
- import { storesOf } from "./stores.mjs";
11
-
12
- const FILE_DB = /\.(?:db|sqlite3?|duckdb)$/i;
13
- // sqlite:///x.db (SQLAlchemy, relative), sqlite:////abs/x.db, sqlite+aiosqlite:///x.db, file:./dev.db (Prisma, libsql).
14
- const FILE_URL = /^(sqlite(?:\+\w+)?:\/\/\/|file:(?:\/\/(?=\/))?)([^?]*)(\?.*)?$/i;
15
-
16
- // What one value names: a database file, with how to write the value again for another path, or nothing.
17
- export function databaseOf(value) {
18
- const v = String(value ?? "").trim();
19
- if (!v) return null;
20
- const url = FILE_URL.exec(v);
21
- if (url) {
22
- const path = decodeURIComponent(url[2]);
23
- if (!path || path === ":memory:" || path.startsWith(":memory")) return null;
24
- return { path, write: (to) => `${url[1]}${to}${url[3] ?? ""}`, prisma: /^file:/i.test(url[1]) };
25
- }
26
- if (!v.includes("://") && FILE_DB.test(v)) return { path: v, write: (to) => to };
27
- return null;
28
- }
29
-
30
- // A folder the app keeps its data in, named by a variable (DATA_DIR=./data): a database file sits
31
- // directly inside it. The whole folder is the database then, uploads and backups beside it.
32
- const FOLDER_NAME = /DATA|DB|DATABASE|STORAGE|STORE|PERSIST|SQLITE|DUCK/i;
33
- function dataFolder(name, value, dirs, appDir, root) {
34
- const v = String(value ?? "").trim();
35
- if (!FOLDER_NAME.test(name) || !v || v.includes("://") || FILE_DB.test(v) || !/^[.~/\w]/.test(v)) return null;
36
- for (const place of [appDir, ...dirs].map((d) => resolve(d, v))) {
37
- if (place === root || place === appDir || !place.startsWith(root + "/")) continue;
38
- let names;
39
- try { if (!statSync(place).isDirectory()) continue; names = readdirSync(place); } catch { continue; }
40
- if (names.some((n) => FILE_DB.test(n))) return place;
41
- }
42
- return null;
43
- }
44
- const sizeOf = (dir, cap) => {
45
- let total = 0;
46
- const walk = (d) => { for (const e of readdirSync(d, { withFileTypes: true })) { if (total > cap) return; const p = join(d, e.name); if (e.isDirectory()) walk(p); else if (e.isFile()) total += statSync(p).size; } };
47
- try { walk(dir); } catch { /* counted what it could */ }
48
- return total;
49
- };
50
-
51
- // A name the code reads with a default written beside it: os.getenv("DATABASE_URL", "sqlite:///./app.db"),
52
- // process.env.DB_PATH ?? "./data.db". With nothing in the environment, the default is the database.
53
- const DEFAULTED = /(?:os\.(?:environ\.get|getenv)|\bgetenv)\(\s*['"]([A-Za-z_]\w*)['"]\s*,\s*['"]([^'"\n]+)['"]|process\.env(?:\.([A-Za-z_]\w*)|\[\s*['"]([A-Za-z_]\w*)['"]\s*\])\s*(?:\?\?|\|\|)\s*['"]([^'"\n]+)['"]/g;
54
- // A database file opened by a path written in the code, with no variable in front of it.
55
- const OPENED = /(?:sqlite3\.connect|aiosqlite\.connect|duckdb\.connect|new\s+(?:sqlite3\.)?Database|create_(?:async_)?engine)\(\s*['"]([^'"\n]+)['"]/g;
56
- export const CODE = /\.(?:[cm]?[jt]sx?|py|rb|go|php)$/;
57
- // A test opens its own scratch file; that is not where the app keeps its data.
58
- export const TEST = /(?:^|\/)(?:tests?|__tests__|spec|e2e|fixtures?)\/|[._-](?:test|spec)\.[^/]+$|(?:^|\/)(?:test_[^/]+|conftest)\.py$/;
59
- // A folder bigger than this is not copied: said instead, so the person decides.
60
- const FOLDER_CAP = 1024 ** 3;
61
-
62
- function scan(sources) {
63
- const defaults = {};
64
- const opened = [];
65
- for (const file of sources) {
66
- if (!CODE.test(file) || TEST.test(file)) continue;
67
- let text = "";
68
- try { if (statSync(file).size > 512_000) continue; text = readFileSync(file, "utf8"); } catch { continue; }
69
- for (const m of text.matchAll(DEFAULTED)) {
70
- const name = m[1] ?? m[3] ?? m[4];
71
- const value = m[2] ?? m[5];
72
- if (name && !(name in defaults) && (databaseOf(value) || storesOf({ [name]: value }).length)) defaults[name] = value;
73
- }
74
- for (const m of text.matchAll(OPENED)) { const db = databaseOf(m[1]); if (db?.path) opened.push({ file, path: db.path }); }
75
- }
76
- return { defaults, opened };
77
- }
78
-
79
- // Where a relative path lands. The app's own folder first, which is where it runs; then the folder
80
- // of the env file that named it, which some settings loaders resolve against; for Prisma, the folder
81
- // its schema lives in. The first place the file exists is the one; none, and it is the app's folder.
82
- function located(db, dirs, appDir) {
83
- if (isAbsolute(db.path)) return db.path;
84
- const prisma = db.prisma ? [join(appDir, "prisma")] : [];
85
- const places = [...prisma, appDir, ...dirs].map((d) => resolve(d, db.path));
86
- return places.find((p) => existsSync(p)) ?? places[0];
87
- }
88
-
89
- // values: the environment the app will start with, names to values. from: the env file each name
90
- // was read from, when there is one. Returns the variables to start the app with, what to say, and
91
- // the settings with the code's own defaults beside them, which the stores are read from.
92
- export function keepData({ values, from = {}, appDir, root = appDir, work, sources = [], started = true }) {
93
- const { defaults, opened } = scan(sources);
94
- const all = { ...defaults, ...values };
95
- const env = {};
96
- const copied = [];
97
- const said = [];
98
- const moved = new Set();
99
- const rel = (p) => relative(root, p) || basename(p);
100
- for (const [name, value] of Object.entries(all)) {
101
- const db = databaseOf(value);
102
- const folder = db ? null : dataFolder(name, value, from[name] ? [dirname(from[name])] : [], appDir, root);
103
- if (folder) {
104
- moved.add(folder);
105
- if (!started) { said.push(`Your app was already running when this command started, so it writes to its own data folder, ${rel(folder)}/: test conversations write into it. Stop your app and run the command again to run against a copy.`); continue; }
106
- if (sizeOf(folder, FOLDER_CAP) > FOLDER_CAP) { said.push(`Your data folder ${rel(folder)}/ (${name}) is over 1 GB, so it is not copied: test conversations write into it.`); continue; }
107
- const to = join(work, "data", String(copied.length), basename(folder));
108
- cpSync(folder, to, { recursive: true });
109
- env[name] = to;
110
- copied.push({ name, from: folder, to });
111
- said.push(`Your app runs against a copy of its data folder ${rel(folder)}/, through ${name}; the original is not touched, and the copy is deleted when this command ends and made again from the original at the next connect.`);
112
- continue;
113
- }
114
- if (!db) continue;
115
- const original = located(db, from[name] ? [dirname(from[name])] : [], appDir);
116
- moved.add(original);
117
- if (!started) {
118
- said.push(`Your app was already running when this command started, so it writes to its own database, ${rel(original)}: test conversations write into it. Stop your app and run the command again to run against a copy.`);
119
- continue;
120
- }
121
- const to = join(work, "data", String(copied.length), basename(original));
122
- mkdirSync(dirname(to), { recursive: true });
123
- // The file and the journal beside it: SQLite keeps recent writes in -wal until a checkpoint.
124
- if (existsSync(original)) for (const tail of ["", "-wal", "-shm", "-journal"]) if (existsSync(original + tail)) copyFileSync(original + tail, to + tail);
125
- env[name] = db.write(to);
126
- copied.push({ name, from: original, to });
127
- said.push(`Your app runs against a copy of ${rel(original)}, through ${name}; the original is not touched, and the copy is deleted when this command ends and made again from the original at the next connect.`);
128
- }
129
- for (const o of opened) {
130
- const path = resolve(dirname(o.file), o.path);
131
- const cwdPath = resolve(appDir, o.path);
132
- if ([path, cwdPath].some((p) => moved.has(p) || [...moved].some((m) => p.startsWith(m + "/")))) continue;
133
- moved.add(cwdPath);
134
- said.push(`Your app opens ${o.path} by a path written in ${rel(o.file)}, with no variable to point it at a copy: test conversations write into that file.`);
135
- }
136
- return { env, copied, said, values: all };
137
- }