agent-led-growth 0.6.4 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agent-led-growth",
3
- "version": "0.6.4",
3
+ "version": "0.8.0",
4
4
  "description": "Score: evals for agents as customers. Runs a real agent through signup, key and first call on your product, before you ship. Track, the analytics half, measures which agents convert after.",
5
5
  "author": {
6
6
  "name": "Kat Laszlo",
package/CHANGELOG.md CHANGED
@@ -1,5 +1,14 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.8.0 (2026-10-23)
4
+
5
+ - `alg connect`: sets up Track's key without copying a secret by hand. It prints a code and opens the browser; once approved it writes `AGENT_TRAFFIC_KEY` (and `AGENT_TRAFFIC_PROJECT_ID`, not a secret) to `.env` or `--env-file`, replacing existing lines for those names and keeping the rest, and creates a new file with mode 0600. It warns when the file is not gitignored in a git repo and offers to add it. In a folder linked to Vercel, or with `--vercel`, it offers to run `vercel env add AGENT_TRAFFIC_KEY production` with the key on stdin. The key is never printed, logged or put on a command line. `--endpoint` or `AGENT_TRAFFIC_ENDPOINT` points it at another Track server, which is then written as `AGENT_TRAFFIC_ENDPOINT`.
6
+
7
+ ## 0.7.0 (2026-10-23)
8
+
9
+ - `test` no longer stops when `alg.yml` has no `verify_call` and the product has no OpenAPI document. At a terminal it asks for a GET that needs a key (and the header that carries it, default `Authorization: Bearer {key}`), checks the answer, saves it to `alg.yml` and continues the run; Enter with nothing skips the check for that run. Answering "n" to an inferred call asks the same way instead of cancelling. Before, it printed `verify_call_invalid` and left the person to edit the file and start over.
10
+ - `test --skip-verify`: the signup runs and the agent's key is not checked, for CI and products with no API call to prove a key. The result passes when the agent saved a key, and says the key was not checked; `--json` carries `checker.skipped: true`. Without a terminal, `verify_call_invalid` names the flag.
11
+
3
12
  ## 0.6.0 (2026-10-22)
4
13
 
5
14
  - One name. The command is `alg` (`npx agent-led-growth` still runs it); the config file is `alg.yml` and the run folder `.alg/`. The skills are `alg` and `alg-test`, the plugin `agent-led-growth`. `agent-ready`, `agent-ready.yml` and `.agent-ready/` are gone.
package/README.md CHANGED
@@ -104,6 +104,14 @@ Add agent traffic to this API: read https://alg.katrinalaszlo.com/SKILL.md and f
104
104
  Add agent traffic to this API: read https://alg.katrinalaszlo.com/SKILL.md and follow it.
105
105
  ```
106
106
 
107
+ To set up the key yourself, run this in your API's folder:
108
+
109
+ ```bash
110
+ npx agent-led-growth connect
111
+ ```
112
+
113
+ It prints a code and opens your browser; approve there, and it writes `AGENT_TRAFFIC_KEY` to `.env` (plus `AGENT_TRAFFIC_PROJECT_ID`, which is not a secret and is optional). Nothing to copy: the key is never printed, logged or put on a command line. If `.env` is not in `.gitignore` it says so and offers to add it. In a folder linked to Vercel (`.vercel/project.json`) it asks whether to also save the key in Vercel production, and passes it to `vercel env add` on stdin. Then restart your server and open `https://alg.katrinalaszlo.com/install` to watch the first event.
114
+
107
115
  ## Full command reference
108
116
 
109
117
  ### The three steps
@@ -142,7 +150,7 @@ Commit `alg.yml` so CI reuses your answers. `test` needs Claude Code installed a
142
150
 
143
151
  ### Installed
144
152
 
145
- `npm i -g agent-led-growth`, then `alg check …` and `alg test …`.
153
+ `npm i -g agent-led-growth`, then `alg check …`, `alg test …` and `alg connect`.
146
154
 
147
155
  ### Flags
148
156
 
@@ -163,11 +171,23 @@ Commit `alg.yml` so CI reuses your answers. `test` needs Claude Code installed a
163
171
  | --- | --- |
164
172
  | `--check` | Show the plan, run nothing. Free. |
165
173
  | `--yes` | Skip the confirmation; save an inferred `verify_call` to `alg.yml`. |
174
+ | `--skip-verify` | Test the signup only. The agent's key is not checked, so no `verify_call` is needed. |
166
175
  | `--max-budget-usd <n>` | Model-use cap, default 5. Checked after each turn, so a run can end slightly above it. |
167
176
  | `--max-turns <n>` | Agent turn budget, default 40. |
168
177
  | `--inbox <address>` | An inbox you relay mail from, for products that email a code. Or set `AGENTMAIL_API_KEY` and each run gets its own. |
169
178
  | `--json` | `agent-ready/verify@1` (or `verify-plan@1` with `--check`) to stdout. |
170
179
 
180
+ `connect`
181
+
182
+ | Flag | Does |
183
+ | --- | --- |
184
+ | `--endpoint <url>` | Track server. Default `AGENT_TRAFFIC_ENDPOINT`, else `https://alg.katrinalaszlo.com`. |
185
+ | `--env-file <path>` | File to write the key to. Default `.env`. Existing lines for the same names are replaced; the rest of the file is kept. A new file is created readable only by you. |
186
+ | `--vercel` | Also save the key in Vercel production. Asked by default when `.vercel/project.json` exists; without a terminal, only this flag does it. |
187
+ | `--no-vercel` | Do not offer Vercel. |
188
+
189
+ `connect` exits `0` when the key is saved and `1` when the sign-in is denied or expires.
190
+
171
191
  Exit codes, both commands: `0` done or passed, `1` fixes found or test failed, `2` usage or setup, `3` the site did not answer or the run was inconclusive, `130` cancelled.
172
192
 
173
193
  ### Cost, accounts and data
@@ -201,7 +221,7 @@ The headline and the seven steps are the same whatever the answers; the answers
201
221
 
202
222
  ### alg.yml
203
223
 
204
- `check` writes it. `test` reads it, and infers a `verify_call` from your OpenAPI document when there is none: a GET that needs a key and no parameters, preferring `/me`, `/account`, `/user`, `/whoami`. A complete file:
224
+ `check` writes it. `test` reads it, and infers a `verify_call` from your OpenAPI document when there is none: a GET that needs a key and no parameters, preferring `/me`, `/account`, `/user`, `/whoami`. With no OpenAPI document, `test` asks you at the prompt for a GET that needs a key (and the header that carries it), saves it to `alg.yml` and goes on; press Enter with nothing to skip the check for that run. Without a terminal, pass `--skip-verify`. A complete file:
205
225
 
206
226
  ```yaml
207
227
  url: yourproduct.com
@@ -253,4 +273,4 @@ npx agent-led-growth test --json --yes # creates a real account; ask the dev
253
273
 
254
274
  Each fix prompt is written for a coding agent: read `files.prompts[0]`, make the change, run its acceptance tests, deploy, run `check` again.
255
275
 
256
- Source and issues: https://github.com/katrinalaszlo/agent-led-growth-cli. MIT.
276
+ Source: https://github.com/katrinalaszlo/score (development) and https://github.com/katrinalaszlo/agent-led-growth-cli (the published build, where issues go). MIT.
package/bin/cli.js CHANGED
@@ -32,8 +32,10 @@ import { runStandards } from "../src/audit/standards.js";
32
32
  import { parseVerifySpec, buildVerifyTask, classify, failedStep, networkFor, describeSpec } from "../src/verify/index.js";
33
33
  import { renderVerify, renderPlan, duration } from "../src/verify/render.js";
34
34
  import { inferVerifyCall, latestInterface, latestAnswers } from "../src/verify/infer.js";
35
+ import { askVerifyCall } from "../src/verify/wizard.js";
35
36
  import { renderPrompt } from "../src/audit/prompts.js";
36
37
  import { patternInfo } from "../src/audit/index.js";
38
+ import { connect, resolveEndpoint } from "../src/connect/index.js";
37
39
  import { createInterface } from "node:readline/promises";
38
40
  import { spawnSync } from "node:child_process";
39
41
 
@@ -276,6 +278,7 @@ program
276
278
  .alias("verify")
277
279
  .description("A real agent (your Claude Code) signs up for and uses your product; a separate checker proves its key. Creates a real account, spends up to $5 of model use, asks first.")
278
280
  .option("-y, --yes", "skip the confirmation and save an inferred verify_call to alg.yml")
281
+ .option("--skip-verify", "test the signup only: the agent's key is not checked, so no verify_call is needed")
279
282
  .option("--inbox <address>", "an inbox you will relay mail from into the run's work/inbox/, for products that email a code")
280
283
  .option("--max-turns <n>", "agent turn budget", (v) => parseInt(v, 10), 40)
281
284
  .option("--max-budget-usd <usd>", "stop the agent once model use passes this many dollars (checked after each turn)", parseBudget, 5)
@@ -289,7 +292,9 @@ program
289
292
  Examples:
290
293
  alg test --check check Claude Code and alg.yml, print the plan, run nothing
291
294
  alg test use url, task and verify_call from alg.yml
292
- (no verify_call: one is taken from the product's OpenAPI document)
295
+ (no verify_call: one is taken from the product's OpenAPI document,
296
+ or you are asked for one at the prompt)
297
+ alg test --skip-verify signup only; the key is not checked
293
298
  alg test --yes --json no confirmation, JSON result (CI)`)
294
299
  .exitOverride(usageExit)
295
300
  .action(async (input, opts) => {
@@ -311,47 +316,77 @@ Examples:
311
316
  const shown = configPath.startsWith(home + sep) ? `~${configPath.slice(home.length)}` : CONFIG_FILE;
312
317
  console.error(` ${style.dim(`Not using ${shown}: it is for ${savedFor || "no product"}.`)}`);
313
318
  }
314
- const addCallHint = `Add the call that proves an agent's key works to ${CONFIG_FILE}, for example: verify_call: GET https://api.example.com/v1/me`;
319
+ const addCallHint = `Add the call that proves an agent's key works to ${CONFIG_FILE}, for example: verify_call: GET https://api.example.com/v1/me, or pass --skip-verify to test the signup without checking the key`;
320
+ const interactive = process.stdin.isTTY && !opts.json;
321
+ const askOnce = async (question) => {
322
+ const rl = createInterface({ input: process.stdin, output: process.stderr });
323
+ const reply = await rl.question(question).catch((err) => {
324
+ if (err.code === "ABORT_ERR") return "";
325
+ throw err;
326
+ });
327
+ rl.close();
328
+ return reply;
329
+ };
315
330
  // No verify_call: take one from the product's own OpenAPI document. It is saved only once accepted, by a
316
- // person at the prompt or by --yes; otherwise it is used for this run alone.
317
- let inferred = null;
318
- if (!(config.verify_call || "").trim()) {
331
+ // person at the prompt or by --yes; otherwise it is used for this run alone. When the docs give none, a person
332
+ // at the terminal is asked for the call, or can skip the check; without a terminal, --skip-verify does that.
333
+ let chosen = null;
334
+ let skipped = Boolean(opts.skipVerify);
335
+ if (!skipped && !(config.verify_call || "").trim()) {
319
336
  const live = !opts.json && Boolean(process.stderr.isTTY);
320
337
  if (live) process.stderr.write(` ${style.dim("Finding a call that proves the agent's key works…")}`);
321
- inferred = await inferVerifyCall({ url, version: pkg.version, cwd: process.cwd() });
338
+ const inferred = await inferVerifyCall({ url, version: pkg.version, cwd: process.cwd() });
322
339
  if (live) process.stderr.write("\r\x1b[2K");
323
- // For another product's file, the call goes in a file of this product's own, never in that one.
324
- if (inferred.error && !useSaved) fail(opts, 2, "verify_call_invalid", `No verify_call for ${host}, and none could be taken from the product's docs: ${inferred.error}.`, `Run "alg check ${host}" in a folder of its own, then add the call that proves an agent's key works to the ${CONFIG_FILE} it writes, for example: verify_call: GET https://api.example.com/v1/me`);
325
- if (inferred.error) fail(opts, 2, "verify_call_invalid", `${CONFIG_FILE} has no verify_call, and none could be taken from the product's docs: ${inferred.error}.`, addCallHint);
326
- const from = `(from ${inferred.specUrl}${inferred.specVia ? `, linked from ${inferred.specVia}` : ""})`;
327
- const willCheck = `Will check with ${inferred.call} on ${inferred.apiHost} (${inferred.headerName}) ${from}.`;
328
- const offSiteNote = inferred.offSite ? `\n This API host is on a different site than ${host}; the key the agent gets is sent there.` : "";
340
+ if (inferred.error && !interactive) {
341
+ // For another product's file, the call goes in a file of this product's own, never in that one.
342
+ if (!useSaved) fail(opts, 2, "verify_call_invalid", `No verify_call for ${host}, and none could be taken from the product's docs: ${inferred.error}.`, `Run "alg check ${host}" in a folder of its own, then add the call that proves an agent's key works to the ${CONFIG_FILE} it writes, for example: verify_call: GET https://api.example.com/v1/me, or pass --skip-verify to test the signup without checking the key`);
343
+ fail(opts, 2, "verify_call_invalid", `${CONFIG_FILE} has no verify_call, and none could be taken from the product's docs: ${inferred.error}.`, addCallHint);
344
+ }
329
345
  let accepted = Boolean(opts.yes);
330
- if (!accepted && process.stdin.isTTY && !opts.json) {
331
- const rl = createInterface({ input: process.stdin, output: process.stderr });
332
- // A person reads the call and where it came from; the "Will check with" line stays for logs and agents.
333
- const ask = `\n To prove the agent's key works, we'll call your API with it after signup:\n ${style.bold(inferred.call)} ${style.dim(`(from ${inferred.specUrl.replace(/^https?:\/\//, "")})`)}${offSiteNote}\n Use this call? [Y/n] `;
334
- const reply = await rl.question(ask).catch((err) => {
335
- if (err.code === "ABORT_ERR") return "n";
336
- throw err;
337
- });
338
- rl.close();
339
- if (!/^(y(es)?)?$/i.test(reply.trim())) {
340
- console.error(` Cancelled. Nothing saved. To use a different call, add it to ${CONFIG_FILE}, for example: verify_call: GET https://api.example.com/v1/me`);
341
- process.exit(130);
346
+ let typed = false;
347
+ if (inferred.error) {
348
+ const answer = await askVerifyCall({ ask: askOnce, say: (l) => console.error(l), url, reason: inferred.error, style });
349
+ if (answer.skip) skipped = true;
350
+ else {
351
+ chosen = { ...answer, source: "typed" };
352
+ accepted = true;
353
+ typed = true;
342
354
  }
343
- accepted = true;
344
- } else if (!opts.json) console.error(`\n ${willCheck}${offSiteNote}`);
345
- if (!useSaved) {
355
+ } else {
356
+ chosen = { ...inferred, source: "inferred" };
357
+ const offSiteNote = inferred.offSite ? `\n This API host is on a different site than ${host}; the key the agent gets is sent there.` : "";
358
+ if (!accepted && interactive) {
359
+ // A person reads the call and where it came from; the "Will check with" line stays for logs and agents.
360
+ const ask = `\n To prove the agent's key works, we'll call your API with it after signup:\n ${style.bold(inferred.call)} ${style.dim(`(from ${inferred.specUrl.replace(/^https?:\/\//, "")})`)}${offSiteNote}\n Use this call? [Y/n] `;
361
+ const reply = await askOnce(ask);
362
+ if (/^(y(es)?)?$/i.test(reply.trim())) accepted = true;
363
+ else {
364
+ // Not that one: the person names the call instead, or skips the check.
365
+ const answer = await askVerifyCall({ ask: askOnce, say: (l) => console.error(l), url, reason: null, style });
366
+ if (answer.skip) {
367
+ chosen = null;
368
+ skipped = true;
369
+ } else {
370
+ chosen = { ...answer, source: "typed" };
371
+ accepted = true;
372
+ typed = true;
373
+ }
374
+ }
375
+ } else if (!opts.json) console.error(`\n Will check with ${inferred.call} on ${inferred.apiHost} (${inferred.headerName}) (from ${inferred.specUrl}${inferred.specVia ? `, linked from ${inferred.specVia}` : ""}).${offSiteNote}`);
376
+ }
377
+ if (skipped) {
378
+ if (!opts.json) console.error(` ${style.dim("Verification skipped: the agent signs up, and its key is not checked.")}`);
379
+ } else if (!useSaved) {
346
380
  if (!opts.json) console.error(` ${style.dim(`Used for this run only: ${CONFIG_FILE} here is not for ${host}.`)}`);
347
381
  } else if (accepted) {
348
- writeVerifyCall(configPath, inferred);
349
- if (!opts.json) console.error(` ${style.dim(`Saved to ${CONFIG_FILE}.`)}`);
382
+ writeVerifyCall(configPath, chosen);
383
+ if (!opts.json) console.error(` ${style.dim(`Saved to ${CONFIG_FILE}${typed ? ": the next test uses it without asking" : ""}.`)}`);
350
384
  } else if (!opts.json) console.error(` ${style.dim(`Not saved to ${CONFIG_FILE}; pass --yes or run in a terminal to save it.`)}`);
351
385
  }
352
- const spec = parseVerifySpec(inferred ? { ...config, verify_call: inferred.call, verify_header: inferred.header } : config);
353
- if (spec.error) fail(opts, 2, "verify_call_invalid", spec.error, addCallHint);
354
- const checker = { inferred: Boolean(inferred), inferredFrom: inferred ? inferred.specUrl : null, ...(inferred?.specVia ? { inferredVia: inferred.specVia } : {}), ...(inferred ? { apiHost: inferred.apiHost, apiHostOffSite: inferred.offSite } : {}) };
386
+ if (skipped && !opts.json && opts.skipVerify) console.error(`\n ${style.dim("Verification skipped (--skip-verify): the agent signs up, and its key is not checked.")}`);
387
+ const spec = skipped ? null : parseVerifySpec(chosen ? { ...config, verify_call: chosen.call, verify_header: chosen.header } : config);
388
+ if (spec?.error) fail(opts, 2, "verify_call_invalid", spec.error, addCallHint);
389
+ const checker = { inferred: chosen?.source === "inferred", inferredFrom: chosen?.source === "inferred" ? chosen.specUrl : null, ...(chosen?.specVia ? { inferredVia: chosen.specVia } : {}), ...(chosen ? { apiHost: chosen.apiHost, apiHostOffSite: chosen.offSite } : {}), ...(skipped ? { skipped: true } : {}) };
355
390
  if (opts.check) {
356
391
  const claude = opts.executor === "claude-print" ? claudeStatus() : { ok: true, detail: `executor ${opts.executor}` };
357
392
  const inbox = process.env.AGENTMAIL_API_KEY ? "AgentMail: a fresh inbox for this run, deleted after" : opts.inbox ? `you relay mail from ${opts.inbox} into work/inbox/` : "none (set AGENTMAIL_API_KEY if the product emails a code or link)";
@@ -361,8 +396,8 @@ Examples:
361
396
  target: { url, host },
362
397
  task,
363
398
  claudeCode: claude,
364
- agent: { hosts: networkFor(url, spec), inbox, maxTurns: opts.maxTurns, maxBudgetUsd: opts.maxBudgetUsd, saves: ["AGENT_READY_KEY", ...spec.fields] },
365
- checker: { call: describeSpec(spec), calls: ["with the agent's key: must answer " + spec.expect, "with no key: must be refused (400, 401 or 403)", "with a wrong key: must be refused"], ...checker },
399
+ agent: { hosts: networkFor(url, spec), inbox, maxTurns: opts.maxTurns, maxBudgetUsd: opts.maxBudgetUsd, saves: ["AGENT_READY_KEY", ...(spec ? spec.fields : [])] },
400
+ checker: { call: describeSpec(spec), calls: spec ? ["with the agent's key: must answer " + spec.expect, "with no key: must be refused (400, 401 or 403)", "with a wrong key: must be refused"] : ["none: a key the agent saves counts, unchecked"], ...checker },
366
401
  };
367
402
  // The onboarding pattern chosen in alg.yml, as the last check's interface.json describes it.
368
403
  const onboarding = latestInterface(process.cwd(), url)?.doc.onboarding;
@@ -435,6 +470,46 @@ Examples:
435
470
  process.exitCode = verdict.exitCode;
436
471
  });
437
472
 
473
+ program
474
+ .command("connect")
475
+ .description("Connect this app to Track: approve in the browser, and the key is written to .env for you (and to Vercel if you say yes). The key is never printed.")
476
+ .option("--endpoint <url>", "Track server (default: AGENT_TRAFFIC_ENDPOINT, else https://alg.katrinalaszlo.com)")
477
+ .option("--env-file <path>", "env file to write the key to", ".env")
478
+ .option("--vercel", "also save the key in Vercel production (asked by default when .vercel/project.json exists)")
479
+ .option("--no-vercel", "do not offer to save the key in Vercel")
480
+ .option("--no-color", "plain output")
481
+ .addHelpText("after", `
482
+ Examples:
483
+ alg connect sign in, write AGENT_TRAFFIC_KEY to .env
484
+ alg connect --env-file .env.local write it to another file
485
+ alg connect --no-vercel skip the Vercel question`)
486
+ .exitOverride(usageExit)
487
+ .action(async (opts) => {
488
+ const endpoint = resolveEndpoint(opts.endpoint, process.env);
489
+ if (!/^https?:\/\/[^/]+/.test(endpoint)) fail(opts, 2, "invalid_address", `"${endpoint}" is not a web address.`, "Try: alg connect --endpoint https://alg.katrinalaszlo.com");
490
+ const style = makeStyle(Boolean(process.stderr.isTTY) && opts.color && !process.env.NO_COLOR);
491
+ const ask = async (question) => {
492
+ const rl = createInterface({ input: process.stdin, output: process.stderr });
493
+ const reply = await rl.question(question).catch((err) => {
494
+ if (err.code === "ABORT_ERR") return "n";
495
+ throw err;
496
+ });
497
+ rl.close();
498
+ return reply;
499
+ };
500
+ process.exitCode = await connect({
501
+ endpoint,
502
+ envFile: resolve(opts.envFile),
503
+ vercel: opts.vercel,
504
+ cwd: process.cwd(),
505
+ interactive: Boolean(process.stdin.isTTY),
506
+ sleep: (ms) => new Promise((done) => setTimeout(done, ms)),
507
+ ask,
508
+ say: (line) => console.error(line),
509
+ style,
510
+ });
511
+ });
512
+
438
513
  // 0.3 kept a Tanso workspace for verify runs. check and test need no Tanso account now; these
439
514
  // commands stay so scripts written for 0.3 get a clear answer instead of "unknown command".
440
515
  for (const name of ["account", "login", "logout"]) {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agent-led-growth",
3
- "version": "0.6.4",
3
+ "version": "0.8.0",
4
4
  "description": "Score: evals for agents as customers. Runs a real agent through signup, key and first call on your product, before you ship. Track, the analytics half, measures which agents convert after.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -15,6 +15,7 @@
15
15
  "bin/cli.js",
16
16
  "src/index.js",
17
17
  "src/audit/",
18
+ "src/connect/",
18
19
  "src/dashboard/strip.js",
19
20
  "src/crash/",
20
21
  "src/harness/",
@@ -14,7 +14,7 @@ A run creates a real account on the product and uses the developer's Claude Code
14
14
 
15
15
  `test` reads `alg.yml` in the current directory. If it is missing, run `npx agent-led-growth check <url> --json --yes` first, which writes it.
16
16
 
17
- With no `verify_call`, `test` infers one from the product's OpenAPI document: a GET that needs a key (bearer, or an API key header) and has no required parameters, preferring `/me`, `/account`, `/user` or `/whoami`, on the document's first server. An API key in the `Authorization` header gets the prefix the document gives (`Authorization: Token {key}` when it says "Prefix your key with 'Token '"); with no documented prefix that call is skipped. The plan shows the call's API host (`checker.apiHost`) and, when it is on a different site than the product (`checker.apiHostOffSite`), says the key the agent gets is sent there. Tell the developer which host the key goes to. When the product's own OpenAPI document has none, it tries the OpenAPI documents its `llms.txt`, `auth.md` and other agent files link to on the same registrable domain (such as `app.` or `api.` hosts). It reuses the latest `.alg/<host>/*/interface.json` from `check`, or reads the public pages again. `--yes` accepts the call and saves it to `alg.yml`; without `--yes` and without a terminal it is used for that run and not saved. With `--json`, the plan and the result carry `checker.inferred: true` and `checker.inferredFrom`, and `checker.inferredVia` (the page that linked the document) when it came from a link. Show the developer the inferred call before a real run. When nothing fits, `test` exits `2` with `verify_call_invalid`; then add the call yourself.
17
+ With no `verify_call`, `test` infers one from the product's OpenAPI document: a GET that needs a key (bearer, or an API key header) and has no required parameters, preferring `/me`, `/account`, `/user` or `/whoami`, on the document's first server. An API key in the `Authorization` header gets the prefix the document gives (`Authorization: Token {key}` when it says "Prefix your key with 'Token '"); with no documented prefix that call is skipped. The plan shows the call's API host (`checker.apiHost`) and, when it is on a different site than the product (`checker.apiHostOffSite`), says the key the agent gets is sent there. Tell the developer which host the key goes to. When the product's own OpenAPI document has none, it tries the OpenAPI documents its `llms.txt`, `auth.md` and other agent files link to on the same registrable domain (such as `app.` or `api.` hosts). It reuses the latest `.alg/<host>/*/interface.json` from `check`, or reads the public pages again. `--yes` accepts the call and saves it to `alg.yml`; without `--yes` and without a terminal it is used for that run and not saved. With `--json`, the plan and the result carry `checker.inferred: true` and `checker.inferredFrom`, and `checker.inferredVia` (the page that linked the document) when it came from a link. Show the developer the inferred call before a real run. When nothing fits and a person is at the terminal, `test` asks for the call (a GET that needs a key, then the header that carries it), saves it to `alg.yml` and goes on; Enter with nothing skips the check for that run, and the result says the key was not checked (`checker.skipped: true`). Without a terminal, or with `--json`, it exits `2` with `verify_call_invalid`; pass `--skip-verify` to run the signup without checking the key, or add the call to `alg.yml`.
18
18
 
19
19
  To choose the call, add it. Pick an authenticated read from the product's own API docs that refuses a request without a key:
20
20
 
@@ -63,7 +63,7 @@ The JSON is `agent-ready/verify@1`. Report `outcome`, `reason`, each entry in `c
63
63
  | --- | --- | --- |
64
64
  | `0` | `passed` or `handoff` | The key works and no key and a wrong key were refused; or the agent correctly stopped where the onboarding model says a person sets access up first. |
65
65
  | `1` | `failed` | The agent got no key, or the checker's calls did not pass. `prompt` names a fix prompt in the run folder. |
66
- | `2` | | Usage or setup: no `verify_call` and none could be inferred, or Claude Code missing or signed out. |
66
+ | `2` | | Usage or setup: no `verify_call` and none could be inferred (without a terminal; `--skip-verify` runs the signup anyway), or Claude Code missing or signed out. |
67
67
  | `3` | `inconclusive` | Not a result about the product: server errors, a call that answers without a key, the agent could not connect, the spending cap, or a run that did not finish. Say why and suggest what to change. |
68
68
  | `130` | | Cancelled. |
69
69
 
@@ -0,0 +1,209 @@
1
+ import { existsSync, readFileSync, writeFileSync, appendFileSync } from "node:fs";
2
+ import { basename, dirname, join, relative } from "node:path";
3
+ import { spawnSync } from "node:child_process";
4
+
5
+ // `alg connect`: a device login against Track. The person approves in the browser, and the key Track returns is
6
+ // written to the env file (and, if they say yes, to Vercel) without ever being printed, logged or put on a command
7
+ // line. Everything that touches the outside (fetch, child processes, the clock, the prompt) is passed in, so tests
8
+ // run it against a fake server.
9
+
10
+ export const DEFAULT_ENDPOINT = "https://alg.katrinalaszlo.com";
11
+
12
+ export function resolveEndpoint(flag, env) {
13
+ return (flag || env.AGENT_TRAFFIC_ENDPOINT || DEFAULT_ENDPOINT).replace(/\/+$/, "");
14
+ }
15
+
16
+ const defaultRun = (cmd, args, options = {}) => spawnSync(cmd, args, { encoding: "utf8", ...options });
17
+
18
+ async function postJson(fetchImpl, url, body) {
19
+ let res;
20
+ try {
21
+ res = await fetchImpl(url, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) });
22
+ } catch (err) {
23
+ throw new Error(`Could not reach ${url}: ${err.cause?.message || err.message}`);
24
+ }
25
+ // A proxy or a 5xx page can answer in HTML: only a JSON answer is read, and the status alone says what happened.
26
+ const isJson = (res.headers.get("content-type") || "").includes("json");
27
+ return { status: res.status, body: isJson ? await res.json() : null };
28
+ }
29
+
30
+ export async function startDeviceLogin({ endpoint, fetchImpl }) {
31
+ const res = await postJson(fetchImpl, `${endpoint}/v1/connect/start`, { client: "agent-led-growth" });
32
+ if (res.status !== 201 || !res.body?.device_code || !res.body?.user_code || !res.body?.verification_url) {
33
+ throw new Error(`${endpoint} did not start a sign-in (${res.status}${res.body?.error ? ` ${res.body.error}` : ""}).`);
34
+ }
35
+ return res.body;
36
+ }
37
+
38
+ // Polls until the person approves, denies, or the code expires. Returns { ok: true, result } or
39
+ // { ok: false, reason, message }.
40
+ export async function pollForKey({ endpoint, start, fetchImpl, sleep, now }) {
41
+ let intervalSeconds = Number(start.interval) || 5;
42
+ const deadline = now() + (Number(start.expires_in) || 600) * 1000;
43
+ const expired = { ok: false, reason: "expired", message: "The sign-in code expired before it was approved. Run alg connect again." };
44
+ for (;;) {
45
+ await sleep(intervalSeconds * 1000);
46
+ if (now() >= deadline) return expired;
47
+ const res = await postJson(fetchImpl, `${endpoint}/v1/connect/poll`, { device_code: start.device_code });
48
+ if (res.status === 202) continue;
49
+ if (res.status === 429) {
50
+ intervalSeconds += 2;
51
+ continue;
52
+ }
53
+ if (res.status === 200 && res.body?.key) return { ok: true, result: res.body };
54
+ if (res.status === 403) return { ok: false, reason: "denied", message: "The sign-in was denied in the browser. Nothing was saved." };
55
+ if (res.status === 410) return expired;
56
+ return { ok: false, reason: "unexpected", message: `${endpoint} answered ${res.status}${res.body?.error ? ` ${res.body.error}` : ""} while waiting for approval. Nothing was saved.` };
57
+ }
58
+ }
59
+
60
+ // Sets each name in values in the env file: an existing line for that name is replaced in place (a second one is
61
+ // dropped), every other line is kept as it is, and a missing name is added at the end. A new file is created
62
+ // readable only by its owner. Returns { created }.
63
+ export function writeEnvFile(path, values) {
64
+ const created = !existsSync(path);
65
+ const lines = created ? [] : readFileSync(path, "utf8").split("\n");
66
+ if (lines.length > 0 && lines[lines.length - 1] === "") lines.pop();
67
+ const written = new Set();
68
+ const out = [];
69
+ for (const line of lines) {
70
+ const name = /^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=/.exec(line)?.[1];
71
+ if (name && name in values) {
72
+ if (written.has(name)) continue;
73
+ out.push(`${name}=${values[name]}`);
74
+ written.add(name);
75
+ continue;
76
+ }
77
+ out.push(line);
78
+ }
79
+ for (const [name, value] of Object.entries(values)) if (!written.has(name)) out.push(`${name}=${value}`);
80
+ const text = out.join("\n") + "\n";
81
+ if (created) writeFileSync(path, text, { mode: 0o600 });
82
+ else writeFileSync(path, text);
83
+ return { created };
84
+ }
85
+
86
+ // "ignored", "not_ignored", "no_repo" (not inside a git work tree) or "no_git" (git is not installed).
87
+ export function gitIgnoreStatus(path, run = defaultRun) {
88
+ const cwd = dirname(path);
89
+ const inside = run("git", ["rev-parse", "--is-inside-work-tree"], { cwd });
90
+ if (inside.error) return "no_git";
91
+ if (inside.status !== 0 || inside.stdout.trim() !== "true") return "no_repo";
92
+ const check = run("git", ["check-ignore", "-q", basename(path)], { cwd });
93
+ return check.status === 0 ? "ignored" : "not_ignored";
94
+ }
95
+
96
+ // Adds the file's name to the .gitignore next to it, creating that .gitignore if there is none.
97
+ export function addToGitignore(path) {
98
+ const gitignore = join(dirname(path), ".gitignore");
99
+ const before = existsSync(gitignore) ? readFileSync(gitignore, "utf8") : "";
100
+ appendFileSync(gitignore, `${before === "" || before.endsWith("\n") ? "" : "\n"}${basename(path)}\n`);
101
+ return gitignore;
102
+ }
103
+
104
+ export function openBrowser(url, run = defaultRun, platform = process.platform) {
105
+ if (platform === "darwin") return run("open", [url]);
106
+ if (platform === "win32") return run("cmd", ["/c", "start", '""', url.replace(/&/g, "^&")], { windowsVerbatimArguments: true });
107
+ return run("xdg-open", [url]);
108
+ }
109
+
110
+ const vercelByHand = "To add it yourself: run vercel env add AGENT_TRAFFIC_KEY production and paste the AGENT_TRAFFIC_KEY value from your env file when asked, or add it in the Vercel dashboard under Settings, Environment Variables.";
111
+
112
+ // The key goes to the child on stdin only. Child output is shown only on failure, with the key cut out.
113
+ export async function saveKeyInVercel({ key, ask, say, run = defaultRun, cwd }) {
114
+ const hide = (text) => String(text || "").split(key).join("[key]").trim();
115
+ const version = run("vercel", ["--version"], { cwd });
116
+ if (version.error) {
117
+ say(" The vercel command was not found, so the key was not saved in Vercel.");
118
+ say(` ${vercelByHand}`);
119
+ return "vercel_missing";
120
+ }
121
+ const list = run("vercel", ["env", "ls", "production"], { cwd });
122
+ if (list.status !== 0) {
123
+ say(` vercel env ls failed, so the key was not saved in Vercel: ${hide(list.stderr) || `exit ${list.status}`}`);
124
+ say(` ${vercelByHand}`);
125
+ return "failed";
126
+ }
127
+ if (/(^|\s)AGENT_TRAFFIC_KEY(\s|$)/m.test(list.stdout)) {
128
+ const reply = await ask(" AGENT_TRAFFIC_KEY is already set in Vercel production. Replace it? (Y/n) ");
129
+ if (!/^(y(es)?)?$/i.test(reply.trim())) {
130
+ say(" Kept the existing AGENT_TRAFFIC_KEY in Vercel production.");
131
+ return "kept";
132
+ }
133
+ const removed = run("vercel", ["env", "rm", "AGENT_TRAFFIC_KEY", "production", "--yes"], { cwd });
134
+ if (removed.status !== 0) {
135
+ say(` vercel env rm failed, so the key was not replaced in Vercel: ${hide(removed.stderr) || `exit ${removed.status}`}`);
136
+ say(` ${vercelByHand}`);
137
+ return "failed";
138
+ }
139
+ }
140
+ const added = run("vercel", ["env", "add", "AGENT_TRAFFIC_KEY", "production"], { cwd, input: key });
141
+ if (added.status !== 0) {
142
+ say(` vercel env add failed, so the key was not saved in Vercel: ${hide(added.stderr) || `exit ${added.status}`}`);
143
+ say(` ${vercelByHand}`);
144
+ return "failed";
145
+ }
146
+ say(" Saved AGENT_TRAFFIC_KEY in Vercel production. It takes effect on the next deploy.");
147
+ return "saved";
148
+ }
149
+
150
+ // Runs the whole flow and returns the exit code: 0 connected, 1 denied, expired or failed.
151
+ export async function connect({ endpoint, envFile, vercel, cwd, interactive, fetchImpl = fetch, sleep, now = Date.now, run = defaultRun, ask, say, style }) {
152
+ const bold = style?.bold || ((s) => s);
153
+ const dim = style?.dim || ((s) => s);
154
+ say("");
155
+ say(` ${bold("alg connect")} · ${new URL(endpoint).host}`);
156
+ const start = await startDeviceLogin({ endpoint, fetchImpl });
157
+ say("");
158
+ say(` Your code: ${bold(start.user_code)}`);
159
+ const opened = openBrowser(start.verification_url, run);
160
+ if (opened.error || opened.status) say(` Open ${start.verification_url} in your browser.`);
161
+ else say(` Opened ${start.verification_url} in your browser.`);
162
+ say(` Check the code there matches, then approve. ${dim("Waiting…")}`);
163
+
164
+ const polled = await pollForKey({ endpoint, start, fetchImpl, sleep, now });
165
+ if (!polled.ok) {
166
+ say("");
167
+ say(` ${polled.message}`);
168
+ return 1;
169
+ }
170
+ const { key, project_id: projectId, project_name: projectName } = polled.result;
171
+ const trackEndpoint = (polled.result.endpoint || endpoint).replace(/\/+$/, "");
172
+
173
+ const values = { AGENT_TRAFFIC_KEY: key };
174
+ if (projectId) values.AGENT_TRAFFIC_PROJECT_ID = projectId;
175
+ // The middleware sends to the default host unless told otherwise.
176
+ if (trackEndpoint !== DEFAULT_ENDPOINT) values.AGENT_TRAFFIC_ENDPOINT = trackEndpoint;
177
+ const { created } = writeEnvFile(envFile, values);
178
+ const shown = relative(cwd, envFile).startsWith("..") ? envFile : relative(cwd, envFile);
179
+ say("");
180
+ say(` Connected to ${bold(projectName || projectId || "your Track project")}.`);
181
+ say(` Saved ${Object.keys(values).join(", ")} to ${shown}${created ? " (new file, readable only by you)" : ""}.`);
182
+
183
+ const ignored = gitIgnoreStatus(envFile, run);
184
+ if (ignored === "not_ignored") {
185
+ say(` ${shown} is not in .gitignore, so git would commit the key.`);
186
+ const reply = interactive ? await ask(` Add ${basename(envFile)} to .gitignore? (Y/n) `) : "n";
187
+ if (/^(y(es)?)?$/i.test(reply.trim())) {
188
+ const gitignore = addToGitignore(envFile);
189
+ say(` Added ${basename(envFile)} to ${relative(cwd, gitignore).startsWith("..") ? gitignore : relative(cwd, gitignore)}.`);
190
+ } else say(` Add ${basename(envFile)} to .gitignore before you commit.`);
191
+ } else if (ignored === "no_git") say(` ${dim("git was not found, so .gitignore was not checked.")}`);
192
+
193
+ const linked = existsSync(join(cwd, ".vercel", "project.json"));
194
+ if (vercel === true || (vercel !== false && linked)) {
195
+ say("");
196
+ if (interactive) {
197
+ const reply = await ask(" Also save the key in Vercel production? (Y/n) ");
198
+ if (/^(y(es)?)?$/i.test(reply.trim())) await saveKeyInVercel({ key, ask, say, run, cwd });
199
+ else say(" Not saved in Vercel.");
200
+ } else if (vercel === true) {
201
+ await saveKeyInVercel({ key, ask: async () => "y", say, run, cwd });
202
+ } else say(` ${dim("This folder is linked to Vercel. Run alg connect --vercel to save the key there too.")}`);
203
+ }
204
+
205
+ say("");
206
+ say(` Next: restart your server, then open ${trackEndpoint}/install to watch the first event.`);
207
+ say("");
208
+ return 0;
209
+ }
@@ -54,7 +54,9 @@ export function parseVerifySpec(config) {
54
54
  return { method: (match[1] || "GET").toUpperCase(), url: match[2], body: config.verify_body || null, header: { name: header.slice(0, colon).trim(), template: header.slice(colon + 1).trim() }, expect, assert, fields, exchange, hosts, cli: registries };
55
55
  }
56
56
 
57
+ // spec is null when verification was skipped: the agent signs up, and its key is not checked.
57
58
  export function describeSpec(spec) {
59
+ if (!spec) return "none: verification skipped, the agent's key is not checked";
58
60
  const assertion = spec.assert ? (spec.assert.equals === null ? `, ${spec.assert.path} present` : `, ${spec.assert.path} = ${spec.assert.equals}`) : "";
59
61
  const first = spec.exchange ? `${spec.exchange.method} ${spec.exchange.url} for a token (${spec.exchange.tokenPath}), then ` : "";
60
62
  return `${first}${spec.method} ${spec.url} with ${spec.header.name}: ${spec.header.template}; expect ${spec.expect}${assertion}`;
@@ -155,23 +157,30 @@ export async function checkKey(spec, key, { fetchImpl = fetch, fields = {} } = {
155
157
  export function networkFor(targetUrl, spec) {
156
158
  const host = new URL(targetUrl).hostname;
157
159
  const apex = registrableDomain(host);
158
- const hosts = [host, apex, ...PRODUCT_SUBDOMAINS.map((s) => `${s}.${apex}`), new URL(spec.url.replace(/\{[^}]+\}/g, "x")).hostname, ...(spec.exchange ? [new URL(spec.exchange.url.replace(/\{[^}]+\}/g, "x")).hostname] : []), ...(spec.hosts || [])];
160
+ const hosts = [host, apex, ...PRODUCT_SUBDOMAINS.map((s) => `${s}.${apex}`)];
161
+ if (spec) hosts.push(new URL(spec.url.replace(/\{[^}]+\}/g, "x")).hostname, ...(spec.exchange ? [new URL(spec.exchange.url.replace(/\{[^}]+\}/g, "x")).hostname] : []), ...(spec.hosts || []));
159
162
  return [...new Set(hosts)];
160
163
  }
161
164
 
165
+ // With no spec the check is skipped: a key the agent saved counts, and the result says it was not checked.
166
+ async function skippedCheck() {
167
+ const checks = [{ id: "key_works", label: "Key works", pass: null, detail: "not checked: verification skipped (no verify_call)" }];
168
+ return { ok: true, status: 0, detail: checks[0].detail, objects: { checks, checkerInvalid: false, skipped: true } };
169
+ }
170
+
162
171
  export function buildVerifyTask({ url, task, spec, fetchImpl = fetch }) {
163
172
  const host = new URL(url).hostname;
164
173
  return defineSignupTask({
165
174
  id: "verify",
166
175
  name: host,
167
176
  url,
168
- apiHost: new URL(spec.url.replace(/\{[^}]+\}/g, "x")).hostname,
177
+ apiHost: spec ? new URL(spec.url.replace(/\{[^}]+\}/g, "x")).hostname : host,
169
178
  network: networkFor(url, spec),
170
179
  credentialEnvName: KEY_ENV,
171
- extraFields: spec.fields,
180
+ extraFields: spec ? spec.fields : [],
172
181
  keyPattern: /^\S{8,}$/,
173
182
  taskText: `${task}. The product is ${host}.`,
174
- verify: { describe: describeSpec(spec), call: (key, runId, fields) => checkKey(spec, key, { fetchImpl, fields }) },
183
+ verify: { describe: describeSpec(spec), call: spec ? (key, runId, fields) => checkKey(spec, key, { fetchImpl, fields }) : skippedCheck },
175
184
  });
176
185
  }
177
186
 
@@ -184,6 +193,7 @@ const REPORTED_5XX = /\b(?:HTTP\s*(?:status)?|status(?:\s*code)?)\W{0,4}5\d\d\b|
184
193
  export function classify(result, onboarding, resultMd = "") {
185
194
  const evaluation = result.evaluation || {};
186
195
  const stages = result.reconciled?.stages || [];
196
+ if (evaluation.success && evaluation.objects?.skipped) return { outcome: "passed", exitCode: 0, reason: "A real agent got its own key. The key was not checked: verification was skipped. Add verify_call to alg.yml to prove it works." };
187
197
  if (evaluation.success) return { outcome: "passed", exitCode: 0, reason: "A real agent got its own key and the checker confirmed it works." };
188
198
  if (evaluation.objects?.checkerInvalid) return { outcome: "inconclusive", exitCode: 3, reason: "The verify call answers without a key, so it cannot prove the agent's key works. Pick a call that needs authentication." };
189
199
  if (stages.some((s) => s.agent?.outcome === "product_unavailable")) return { outcome: "inconclusive", exitCode: 3, reason: "The product returned server errors during the run. Try again later." };
@@ -0,0 +1,41 @@
1
+ import { parseVerifySpec } from "./index.js";
2
+ import { sameSite } from "../interface/sources.js";
3
+
4
+ // `test` found no verify_call in alg.yml and none in the product's docs. Instead of stopping, it asks the person
5
+ // at the terminal for the call, or lets them skip the check. ask(question) returns the typed line; say(line)
6
+ // prints one. Returns { call, header, apiHost, offSite } for a call, or { skip: true }.
7
+
8
+ export const EXAMPLE_CALL = "GET https://api.example.com/v1/me";
9
+ export const DEFAULT_HEADER = "Authorization: Bearer {key}";
10
+
11
+ export async function askVerifyCall({ ask, say, url, reason, style }) {
12
+ const dim = style?.dim || ((s) => s);
13
+ const bold = style?.bold || ((s) => s);
14
+ say("");
15
+ say(` We couldn't find an OpenAPI spec to figure out how to test your API key${reason ? ` ${dim(`(${reason})`)}` : ""}.`);
16
+ say(" What's a simple GET endpoint we can use to verify the key works?");
17
+ say(` ${dim(`e.g. ${EXAMPLE_CALL}. Press Enter with nothing to skip verification: the agent signs up, and its key is not checked.`)}`);
18
+ let spec;
19
+ for (;;) {
20
+ const reply = (await ask(" Call: ")).trim();
21
+ if (!reply || /^skip$/i.test(reply)) return { skip: true };
22
+ spec = parseVerifySpec({ verify_call: reply });
23
+ if (!spec.error) break;
24
+ say(` ${spec.error.replace(/^alg\.yml has no verify_call\. /, "")}`);
25
+ }
26
+ const call = `${spec.method} ${spec.url}`;
27
+ let header;
28
+ for (;;) {
29
+ const reply = (await ask(` Header that carries the key ${dim(`[${DEFAULT_HEADER}]`)}: `)).trim() || DEFAULT_HEADER;
30
+ const withHeader = parseVerifySpec({ verify_call: call, verify_header: reply });
31
+ if (!withHeader.error) {
32
+ header = reply;
33
+ break;
34
+ }
35
+ say(` ${withHeader.error}`);
36
+ }
37
+ const apiHost = new URL(spec.url.replace(/\{[^}]+\}/g, "x")).hostname;
38
+ const offSite = !sameSite(spec.url, url);
39
+ say(` ${dim("Will check with")} ${bold(call)} ${dim(`(${header})`)}`);
40
+ return { call, header, apiHost, offSite };
41
+ }