agent-led-growth 0.8.2 → 0.8.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +14 -0
- package/README.md +2 -0
- package/bin/cli.js +10 -5
- package/package.json +1 -1
- package/skills/alg/SKILL.md +1 -1
- package/src/audit/index.js +8 -2
- package/src/audit/render.js +33 -1
- package/src/dashboard/strip.js +4 -3
- package/src/interface/evaluate.js +7 -0
- package/src/interface/extract.js +3 -1
- package/src/interface/funnel.js +46 -21
- package/src/interface/onboarding.js +71 -4
- package/src/interface/schema.js +2 -2
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-led-growth",
|
|
3
|
-
"version": "0.8.
|
|
3
|
+
"version": "0.8.3",
|
|
4
4
|
"description": "Score: evals for agents as customers. Runs a real agent through signup, key and first call on your product, before you ship. Track, the analytics half, measures which agents convert after.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Kat Laszlo",
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.8.3 (2026-10-24)
|
|
4
|
+
|
|
5
|
+
False findings fixed. `check` read Track (alg.katrinalaszlo.com), which is free and takes no email, as charging per request, needing an inbox and having a plan to change.
|
|
6
|
+
|
|
7
|
+
- Rule: a step that says the product needs or requires something (an inbox, a payment, a person at checkout, a shell for its CLI) is reported only with the sentence it came from, and that sentence is printed under the step with its page. A need with no quote is not reported. A negated sentence ("takes no email", "cannot sign in to the dashboard", "nothing is deleted"), an example ("such as opening a verification email"), a line of a link index, an optional step and text about the reader's own customers never count.
|
|
8
|
+
- Pay reads machine-readable pricing first. When every plan in /pricing.json costs 0 and no purchase is documented, Pay is `not_applicable`: "Nothing to buy: every plan is free (pricing.json)." Before, x402 named in Track's docs (as a task Score tests) made Pay "The docs mention paying per request", and a demo's "$49.00" on its homepage would have read as a paid plan. "per request" counts as a usage price only beside a price word ("priced per request", "$0.01 per request"); "4 KB per request" is a limit.
|
|
9
|
+
- Manage is `not_applicable` when pricing.json lists exactly one plan and it is free: "One free plan: no plan or limit to change." Otherwise unchanged.
|
|
10
|
+
- Inbox: an emailed code or link counts only in a sentence about the product's own signup or sign-in, and an explicit negative on the product's pages ("takes no email", "needs no inbox", "no account, email or browser") clears it. `test --check` warns about a missing inbox only with that sentence, which it quotes; an `interface.json` from before 0.8.3 carries none, so it no longer warns from one.
|
|
11
|
+
- Other rules audited for the same failure (text that mentions a thing read as the product requiring it), each with a test: expiry ignores "nothing is deleted" and "never expire"; restricted access, a person setting access up, and x402 ignore a negation in the same clause; x402 is ignored when pricing.json lists only free plans; a CLI signup needs the flag or subcommand within four words of the command, so "npx agent-led-growth check <url> runs a real agent through signup" is not one.
|
|
12
|
+
- The step count skips steps that do not apply: "Your public pages document 5 of 5 applicable steps; Pay and Manage don't apply." A not-applicable step is marked `-` and tagged "from your files".
|
|
13
|
+
- After a passing `test`, `check` no longer ends with "Step 3 of 3: test". Next says what is left ("Your latest test passed. Nothing left to fix or test. Pay and Manage don't apply.") and how to keep it passing in CI, and the What next? menu is not offered.
|
|
14
|
+
- JSON, backward compatible (`agent-ready/audit-report@1` keeps its name): `steps[].state` can be `not_applicable`; steps can carry `quotes: [{ for, obs, quote, url }]`; the report adds `applicable`, the number of steps that apply. `documented`, `verified` and `of` (7) are unchanged; the headline text changes only when a step does not apply. In `interface.json`, onboarding patterns add `needsEvidence`.
|
|
15
|
+
- Tests: Track's live pages are recorded in fixtures/onboarding/alg.katrinalaszlo.com.json.gz and its whole check result is asserted, with contrasting products (paid per request over x402, a price per request in prose, an email-gated signup) where each rule must still fire, and a test over every recorded product that each stated requirement carries a quote. Re-checked against the recorded products, the inbox need is no longer reported for firecrawl.dev ("for example OTP flows"), mem0.ai ("no email, no dashboard, no OTP required"), moltbook.com (a verification code for a math challenge), neon.com (a link to its Email OTP feature) and paywithlocus.com ("an optional AgentMail OTP"); it is kept for telnyx.com, inkbox.ai, agentmail.to, aisend.app, www.cosmicjs.com and agentline.cloud.
|
|
16
|
+
|
|
3
17
|
## 0.8.2 (2026-10-24)
|
|
4
18
|
|
|
5
19
|
- `alg connect`: Track's /connect page no longer fills in the code, so the CLI now asks you to type the code it prints on that page, check where the page says the request came from, then approve. The code is printed on its own line. The CLI opens the `verification_url` Track returns as it is and does not expect the code in it.
|
package/README.md
CHANGED
|
@@ -53,6 +53,8 @@ $ npx agent-led-growth check resend.com
|
|
|
53
53
|
|
|
54
54
|
Six seconds. No account, no Docker, nothing sent to your product but GET requests. Needs Node 20.9 or later.
|
|
55
55
|
|
|
56
|
+
When a step says your product needs or requires something (an inbox, a payment, a person at checkout), the sentence it came from is printed under the step with the page it is on. A sentence that says the opposite ("takes no email"), an example ("such as a verification email") or text about your own customers never counts. A step that does not apply is marked `-`: Pay when every plan in /pricing.json is free, Manage when there is one free plan.
|
|
57
|
+
|
|
56
58
|
Then fix and test:
|
|
57
59
|
|
|
58
60
|
```bash
|
package/bin/cli.js
CHANGED
|
@@ -25,7 +25,7 @@ import { runHarness, TASKS } from "../src/harness/index.js";
|
|
|
25
25
|
import { inspect, buildAudit, writeAudit, parseTarget, unreachableReason, DEFAULT_TASK } from "../src/audit/index.js";
|
|
26
26
|
import { defaultAnswers, askQuestions, QUESTIONS } from "../src/audit/questions.js";
|
|
27
27
|
import { readConfig, writeConfig, writeVerifyCall, configHost, CONFIG_FILE } from "../src/audit/config.js";
|
|
28
|
-
import { makeStyle, renderAnswer, renderVerdict, renderSteps, renderSummary, renderNext, renderStandardsStart, renderStandards, renderStrip } from "../src/audit/render.js";
|
|
28
|
+
import { makeStyle, renderAnswer, renderVerdict, renderSteps, renderSummary, renderNext, renderStandardsStart, renderStandards, renderStrip, latestTestOutcome } from "../src/audit/render.js";
|
|
29
29
|
import { localJourney, localTestRuns } from "../src/verify/journey.js";
|
|
30
30
|
import { applyEvidence } from "../src/interface/evidence.js";
|
|
31
31
|
import { runStandards } from "../src/audit/standards.js";
|
|
@@ -184,7 +184,9 @@ Examples:
|
|
|
184
184
|
if (!opts.json) for (const line of renderSummary(audit, paths, style, rel, savedTo)) say(line);
|
|
185
185
|
// Skipped, the benchmarks get one line above Next, so Next stays the last thing on screen.
|
|
186
186
|
if (!opts.json && !opts.standards) for (const line of renderStandards(null, style)) say(line);
|
|
187
|
-
|
|
187
|
+
const latestTest = latestTestOutcome(funnel);
|
|
188
|
+
const testPassed = latestTest === "passed" || latestTest === "handoff";
|
|
189
|
+
if (!opts.json) for (const line of renderNext(audit, paths, style, rel, latestTest)) say(line);
|
|
188
190
|
|
|
189
191
|
let standardsFile = null;
|
|
190
192
|
if (opts.standards) {
|
|
@@ -216,7 +218,7 @@ Examples:
|
|
|
216
218
|
if (!opts.json) for (const line of renderStandards(standards, style, raw ? rel(raw) : null)) say(line);
|
|
217
219
|
// The last line on screen is the next action, not the benchmark table.
|
|
218
220
|
if (!opts.json) {
|
|
219
|
-
const next = audit.findings.length ? " Next: the fix prompt above, then check again." : ` Next: ${style.bold(`${TEST_CMD} ${host} --check`)}`;
|
|
221
|
+
const next = audit.findings.length ? " Next: the fix prompt above, then check again." : testPassed ? " Your latest test passed; Next is above." : ` Next: ${style.bold(`${TEST_CMD} ${host} --check`)}`;
|
|
220
222
|
say(` Done in ${duration(Date.now() - started)}.${next}`);
|
|
221
223
|
say("");
|
|
222
224
|
}
|
|
@@ -227,7 +229,7 @@ Examples:
|
|
|
227
229
|
if (opts.failOn && audit.findings.some((f) => FAIL_ON[opts.failOn].includes(f.severity))) process.exitCode = 1;
|
|
228
230
|
// With nothing to fix, the next step is a test, and a person at a terminal can start it here.
|
|
229
231
|
const canAsk = Boolean(process.stdin.isTTY && process.stderr.isTTY) && !opts.yes && !opts.json;
|
|
230
|
-
if (canAsk && !audit.findings.length) await offerTest({ audit, style, say });
|
|
232
|
+
if (canAsk && !audit.findings.length && !testPassed) await offerTest({ audit, style, say });
|
|
231
233
|
});
|
|
232
234
|
|
|
233
235
|
// check reads public pages; test has a real agent do the steps. With nothing to fix, a person at a terminal can
|
|
@@ -402,7 +404,10 @@ Examples:
|
|
|
402
404
|
// The onboarding pattern chosen in alg.yml, as the last check's interface.json describes it.
|
|
403
405
|
const onboarding = latestInterface(process.cwd(), url)?.doc.onboarding;
|
|
404
406
|
const chosen = onboarding?.patterns?.find((p) => p.id === (config.onboarding || onboarding.primary));
|
|
405
|
-
|
|
407
|
+
// Only with the sentence it came from: a need without a quote is not reported (interface.json from before 0.8.3
|
|
408
|
+
// has none).
|
|
409
|
+
const inboxQuote = chosen?.needs?.includes("inbox") ? chosen.needsEvidence?.inbox?.[0]?.quote : null;
|
|
410
|
+
if (inboxQuote && !process.env.AGENTMAIL_API_KEY && !opts.inbox) plan.warnings = [`The product's signup sends an emailed code or link ("${inboxQuote}") and no inbox is set. Set AGENTMAIL_API_KEY or pass --inbox, or the run may end inconclusive.`];
|
|
406
411
|
if (opts.json) process.stdout.write(JSON.stringify(plan, null, 2) + "\n");
|
|
407
412
|
else for (const line of renderPlan(plan, style)) console.error(line);
|
|
408
413
|
process.exit(plan.ready ? 0 : 2);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-led-growth",
|
|
3
|
-
"version": "0.8.
|
|
3
|
+
"version": "0.8.3",
|
|
4
4
|
"description": "Score: evals for agents as customers. Runs a real agent through signup, key and first call on your product, before you ship. Track, the analytics half, measures which agents convert after.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
package/skills/alg/SKILL.md
CHANGED
|
@@ -41,7 +41,7 @@ The fix prompts build toward the chosen model. Say which answers you used and th
|
|
|
41
41
|
|
|
42
42
|
1. Lead with `headline` ("Your public pages document N of 7 steps. No step has been tested yet: a test runs a real agent."). `documented` and `verified` hold the two counts; a check never verifies, so `verified` is 0.
|
|
43
43
|
2. List each finding: its step, its title and its reason.
|
|
44
|
-
3. Never call a step verified or confirmed: a check only reads pages. Say how each step was found (`basis`): `observed` comes from a structured file or an HTTP status, `heuristic` from matching page text (it can be wrong), and `not_checked` means public pages cannot show it. Only a test with a real agent shows a step works. `state`: `agent_can`/`agent_did` pass, `handoff` passes with one person step, `needs_person`/`blocked` are gaps, `not_checked` needs a test.
|
|
44
|
+
3. Never call a step verified or confirmed: a check only reads pages. Say how each step was found (`basis`): `observed` comes from a structured file or an HTTP status, `heuristic` from matching page text (it can be wrong), and `not_checked` means public pages cannot show it. Only a test with a real agent shows a step works. `state`: `agent_can`/`agent_did` pass, `handoff` passes with one person step, `needs_person`/`blocked` are gaps, `not_checked` needs a test, `not_applicable` has nothing to do (Pay with only free plans in pricing.json, Manage with one free plan). A step that says the product needs or requires something carries `quotes`, the sentences it came from; quote them rather than restating the claim.
|
|
45
45
|
4. After the findings, summarize `standards` in a few lines: each benchmark's passed/total and score, then the failed check names the developer asks about. Say they are third-party benchmarks and separate from the seven steps. If `standards.error` is set, tell the developer aeo-ready did not run and quote it.
|
|
46
46
|
5. Give the paths in `files`: `files.prompts` (one fix prompt per finding), `files.brief` (a one-page brief for security, legal and billing), `files.config` and `files.standards` (aeo-ready's full output).
|
|
47
47
|
6. Then tell the developer the next step is a test, and offer the free plan: `npx agent-led-growth test <url> --check --json`. Do not run `test` itself; see 'If the developer asks for a test'.
|
package/src/audit/index.js
CHANGED
|
@@ -15,6 +15,7 @@ export const DEFAULT_TASK = "Sign up, get an API key, and make one authenticated
|
|
|
15
15
|
// reads Agent did.
|
|
16
16
|
export function basisFor(step, doc) {
|
|
17
17
|
if (step.live) return "tested";
|
|
18
|
+
if (step.state === "not_applicable") return "observed";
|
|
18
19
|
if (step.state === "not_checked") return "not_checked";
|
|
19
20
|
if (step.id === "discover") return "observed";
|
|
20
21
|
if (step.id === "understand") return doc.pricing?.agentCanDetermineCost?.verdict === "yes" ? "observed" : "heuristic";
|
|
@@ -85,7 +86,11 @@ export function patternInfo(id) {
|
|
|
85
86
|
// Everything the audit reports, from the inspection and the answers. Pure, so it is tested without a network.
|
|
86
87
|
export function buildAudit({ doc, funnel, answers, task, runId, version }) {
|
|
87
88
|
const pattern = patternInfo(answers.onboarding.value);
|
|
88
|
-
|
|
89
|
+
// quotes: the sentences behind a claim that the product requires something, with the page each is on. A step a test
|
|
90
|
+
// decided says what the test did instead, so it carries none.
|
|
91
|
+
const urlOf = (obs) => doc.observations.find((o) => o.id === obs)?.url || null;
|
|
92
|
+
const quotesOf = (s) => (!s.live && s.quotes?.length ? { quotes: s.quotes.map((q) => ({ ...q, url: urlOf(q.obs) })) } : {});
|
|
93
|
+
const steps = funnel.steps.map((s) => ({ id: s.id, name: s.name, state: s.state, basis: basisFor(s, doc), reason: s.reason, basedOn: s.basedOn, ...quotesOf(s), ...(s.fix ? { fix: s.fix } : {}) }));
|
|
89
94
|
// The chosen model is the target. A product whose docs only describe a person handing over a key passes Sign up
|
|
90
95
|
// as a handoff, but if the owner chose an agent-first model, that handoff is the gap to close. Agent identity
|
|
91
96
|
// counts: the agent is meant to sign in as itself (for example with AgentID), not take a key from a person.
|
|
@@ -108,6 +113,7 @@ export function buildAudit({ doc, funnel, answers, task, runId, version }) {
|
|
|
108
113
|
headline: funnel.headline,
|
|
109
114
|
documented: funnel.documented,
|
|
110
115
|
verified: funnel.verified,
|
|
116
|
+
applicable: funnel.applicable ?? funnel.steps.filter((s) => s.state !== "not_applicable").length,
|
|
111
117
|
pagesRead: doc.observations.filter((o) => o.ok).length,
|
|
112
118
|
steps,
|
|
113
119
|
onboarding: {
|
|
@@ -151,7 +157,7 @@ export function renderBrief(audit, doc) {
|
|
|
151
157
|
lines.push(`Conflict: a verified person must exist before any use, but ${audit.onboarding.chosen.name} lets an agent act first. Consider "A person sets up access first".`, "");
|
|
152
158
|
}
|
|
153
159
|
lines.push("## Steps", "", "| Step | State | How we know | Evidence |", "| --- | --- | --- | --- |");
|
|
154
|
-
for (const s of audit.steps) lines.push(`| ${s.name} | ${s.state.replace("_", " ")} | ${s.basis.replace("_", " ")} | ${evidence(s.basedOn).join(", ") || "none"} |`);
|
|
160
|
+
for (const s of audit.steps) lines.push(`| ${s.name} | ${s.state.replace("_", " ")} | ${s.basis.replace("_", " ")} | ${evidence(s.basedOn).join(", ") || "none"}${(s.quotes || []).map((q) => `<br>"${q.quote.replace(/\|/g, "\\|")}"`).join("")} |`);
|
|
155
161
|
lines.push("", "## Fixes", "");
|
|
156
162
|
if (!audit.findings.length) lines.push("None from public pages. A test with a real agent is the next step: `npx agent-led-growth test --check` shows the plan for free, `npx agent-led-growth test` runs it.");
|
|
157
163
|
for (const f of audit.findings) lines.push(`${f.n}. **${f.title}** (${f.severity}, ${f.name}). Prompt: prompts/${f.file}`);
|
package/src/audit/render.js
CHANGED
|
@@ -9,6 +9,15 @@ export function makeStyle(enabled) {
|
|
|
9
9
|
import { PASSING, mark, label } from "../dashboard/strip.js";
|
|
10
10
|
const TAG = { observed: "from your files", heuristic: "from page text", not_checked: "needs a test run", tested: "from your test" };
|
|
11
11
|
|
|
12
|
+
// The page a quote is on, as a path: "/docs.md".
|
|
13
|
+
const pathOf = (url) => {
|
|
14
|
+
try {
|
|
15
|
+
return new URL(url).pathname;
|
|
16
|
+
} catch {
|
|
17
|
+
return url || "";
|
|
18
|
+
}
|
|
19
|
+
};
|
|
20
|
+
|
|
12
21
|
|
|
13
22
|
// Breaks text into lines of at most `width` characters, at spaces. A word longer than a line (a long URL) is split.
|
|
14
23
|
export function wrap(text, width) {
|
|
@@ -26,6 +35,14 @@ export function wrap(text, width) {
|
|
|
26
35
|
}
|
|
27
36
|
|
|
28
37
|
const plural = (n, one, many) => `${n} ${n === 1 ? one : many}`;
|
|
38
|
+
const listNames = (names) => (names.length < 2 ? names.join("") : `${names.slice(0, -1).join(", ")} and ${names.at(-1)}`);
|
|
39
|
+
|
|
40
|
+
// The outcome of the newest test of this product on this machine, from the funnel's evidence, or null.
|
|
41
|
+
export function latestTestOutcome(funnel) {
|
|
42
|
+
const tests = (funnel?.steps || []).flatMap((s) => (s.evidence || []).filter((e) => e.kind === "test"));
|
|
43
|
+
tests.sort((a, b) => (b.at > a.at ? 1 : b.at < a.at ? -1 : 0));
|
|
44
|
+
return tests[0]?.outcome || null;
|
|
45
|
+
}
|
|
29
46
|
|
|
30
47
|
// The answer to "can an agent do it?", before the tally: which steps the pages leave an agent without.
|
|
31
48
|
const DOES = { discover: "finds you", understand: "understands what you sell", signup: "signs up", access: "gets a key", use: "uses the product", pay: "pays", manage: "changes a plan" };
|
|
@@ -59,6 +76,11 @@ export function renderSteps(audit, style) {
|
|
|
59
76
|
const [first = "", ...rest] = wrap(s.reason, REASON);
|
|
60
77
|
out.push(` ${mark(s.state, style)} ${s.name.padEnd(NAME)} ${first.padEnd(REASON)} ${style.dim(TAG[s.basis] || s.basis)}`);
|
|
61
78
|
for (const line of rest) out.push(` ${" ".repeat(NAME + 4)}${line}`);
|
|
79
|
+
// A claim that the product requires something is shown with the sentence it came from, so a reader can check it.
|
|
80
|
+
for (const q of s.quotes || []) {
|
|
81
|
+
const text = q.quote.length > 160 ? `${q.quote.slice(0, 157).replace(/\s+\S*$/, "")}...` : q.quote;
|
|
82
|
+
for (const line of wrap(`"${text}" (${pathOf(q.url)})`, 60)) out.push(` ${" ".repeat(NAME + 4)}${style.dim(line)}`);
|
|
83
|
+
}
|
|
62
84
|
}
|
|
63
85
|
return out;
|
|
64
86
|
}
|
|
@@ -85,10 +107,20 @@ export function renderSummary(audit, paths, style, relative, savedTo) {
|
|
|
85
107
|
// What to do next, as the README's three steps: 1 check, 2 fix, 3 test. With fixes, step 2 names the first prompt
|
|
86
108
|
// by its full path, so it can be copied; the test commands follow for later. They name the product, so test runs on
|
|
87
109
|
// it whatever alg.yml the folder holds.
|
|
88
|
-
|
|
110
|
+
// latestTest: the outcome of the newest local test of this product (latestTestOutcome), or null. After a passing test
|
|
111
|
+
// with nothing to fix, Next says what is left instead of suggesting the same test again.
|
|
112
|
+
export function renderNext(audit, paths, style, relative, latestTest = null) {
|
|
89
113
|
const cmd = `npx agent-led-growth test ${audit.target.host}`;
|
|
90
114
|
const test = [` ${style.bold(`${cmd} --check`)}`, ` ${style.bold(cmd)}`, ""];
|
|
91
115
|
const step3 = "Step 3 of 3: test. The plan is free; a signup uses your Claude Code.";
|
|
116
|
+
if (!audit.findings.length && (latestTest === "passed" || latestTest === "handoff")) {
|
|
117
|
+
const skipped = audit.steps.filter((s) => s.state === "not_applicable").map((s) => s.name);
|
|
118
|
+
const open = audit.steps.filter((s) => s.state === "not_checked").map((s) => s.name);
|
|
119
|
+
const left = open.length ? `Not shown by a test yet: ${listNames(open)}.` : "Nothing left to fix or test.";
|
|
120
|
+
const na = skipped.length ? ` ${listNames(skipped)} ${skipped.length === 1 ? "doesn't" : "don't"} apply.` : "";
|
|
121
|
+
const [first, ...rest] = wrap(`Your latest test passed. ${left}${na} Keep it passing in CI:`, 68);
|
|
122
|
+
return [` ${style.bold("Next")} ${first}`, ...rest.map((l) => ` ${l}`), ` ${style.bold(`${cmd} --yes --json`)}`, ""];
|
|
123
|
+
}
|
|
92
124
|
if (!audit.findings.length) return [` ${style.bold("Next")} ${step3}`, ...test];
|
|
93
125
|
return [
|
|
94
126
|
` ${style.bold("Next")} Step 2 of 3: fix. Paste a prompt into your coding agent, deploy,`,
|
package/src/dashboard/strip.js
CHANGED
|
@@ -6,9 +6,9 @@ export const PASSING = new Set(["agent_did", "agent_can", "handoff"]);
|
|
|
6
6
|
export const FAILING = new Set(["needs_person", "blocked"]);
|
|
7
7
|
|
|
8
8
|
// Who the colour stands for. A handoff is a person stepping in at a documented point, so it reads as a person.
|
|
9
|
-
export const WHO = { agent_did: "agent", agent_can: "agent", handoff: "person", needs_person: "person", blocked: "blocked", not_checked: "none" };
|
|
10
|
-
export const LABEL = { agent_did: "Agent did", agent_can: "Agent can", handoff: "Handoff", needs_person: "Needs a person", blocked: "Blocked", not_checked: "Needs a test" };
|
|
11
|
-
export const KIND = { agent_did: "success", agent_can: "success", handoff: "success", needs_person: "warning", blocked: "error", not_checked: "neutral" };
|
|
9
|
+
export const WHO = { agent_did: "agent", agent_can: "agent", handoff: "person", needs_person: "person", blocked: "blocked", not_checked: "none", not_applicable: "none" };
|
|
10
|
+
export const LABEL = { agent_did: "Agent did", agent_can: "Agent can", handoff: "Handoff", needs_person: "Needs a person", blocked: "Blocked", not_checked: "Needs a test", not_applicable: "Doesn't apply" };
|
|
11
|
+
export const KIND = { agent_did: "success", agent_can: "success", handoff: "success", needs_person: "warning", blocked: "error", not_checked: "neutral", not_applicable: "neutral" };
|
|
12
12
|
export const EVIDENCE_KIND = { check: "Check of public pages", test: "Test with a real agent", pay: "Payment run", signal: "Your server", stuck: "Stuck report" };
|
|
13
13
|
|
|
14
14
|
export const who = (state) => WHO[state] || "none";
|
|
@@ -20,6 +20,7 @@ export function mark(state, style = null) {
|
|
|
20
20
|
const paint = (name, s) => (style ? style[name](s) : s);
|
|
21
21
|
if (PASSING.has(state)) return paint("green", "✓");
|
|
22
22
|
if (FAILING.has(state)) return paint("red", "✗");
|
|
23
|
+
if (state === "not_applicable") return paint("dim", "-");
|
|
23
24
|
return paint("dim", "·");
|
|
24
25
|
}
|
|
25
26
|
export const markOk = (ok, style = null) => (style ? (ok ? style.green("✓") : style.red("✗")) : ok ? "✓" : "✗");
|
|
@@ -227,6 +227,13 @@ export function evaluateAuthentication(x, observations) {
|
|
|
227
227
|
return { methods, mentions, requirement, agentCanUnderstandSetup: setup, friction };
|
|
228
228
|
}
|
|
229
229
|
|
|
230
|
+
// pricing.json lists plans and every one costs 0: nothing to buy. Only machine-readable plans count; a price in prose
|
|
231
|
+
// can be a demo's sample data (Track's homepage shows a customer paying $49.00).
|
|
232
|
+
export function freePlansOnly(pricing) {
|
|
233
|
+
const plans = pricing?.plans || [];
|
|
234
|
+
return plans.length > 0 && plans.every((p) => p.amount === 0);
|
|
235
|
+
}
|
|
236
|
+
|
|
230
237
|
export function evaluatePricing(x, observations) {
|
|
231
238
|
const pj = x.pricingJson;
|
|
232
239
|
const pagesFound = x.pricingPages;
|
package/src/interface/extract.js
CHANGED
|
@@ -172,7 +172,9 @@ export function extractPricingJson(observations, bodies) {
|
|
|
172
172
|
|
|
173
173
|
const PRICE_RE = /(?:\$|€|£|USD\s?)\s?\d[\d,]*(?:\.\d+)?(?:\s?(?:\/|per)\s?(?:mo(?:nth)?|yr|year|seat|user|request|call|1k|1000|k)\b)?/gi;
|
|
174
174
|
const AMBIGUITY_RE = /contact (?:us|sales)|custom pricing|talk to sales|get a quote|enterprise pricing|request a demo/gi;
|
|
175
|
-
|
|
175
|
+
// "per request" is a price only beside a price word: "priced per request", "billed per call". Alone it is usually a
|
|
176
|
+
// limit: Track's "4 KB per request", "What it sends, per request".
|
|
177
|
+
const USAGE_RE = /usage[- ]based|pay[- ]as[- ]you[- ]go|\b(?:priced|charged|billed|paid|pay|costs?|fee|price)\s+(?:\w+\s+)?per (?:request|call|token|seat|user)\b|metered billing/gi;
|
|
176
178
|
|
|
177
179
|
export function extractPricingPages(observations, bodies) {
|
|
178
180
|
const out = [];
|
package/src/interface/funnel.js
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
import { STAGES, PILLAR } from "../schema/stages.js";
|
|
2
2
|
import { AGENTID_NEEDS } from "./onboarding.js";
|
|
3
3
|
import { PASSING } from "../dashboard/strip.js";
|
|
4
|
+
import { freePlansOnly } from "./evaluate.js";
|
|
4
5
|
|
|
5
6
|
// The seven-step answer for one public check: how far an agent can get, where it stops, and what to build.
|
|
6
7
|
// Rule-based over the interface document only. Public evidence can show a path is documented; only a live
|
|
@@ -8,7 +9,9 @@ import { PASSING } from "../dashboard/strip.js";
|
|
|
8
9
|
export const FUNNEL_SCHEMA = "agent-ready/funnel@1";
|
|
9
10
|
// handoff: a person steps in at a documented point and the agent carries on. That is a working path, not a failure.
|
|
10
11
|
// agent_did: a live test proved the step. Only a live test sets it.
|
|
11
|
-
|
|
12
|
+
// not_applicable: the product's machine-readable files say the step has nothing to do (every plan in pricing.json is
|
|
13
|
+
// free, so there is nothing to pay for). It neither passes nor fails and is left out of the counts.
|
|
14
|
+
export const STEP_STATES = ["agent_did", "agent_can", "handoff", "needs_person", "blocked", "not_checked", "not_applicable"];
|
|
12
15
|
// Kept under this name for callers; the order itself lives in schema/stages.js.
|
|
13
16
|
export const FUNNEL_ORDER = STAGES;
|
|
14
17
|
|
|
@@ -52,15 +55,23 @@ const NEEDS_SENTENCE = {
|
|
|
52
55
|
wallet: "Needs a wallet the agent can pay from.",
|
|
53
56
|
agent_identity: "Needs an identity provider the agent can sign in with.",
|
|
54
57
|
};
|
|
55
|
-
|
|
56
|
-
|
|
58
|
+
// A step that says the product needs something carries the sentence it came from. A need with no quote is not said:
|
|
59
|
+
// the rule is that a requirement is reported only with the exact text that states it.
|
|
60
|
+
function needsOf(pattern) {
|
|
61
|
+
const quoted = (need) => (pattern.needsEvidence?.[need] || []).filter((e) => e.quote);
|
|
62
|
+
const kept = pattern.needs.filter((need) => quoted(need).length);
|
|
63
|
+
const sentence = kept.map((need) => {
|
|
57
64
|
if (!NEEDS_SENTENCE[need]) throw new Error(`no sentence for need "${need}"`);
|
|
58
65
|
return NEEDS_SENTENCE[need];
|
|
59
66
|
}).join(" ");
|
|
67
|
+
const quotes = kept.flatMap((need) => quoted(need).slice(0, 1).map((e) => ({ for: need, obs: e.obs, quote: e.quote })));
|
|
68
|
+
return { kept, sentence, quotes };
|
|
60
69
|
}
|
|
61
70
|
|
|
62
|
-
|
|
71
|
+
// quotes: [{ for, obs, quote }], the sentences behind a claim that the product requires something, shown with the step.
|
|
72
|
+
function step(id, state, reason, basedOn = [], tip = null, quotes = []) {
|
|
63
73
|
const out = { id, name: STEP_NAME[id], area: PILLAR[id], question: STEP_QUESTION[id], state, reason, basedOn: [...new Set(basedOn.filter(Boolean))] };
|
|
74
|
+
if (quotes.length) out.quotes = quotes;
|
|
64
75
|
// Only a step known to fail gets a fix. An unchecked step needs a live test first.
|
|
65
76
|
if (state === "needs_person" || state === "blocked") out.fix = FIX[id];
|
|
66
77
|
// A passing step can still be easier for agents; that is a tip, not a fix.
|
|
@@ -103,8 +114,9 @@ function signup(doc) {
|
|
|
103
114
|
const existing = patterns.find((p) => p.id === "existing_account");
|
|
104
115
|
const refs = (p) => (p?.evidence || []).map((e) => e.obs);
|
|
105
116
|
// The pattern's reason already names AgentID, what the agent needs, and whether the owner is shared.
|
|
106
|
-
|
|
107
|
-
if (self
|
|
117
|
+
const needs = self ? needsOf(self) : null;
|
|
118
|
+
if (self?.provider === "AgentID") return step("signup", "agent_can", self.reason, refs(self), null, needs.quotes);
|
|
119
|
+
if (self && needs.kept.length) return step("signup", "agent_can", `The docs describe an agent signing up on its own (${self.name}). ${needs.sentence}`, [...refs(self), ...needs.quotes.map((q) => q.obs)], null, needs.quotes);
|
|
108
120
|
// "Agent is the customer" has no person step, and its boundary text says so ("None documented. ..."), which does
|
|
109
121
|
// not read after "A person steps in".
|
|
110
122
|
const boundary = !self?.humanBoundary ? "" : /^None\b/.test(self.humanBoundary) ? " No person step is documented: the agent holds the account itself." : ` A person steps in ${self.humanBoundary.charAt(0).toLowerCase()}${self.humanBoundary.slice(1)}`;
|
|
@@ -149,24 +161,29 @@ function pay(doc) {
|
|
|
149
161
|
// agent_can, which would say no person is needed, and not needs_person, which would say a person checks out each time.
|
|
150
162
|
const purchase = doc.pricing?.agentPurchase;
|
|
151
163
|
const purchaseRefs = (purchase?.evidence || []).map((e) => e.obs);
|
|
164
|
+
const purchaseQuotes = (purchase?.evidence || []).filter((e) => e.quote).slice(0, 1).map((e) => ({ for: "payment", obs: e.obs, quote: e.quote }));
|
|
152
165
|
const perRequestCall = purchase?.rule === "pay_per_request_own_api";
|
|
153
166
|
const checkoutCall = purchase?.rule === "checkout_protocol_own_api";
|
|
154
167
|
const call = purchase?.call ? ` (${purchase.call})` : "";
|
|
168
|
+
const quote = (entries, what) => (entries || []).filter((e) => e.quote).slice(0, 1).map((e) => ({ for: what, obs: e.obs, quote: e.quote }));
|
|
155
169
|
const over = purchase?.protocol ? ` over ${purchase.protocol}` : "";
|
|
156
170
|
if (purchase?.approval === "once") {
|
|
157
171
|
const how = perRequestCall ? `pays over ${purchase.protocol || "x402 or MPP"}${call}` : checkoutCall ? `checks out over ${purchase.protocol}${call}` : purchase.call ? `buys through ${purchase.call}${over}` : `buys through the API${over}`;
|
|
158
|
-
return step("pay", "handoff", `A person sets payment up once (an approval link, a spending cap, a saved card or a delegation), then the agent ${how}, with no browser checkout. The docs say so; a test shows whether it works.`, purchaseRefs);
|
|
172
|
+
return step("pay", "handoff", `A person sets payment up once (an approval link, a spending cap, a saved card or a delegation), then the agent ${how}, with no browser checkout. The docs say so; a test shows whether it works.`, purchaseRefs, null, purchaseQuotes);
|
|
159
173
|
}
|
|
160
|
-
if (purchase?.rule === "agent_buys_through_api") return step("pay", "agent_can", `The docs describe an agent buying through the API${call}${over}, and no person approving first. A test shows whether it works.`, purchaseRefs);
|
|
161
|
-
if (perRequestCall) return step("pay", "agent_can", `The docs show a paid call over ${purchase.protocol || "x402 or MPP"} on the product's own API${call}, with no browser checkout. The agent needs a funded wallet; a test shows whether the payment goes through.`, purchaseRefs);
|
|
162
|
-
if (checkoutCall && purchase.call) return step("pay", "agent_can", `The docs show a checkout an agent completes over ${purchase.protocol} on the product's own API${call}, with no browser checkout. A test shows whether a purchase goes through.`, purchaseRefs);
|
|
163
|
-
if (checkoutCall) return step("pay", "agent_can", `The product publishes a UCP business profile at /.well-known/ucp, so an agent can check out over UCP, with no browser checkout. A test shows whether a purchase goes through.`, purchaseRefs);
|
|
174
|
+
if (purchase?.rule === "agent_buys_through_api") return step("pay", "agent_can", `The docs describe an agent buying through the API${call}${over}, and no person approving first. A test shows whether it works.`, purchaseRefs, null, purchaseQuotes);
|
|
175
|
+
if (perRequestCall) return step("pay", "agent_can", `The docs show a paid call over ${purchase.protocol || "x402 or MPP"} on the product's own API${call}, with no browser checkout. The agent needs a funded wallet; a test shows whether the payment goes through.`, purchaseRefs, null, purchaseQuotes);
|
|
176
|
+
if (checkoutCall && purchase.call) return step("pay", "agent_can", `The docs show a checkout an agent completes over ${purchase.protocol} on the product's own API${call}, with no browser checkout. A test shows whether a purchase goes through.`, purchaseRefs, null, purchaseQuotes);
|
|
177
|
+
if (checkoutCall) return step("pay", "agent_can", `The product publishes a UCP business profile at /.well-known/ucp, so an agent can check out over UCP, with no browser checkout. A test shows whether a purchase goes through.`, purchaseRefs, null, purchaseQuotes);
|
|
178
|
+
// Machine-readable pricing comes before prose: with every plan in pricing.json at 0 and no purchase documented,
|
|
179
|
+
// there is nothing to pay for, whatever the page text says (a demo's "$49.00", x402 named as something tested).
|
|
180
|
+
if (freePlansOnly(doc.pricing)) return step("pay", "not_applicable", "Nothing to buy: every plan is free (pricing.json).", plans.flatMap((p) => (p.evidence || []).map((e) => e.obs)));
|
|
164
181
|
// Pay-per-request wording is easy to misread: a billing product describing its own
|
|
165
182
|
// customers' metering reads the same as an agent paying. Only a live test settles it.
|
|
166
|
-
if (perRequest) return step("pay", "not_checked", "The docs mention paying per request. A test with a real agent shows whether it can actually pay.", (perRequest.evidence || []).map((e) => e.obs));
|
|
183
|
+
if (perRequest) return step("pay", "not_checked", "The docs mention paying per request over x402. A test with a real agent shows whether it can actually pay.", (perRequest.evidence || []).map((e) => e.obs), null, quote(perRequest.evidence, "payment"));
|
|
167
184
|
// Only the product's own domain is read: a purchase path on a payments provider's site (Stripe Projects lists
|
|
168
185
|
// Cloudflare, Neon, Turso and others on stripe.com) is not seen here, so the sentence says what was read.
|
|
169
|
-
if (paidPlans.length || paidText.length) return step("pay", "needs_person", "Paid plans are published. The pages read document no way for an agent to buy one, so a person may have to check out.", refs);
|
|
186
|
+
if (paidPlans.length || paidText.length) return step("pay", "needs_person", "Paid plans are published. The pages read document no way for an agent to buy one, so a person may have to check out.", refs, null, quote(paidPlans.length ? paidPlans[0].evidence : paidText[0].evidence, "payment"));
|
|
170
187
|
if (plans.length || observed.length) return step("pay", "not_checked", "Only free plans were found, so there is nothing to pay for yet.", refs);
|
|
171
188
|
return step("pay", "not_checked", "No plans were found in the pages read.");
|
|
172
189
|
}
|
|
@@ -186,7 +203,9 @@ function use(doc) {
|
|
|
186
203
|
return step("use", "not_checked", "No API actions were found in the pages read.", refs);
|
|
187
204
|
}
|
|
188
205
|
|
|
189
|
-
function manage() {
|
|
206
|
+
function manage(doc) {
|
|
207
|
+
const plans = doc.pricing?.plans || [];
|
|
208
|
+
if (plans.length === 1 && freePlansOnly(doc.pricing) && !doc.pricing.agentPurchase) return step("manage", "not_applicable", "One free plan: no plan or limit to change.", (plans[0].evidence || []).map((e) => e.obs));
|
|
190
209
|
return step("manage", "not_checked", "Changing a plan or a limit can only be proven by a test run.");
|
|
191
210
|
}
|
|
192
211
|
|
|
@@ -199,14 +218,15 @@ function withAgentId(doc, s, sentence) {
|
|
|
199
218
|
const self = patterns.find((p) => SIGNUP_PATTERNS.has(p.id));
|
|
200
219
|
const agentId = patterns.find((p) => p.provider === "AgentID");
|
|
201
220
|
if (!agentId || agentId === self) return s;
|
|
202
|
-
|
|
221
|
+
const quotes = [...(s.quotes || []), ...agentId.evidence.filter((e) => e.quote).slice(0, 1).map((e) => ({ for: "agent_identity", obs: e.obs, quote: e.quote }))];
|
|
222
|
+
return { ...s, reason: `${s.reason} ${sentence}`, basedOn: [...new Set([...s.basedOn, ...agentId.evidence.map((e) => e.obs)])], quotes };
|
|
203
223
|
}
|
|
204
224
|
|
|
205
225
|
export function buildFunnel(doc) {
|
|
206
226
|
const d = discover(doc);
|
|
207
227
|
const s = withAgentId(doc, signup(doc), `The docs also describe the agent signing in with AgentID as itself, which needs ${AGENTID_NEEDS}.`);
|
|
208
228
|
const a = withAgentId(doc, access(doc, s), "Signed in with AgentID, the agent would hold its own account and get its key or session from it.");
|
|
209
|
-
const byId = { discover: d, understand: understand(doc), signup: s, access: a, pay: pay(doc), use: use(doc), manage: manage() };
|
|
229
|
+
const byId = { discover: d, understand: understand(doc), signup: s, access: a, pay: pay(doc), use: use(doc), manage: manage(doc) };
|
|
210
230
|
const steps = FUNNEL_ORDER.map((id) => byId[id]);
|
|
211
231
|
const chosen = choosePath(doc);
|
|
212
232
|
const paths = (doc.onboarding?.patterns || []).map((p) => ({ id: p.id, name: p.name, status: p.status, humanBoundary: p.humanBoundary || null, needs: p.needs, followed: p === chosen }));
|
|
@@ -217,20 +237,25 @@ export function buildFunnel(doc) {
|
|
|
217
237
|
// that does not pass, and whether that step is a real stop or only unproven.
|
|
218
238
|
// The headline keeps two counts apart: steps the public pages document (agent_can, handoff) and steps a test with a
|
|
219
239
|
// real agent verified (agent_did). Reading pages is not running a task, so a check alone never says a step works.
|
|
240
|
+
// A step that does not apply is skipped: it neither passes nor stops the walk, and applicable counts the rest.
|
|
220
241
|
export function summarize(steps) {
|
|
221
|
-
const
|
|
222
|
-
const
|
|
242
|
+
const applicable = steps.filter((x) => x.state !== "not_applicable");
|
|
243
|
+
const firstOpen = applicable.findIndex((x) => !PASSING.has(x.state));
|
|
244
|
+
const passed = firstOpen === -1 ? applicable.length : firstOpen;
|
|
223
245
|
const stop = steps.find((x) => x.state === "blocked" || x.state === "needs_person") || null;
|
|
224
246
|
const verified = steps.filter((x) => x.state === "agent_did").length;
|
|
225
247
|
const documented = steps.filter((x) => x.state === "agent_can" || x.state === "handoff").length;
|
|
226
248
|
// stepsPassing (from #78) is every passing step, the total the headline gives: documented plus verified.
|
|
227
|
-
return { headline: headlineFor(steps, documented, verified), passed, stepsPassing: documented + verified, documented, verified, of: steps.length, stopsAt: stop?.id || null, fixes: steps.filter((x) => x.fix).map((x) => ({ step: x.id, name: x.name, fix: x.fix })) };
|
|
249
|
+
return { headline: headlineFor(steps, documented, verified), passed, stepsPassing: documented + verified, documented, verified, of: steps.length, applicable: applicable.length, stopsAt: stop?.id || null, fixes: steps.filter((x) => x.fix).map((x) => ({ step: x.id, name: x.name, fix: x.fix })) };
|
|
228
250
|
}
|
|
229
251
|
|
|
230
252
|
function headlineFor(steps, documented, verified) {
|
|
231
|
-
|
|
253
|
+
const skipped = steps.filter((x) => x.state === "not_applicable").map((x) => x.name);
|
|
254
|
+
const of = skipped.length ? `${steps.length - skipped.length} applicable steps` : `${steps.length} steps`;
|
|
255
|
+
const dontApply = skipped.length ? `; ${listNames(skipped)} ${skipped.length === 1 ? "doesn't" : "don't"} apply` : "";
|
|
256
|
+
if (!steps.some((x) => x.live)) return `Your public pages document ${documented} of ${of}${dontApply}. No step has been tested yet: a test runs a real agent.`;
|
|
232
257
|
const failed = steps.filter((x) => x.live && x.state === "blocked").map((x) => x.name);
|
|
233
|
-
const parts = [`${verified} of ${
|
|
258
|
+
const parts = [`${verified} of ${of} verified by a test${dontApply}.`];
|
|
234
259
|
if (documented) parts.push(`Your public pages document ${documented} more.`);
|
|
235
260
|
if (failed.length) parts.push(`${listNames(failed)} failed a test.`);
|
|
236
261
|
return parts.join(" ");
|
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
import { htmlToText, registrableDomain, sameSite } from "./sources.js";
|
|
2
|
+
import { freePlansOnly } from "./evaluate.js";
|
|
2
3
|
|
|
3
4
|
// Which ways can an agent start using this product? Read from public docs only, so every pattern is a claim the docs
|
|
4
5
|
// make, with the quotes that make it; none is a result. A product can support several. Rules, not a model: each signal
|
|
@@ -41,7 +42,9 @@ const SIGNALS = {
|
|
|
41
42
|
// use it to mean "claim first" (Cosmic) or "over your plan" (Inkbox).
|
|
42
43
|
payPerRequest: /\bx402\b|\bX-PAYMENT\b|\bPAYMENT-REQUIRED\b/,
|
|
43
44
|
// The documented way in is a command-line tool rather than an HTTP call.
|
|
44
|
-
|
|
45
|
+
// At most four words between the tool and the flag or subcommand, so it is one command line: Track's docs say
|
|
46
|
+
// "npx agent-led-growth check <url> runs a real agent through signup", which is prose after a command, not a signup.
|
|
47
|
+
cliBootstrap: /\b(?:npx|wrangler|mem0|stripe|npm i(?:nstall)?|pip install|brew install)\b(?:[ \t]+[^\s`]+){0,4}?[ \t]+(?:--temporary|--agent|init\b|sign-?up\b|projects init)/i,
|
|
45
48
|
// The product's CLI takes a token without a browser, so an agent can run it unattended once a person has made the
|
|
46
49
|
// token: Vercel's "Set the VERCEL_TOKEN environment variable" or "Pass the --token option", GitHub's "--with-token".
|
|
47
50
|
// An environment variable counts only where the text calls it one: code samples are full of other _API_KEY names.
|
|
@@ -245,12 +248,51 @@ const EXAMPLE_PAGE = /\/(?:templates?|examples?|starters?)\//i;
|
|
|
245
248
|
// OAuth dynamic client registration (POST /oauth/register, /oauth2/register, /connect/register) registers a client app
|
|
246
249
|
// with an authorization server; it is not an agent getting an account.
|
|
247
250
|
const OAUTH_REGISTRATION = /\/(?:oauth2?|connect|clients?|dcr)\/regist(?:er|ration)\b/i;
|
|
251
|
+
// A signal that says the product requires something (an inbox, a person setting access up, restricted access, a
|
|
252
|
+
// payment) counts only from a sentence that says so about the product itself. Text that merely mentions the thing never
|
|
253
|
+
// counts: a negation ("No project key can sign in to the dashboard"), an example ("such as opening a verification
|
|
254
|
+
// email") or a sentence about the reader's own customers. Each kept match is quoted, and the quote is shown with the
|
|
255
|
+
// step it decides.
|
|
256
|
+
// A negation in the same clause just before the match: "A project key cannot sign in to the dashboard". A comma ends
|
|
257
|
+
// the clause, so "If you don't have one, create an API key in the dashboard" still counts.
|
|
258
|
+
const CLAUSE_NEGATION = /\b(?:no|not|never|cannot|can't|can not|without|don't|doesn't|won't|isn't|aren't|nothing)\b[^.,;:!?\n]{0,40}$/i;
|
|
259
|
+
const negatedBefore = (sentence, match) => {
|
|
260
|
+
const at = sentence.indexOf(match.replace(/\s+/g, " "));
|
|
261
|
+
return at >= 0 && CLAUSE_NEGATION.test(sentence.slice(0, at));
|
|
262
|
+
};
|
|
263
|
+
// Expiry said not to happen: Track's "Unclaimed, nothing is deleted", "the workspace and key never expire".
|
|
264
|
+
const NO_EXPIRY = /\b(?:nothing|never|not|no\s+\w+)\s+(?:is\s+|are\s+|will\s+|gets?\s+|be\s+|ever\s+)*(?:deleted|removed|expires?)\b|\bdo(?:es)?(?: not|n't) expire\b/i;
|
|
265
|
+
// An example of something, not the product's own step: Track's "For a step your API never sees, such as opening a
|
|
266
|
+
// verification email, send an onboarding_step event".
|
|
267
|
+
const EXAMPLE = /\b(?:such as|for example|for instance|e\.g\.|i\.e\.)/i;
|
|
268
|
+
// An inbox step counts only in a sentence about the product's own signup or sign-in path. The words are looked for
|
|
269
|
+
// outside the match, so "Verification email never opened" (a sample error message in Track's docs) is not its own
|
|
270
|
+
// context.
|
|
271
|
+
const SIGNUP_CONTEXT = /\bsign(?:s|ing|ed)?[- ]?(?:up|in)\b|\bsignups?\b|\bregist(?:er|ers|ered|ering|ration)\b|\bcreat(?:e|es|ing) (?:an? |your |the )?(?:new )?account\b|\bonboard(?:ing)?\b|\bverif(?:y|ies|ied|ication)\b|\bconfirm(?:s|ed)?\b|\blog ?in\b|\bget (?:an? |your )?(?:API )?key\b/i;
|
|
272
|
+
// The product saying there is no inbox step. Authoritative: one such sentence on the product's own pages clears every
|
|
273
|
+
// inbox match. Track's "POST /v1/signup returns a key with no account, email or browser", "Track needs no inbox
|
|
274
|
+
// verification", "takes no email and sends nothing to confirm". Not "without email": Telnyx's "Without email access the
|
|
275
|
+
// flow falls back to..." is a condition, and its signup does send a magic link.
|
|
276
|
+
export const NO_INBOX = /\b(?:takes|needs|requires|asks for|sends) no (?:e-?mail|inbox)\b|\bneeds? no inbox\b|\bno inbox (?:verification|step|needed|required)\b|\bno (?:account, )?e-?mail(?:,| or| needed| required)\b|\bno (?:e-?mail )?verification (?:e-?mail|code|link|step)\b|\bsends nothing to confirm\b/i;
|
|
277
|
+
const withoutMatch = (sentence, match) => sentence.replace(match.replace(/\s+/g, " "), " ");
|
|
278
|
+
// A line of a link index ("- [Email OTP](https://neon.com/docs/auth/...): Sign in and verify email addresses") describes
|
|
279
|
+
// another page, often a feature the product sells, not the product's own signup.
|
|
280
|
+
const INDEX_LINE = /^\s*(?:[-*]\s*)?\[[^\]]+\]\([^)]+\)/;
|
|
281
|
+
// Signing up somewhere else: AgentMail's "Enable browser agents to sign up for services by extracting OTP codes".
|
|
282
|
+
// AgentLine's "Receive OTP codes so your agent can sign up and verify" is a feature for signing up elsewhere too.
|
|
283
|
+
const SIGNUP_ELSEWHERE = /\bsign(?:s|ing)? up for (?:services|other|any|third-party|apps|sites|websites)\b|\b(?:other|third-party) (?:services|apps|sites|websites)\b|\bso (?:your|the|an) agents? can sign up\b/i;
|
|
284
|
+
// An optional step is not a requirement: Locus's "the human is needed only for an optional AgentMail OTP".
|
|
285
|
+
const OPTIONAL = /\boptional(?:ly)?\b/i;
|
|
286
|
+
|
|
248
287
|
// Whether a sentence holding a match is about something else; such a match is skipped and the next one tried.
|
|
249
288
|
const SKIP = {
|
|
250
289
|
bootstrap: (sentence, ownDomain, match) => DOMAIN_ACTORS.test(sentence) || SIGNUP_OFF.test(sentence) || PERSON_WITHOUT_ACCOUNT.test(sentence) || PAYMENT_WAY_IN.test(sentence) || CLIENT_REGISTRATION.test(sentence) || BROKERED.test(sentence) || (/^temporary\b/i.test(match) && STORED_THING.test(sentence)),
|
|
251
|
-
expiry: (sentence) => !ACCOUNT_THING.test(sentence) || STORED_THING.test(sentence),
|
|
290
|
+
expiry: (sentence) => !ACCOUNT_THING.test(sentence) || STORED_THING.test(sentence) || NO_EXPIRY.test(sentence),
|
|
291
|
+
restricted: (sentence, ownDomain, match) => DOMAIN_ACTORS.test(sentence) || EXAMPLE.test(sentence) || negatedBefore(sentence, match),
|
|
292
|
+
emailStep: (sentence, ownDomain, match) => DOMAIN_ACTORS.test(sentence) || EXAMPLE.test(sentence) || INDEX_LINE.test(sentence) || SIGNUP_ELSEWHERE.test(sentence) || OPTIONAL.test(sentence) || NO_INBOX.test(sentence) || negatedBefore(sentence, match) || !SIGNUP_CONTEXT.test(withoutMatch(sentence, match)),
|
|
293
|
+
humanFirst: (sentence, ownDomain, match) => DOMAIN_ACTORS.test(sentence) || EXAMPLE.test(sentence) || negatedBefore(sentence, match),
|
|
252
294
|
httpBootstrap: (sentence, ownDomain, match) => DOMAIN_ACTORS.test(sentence) || OAUTH_REGISTRATION.test(match),
|
|
253
|
-
payPerRequest: (sentence, ownDomain) => DOMAIN_ACTORS.test(sentence) || SELLER.test(sentence) || PROTOCOL_DESCRIPTION.test(sentence) || paysElsewhere(sentence, ownDomain),
|
|
295
|
+
payPerRequest: (sentence, ownDomain, match) => DOMAIN_ACTORS.test(sentence) || SELLER.test(sentence) || PROTOCOL_DESCRIPTION.test(sentence) || paysElsewhere(sentence, ownDomain) || negatedBefore(sentence, match),
|
|
254
296
|
};
|
|
255
297
|
const GLOBAL = Object.fromEntries(Object.keys(SKIP).map((name) => [name, new RegExp(SIGNALS[name].source, `${SIGNALS[name].flags}g`)]));
|
|
256
298
|
|
|
@@ -288,6 +330,18 @@ function findSignals(observations, bodies, targetUrl) {
|
|
|
288
330
|
return found;
|
|
289
331
|
}
|
|
290
332
|
|
|
333
|
+
// The first sentence on the product's own pages saying signup has no inbox step, or null.
|
|
334
|
+
function findNoInbox(doc, bodies) {
|
|
335
|
+
for (const obs of doc.observations) {
|
|
336
|
+
const body = bodies.get(obs.id);
|
|
337
|
+
if (!body || !sameSite(obs.url, doc.target.url)) continue;
|
|
338
|
+
const text = body.html ? htmlToText(body.text) : body.text;
|
|
339
|
+
const m = NO_INBOX.exec(text);
|
|
340
|
+
if (m) return { obs: obs.id, quote: quoteAround(text, m.index) };
|
|
341
|
+
}
|
|
342
|
+
return null;
|
|
343
|
+
}
|
|
344
|
+
|
|
291
345
|
// What testing this pattern end to end requires beyond HTTP requests: an inbox for an
|
|
292
346
|
// email step, a command-line runner, a funded wallet. Whether a deployment has each is
|
|
293
347
|
// decided where the report is shown, not here. Agent identity has no runner support.
|
|
@@ -302,6 +356,13 @@ function needsFor(id, found) {
|
|
|
302
356
|
return needs;
|
|
303
357
|
}
|
|
304
358
|
|
|
359
|
+
// The quotes behind each need, so a step that says the product needs something can show the sentence it came from.
|
|
360
|
+
const NEED_SIGNALS = { inbox: ["emailStep"], cli: ["cliBootstrap"], wallet: ["payPerRequest"], agent_identity: ["agentIdSignIn", "agentIdentity"] };
|
|
361
|
+
function needsEvidenceFor(needs, found) {
|
|
362
|
+
const ordered = (entries) => [...entries.filter((e) => e.agentFacing), ...entries.filter((e) => !e.agentFacing)];
|
|
363
|
+
return Object.fromEntries(needs.map((need) => [need, ordered(NEED_SIGNALS[need].flatMap((name) => found[name] || [])).map(({ obs, quote }) => ({ obs, quote }))]));
|
|
364
|
+
}
|
|
365
|
+
|
|
305
366
|
function pattern(id, found, signalNames, reason) {
|
|
306
367
|
// The product's agent-facing pages first: a quote from llms.txt or an agent-signup
|
|
307
368
|
// page says more than the same words in homepage navigation.
|
|
@@ -316,6 +377,7 @@ function pattern(id, found, signalNames, reason) {
|
|
|
316
377
|
evidence,
|
|
317
378
|
humanBoundary: PATTERNS[id].humanBoundary,
|
|
318
379
|
needs: needsFor(id, found),
|
|
380
|
+
needsEvidence: needsEvidenceFor(needsFor(id, found), found),
|
|
319
381
|
};
|
|
320
382
|
}
|
|
321
383
|
|
|
@@ -352,6 +414,9 @@ export function detectOnboarding(doc, bodies) {
|
|
|
352
414
|
// A signup call documented on a page written for agents is the product saying an agent can sign itself up, even
|
|
353
415
|
// without the words "agent signup": Moltbook's skill.md says "register" and shows POST /api/v1/agents/register.
|
|
354
416
|
found.bootstrap.push(...found.httpBootstrap.filter((e) => e.agentFacing && !found.bootstrap.some((b) => b.obs === e.obs)));
|
|
417
|
+
// The product saying it has no inbox step outweighs any mention of one.
|
|
418
|
+
const noInbox = findNoInbox(doc, bodies);
|
|
419
|
+
if (noInbox) found.emailStep = [];
|
|
355
420
|
const has = (name) => found[name].length > 0;
|
|
356
421
|
const patterns = [];
|
|
357
422
|
|
|
@@ -378,7 +443,9 @@ export function detectOnboarding(doc, bodies) {
|
|
|
378
443
|
patterns.push(p);
|
|
379
444
|
}
|
|
380
445
|
|
|
381
|
-
|
|
446
|
+
// Machine-readable pricing first: when pricing.json lists only free plans, x402 in prose is not a price. Track's docs
|
|
447
|
+
// name x402 as one of the tasks its sister product tests, and that read as Track charging per request.
|
|
448
|
+
if (has("payPerRequest") && !freePlansOnly(doc.pricing)) patterns.push(pattern("pay_per_request", found, ["payPerRequest"], "The docs describe paying per request over HTTP 402, with no account needed first."));
|
|
382
449
|
|
|
383
450
|
// Only the signup side says a person sets access up: a key made in a dashboard, a token a person creates. An auth
|
|
384
451
|
// scheme in the OpenAPI document (Turso's `bearerAuth: http bearer`) says how a request is signed, not who gets the
|
package/src/interface/schema.js
CHANGED
|
@@ -23,8 +23,8 @@ export const SCHEMA_DOC = {
|
|
|
23
23
|
authentication_requirement: "{ value: none|required|mixed|not_declared|unknown, reason, basedOn, rule, operations: [{ operation, value, reason, basedOn, rule, source: operation|global|absent, schemeNames }] }; operation security overrides global security; none requires an explicit operation [] or anonymous {} alternative; absent or empty global security is not_declared; unresolved declarations or incomplete operation coverage are unknown",
|
|
24
24
|
pricing: "model, plans, observedPrices, agentCanDetermineCost, ambiguities, agentPurchase. agentPurchase is null or { rule: agent_buys_through_api|payment_required_approval|pay_per_request_own_api|checkout_protocol_own_api, rules, approval: once|none_documented, call, protocol: x402|MPP|ACP|AP2|UCP|null, evidence: [{ obs, quote }] }: a way for an agent to buy from the product through an API, as the product's own pages describe it (an agent buying with a purchase call, a cap or an approval; 402 payment_required with an approval link; x402 or MPP on a call to the product's own API; an ACP or UCP checkout session call on the product's own API, or a UCP business profile at /.well-known/ucp). protocol names the payment or checkout protocol the quoted sentence names, when it names one. approval once means a person approves first and the agent buys after that. A documented claim, never a result.",
|
|
25
25
|
machineAccess: "robots, llms.txt, agent.json facts plus aiCrawlersAllowed, hasAgentReadableIndex and agentAuthDiscoverable",
|
|
26
|
-
onboarding: "{ patterns: [{ id: try_then_claim|limited_until_claimed|agent_is_customer|agent_identity|existing_account|pay_per_request, name, status: documented|inferred, reason, evidence: [{ obs, quote }], humanBoundary, needs: [inbox|cli|wallet|agent_identity], provider?: AgentID }], apiHosts: [{ domain, host, example, evidence }], primary, reason }. apiHosts are API URLs on other domains that the product's agent-facing pages point to. Ways an agent can start, as the public docs describe them; needs lists what testing that way end to end requires beyond HTTP requests, which a given deployment may or may not provide. provider names the identity provider an agent_identity pattern uses, when the docs name one the detector knows (AgentID). A pattern is a documented claim, never a result.",
|
|
27
|
-
funnel: "{ headline, passed, stepsPassing, documented, verified, of, stopsAt, path, paths, steps: [{ id: discover|understand|signup|access|use|pay|manage, name, area: web|onboarding|monetization, question, state: agent_did|agent_can|handoff|needs_person|blocked|not_checked, reason, fix?, basedOn, live?, evidence? }], fixes }. The seven steps an agent takes, from public evidence only. handoff means a person steps in at a documented point and the agent carries on, which counts as passing. agent_did appears only when the latest test run for that step on this product passed; a blocked test run sets blocked. live names that test. evidence lists, newest first, what else set or touched the step: { kind: test|pay|signal|stuck, at, reason, runId?, outcome?, count? }; a test or payment or the product's own signals can decide the state, a stuck report only attaches. documented counts agent_can and handoff steps, verified counts agent_did steps; the headline gives both and never calls a documented step verified. passed counts the passing steps before the first one that does not pass; stepsPassing counts every passing step (documented plus verified), the total the headline gives.",
|
|
26
|
+
onboarding: "{ patterns: [{ id: try_then_claim|limited_until_claimed|agent_is_customer|agent_identity|existing_account|pay_per_request, name, status: documented|inferred, reason, evidence: [{ obs, quote }], humanBoundary, needs: [inbox|cli|wallet|agent_identity], needsEvidence?: { <need>: [{ obs, quote }] }, provider?: AgentID }], apiHosts: [{ domain, host, example, evidence }], primary, reason }. apiHosts are API URLs on other domains that the product's agent-facing pages point to. Ways an agent can start, as the public docs describe them; needs lists what testing that way end to end requires beyond HTTP requests, which a given deployment may or may not provide. provider names the identity provider an agent_identity pattern uses, when the docs name one the detector knows (AgentID). A pattern is a documented claim, never a result.",
|
|
27
|
+
funnel: "{ headline, passed, stepsPassing, documented, verified, of, stopsAt, path, paths, steps: [{ id: discover|understand|signup|access|use|pay|manage, name, area: web|onboarding|monetization, question, state: agent_did|agent_can|handoff|needs_person|blocked|not_checked|not_applicable, reason, fix?, basedOn, quotes?, live?, evidence? }], applicable, fixes }. not_applicable: the product's machine-readable files say the step has nothing to do (Pay when every plan in pricing.json costs 0 and no purchase is documented; Manage when that is one plan); it neither passes nor stops the walk, and applicable counts the other steps. quotes: [{ for, obs, quote }], the sentences behind a claim that the product needs or requires something; a need with no quote is not reported. The seven steps an agent takes, from public evidence only. handoff means a person steps in at a documented point and the agent carries on, which counts as passing. agent_did appears only when the latest test run for that step on this product passed; a blocked test run sets blocked. live names that test. evidence lists, newest first, what else set or touched the step: { kind: test|pay|signal|stuck, at, reason, runId?, outcome?, count? }; a test or payment or the product's own signals can decide the state, a stuck report only attaches. documented counts agent_can and handoff steps, verified counts agent_did steps; the headline gives both and never calls a documented step verified. passed counts the passing steps before the first one that does not pass; stepsPassing counts every passing step (documented plus verified), the total the headline gives.",
|
|
28
28
|
unknowns: "every field whose verdict is unknown or whose fact is null, with the reason",
|
|
29
29
|
limits: "what this run could not do (no JS rendering, no login, no model interpretation)",
|
|
30
30
|
},
|