agent-led-growth 0.8.2 → 0.8.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +28 -0
- package/README.md +2 -0
- package/bin/cli.js +11 -5
- package/package.json +1 -1
- package/skills/alg/SKILL.md +1 -1
- package/src/audit/index.js +8 -2
- package/src/audit/render.js +33 -1
- package/src/dashboard/strip.js +4 -3
- package/src/interface/evaluate.js +8 -1
- package/src/interface/extract.js +20 -2
- package/src/interface/funnel.js +67 -22
- package/src/interface/onboarding.js +190 -27
- package/src/interface/purchase.js +8 -4
- package/src/interface/quote.js +255 -0
- package/src/interface/schema.js +2 -2
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-led-growth",
|
|
3
|
-
"version": "0.8.
|
|
3
|
+
"version": "0.8.4",
|
|
4
4
|
"description": "Score: evals for agents as customers. Runs a real agent through signup, key and first call on your product, before you ship. Track, the analytics half, measures which agents convert after.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Kat Laszlo",
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,33 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.8.4 (2026-10-24)
|
|
4
|
+
|
|
5
|
+
Quotes you can check. 0.8.3 printed each requirement with the text it came from, but the text was cut from the page as flat words, so on resend.com Pay quoted "scale as you grow. Are you an AI agent? See pricing Add-ons if you send from a": a subtitle, a link and a heading run together.
|
|
6
|
+
|
|
7
|
+
- Rule: a quote is one or two whole sentences from one element of the page (a paragraph, a list item, a table cell), cut only at sentence ends, and it holds the words that triggered the rule. When those words sit in an element that is not a sentence, the quote is that element with a label: `Heading: "Domains - $20 / mo" (/pricing)`, `Table cell: "/v1/x402/search"`, `List item: "Identity provider: AgentID OpenID Connect"`, `OpenAPI path: "/v1/x402/numbers/buy"`, `pricing.json: "Pro: 20 USD per month"`. Words that run across two elements are never a quote, and the rule tries its next match instead. Quotes are printed whole; the 160-character cut with "..." is gone.
|
|
8
|
+
- The rules read the same sentence they quote, so a guard never decides on words from a neighbouring element.
|
|
9
|
+
- Pay quotes a paid price, from the pricing page before the homepage: firecrawl.dev's quote was its "$75M Series B" announcement and is now "Standard: $83/month" from /pricing. A price that can only be read out of a script's data and never appears in a sentence or table no longer makes Pay "needs a person"; it says the prices could not be quoted and a test decides. No recorded product changed state from this.
|
|
10
|
+
- Inbox: an emailed code or link counts only when the sentence is about the product's own signup: it names the product's signup or verify call, names the product ("Sign up for Acme", "Acme emails"), says "we send" or "our signup", sits on the product's agent-signup or auth page, or sits in a signup section of a page (a "Sign up" or "Quick start" heading, or right after the product's signup call). A narrative about agents in general ("An agent signs up with its owner's email, reads the verification code out of the owner's inbox") does not count, except on the product's own signup page. www.agentid.com no longer needs an inbox; telnyx.com, inkbox.ai, agentmail.to, aisend.app, www.cosmicjs.com and agentline.cloud still do, each quoting its own signup sentence.
|
|
11
|
+
- CLI signup: a bare `init` no longer counts as signing up; it needs `--agent`, `--temporary` or a signup subcommand. AgentID's `npx @agentmail/agentid-cli init` registers a developer's app and read as AgentID's signup needing a shell. Cloudflare's CLI quote is now its `wrangler deploy --temporary` sentence instead of `wrangler init my-worker`.
|
|
12
|
+
- Signup: www.agentid.com read as "Try first, claim later" from three things that say nothing about its own signup: a blog link ("[can an AI agent sign up for a SaaS](/blog/can-an-ai-agent-sign-up-for-a-saas)", also in its sitemap), "owner claim", which on AgentID is an OpenID Connect field in a token, and the question "How can you tell an agent signup from a human one?". Now a match inside link text or a URL slug, a sitemap entry, or a question does not count as an agent signup, and the claim rule needs a person doing it ("the owner claims the workspace"). HTML written inline in a text file is split at its tags like an HTML page, so a quote never shows raw `<td className=...>`.
|
|
13
|
+
- Signup elsewhere: when the product's agent-facing pages send the agent to sign up on another company's site (AgentID's llms.txt: "Create an AgentMail inbox to get an AgentID", linking console.agentmail.to/sign-up), Sign up and Access say so and need a test run, instead of saying a person has to create the account. AgentID's Sign up and Access moved from "agent can" to "needs a test run"; no other recorded product changed state. JSON: `onboarding.signupElsewhere`.
|
|
14
|
+
- JSON: quotes can carry `label` (`steps[].quotes[]`, `needsEvidence`, pattern `evidence`); `observedPrices[]` can carry `paidQuote`. Nothing is removed.
|
|
15
|
+
- Tests: resend.com is recorded in fixtures/onboarding. Over every recorded product, each quote is a whole sentence ending in terminal punctuation or a labelled element, sits inside one block of its page, starts a sentence and holds its trigger. The inbox products above are pinned with their exact sentences, and AgentID, firecrawl.dev, mem0.ai, moltbook.com, neon.com, paywithlocus.com, Track and Resend with none.
|
|
16
|
+
|
|
17
|
+
## 0.8.3 (2026-10-24)
|
|
18
|
+
|
|
19
|
+
False findings fixed. `check` read Track (alg.katrinalaszlo.com), which is free and takes no email, as charging per request, needing an inbox and having a plan to change.
|
|
20
|
+
|
|
21
|
+
- Rule: a step that says the product needs or requires something (an inbox, a payment, a person at checkout, a shell for its CLI) is reported only with the sentence it came from, and that sentence is printed under the step with its page. A need with no quote is not reported. A negated sentence ("takes no email", "cannot sign in to the dashboard", "nothing is deleted"), an example ("such as opening a verification email"), a line of a link index, an optional step and text about the reader's own customers never count.
|
|
22
|
+
- Pay reads machine-readable pricing first. When every plan in /pricing.json costs 0 and no purchase is documented, Pay is `not_applicable`: "Nothing to buy: every plan is free (pricing.json)." Before, x402 named in Track's docs (as a task Score tests) made Pay "The docs mention paying per request", and a demo's "$49.00" on its homepage would have read as a paid plan. "per request" counts as a usage price only beside a price word ("priced per request", "$0.01 per request"); "4 KB per request" is a limit.
|
|
23
|
+
- Manage is `not_applicable` when pricing.json lists exactly one plan and it is free: "One free plan: no plan or limit to change." Otherwise unchanged.
|
|
24
|
+
- Inbox: an emailed code or link counts only in a sentence about the product's own signup or sign-in, and an explicit negative on the product's pages ("takes no email", "needs no inbox", "no account, email or browser") clears it. `test --check` warns about a missing inbox only with that sentence, which it quotes; an `interface.json` from before 0.8.3 carries none, so it no longer warns from one.
|
|
25
|
+
- Other rules audited for the same failure (text that mentions a thing read as the product requiring it), each with a test: expiry ignores "nothing is deleted" and "never expire"; restricted access, a person setting access up, and x402 ignore a negation in the same clause; x402 is ignored when pricing.json lists only free plans; a CLI signup needs the flag or subcommand within four words of the command, so "npx agent-led-growth check <url> runs a real agent through signup" is not one.
|
|
26
|
+
- The step count skips steps that do not apply: "Your public pages document 5 of 5 applicable steps; Pay and Manage don't apply." A not-applicable step is marked `-` and tagged "from your files".
|
|
27
|
+
- After a passing `test`, `check` no longer ends with "Step 3 of 3: test". Next says what is left ("Your latest test passed. Nothing left to fix or test. Pay and Manage don't apply.") and how to keep it passing in CI, and the What next? menu is not offered.
|
|
28
|
+
- JSON, backward compatible (`agent-ready/audit-report@1` keeps its name): `steps[].state` can be `not_applicable`; steps can carry `quotes: [{ for, obs, quote, url }]`; the report adds `applicable`, the number of steps that apply. `documented`, `verified` and `of` (7) are unchanged; the headline text changes only when a step does not apply. In `interface.json`, onboarding patterns add `needsEvidence`.
|
|
29
|
+
- Tests: Track's live pages are recorded in fixtures/onboarding/alg.katrinalaszlo.com.json.gz and its whole check result is asserted, with contrasting products (paid per request over x402, a price per request in prose, an email-gated signup) where each rule must still fire, and a test over every recorded product that each stated requirement carries a quote. Re-checked against the recorded products, the inbox need is no longer reported for firecrawl.dev ("for example OTP flows"), mem0.ai ("no email, no dashboard, no OTP required"), moltbook.com (a verification code for a math challenge), neon.com (a link to its Email OTP feature) and paywithlocus.com ("an optional AgentMail OTP"); it is kept for telnyx.com, inkbox.ai, agentmail.to, aisend.app, www.cosmicjs.com and agentline.cloud.
|
|
30
|
+
|
|
3
31
|
## 0.8.2 (2026-10-24)
|
|
4
32
|
|
|
5
33
|
- `alg connect`: Track's /connect page no longer fills in the code, so the CLI now asks you to type the code it prints on that page, check where the page says the request came from, then approve. The code is printed on its own line. The CLI opens the `verification_url` Track returns as it is and does not expect the code in it.
|
package/README.md
CHANGED
|
@@ -53,6 +53,8 @@ $ npx agent-led-growth check resend.com
|
|
|
53
53
|
|
|
54
54
|
Six seconds. No account, no Docker, nothing sent to your product but GET requests. Needs Node 20.9 or later.
|
|
55
55
|
|
|
56
|
+
When a step says your product needs or requires something (an inbox, a payment, a person at checkout), the sentence it came from is printed under the step with the page it is on. A sentence that says the opposite ("takes no email"), an example ("such as a verification email") or text about your own customers never counts. A step that does not apply is marked `-`: Pay when every plan in /pricing.json is free, Manage when there is one free plan.
|
|
57
|
+
|
|
56
58
|
Then fix and test:
|
|
57
59
|
|
|
58
60
|
```bash
|
package/bin/cli.js
CHANGED
|
@@ -25,7 +25,7 @@ import { runHarness, TASKS } from "../src/harness/index.js";
|
|
|
25
25
|
import { inspect, buildAudit, writeAudit, parseTarget, unreachableReason, DEFAULT_TASK } from "../src/audit/index.js";
|
|
26
26
|
import { defaultAnswers, askQuestions, QUESTIONS } from "../src/audit/questions.js";
|
|
27
27
|
import { readConfig, writeConfig, writeVerifyCall, configHost, CONFIG_FILE } from "../src/audit/config.js";
|
|
28
|
-
import { makeStyle, renderAnswer, renderVerdict, renderSteps, renderSummary, renderNext, renderStandardsStart, renderStandards, renderStrip } from "../src/audit/render.js";
|
|
28
|
+
import { makeStyle, renderAnswer, renderVerdict, renderSteps, renderSummary, renderNext, renderStandardsStart, renderStandards, renderStrip, latestTestOutcome } from "../src/audit/render.js";
|
|
29
29
|
import { localJourney, localTestRuns } from "../src/verify/journey.js";
|
|
30
30
|
import { applyEvidence } from "../src/interface/evidence.js";
|
|
31
31
|
import { runStandards } from "../src/audit/standards.js";
|
|
@@ -184,7 +184,9 @@ Examples:
|
|
|
184
184
|
if (!opts.json) for (const line of renderSummary(audit, paths, style, rel, savedTo)) say(line);
|
|
185
185
|
// Skipped, the benchmarks get one line above Next, so Next stays the last thing on screen.
|
|
186
186
|
if (!opts.json && !opts.standards) for (const line of renderStandards(null, style)) say(line);
|
|
187
|
-
|
|
187
|
+
const latestTest = latestTestOutcome(funnel);
|
|
188
|
+
const testPassed = latestTest === "passed" || latestTest === "handoff";
|
|
189
|
+
if (!opts.json) for (const line of renderNext(audit, paths, style, rel, latestTest)) say(line);
|
|
188
190
|
|
|
189
191
|
let standardsFile = null;
|
|
190
192
|
if (opts.standards) {
|
|
@@ -216,7 +218,7 @@ Examples:
|
|
|
216
218
|
if (!opts.json) for (const line of renderStandards(standards, style, raw ? rel(raw) : null)) say(line);
|
|
217
219
|
// The last line on screen is the next action, not the benchmark table.
|
|
218
220
|
if (!opts.json) {
|
|
219
|
-
const next = audit.findings.length ? " Next: the fix prompt above, then check again." : ` Next: ${style.bold(`${TEST_CMD} ${host} --check`)}`;
|
|
221
|
+
const next = audit.findings.length ? " Next: the fix prompt above, then check again." : testPassed ? " Your latest test passed; Next is above." : ` Next: ${style.bold(`${TEST_CMD} ${host} --check`)}`;
|
|
220
222
|
say(` Done in ${duration(Date.now() - started)}.${next}`);
|
|
221
223
|
say("");
|
|
222
224
|
}
|
|
@@ -227,7 +229,7 @@ Examples:
|
|
|
227
229
|
if (opts.failOn && audit.findings.some((f) => FAIL_ON[opts.failOn].includes(f.severity))) process.exitCode = 1;
|
|
228
230
|
// With nothing to fix, the next step is a test, and a person at a terminal can start it here.
|
|
229
231
|
const canAsk = Boolean(process.stdin.isTTY && process.stderr.isTTY) && !opts.yes && !opts.json;
|
|
230
|
-
if (canAsk && !audit.findings.length) await offerTest({ audit, style, say });
|
|
232
|
+
if (canAsk && !audit.findings.length && !testPassed) await offerTest({ audit, style, say });
|
|
231
233
|
});
|
|
232
234
|
|
|
233
235
|
// check reads public pages; test has a real agent do the steps. With nothing to fix, a person at a terminal can
|
|
@@ -402,7 +404,11 @@ Examples:
|
|
|
402
404
|
// The onboarding pattern chosen in alg.yml, as the last check's interface.json describes it.
|
|
403
405
|
const onboarding = latestInterface(process.cwd(), url)?.doc.onboarding;
|
|
404
406
|
const chosen = onboarding?.patterns?.find((p) => p.id === (config.onboarding || onboarding.primary));
|
|
405
|
-
|
|
407
|
+
// Only with the sentence it came from: a need without a quote is not reported (interface.json from before 0.8.3
|
|
408
|
+
// has none).
|
|
409
|
+
const inboxEvidence = chosen?.needs?.includes("inbox") ? chosen.needsEvidence?.inbox?.[0] : null;
|
|
410
|
+
const inboxQuote = inboxEvidence?.quote ? `${inboxEvidence.label ? `${inboxEvidence.label}: ` : ""}"${inboxEvidence.quote}"` : null;
|
|
411
|
+
if (inboxQuote && !process.env.AGENTMAIL_API_KEY && !opts.inbox) plan.warnings = [`The product's signup sends an emailed code or link (${inboxQuote}) and no inbox is set. Set AGENTMAIL_API_KEY or pass --inbox, or the run may end inconclusive.`];
|
|
406
412
|
if (opts.json) process.stdout.write(JSON.stringify(plan, null, 2) + "\n");
|
|
407
413
|
else for (const line of renderPlan(plan, style)) console.error(line);
|
|
408
414
|
process.exit(plan.ready ? 0 : 2);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-led-growth",
|
|
3
|
-
"version": "0.8.
|
|
3
|
+
"version": "0.8.4",
|
|
4
4
|
"description": "Score: evals for agents as customers. Runs a real agent through signup, key and first call on your product, before you ship. Track, the analytics half, measures which agents convert after.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
package/skills/alg/SKILL.md
CHANGED
|
@@ -41,7 +41,7 @@ The fix prompts build toward the chosen model. Say which answers you used and th
|
|
|
41
41
|
|
|
42
42
|
1. Lead with `headline` ("Your public pages document N of 7 steps. No step has been tested yet: a test runs a real agent."). `documented` and `verified` hold the two counts; a check never verifies, so `verified` is 0.
|
|
43
43
|
2. List each finding: its step, its title and its reason.
|
|
44
|
-
3. Never call a step verified or confirmed: a check only reads pages. Say how each step was found (`basis`): `observed` comes from a structured file or an HTTP status, `heuristic` from matching page text (it can be wrong), and `not_checked` means public pages cannot show it. Only a test with a real agent shows a step works. `state`: `agent_can`/`agent_did` pass, `handoff` passes with one person step, `needs_person`/`blocked` are gaps, `not_checked` needs a test.
|
|
44
|
+
3. Never call a step verified or confirmed: a check only reads pages. Say how each step was found (`basis`): `observed` comes from a structured file or an HTTP status, `heuristic` from matching page text (it can be wrong), and `not_checked` means public pages cannot show it. Only a test with a real agent shows a step works. `state`: `agent_can`/`agent_did` pass, `handoff` passes with one person step, `needs_person`/`blocked` are gaps, `not_checked` needs a test, `not_applicable` has nothing to do (Pay with only free plans in pricing.json, Manage with one free plan). A step that says the product needs or requires something carries `quotes`, the sentences it came from; quote them rather than restating the claim.
|
|
45
45
|
4. After the findings, summarize `standards` in a few lines: each benchmark's passed/total and score, then the failed check names the developer asks about. Say they are third-party benchmarks and separate from the seven steps. If `standards.error` is set, tell the developer aeo-ready did not run and quote it.
|
|
46
46
|
5. Give the paths in `files`: `files.prompts` (one fix prompt per finding), `files.brief` (a one-page brief for security, legal and billing), `files.config` and `files.standards` (aeo-ready's full output).
|
|
47
47
|
6. Then tell the developer the next step is a test, and offer the free plan: `npx agent-led-growth test <url> --check --json`. Do not run `test` itself; see 'If the developer asks for a test'.
|
package/src/audit/index.js
CHANGED
|
@@ -15,6 +15,7 @@ export const DEFAULT_TASK = "Sign up, get an API key, and make one authenticated
|
|
|
15
15
|
// reads Agent did.
|
|
16
16
|
export function basisFor(step, doc) {
|
|
17
17
|
if (step.live) return "tested";
|
|
18
|
+
if (step.state === "not_applicable") return "observed";
|
|
18
19
|
if (step.state === "not_checked") return "not_checked";
|
|
19
20
|
if (step.id === "discover") return "observed";
|
|
20
21
|
if (step.id === "understand") return doc.pricing?.agentCanDetermineCost?.verdict === "yes" ? "observed" : "heuristic";
|
|
@@ -85,7 +86,11 @@ export function patternInfo(id) {
|
|
|
85
86
|
// Everything the audit reports, from the inspection and the answers. Pure, so it is tested without a network.
|
|
86
87
|
export function buildAudit({ doc, funnel, answers, task, runId, version }) {
|
|
87
88
|
const pattern = patternInfo(answers.onboarding.value);
|
|
88
|
-
|
|
89
|
+
// quotes: the sentences behind a claim that the product requires something, with the page each is on. A step a test
|
|
90
|
+
// decided says what the test did instead, so it carries none.
|
|
91
|
+
const urlOf = (obs) => doc.observations.find((o) => o.id === obs)?.url || null;
|
|
92
|
+
const quotesOf = (s) => (!s.live && s.quotes?.length ? { quotes: s.quotes.map((q) => ({ ...q, url: urlOf(q.obs) })) } : {});
|
|
93
|
+
const steps = funnel.steps.map((s) => ({ id: s.id, name: s.name, state: s.state, basis: basisFor(s, doc), reason: s.reason, basedOn: s.basedOn, ...quotesOf(s), ...(s.fix ? { fix: s.fix } : {}) }));
|
|
89
94
|
// The chosen model is the target. A product whose docs only describe a person handing over a key passes Sign up
|
|
90
95
|
// as a handoff, but if the owner chose an agent-first model, that handoff is the gap to close. Agent identity
|
|
91
96
|
// counts: the agent is meant to sign in as itself (for example with AgentID), not take a key from a person.
|
|
@@ -108,6 +113,7 @@ export function buildAudit({ doc, funnel, answers, task, runId, version }) {
|
|
|
108
113
|
headline: funnel.headline,
|
|
109
114
|
documented: funnel.documented,
|
|
110
115
|
verified: funnel.verified,
|
|
116
|
+
applicable: funnel.applicable ?? funnel.steps.filter((s) => s.state !== "not_applicable").length,
|
|
111
117
|
pagesRead: doc.observations.filter((o) => o.ok).length,
|
|
112
118
|
steps,
|
|
113
119
|
onboarding: {
|
|
@@ -151,7 +157,7 @@ export function renderBrief(audit, doc) {
|
|
|
151
157
|
lines.push(`Conflict: a verified person must exist before any use, but ${audit.onboarding.chosen.name} lets an agent act first. Consider "A person sets up access first".`, "");
|
|
152
158
|
}
|
|
153
159
|
lines.push("## Steps", "", "| Step | State | How we know | Evidence |", "| --- | --- | --- | --- |");
|
|
154
|
-
for (const s of audit.steps) lines.push(`| ${s.name} | ${s.state.replace("_", " ")} | ${s.basis.replace("_", " ")} | ${evidence(s.basedOn).join(", ") || "none"} |`);
|
|
160
|
+
for (const s of audit.steps) lines.push(`| ${s.name} | ${s.state.replace("_", " ")} | ${s.basis.replace("_", " ")} | ${evidence(s.basedOn).join(", ") || "none"}${(s.quotes || []).map((q) => `<br>${q.label ? `${q.label}: ` : ""}"${q.quote.replace(/\|/g, "\\|")}"`).join("")} |`);
|
|
155
161
|
lines.push("", "## Fixes", "");
|
|
156
162
|
if (!audit.findings.length) lines.push("None from public pages. A test with a real agent is the next step: `npx agent-led-growth test --check` shows the plan for free, `npx agent-led-growth test` runs it.");
|
|
157
163
|
for (const f of audit.findings) lines.push(`${f.n}. **${f.title}** (${f.severity}, ${f.name}). Prompt: prompts/${f.file}`);
|
package/src/audit/render.js
CHANGED
|
@@ -9,6 +9,15 @@ export function makeStyle(enabled) {
|
|
|
9
9
|
import { PASSING, mark, label } from "../dashboard/strip.js";
|
|
10
10
|
const TAG = { observed: "from your files", heuristic: "from page text", not_checked: "needs a test run", tested: "from your test" };
|
|
11
11
|
|
|
12
|
+
// The page a quote is on, as a path: "/docs.md".
|
|
13
|
+
const pathOf = (url) => {
|
|
14
|
+
try {
|
|
15
|
+
return new URL(url).pathname;
|
|
16
|
+
} catch {
|
|
17
|
+
return url || "";
|
|
18
|
+
}
|
|
19
|
+
};
|
|
20
|
+
|
|
12
21
|
|
|
13
22
|
// Breaks text into lines of at most `width` characters, at spaces. A word longer than a line (a long URL) is split.
|
|
14
23
|
export function wrap(text, width) {
|
|
@@ -26,6 +35,14 @@ export function wrap(text, width) {
|
|
|
26
35
|
}
|
|
27
36
|
|
|
28
37
|
const plural = (n, one, many) => `${n} ${n === 1 ? one : many}`;
|
|
38
|
+
const listNames = (names) => (names.length < 2 ? names.join("") : `${names.slice(0, -1).join(", ")} and ${names.at(-1)}`);
|
|
39
|
+
|
|
40
|
+
// The outcome of the newest test of this product on this machine, from the funnel's evidence, or null.
|
|
41
|
+
export function latestTestOutcome(funnel) {
|
|
42
|
+
const tests = (funnel?.steps || []).flatMap((s) => (s.evidence || []).filter((e) => e.kind === "test"));
|
|
43
|
+
tests.sort((a, b) => (b.at > a.at ? 1 : b.at < a.at ? -1 : 0));
|
|
44
|
+
return tests[0]?.outcome || null;
|
|
45
|
+
}
|
|
29
46
|
|
|
30
47
|
// The answer to "can an agent do it?", before the tally: which steps the pages leave an agent without.
|
|
31
48
|
const DOES = { discover: "finds you", understand: "understands what you sell", signup: "signs up", access: "gets a key", use: "uses the product", pay: "pays", manage: "changes a plan" };
|
|
@@ -59,6 +76,11 @@ export function renderSteps(audit, style) {
|
|
|
59
76
|
const [first = "", ...rest] = wrap(s.reason, REASON);
|
|
60
77
|
out.push(` ${mark(s.state, style)} ${s.name.padEnd(NAME)} ${first.padEnd(REASON)} ${style.dim(TAG[s.basis] || s.basis)}`);
|
|
61
78
|
for (const line of rest) out.push(` ${" ".repeat(NAME + 4)}${line}`);
|
|
79
|
+
// A claim that the product requires something is shown with the sentence it came from, so a reader can check it.
|
|
80
|
+
// The quote is whole (src/interface/quote.js), never cut here; an element that is not a sentence carries its label.
|
|
81
|
+
for (const q of s.quotes || []) {
|
|
82
|
+
for (const line of wrap(`${q.label ? `${q.label}: ` : ""}"${q.quote}" (${pathOf(q.url)})`, 60)) out.push(` ${" ".repeat(NAME + 4)}${style.dim(line)}`);
|
|
83
|
+
}
|
|
62
84
|
}
|
|
63
85
|
return out;
|
|
64
86
|
}
|
|
@@ -85,10 +107,20 @@ export function renderSummary(audit, paths, style, relative, savedTo) {
|
|
|
85
107
|
// What to do next, as the README's three steps: 1 check, 2 fix, 3 test. With fixes, step 2 names the first prompt
|
|
86
108
|
// by its full path, so it can be copied; the test commands follow for later. They name the product, so test runs on
|
|
87
109
|
// it whatever alg.yml the folder holds.
|
|
88
|
-
|
|
110
|
+
// latestTest: the outcome of the newest local test of this product (latestTestOutcome), or null. After a passing test
|
|
111
|
+
// with nothing to fix, Next says what is left instead of suggesting the same test again.
|
|
112
|
+
export function renderNext(audit, paths, style, relative, latestTest = null) {
|
|
89
113
|
const cmd = `npx agent-led-growth test ${audit.target.host}`;
|
|
90
114
|
const test = [` ${style.bold(`${cmd} --check`)}`, ` ${style.bold(cmd)}`, ""];
|
|
91
115
|
const step3 = "Step 3 of 3: test. The plan is free; a signup uses your Claude Code.";
|
|
116
|
+
if (!audit.findings.length && (latestTest === "passed" || latestTest === "handoff")) {
|
|
117
|
+
const skipped = audit.steps.filter((s) => s.state === "not_applicable").map((s) => s.name);
|
|
118
|
+
const open = audit.steps.filter((s) => s.state === "not_checked").map((s) => s.name);
|
|
119
|
+
const left = open.length ? `Not shown by a test yet: ${listNames(open)}.` : "Nothing left to fix or test.";
|
|
120
|
+
const na = skipped.length ? ` ${listNames(skipped)} ${skipped.length === 1 ? "doesn't" : "don't"} apply.` : "";
|
|
121
|
+
const [first, ...rest] = wrap(`Your latest test passed. ${left}${na} Keep it passing in CI:`, 68);
|
|
122
|
+
return [` ${style.bold("Next")} ${first}`, ...rest.map((l) => ` ${l}`), ` ${style.bold(`${cmd} --yes --json`)}`, ""];
|
|
123
|
+
}
|
|
92
124
|
if (!audit.findings.length) return [` ${style.bold("Next")} ${step3}`, ...test];
|
|
93
125
|
return [
|
|
94
126
|
` ${style.bold("Next")} Step 2 of 3: fix. Paste a prompt into your coding agent, deploy,`,
|
package/src/dashboard/strip.js
CHANGED
|
@@ -6,9 +6,9 @@ export const PASSING = new Set(["agent_did", "agent_can", "handoff"]);
|
|
|
6
6
|
export const FAILING = new Set(["needs_person", "blocked"]);
|
|
7
7
|
|
|
8
8
|
// Who the colour stands for. A handoff is a person stepping in at a documented point, so it reads as a person.
|
|
9
|
-
export const WHO = { agent_did: "agent", agent_can: "agent", handoff: "person", needs_person: "person", blocked: "blocked", not_checked: "none" };
|
|
10
|
-
export const LABEL = { agent_did: "Agent did", agent_can: "Agent can", handoff: "Handoff", needs_person: "Needs a person", blocked: "Blocked", not_checked: "Needs a test" };
|
|
11
|
-
export const KIND = { agent_did: "success", agent_can: "success", handoff: "success", needs_person: "warning", blocked: "error", not_checked: "neutral" };
|
|
9
|
+
export const WHO = { agent_did: "agent", agent_can: "agent", handoff: "person", needs_person: "person", blocked: "blocked", not_checked: "none", not_applicable: "none" };
|
|
10
|
+
export const LABEL = { agent_did: "Agent did", agent_can: "Agent can", handoff: "Handoff", needs_person: "Needs a person", blocked: "Blocked", not_checked: "Needs a test", not_applicable: "Doesn't apply" };
|
|
11
|
+
export const KIND = { agent_did: "success", agent_can: "success", handoff: "success", needs_person: "warning", blocked: "error", not_checked: "neutral", not_applicable: "neutral" };
|
|
12
12
|
export const EVIDENCE_KIND = { check: "Check of public pages", test: "Test with a real agent", pay: "Payment run", signal: "Your server", stuck: "Stuck report" };
|
|
13
13
|
|
|
14
14
|
export const who = (state) => WHO[state] || "none";
|
|
@@ -20,6 +20,7 @@ export function mark(state, style = null) {
|
|
|
20
20
|
const paint = (name, s) => (style ? style[name](s) : s);
|
|
21
21
|
if (PASSING.has(state)) return paint("green", "✓");
|
|
22
22
|
if (FAILING.has(state)) return paint("red", "✗");
|
|
23
|
+
if (state === "not_applicable") return paint("dim", "-");
|
|
23
24
|
return paint("dim", "·");
|
|
24
25
|
}
|
|
25
26
|
export const markOk = (ok, style = null) => (style ? (ok ? style.green("✓") : style.red("✗")) : ok ? "✓" : "✗");
|
|
@@ -227,6 +227,13 @@ export function evaluateAuthentication(x, observations) {
|
|
|
227
227
|
return { methods, mentions, requirement, agentCanUnderstandSetup: setup, friction };
|
|
228
228
|
}
|
|
229
229
|
|
|
230
|
+
// pricing.json lists plans and every one costs 0: nothing to buy. Only machine-readable plans count; a price in prose
|
|
231
|
+
// can be a demo's sample data (Track's homepage shows a customer paying $49.00).
|
|
232
|
+
export function freePlansOnly(pricing) {
|
|
233
|
+
const plans = pricing?.plans || [];
|
|
234
|
+
return plans.length > 0 && plans.every((p) => p.amount === 0);
|
|
235
|
+
}
|
|
236
|
+
|
|
230
237
|
export function evaluatePricing(x, observations) {
|
|
231
238
|
const pj = x.pricingJson;
|
|
232
239
|
const pagesFound = x.pricingPages;
|
|
@@ -247,7 +254,7 @@ export function evaluatePricing(x, observations) {
|
|
|
247
254
|
return {
|
|
248
255
|
model: pj?.value.revenueModel ? { value: pj.value.revenueModel, method: "extracted", evidence: pj.evidence } : null,
|
|
249
256
|
plans: plans.map((p) => ({ ...p, evidence: pj.evidence })),
|
|
250
|
-
observedPrices: pagesFound.map((p) => ({ prices: p.value.prices, evidence: p.evidence })),
|
|
257
|
+
observedPrices: pagesFound.map((p) => ({ prices: p.value.prices, evidence: p.evidence, ...(p.value.paidQuote ? { paidQuote: p.value.paidQuote } : {}) })),
|
|
251
258
|
agentCanDetermineCost: cost,
|
|
252
259
|
ambiguities,
|
|
253
260
|
};
|
package/src/interface/extract.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import { parseRobots, AI_BOTS } from "../probe/index.js";
|
|
2
2
|
import { htmlToText } from "./sources.js";
|
|
3
|
+
import { quoteAt } from "./quote.js";
|
|
3
4
|
|
|
4
5
|
// Deterministic extraction. Every fact carries the observation it came from and a quote from that page.
|
|
5
6
|
// A fact that cannot be extracted is null; callers turn nulls into unknowns. No inference happens here.
|
|
@@ -172,7 +173,21 @@ export function extractPricingJson(observations, bodies) {
|
|
|
172
173
|
|
|
173
174
|
const PRICE_RE = /(?:\$|€|£|USD\s?)\s?\d[\d,]*(?:\.\d+)?(?:\s?(?:\/|per)\s?(?:mo(?:nth)?|yr|year|seat|user|request|call|1k|1000|k)\b)?/gi;
|
|
174
175
|
const AMBIGUITY_RE = /contact (?:us|sales)|custom pricing|talk to sales|get a quote|enterprise pricing|request a demo/gi;
|
|
175
|
-
|
|
176
|
+
// "per request" is a price only beside a price word: "priced per request", "billed per call". Alone it is usually a
|
|
177
|
+
// limit: Track's "4 KB per request", "What it sends, per request".
|
|
178
|
+
const USAGE_RE = /usage[- ]based|pay[- ]as[- ]you[- ]go|\b(?:priced|charged|billed|paid|pay|costs?|fee|price)\s+(?:\w+\s+)?per (?:request|call|token|seat|user)\b|metered billing/gi;
|
|
179
|
+
|
|
180
|
+
function paidPriceQuote(page, text) {
|
|
181
|
+
let tried = 0;
|
|
182
|
+
for (const m of text.matchAll(PRICE_RE)) {
|
|
183
|
+
// "$75M Series B" is a funding round, not a plan.
|
|
184
|
+
if (!/[1-9]/.test(m[0]) || /^[MB]\b/i.test(text.slice(m.index + m[0].length))) continue;
|
|
185
|
+
const q = quoteAt(page, text, m.index, m.index + m[0].length, page.obs.role);
|
|
186
|
+
if (q) return q;
|
|
187
|
+
if (++tried >= 200) return null;
|
|
188
|
+
}
|
|
189
|
+
return null;
|
|
190
|
+
}
|
|
176
191
|
|
|
177
192
|
export function extractPricingPages(observations, bodies) {
|
|
178
193
|
const out = [];
|
|
@@ -182,7 +197,10 @@ export function extractPricingPages(observations, bodies) {
|
|
|
182
197
|
const ambiguities = [...new Set((text.match(AMBIGUITY_RE) || []).map((s) => s.toLowerCase()))];
|
|
183
198
|
const usage = [...new Set((text.match(USAGE_RE) || []).map((s) => s.toLowerCase()))];
|
|
184
199
|
if (page.obs.role === "homepage" && !prices.length) continue;
|
|
185
|
-
|
|
200
|
+
// A paid price is what Pay quotes when it says paid plans are published, so it gets a clean quote: the sentence or
|
|
201
|
+
// labelled element holding the first price above zero that has one (src/interface/quote.js), or null.
|
|
202
|
+
const paid = paidPriceQuote(page, text);
|
|
203
|
+
out.push(fact({ prices, ambiguities, usageTerms: usage, paidQuote: paid ? { obs: page.obs.id, ...paid } : null }, page.obs.id, quoteAround(text, /(\$|€|£)\s?\d/) || quoteAround(text, AMBIGUITY_RE) || text.slice(0, 140)));
|
|
186
204
|
}
|
|
187
205
|
return out;
|
|
188
206
|
}
|
package/src/interface/funnel.js
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
import { STAGES, PILLAR } from "../schema/stages.js";
|
|
2
2
|
import { AGENTID_NEEDS } from "./onboarding.js";
|
|
3
3
|
import { PASSING } from "../dashboard/strip.js";
|
|
4
|
+
import { freePlansOnly } from "./evaluate.js";
|
|
4
5
|
|
|
5
6
|
// The seven-step answer for one public check: how far an agent can get, where it stops, and what to build.
|
|
6
7
|
// Rule-based over the interface document only. Public evidence can show a path is documented; only a live
|
|
@@ -8,7 +9,9 @@ import { PASSING } from "../dashboard/strip.js";
|
|
|
8
9
|
export const FUNNEL_SCHEMA = "agent-ready/funnel@1";
|
|
9
10
|
// handoff: a person steps in at a documented point and the agent carries on. That is a working path, not a failure.
|
|
10
11
|
// agent_did: a live test proved the step. Only a live test sets it.
|
|
11
|
-
|
|
12
|
+
// not_applicable: the product's machine-readable files say the step has nothing to do (every plan in pricing.json is
|
|
13
|
+
// free, so there is nothing to pay for). It neither passes nor fails and is left out of the counts.
|
|
14
|
+
export const STEP_STATES = ["agent_did", "agent_can", "handoff", "needs_person", "blocked", "not_checked", "not_applicable"];
|
|
12
15
|
// Kept under this name for callers; the order itself lives in schema/stages.js.
|
|
13
16
|
export const FUNNEL_ORDER = STAGES;
|
|
14
17
|
|
|
@@ -52,15 +55,23 @@ const NEEDS_SENTENCE = {
|
|
|
52
55
|
wallet: "Needs a wallet the agent can pay from.",
|
|
53
56
|
agent_identity: "Needs an identity provider the agent can sign in with.",
|
|
54
57
|
};
|
|
55
|
-
|
|
56
|
-
|
|
58
|
+
// A step that says the product needs something carries the sentence it came from. A need with no quote is not said:
|
|
59
|
+
// the rule is that a requirement is reported only with the exact text that states it.
|
|
60
|
+
function needsOf(pattern) {
|
|
61
|
+
const quoted = (need) => (pattern.needsEvidence?.[need] || []).filter((e) => e.quote);
|
|
62
|
+
const kept = pattern.needs.filter((need) => quoted(need).length);
|
|
63
|
+
const sentence = kept.map((need) => {
|
|
57
64
|
if (!NEEDS_SENTENCE[need]) throw new Error(`no sentence for need "${need}"`);
|
|
58
65
|
return NEEDS_SENTENCE[need];
|
|
59
66
|
}).join(" ");
|
|
67
|
+
const quotes = kept.flatMap((need) => quoted(need).slice(0, 1).map((e) => ({ for: need, obs: e.obs, quote: e.quote, ...(e.label ? { label: e.label } : {}) })));
|
|
68
|
+
return { kept, sentence, quotes };
|
|
60
69
|
}
|
|
61
70
|
|
|
62
|
-
|
|
71
|
+
// quotes: [{ for, obs, quote }], the sentences behind a claim that the product requires something, shown with the step.
|
|
72
|
+
function step(id, state, reason, basedOn = [], tip = null, quotes = []) {
|
|
63
73
|
const out = { id, name: STEP_NAME[id], area: PILLAR[id], question: STEP_QUESTION[id], state, reason, basedOn: [...new Set(basedOn.filter(Boolean))] };
|
|
74
|
+
if (quotes.length) out.quotes = quotes;
|
|
64
75
|
// Only a step known to fail gets a fix. An unchecked step needs a live test first.
|
|
65
76
|
if (state === "needs_person" || state === "blocked") out.fix = FIX[id];
|
|
66
77
|
// A passing step can still be easier for agents; that is a tip, not a fix.
|
|
@@ -103,8 +114,9 @@ function signup(doc) {
|
|
|
103
114
|
const existing = patterns.find((p) => p.id === "existing_account");
|
|
104
115
|
const refs = (p) => (p?.evidence || []).map((e) => e.obs);
|
|
105
116
|
// The pattern's reason already names AgentID, what the agent needs, and whether the owner is shared.
|
|
106
|
-
|
|
107
|
-
if (self
|
|
117
|
+
const needs = self ? needsOf(self) : null;
|
|
118
|
+
if (self?.provider === "AgentID") return step("signup", "agent_can", self.reason, refs(self), null, needs.quotes);
|
|
119
|
+
if (self && needs.kept.length) return step("signup", "agent_can", `The docs describe an agent signing up on its own (${self.name}). ${needs.sentence}`, [...refs(self), ...needs.quotes.map((q) => q.obs)], null, needs.quotes);
|
|
108
120
|
// "Agent is the customer" has no person step, and its boundary text says so ("None documented. ..."), which does
|
|
109
121
|
// not read after "A person steps in".
|
|
110
122
|
const boundary = !self?.humanBoundary ? "" : /^None\b/.test(self.humanBoundary) ? " No person step is documented: the agent holds the account itself." : ` A person steps in ${self.humanBoundary.charAt(0).toLowerCase()}${self.humanBoundary.slice(1)}`;
|
|
@@ -112,6 +124,9 @@ function signup(doc) {
|
|
|
112
124
|
if (existing?.cli === "token") return step("signup", "handoff", "A person creates the account and a token. The agent's CLI takes the token from a flag or an environment variable, with no browser.", refs(existing));
|
|
113
125
|
if (existing?.cli === "browser") return step("signup", "handoff", "A person creates the account and signs the CLI in through a browser; no token option was found in the pages read. The agent carries on from there.", refs(existing));
|
|
114
126
|
if (existing) return step("signup", "handoff", "A person creates the account and gives the agent a key. The agent carries on from there.", refs(existing));
|
|
127
|
+
// The product sends the agent to sign up on another company's site, which is not read: unknown, not a gap.
|
|
128
|
+
const elsewhere = doc.onboarding?.signupElsewhere;
|
|
129
|
+
if (elsewhere) return step("signup", "not_checked", `The docs send the agent to sign up on ${elsewhere.domain}, another company's site, which was not read. Only a test run shows whether an agent can sign up there.`, elsewhere.evidence.map((e) => e.obs));
|
|
115
130
|
return step("signup", "needs_person", "The pages read document no way for an agent to sign up. A person may have to create the account; only a real agent run can confirm.");
|
|
116
131
|
}
|
|
117
132
|
|
|
@@ -126,6 +141,8 @@ function access(doc, signupStep) {
|
|
|
126
141
|
const self = (doc.onboarding?.patterns || []).find((p) => SIGNUP_PATTERNS.has(p.id));
|
|
127
142
|
if (signupStep.state === "agent_can" && self?.provider === "AgentID") return step("access", "agent_can", "Signed in with AgentID, the agent holds its own account and gets its key or session from it, not from a person.", [...refs, ...signupStep.basedOn]);
|
|
128
143
|
if (signupStep.state === "agent_can") return step("access", "agent_can", "The agent signup path hands back a key the agent can use.", [...refs, ...signupStep.basedOn]);
|
|
144
|
+
const elsewhere = doc.onboarding?.signupElsewhere;
|
|
145
|
+
if (elsewhere && signupStep.state === "not_checked") return step("access", "not_checked", `The agent gets its credential where it signs up, on ${elsewhere.domain}. Only a test run shows whether it can.`, [...refs, ...signupStep.basedOn]);
|
|
129
146
|
if (methods.length || terms.length) return step("access", "needs_person", `The docs describe ${describeAuth(methods, terms)}, but the pages read do not show how an agent gets one.`, refs);
|
|
130
147
|
return step("access", "not_checked", "The docs read do not say how an agent authenticates.", refs);
|
|
131
148
|
}
|
|
@@ -142,35 +159,55 @@ function pay(doc) {
|
|
|
142
159
|
const plans = doc.pricing?.plans || [];
|
|
143
160
|
const observed = doc.pricing?.observedPrices || [];
|
|
144
161
|
const paidPlans = plans.filter((p) => typeof p.amount === "number" && p.amount > 0);
|
|
145
|
-
|
|
162
|
+
// The pricing page first: a homepage price can be anything (a demo's "$49.00", a funding round).
|
|
163
|
+
const roleOf = (o) => doc.observations?.find((x) => x.id === o.evidence?.[0]?.obs)?.role;
|
|
164
|
+
const paidText = observed.filter((o) => (o.prices || []).some((price) => /[1-9]/.test(price))).sort((a, b) => Number(roleOf(b) === "pricing") - Number(roleOf(a) === "pricing"));
|
|
146
165
|
const refs = [...plans.flatMap((p) => (p.evidence || []).map((e) => e.obs)), ...paidText.flatMap((o) => (o.evidence || []).map((e) => e.obs))];
|
|
147
166
|
// A purchase the product's own pages document (src/interface/purchase.js). One approval by a person, then the agent
|
|
148
167
|
// buys on its own, is a handoff: a person steps in at a documented point and the agent carries on. It is not
|
|
149
168
|
// agent_can, which would say no person is needed, and not needs_person, which would say a person checks out each time.
|
|
169
|
+
const quote = (entries, what) => (entries || []).filter((e) => e.quote).slice(0, 1).map((e) => ({ for: what, obs: e.obs, quote: e.quote, ...(e.label ? { label: e.label } : {}) }));
|
|
150
170
|
const purchase = doc.pricing?.agentPurchase;
|
|
151
171
|
const purchaseRefs = (purchase?.evidence || []).map((e) => e.obs);
|
|
172
|
+
const purchaseQuotes = quote(purchase?.evidence, "payment");
|
|
152
173
|
const perRequestCall = purchase?.rule === "pay_per_request_own_api";
|
|
153
174
|
const checkoutCall = purchase?.rule === "checkout_protocol_own_api";
|
|
154
175
|
const call = purchase?.call ? ` (${purchase.call})` : "";
|
|
155
176
|
const over = purchase?.protocol ? ` over ${purchase.protocol}` : "";
|
|
156
177
|
if (purchase?.approval === "once") {
|
|
157
178
|
const how = perRequestCall ? `pays over ${purchase.protocol || "x402 or MPP"}${call}` : checkoutCall ? `checks out over ${purchase.protocol}${call}` : purchase.call ? `buys through ${purchase.call}${over}` : `buys through the API${over}`;
|
|
158
|
-
return step("pay", "handoff", `A person sets payment up once (an approval link, a spending cap, a saved card or a delegation), then the agent ${how}, with no browser checkout. The docs say so; a test shows whether it works.`, purchaseRefs);
|
|
179
|
+
return step("pay", "handoff", `A person sets payment up once (an approval link, a spending cap, a saved card or a delegation), then the agent ${how}, with no browser checkout. The docs say so; a test shows whether it works.`, purchaseRefs, null, purchaseQuotes);
|
|
159
180
|
}
|
|
160
|
-
if (purchase?.rule === "agent_buys_through_api") return step("pay", "agent_can", `The docs describe an agent buying through the API${call}${over}, and no person approving first. A test shows whether it works.`, purchaseRefs);
|
|
161
|
-
if (perRequestCall) return step("pay", "agent_can", `The docs show a paid call over ${purchase.protocol || "x402 or MPP"} on the product's own API${call}, with no browser checkout. The agent needs a funded wallet; a test shows whether the payment goes through.`, purchaseRefs);
|
|
162
|
-
if (checkoutCall && purchase.call) return step("pay", "agent_can", `The docs show a checkout an agent completes over ${purchase.protocol} on the product's own API${call}, with no browser checkout. A test shows whether a purchase goes through.`, purchaseRefs);
|
|
163
|
-
if (checkoutCall) return step("pay", "agent_can", `The product publishes a UCP business profile at /.well-known/ucp, so an agent can check out over UCP, with no browser checkout. A test shows whether a purchase goes through.`, purchaseRefs);
|
|
181
|
+
if (purchase?.rule === "agent_buys_through_api") return step("pay", "agent_can", `The docs describe an agent buying through the API${call}${over}, and no person approving first. A test shows whether it works.`, purchaseRefs, null, purchaseQuotes);
|
|
182
|
+
if (perRequestCall) return step("pay", "agent_can", `The docs show a paid call over ${purchase.protocol || "x402 or MPP"} on the product's own API${call}, with no browser checkout. The agent needs a funded wallet; a test shows whether the payment goes through.`, purchaseRefs, null, purchaseQuotes);
|
|
183
|
+
if (checkoutCall && purchase.call) return step("pay", "agent_can", `The docs show a checkout an agent completes over ${purchase.protocol} on the product's own API${call}, with no browser checkout. A test shows whether a purchase goes through.`, purchaseRefs, null, purchaseQuotes);
|
|
184
|
+
if (checkoutCall) return step("pay", "agent_can", `The product publishes a UCP business profile at /.well-known/ucp, so an agent can check out over UCP, with no browser checkout. A test shows whether a purchase goes through.`, purchaseRefs, null, purchaseQuotes);
|
|
185
|
+
// Machine-readable pricing comes before prose: with every plan in pricing.json at 0 and no purchase documented,
|
|
186
|
+
// there is nothing to pay for, whatever the page text says (a demo's "$49.00", x402 named as something tested).
|
|
187
|
+
if (freePlansOnly(doc.pricing)) return step("pay", "not_applicable", "Nothing to buy: every plan is free (pricing.json).", plans.flatMap((p) => (p.evidence || []).map((e) => e.obs)));
|
|
164
188
|
// Pay-per-request wording is easy to misread: a billing product describing its own
|
|
165
189
|
// customers' metering reads the same as an agent paying. Only a live test settles it.
|
|
166
|
-
if (perRequest) return step("pay", "not_checked", "The docs mention paying per request. A test with a real agent shows whether it can actually pay.", (perRequest.evidence || []).map((e) => e.obs));
|
|
190
|
+
if (perRequest) return step("pay", "not_checked", "The docs mention paying per request over x402. A test with a real agent shows whether it can actually pay.", (perRequest.evidence || []).map((e) => e.obs), null, quote(perRequest.evidence, "payment"));
|
|
167
191
|
// Only the product's own domain is read: a purchase path on a payments provider's site (Stripe Projects lists
|
|
168
192
|
// Cloudflare, Neon, Turso and others on stripe.com) is not seen here, so the sentence says what was read.
|
|
169
|
-
|
|
193
|
+
// A paid price in prose counts only with a clean quote (the sentence or labelled element it is in): Cloudflare's
|
|
194
|
+
// pricing page carries its prices only inside a script's JSON, which is no sentence to show.
|
|
195
|
+
const quotedText = paidText.filter((o) => o.paidQuote);
|
|
196
|
+
if (paidPlans.length || quotedText.length) return step("pay", "needs_person", "Paid plans are published. The pages read document no way for an agent to buy one, so a person may have to check out.", refs, null, paidPlans.length ? planQuote(paidPlans[0]) : quote([quotedText[0].paidQuote], "payment"));
|
|
197
|
+
if (paidText.length) return step("pay", "not_checked", "Prices appear in the pages read, but not in a sentence or table that can be quoted. A test shows whether an agent can pay.", refs);
|
|
170
198
|
if (plans.length || observed.length) return step("pay", "not_checked", "Only free plans were found, so there is nothing to pay for yet.", refs);
|
|
171
199
|
return step("pay", "not_checked", "No plans were found in the pages read.");
|
|
172
200
|
}
|
|
173
201
|
|
|
202
|
+
// A paid plan from pricing.json, quoted as the plan it is rather than a cut of the file's JSON:
|
|
203
|
+
// pricing.json: "Pro: 20 USD per month".
|
|
204
|
+
function planQuote(plan) {
|
|
205
|
+
const obs = plan.evidence?.[0]?.obs;
|
|
206
|
+
if (!obs) return [];
|
|
207
|
+
const price = [plan.amount, plan.currency, plan.period ? `per ${plan.period}` : ""].filter((x) => x !== null && x !== "").join(" ");
|
|
208
|
+
return [{ for: "payment", obs, quote: `${plan.name || plan.id || "Plan"}: ${price}`, label: "pricing.json" }];
|
|
209
|
+
}
|
|
210
|
+
|
|
174
211
|
function use(doc) {
|
|
175
212
|
const caps = doc.capabilities || [];
|
|
176
213
|
const api = doc.interfaces?.api;
|
|
@@ -186,7 +223,9 @@ function use(doc) {
|
|
|
186
223
|
return step("use", "not_checked", "No API actions were found in the pages read.", refs);
|
|
187
224
|
}
|
|
188
225
|
|
|
189
|
-
function manage() {
|
|
226
|
+
function manage(doc) {
|
|
227
|
+
const plans = doc.pricing?.plans || [];
|
|
228
|
+
if (plans.length === 1 && freePlansOnly(doc.pricing) && !doc.pricing.agentPurchase) return step("manage", "not_applicable", "One free plan: no plan or limit to change.", (plans[0].evidence || []).map((e) => e.obs));
|
|
190
229
|
return step("manage", "not_checked", "Changing a plan or a limit can only be proven by a test run.");
|
|
191
230
|
}
|
|
192
231
|
|
|
@@ -199,14 +238,15 @@ function withAgentId(doc, s, sentence) {
|
|
|
199
238
|
const self = patterns.find((p) => SIGNUP_PATTERNS.has(p.id));
|
|
200
239
|
const agentId = patterns.find((p) => p.provider === "AgentID");
|
|
201
240
|
if (!agentId || agentId === self) return s;
|
|
202
|
-
|
|
241
|
+
const quotes = [...(s.quotes || []), ...agentId.evidence.filter((e) => e.quote).slice(0, 1).map((e) => ({ for: "agent_identity", obs: e.obs, quote: e.quote, ...(e.label ? { label: e.label } : {}) }))];
|
|
242
|
+
return { ...s, reason: `${s.reason} ${sentence}`, basedOn: [...new Set([...s.basedOn, ...agentId.evidence.map((e) => e.obs)])], quotes };
|
|
203
243
|
}
|
|
204
244
|
|
|
205
245
|
export function buildFunnel(doc) {
|
|
206
246
|
const d = discover(doc);
|
|
207
247
|
const s = withAgentId(doc, signup(doc), `The docs also describe the agent signing in with AgentID as itself, which needs ${AGENTID_NEEDS}.`);
|
|
208
248
|
const a = withAgentId(doc, access(doc, s), "Signed in with AgentID, the agent would hold its own account and get its key or session from it.");
|
|
209
|
-
const byId = { discover: d, understand: understand(doc), signup: s, access: a, pay: pay(doc), use: use(doc), manage: manage() };
|
|
249
|
+
const byId = { discover: d, understand: understand(doc), signup: s, access: a, pay: pay(doc), use: use(doc), manage: manage(doc) };
|
|
210
250
|
const steps = FUNNEL_ORDER.map((id) => byId[id]);
|
|
211
251
|
const chosen = choosePath(doc);
|
|
212
252
|
const paths = (doc.onboarding?.patterns || []).map((p) => ({ id: p.id, name: p.name, status: p.status, humanBoundary: p.humanBoundary || null, needs: p.needs, followed: p === chosen }));
|
|
@@ -217,20 +257,25 @@ export function buildFunnel(doc) {
|
|
|
217
257
|
// that does not pass, and whether that step is a real stop or only unproven.
|
|
218
258
|
// The headline keeps two counts apart: steps the public pages document (agent_can, handoff) and steps a test with a
|
|
219
259
|
// real agent verified (agent_did). Reading pages is not running a task, so a check alone never says a step works.
|
|
260
|
+
// A step that does not apply is skipped: it neither passes nor stops the walk, and applicable counts the rest.
|
|
220
261
|
export function summarize(steps) {
|
|
221
|
-
const
|
|
222
|
-
const
|
|
262
|
+
const applicable = steps.filter((x) => x.state !== "not_applicable");
|
|
263
|
+
const firstOpen = applicable.findIndex((x) => !PASSING.has(x.state));
|
|
264
|
+
const passed = firstOpen === -1 ? applicable.length : firstOpen;
|
|
223
265
|
const stop = steps.find((x) => x.state === "blocked" || x.state === "needs_person") || null;
|
|
224
266
|
const verified = steps.filter((x) => x.state === "agent_did").length;
|
|
225
267
|
const documented = steps.filter((x) => x.state === "agent_can" || x.state === "handoff").length;
|
|
226
268
|
// stepsPassing (from #78) is every passing step, the total the headline gives: documented plus verified.
|
|
227
|
-
return { headline: headlineFor(steps, documented, verified), passed, stepsPassing: documented + verified, documented, verified, of: steps.length, stopsAt: stop?.id || null, fixes: steps.filter((x) => x.fix).map((x) => ({ step: x.id, name: x.name, fix: x.fix })) };
|
|
269
|
+
return { headline: headlineFor(steps, documented, verified), passed, stepsPassing: documented + verified, documented, verified, of: steps.length, applicable: applicable.length, stopsAt: stop?.id || null, fixes: steps.filter((x) => x.fix).map((x) => ({ step: x.id, name: x.name, fix: x.fix })) };
|
|
228
270
|
}
|
|
229
271
|
|
|
230
272
|
function headlineFor(steps, documented, verified) {
|
|
231
|
-
|
|
273
|
+
const skipped = steps.filter((x) => x.state === "not_applicable").map((x) => x.name);
|
|
274
|
+
const of = skipped.length ? `${steps.length - skipped.length} applicable steps` : `${steps.length} steps`;
|
|
275
|
+
const dontApply = skipped.length ? `; ${listNames(skipped)} ${skipped.length === 1 ? "doesn't" : "don't"} apply` : "";
|
|
276
|
+
if (!steps.some((x) => x.live)) return `Your public pages document ${documented} of ${of}${dontApply}. No step has been tested yet: a test runs a real agent.`;
|
|
232
277
|
const failed = steps.filter((x) => x.live && x.state === "blocked").map((x) => x.name);
|
|
233
|
-
const parts = [`${verified} of ${
|
|
278
|
+
const parts = [`${verified} of ${of} verified by a test${dontApply}.`];
|
|
234
279
|
if (documented) parts.push(`Your public pages document ${documented} more.`);
|
|
235
280
|
if (failed.length) parts.push(`${listNames(failed)} failed a test.`);
|
|
236
281
|
return parts.join(" ");
|
|
@@ -1,4 +1,6 @@
|
|
|
1
1
|
import { htmlToText, registrableDomain, sameSite } from "./sources.js";
|
|
2
|
+
import { freePlansOnly } from "./evaluate.js";
|
|
3
|
+
import { blockContext, quoteAt } from "./quote.js";
|
|
2
4
|
|
|
3
5
|
// Which ways can an agent start using this product? Read from public docs only, so every pattern is a claim the docs
|
|
4
6
|
// make, with the quotes that make it; none is a result. A product can support several. Rules, not a model: each signal
|
|
@@ -13,8 +15,9 @@ const SIGNALS = {
|
|
|
13
15
|
// Agent-infra products use the same words for signing up somewhere else: AgentLine's "so your agent can sign up and
|
|
14
16
|
// verify", AgentID's "apps where agents can sign up and sign in with AgentID". "No agent can sign up" is the opposite.
|
|
15
17
|
bootstrap: /\bagent[- ]?sign[- ]?up\b|(?<!\b(?:no|not|never|where)\s+(?:\w+\s+){0,2})\bagents? (?:can )?(?:sign(?:s)? (?:itself |themselves )?up|creates? (?:its|their) own (?:accounts?|API keys?))(?:\*\*)?(?: here| for an account| in one (?:API )?(?:call|request)| and get (?:an? |its |their )?(?:own )?(?:API )?key| \(beta\)| instantly| immediately| automatically| programmatically)*(?=(?:\*\*)?\s*(?:[.;!)\n]|,\s*(?:and )?get\b|[—–]|$))|\/agents?\/sign-?up\b|\bsign(?:s|ing)? (?:itself|themselves) up\b|\bregisters? itself\b|\bno (?:human|account|signup|sign-up)(?: or API key)? (?:is )?(?:needed|required)\b|\bwithout (?:a |an )?(?:human|account)\b|\banonymous(?:ly)? (?:create|creates|provision|access|account|project|identit)|\btemporary (?:account|project|deployment)s?\b|--temporary\b|\binit --agent\b|\bagent_mode\b|\bbot[_ ]signup\b|\bidentity_types_supported\b/i,
|
|
16
|
-
// A person can take ownership afterwards.
|
|
17
|
-
claim
|
|
18
|
+
// A person can take ownership afterwards. "The owner claims the workspace", with an object: an OpenID Connect "owner
|
|
19
|
+
// claim" is a field in a token (AgentID's llms-full.txt "the requested owner claim quietly missing"), not a person.
|
|
20
|
+
claim: /\bclaim(?:s|ed|ing)? (?:it|this|the|your|their) (?:account|project|workspace|deployment|bucket|inbox|resources?)\b|\bclaim (?:url|code|link|token|flow|page|email)\b|\bclaim_url\b|\bclaimUrl\b|\/claim\b|\bunclaimed\b|\bclaimable\b|\b(?:human|owner|person) (?:can |may |will |must |then )?(?:later )?claims? (?:it|this|the|your|their|its)\b|\battach (?:a |the )?(?:human )?owner\b|\badopt (?:the |this )?workspace\b/i,
|
|
18
21
|
// What the agent made goes away unless someone keeps it.
|
|
19
22
|
// Tied to the account or resource: "quotes expire after 5 minutes" is not this. The sentence must name an account,
|
|
20
23
|
// project, workspace, key or token and not a stored thing (STORED_THING below): Supermemory's "temporary facts drop
|
|
@@ -41,7 +44,9 @@ const SIGNALS = {
|
|
|
41
44
|
// use it to mean "claim first" (Cosmic) or "over your plan" (Inkbox).
|
|
42
45
|
payPerRequest: /\bx402\b|\bX-PAYMENT\b|\bPAYMENT-REQUIRED\b/,
|
|
43
46
|
// The documented way in is a command-line tool rather than an HTTP call.
|
|
44
|
-
|
|
47
|
+
// At most four words between the tool and the flag or subcommand, so it is one command line: Track's docs say
|
|
48
|
+
// "npx agent-led-growth check <url> runs a real agent through signup", which is prose after a command, not a signup.
|
|
49
|
+
cliBootstrap: /\b(?:npx|wrangler|mem0|stripe|npm i(?:nstall)?|pip install|brew install)\b(?:[ \t]+[^\s`]+){0,4}?[ \t]+(?:--temporary|--agent|sign-?up\b|projects init)/i,
|
|
45
50
|
// The product's CLI takes a token without a browser, so an agent can run it unattended once a person has made the
|
|
46
51
|
// token: Vercel's "Set the VERCEL_TOKEN environment variable" or "Pass the --token option", GitHub's "--with-token".
|
|
47
52
|
// An environment variable counts only where the text calls it one: code samples are full of other _API_KEY names.
|
|
@@ -192,9 +197,19 @@ function findAgentIdSignIn(doc, bodies) {
|
|
|
192
197
|
...(agentFacing ? [...text.matchAll(AGENTID_DECLARED)] : []),
|
|
193
198
|
...[...text.matchAll(AGENTID_ROUTE)],
|
|
194
199
|
].sort((x, y) => x.index - y.index);
|
|
195
|
-
const
|
|
200
|
+
const kept = matches.filter((m) => !AGENTID_ELSEWHERE.test(text.slice(Math.max(0, m.index - 60), m.index + m[0].length + 60)) && !announcedByOther(text.slice(Math.max(0, m.index - 60), m.index + m[0].length), doc) && !AGENTID_VENDOR.test(sentenceAround(text, m.index, m.index + m[0].length)));
|
|
201
|
+
// The first kept match with a clean quote: one sentence or labelled element holding the words (src/interface/quote.js).
|
|
202
|
+
let match = null;
|
|
203
|
+
let q = null;
|
|
204
|
+
for (const m of kept) {
|
|
205
|
+
q = quoteAt(body, text, m.index, m.index + m[0].length, obs.role);
|
|
206
|
+
if (q) {
|
|
207
|
+
match = m;
|
|
208
|
+
break;
|
|
209
|
+
}
|
|
210
|
+
}
|
|
196
211
|
if (!match) continue;
|
|
197
|
-
out.push({ obs: obs.id, quote:
|
|
212
|
+
out.push({ obs: obs.id, quote: q.quote, ...(q.label ? { label: q.label } : {}), agentFacing });
|
|
198
213
|
if (AGENTID_OWNER.test(text.slice(Math.max(0, match.index - OWNER_WINDOW), match.index + OWNER_WINDOW))) ownerDocumented = true;
|
|
199
214
|
}
|
|
200
215
|
return { entries: out, ownerDocumented };
|
|
@@ -245,49 +260,165 @@ const EXAMPLE_PAGE = /\/(?:templates?|examples?|starters?)\//i;
|
|
|
245
260
|
// OAuth dynamic client registration (POST /oauth/register, /oauth2/register, /connect/register) registers a client app
|
|
246
261
|
// with an authorization server; it is not an agent getting an account.
|
|
247
262
|
const OAUTH_REGISTRATION = /\/(?:oauth2?|connect|clients?|dcr)\/regist(?:er|ration)\b/i;
|
|
263
|
+
// A signal that says the product requires something (an inbox, a person setting access up, restricted access, a
|
|
264
|
+
// payment) counts only from a sentence that says so about the product itself. Text that merely mentions the thing never
|
|
265
|
+
// counts: a negation ("No project key can sign in to the dashboard"), an example ("such as opening a verification
|
|
266
|
+
// email") or a sentence about the reader's own customers. Each kept match is quoted, and the quote is shown with the
|
|
267
|
+
// step it decides.
|
|
268
|
+
// A negation in the same clause just before the match: "A project key cannot sign in to the dashboard". A comma ends
|
|
269
|
+
// the clause, so "If you don't have one, create an API key in the dashboard" still counts.
|
|
270
|
+
const CLAUSE_NEGATION = /\b(?:no|not|never|cannot|can't|can not|without|don't|doesn't|won't|isn't|aren't|nothing)\b[^.,;:!?\n]{0,40}$/i;
|
|
271
|
+
const negatedBefore = (sentence, match) => {
|
|
272
|
+
const at = sentence.indexOf(match.replace(/\s+/g, " "));
|
|
273
|
+
return at >= 0 && CLAUSE_NEGATION.test(sentence.slice(0, at));
|
|
274
|
+
};
|
|
275
|
+
// Expiry said not to happen: Track's "Unclaimed, nothing is deleted", "the workspace and key never expire".
|
|
276
|
+
const NO_EXPIRY = /\b(?:nothing|never|not|no\s+\w+)\s+(?:is\s+|are\s+|will\s+|gets?\s+|be\s+|ever\s+)*(?:deleted|removed|expires?)\b|\bdo(?:es)?(?: not|n't) expire\b/i;
|
|
277
|
+
// An example of something, not the product's own step: Track's "For a step your API never sees, such as opening a
|
|
278
|
+
// verification email, send an onboarding_step event".
|
|
279
|
+
const EXAMPLE = /\b(?:such as|for example|for instance|e\.g\.|i\.e\.)/i;
|
|
280
|
+
// An inbox step counts only in a sentence about the product's own signup or sign-in path. The words are looked for
|
|
281
|
+
// outside the match, so "Verification email never opened" (a sample error message in Track's docs) is not its own
|
|
282
|
+
// context.
|
|
283
|
+
const SIGNUP_CONTEXT = /\bsign(?:s|ing|ed)?[- ]?(?:up|in)\b|\bsignups?\b|\bregist(?:er|ers|ered|ering|ration)\b|\bcreat(?:e|es|ing) (?:an? |your |the )?(?:new )?account\b|\bonboard(?:ing)?\b|\bverif(?:y|ies|ied|ication)\b|\bconfirm(?:s|ed)?\b|\blog ?in\b|\bget (?:an? |your )?(?:API )?key\b/i;
|
|
284
|
+
// The product saying there is no inbox step. Authoritative: one such sentence on the product's own pages clears every
|
|
285
|
+
// inbox match. Track's "POST /v1/signup returns a key with no account, email or browser", "Track needs no inbox
|
|
286
|
+
// verification", "takes no email and sends nothing to confirm". Not "without email": Telnyx's "Without email access the
|
|
287
|
+
// flow falls back to..." is a condition, and its signup does send a magic link.
|
|
288
|
+
export const NO_INBOX = /\b(?:takes|needs|requires|asks for|sends) no (?:e-?mail|inbox)\b|\bneeds? no inbox\b|\bno inbox (?:verification|step|needed|required)\b|\bno (?:account, )?e-?mail(?:,| or| needed| required)\b|\bno (?:e-?mail )?verification (?:e-?mail|code|link|step)\b|\bsends nothing to confirm\b/i;
|
|
289
|
+
const withoutMatch = (sentence, match) => sentence.replace(match.replace(/\s+/g, " "), " ");
|
|
290
|
+
// A line of a link index ("- [Email OTP](https://neon.com/docs/auth/...): Sign in and verify email addresses") describes
|
|
291
|
+
// another page, often a feature the product sells, not the product's own signup.
|
|
292
|
+
const INDEX_LINE = /^\s*(?:[-*]\s*)?\[[^\]]+\]\([^)]+\)/;
|
|
293
|
+
// Signing up somewhere else: AgentMail's "Enable browser agents to sign up for services by extracting OTP codes".
|
|
294
|
+
// AgentLine's "Receive OTP codes so your agent can sign up and verify" is a feature for signing up elsewhere too.
|
|
295
|
+
const SIGNUP_ELSEWHERE = /\bsign(?:s|ing)? up for (?:services|other|any|third-party|apps|sites|websites)\b|\b(?:other|third-party) (?:services|apps|sites|websites)\b|\bso (?:your|the|an) agents? can sign up\b/i;
|
|
296
|
+
// An optional step is not a requirement: Locus's "the human is needed only for an optional AgentMail OTP".
|
|
297
|
+
const OPTIONAL = /\boptional(?:ly)?\b/i;
|
|
298
|
+
|
|
299
|
+
// An inbox step counts only when its subject is the product's own signup. AgentID's llms-full.txt tells the general
|
|
300
|
+
// problem, "An agent signs up with its owner's email, reads the verification code out of the owner's inbox", and that
|
|
301
|
+
// read as AgentID's own signup needing an inbox. A sentence counts when it says so itself (ownSignup below), or when it
|
|
302
|
+
// sits on the product's own agent-signup or auth page, or in a signup section of a page (under a "Sign up", "Agent
|
|
303
|
+
// Signup", "Registration" or "Quick start" heading, or right after a step that is the product's own signup call).
|
|
304
|
+
// A third-person narrative ("an agent signs up", "AI agents can sign up for a SaaS product") counts only on the
|
|
305
|
+
// product's own signup page.
|
|
306
|
+
// The sentence calls one of the product's signup, sign-in or verify endpoints: AgentLine's "`POST /v1/auth/otp` →
|
|
307
|
+
// `POST /v1/auth/verify`", Cosmic's "(POST /v3/agents/sign-up)", AgentMail's "curl -X POST .../v0/agent/sign-up".
|
|
308
|
+
const SIGNUP_CALL = /\b(?:POST|GET|PUT)\b[^\n]{0,24}?\/[\w.\/{}:-]*?(?:sign[-_]?up|sign[-_]?in|register|registration|verif\w*|otp|auth|login|magic[-_]?link|onboard)\b/i;
|
|
309
|
+
// "We send", "we'll email", "our signup": the product speaking about its own flow.
|
|
310
|
+
const FIRST_PERSON = /\b(?:we|we'll|we will)\s+(?:then\s+)?(?:send|email|text|mail)\b|\bour (?:sign[- ]?up|signup|registration|onboarding|verification|sign[- ]?in|login)\b|\b(?:create an account|sign up|register) here\b/i;
|
|
311
|
+
const SIGNUP_HEADING = /^\W*(?:\w+ )?(?:sign[- ]?up|signup|sign[- ]?in|register|registration|create (?:an |your )?account|get(?:ting)? started|quick ?start|onboarding)\b(?![^?]*\?)/i;
|
|
312
|
+
const GENERIC_NARRATIVE = /\b(?:an?|every|each|any|most|many|some)\s+(?:AI\s+)?(?:agents?|users?|people|persons?|humans?|developers?|bots?)\b(?:\s+\w+){0,3}?\s+(?:sign(?:s|ed|ing)?|can sign|could sign) up\b/i;
|
|
313
|
+
const OWN_SIGNUP_PAGES = new Set(["onboarding", "auth"]);
|
|
314
|
+
const escapeRe = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
|
315
|
+
// The product's names: its host's labels (agentmail, cosmicjs and cosmic) and the first part of its page title.
|
|
316
|
+
function productNames(doc) {
|
|
317
|
+
const labels = doc.target.host.toLowerCase().split(".").slice(0, -1).filter((l) => l !== "www" && l.length > 2);
|
|
318
|
+
const variants = labels.flatMap((l) => [l, l.replace(/(?:js|hq|app|ai|io|dev|labs)$/, "")]).filter((l) => l.length > 2);
|
|
319
|
+
const title = String(doc.product?.name?.value || "").split(/\s+[|\u2014\u2013:-]\s+|\s*[|:]\s*/)[0].trim();
|
|
320
|
+
return [...new Set([...variants, ...(title && title.length <= 40 ? [title.toLowerCase()] : [])])];
|
|
321
|
+
}
|
|
322
|
+
function ownSignupSentence(sentence, names) {
|
|
323
|
+
if (SIGNUP_CALL.test(sentence) || FIRST_PERSON.test(sentence)) return true;
|
|
324
|
+
if (!names.length) return false;
|
|
325
|
+
const name = `(?:${names.map(escapeRe).join("|")})`;
|
|
326
|
+
return new RegExp(`\\b(?:sign(?:s|ed|ing)?[- ]?(?:up|in)|log(?:s|ged|ging)?[- ]?in|register(?:s|ed|ing)?|creat(?:e|es|ed|ing) (?:an? |your )?account)\\s+(?:for|to|with|on|at|into)\\s+(?:an? )?${name}\\b|\\b${name}\\s+(?:sends?|emails?|will (?:send|email)|texts?)\\b|\\b(?:your|an?|the) ${name} account\\b|\\b${name}[\\w.]*\\/(?:sign-?up|register|signin|login)\\b`, "i").test(sentence);
|
|
327
|
+
}
|
|
328
|
+
function ownSignup(sentence, match, page) {
|
|
329
|
+
if (ownSignupSentence(sentence, page.names)) return true;
|
|
330
|
+
// On the product's own signup page the sentence still has to be about signing up: Moltbook's "Verification code
|
|
331
|
+
// expired." is a line of an error list, Locus's "Never ask the human to reveal their ... email verification
|
|
332
|
+
// code" is advice. The words are looked for outside the match.
|
|
333
|
+
if (OWN_SIGNUP_PAGES.has(page.role) && SIGNUP_CONTEXT.test(withoutMatch(sentence, match))) return true;
|
|
334
|
+
if (GENERIC_NARRATIVE.test(sentence)) return false;
|
|
335
|
+
const ctx = page.context();
|
|
336
|
+
if (!ctx) return false;
|
|
337
|
+
const heading = ctx.heading.replace(/[#*_`]/g, " ").trim();
|
|
338
|
+
return SIGNUP_HEADING.test(heading) || ownSignupSentence(heading, page.names) || ownSignupSentence(ctx.previous, page.names);
|
|
339
|
+
}
|
|
340
|
+
|
|
341
|
+
// A question says nothing about this product: AgentID's llms-full.txt "How can you tell an agent signup from a human
|
|
342
|
+
// one?" heads an explainer, and its comparison table's row "Can the agent sign up for a new service on its own" has no
|
|
343
|
+
// question mark. An FAQ's answer, where it says yes, is its own sentence and is tried next.
|
|
344
|
+
const QUESTION = /\?\s*(?:\*\*)?\s*$|^\W*(?:can|could|does|do|is|are|will|should|how|why|what|when|where)\b/i;
|
|
345
|
+
|
|
346
|
+
// The match names a page rather than saying anything: AgentID's "[can an AI agent sign up for a
|
|
347
|
+
// SaaS](/blog/can-an-ai-agent-sign-up-for-a-saas)" is a link to a blog post, in its text and in its slug. A path like
|
|
348
|
+
// /agents/signup starts with a slash and still counts.
|
|
349
|
+
function inSlug(sentence, match) {
|
|
350
|
+
if (match.startsWith("/")) return false;
|
|
351
|
+
const m = escapeRe(match);
|
|
352
|
+
if (new RegExp(`\\[[^\\]]*${m}[^\\]]*\\]\\(`, "i").test(sentence)) return true;
|
|
353
|
+
const re = new RegExp(`[\\w/]-${m}|${m}-[a-z]`, "i");
|
|
354
|
+
return re.test(sentence) && !sentence.includes(` ${match} `) && !sentence.startsWith(`${match} `);
|
|
355
|
+
}
|
|
356
|
+
|
|
248
357
|
// Whether a sentence holding a match is about something else; such a match is skipped and the next one tried.
|
|
358
|
+
// page: { role, names, context() } for the rules that read where the sentence is (the inbox rule).
|
|
249
359
|
const SKIP = {
|
|
250
|
-
bootstrap: (sentence, ownDomain, match) => DOMAIN_ACTORS.test(sentence) || SIGNUP_OFF.test(sentence) || PERSON_WITHOUT_ACCOUNT.test(sentence) || PAYMENT_WAY_IN.test(sentence) || CLIENT_REGISTRATION.test(sentence) || BROKERED.test(sentence) || (/^temporary\b/i.test(match) && STORED_THING.test(sentence)),
|
|
251
|
-
expiry: (sentence) => !ACCOUNT_THING.test(sentence) || STORED_THING.test(sentence),
|
|
360
|
+
bootstrap: (sentence, ownDomain, match) => inSlug(sentence, match) || QUESTION.test(sentence) || DOMAIN_ACTORS.test(sentence) || SIGNUP_OFF.test(sentence) || PERSON_WITHOUT_ACCOUNT.test(sentence) || PAYMENT_WAY_IN.test(sentence) || CLIENT_REGISTRATION.test(sentence) || BROKERED.test(sentence) || (/^temporary\b/i.test(match) && STORED_THING.test(sentence)),
|
|
361
|
+
expiry: (sentence) => !ACCOUNT_THING.test(sentence) || STORED_THING.test(sentence) || NO_EXPIRY.test(sentence),
|
|
362
|
+
restricted: (sentence, ownDomain, match) => DOMAIN_ACTORS.test(sentence) || EXAMPLE.test(sentence) || negatedBefore(sentence, match),
|
|
363
|
+
emailStep: (sentence, ownDomain, match, page, label) => label === "Code" || DOMAIN_ACTORS.test(sentence) || EXAMPLE.test(sentence) || INDEX_LINE.test(sentence) || SIGNUP_ELSEWHERE.test(sentence) || OPTIONAL.test(sentence) || NO_INBOX.test(sentence) || negatedBefore(sentence, match) || !ownSignup(sentence, match, page),
|
|
364
|
+
humanFirst: (sentence, ownDomain, match) => DOMAIN_ACTORS.test(sentence) || EXAMPLE.test(sentence) || negatedBefore(sentence, match),
|
|
252
365
|
httpBootstrap: (sentence, ownDomain, match) => DOMAIN_ACTORS.test(sentence) || OAUTH_REGISTRATION.test(match),
|
|
253
|
-
payPerRequest: (sentence, ownDomain) => DOMAIN_ACTORS.test(sentence) || SELLER.test(sentence) || PROTOCOL_DESCRIPTION.test(sentence) || paysElsewhere(sentence, ownDomain),
|
|
366
|
+
payPerRequest: (sentence, ownDomain, match) => DOMAIN_ACTORS.test(sentence) || SELLER.test(sentence) || PROTOCOL_DESCRIPTION.test(sentence) || paysElsewhere(sentence, ownDomain) || negatedBefore(sentence, match),
|
|
254
367
|
};
|
|
255
|
-
const GLOBAL = Object.fromEntries(Object.keys(
|
|
368
|
+
const GLOBAL = Object.fromEntries(Object.keys(SIGNALS).map((name) => [name, new RegExp(SIGNALS[name].source, `${SIGNALS[name].flags.replace("g", "")}g`)]));
|
|
256
369
|
|
|
257
|
-
// The first match whose sentence is not about something else.
|
|
258
|
-
//
|
|
370
|
+
// The first match that has a clean quote (src/interface/quote.js) and whose sentence is not about something else. The
|
|
371
|
+
// guards read the same sentence the quote shows, so a rule never decides on words from a neighbouring element of the
|
|
372
|
+
// page. Tries at most MAX_TRIED matches a page, so a page that repeats a skipped phrase thousands of times costs no
|
|
373
|
+
// more than one that says it once.
|
|
259
374
|
const MAX_TRIED = 200;
|
|
260
|
-
function firstKept(
|
|
375
|
+
function firstKept(page, re, skip, ownDomain) {
|
|
261
376
|
let tried = 0;
|
|
262
|
-
for (const m of text.matchAll(re)) {
|
|
263
|
-
|
|
377
|
+
for (const m of page.text.matchAll(re)) {
|
|
378
|
+
const q = quoteAt(page.body, page.text, m.index, m.index + m[0].length, page.role);
|
|
379
|
+
const at = { ...page, context: () => blockContext(page.body, page.text, m.index, m.index + m[0].length, page.role) };
|
|
380
|
+
if (q && !(skip && skip(q.quote, ownDomain, m[0], at, q.label))) return { index: m.index, ...q };
|
|
264
381
|
tried += 1;
|
|
265
382
|
if (tried >= MAX_TRIED) return null;
|
|
266
383
|
}
|
|
267
384
|
return null;
|
|
268
385
|
}
|
|
269
386
|
|
|
270
|
-
function findSignals(observations, bodies, targetUrl) {
|
|
387
|
+
function findSignals(observations, bodies, targetUrl, names) {
|
|
271
388
|
const ownDomain = registrableDomain(new URL(targetUrl).hostname);
|
|
272
389
|
const found = Object.fromEntries(Object.keys(SIGNALS).map((name) => [name, []]));
|
|
273
390
|
for (const obs of observations) {
|
|
274
391
|
const body = bodies.get(obs.id);
|
|
275
392
|
if (!body) continue;
|
|
276
|
-
const
|
|
277
|
-
for (const
|
|
393
|
+
const page = { body, role: obs.role, names, text: body.html ? htmlToText(body.text) : body.text };
|
|
394
|
+
for (const name of Object.keys(SIGNALS)) {
|
|
278
395
|
if (found[name].length >= EVIDENCE_PER_SIGNAL) continue;
|
|
279
|
-
// A sitemap's URL slugs ("/posts/x402-launch") are page names,
|
|
280
|
-
// (vercel.com/templates/next.js/x402-ai-starter) is code for the reader's own
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
396
|
+
// A sitemap's URL slugs ("/posts/x402-launch", "/blog/can-an-ai-agent-sign-up-for-a-saas") are page names, not
|
|
397
|
+
// claims. A template or example page (vercel.com/templates/next.js/x402-ai-starter) is code for the reader's own
|
|
398
|
+
// app: it does not document paying this product.
|
|
399
|
+
if (obs.role === "sitemap") continue;
|
|
400
|
+
if (name === "payPerRequest" && EXAMPLE_PAGE.test(new URL(obs.url).pathname)) continue;
|
|
401
|
+
// Every match is tried, so one sentence about something else, or words run together from two elements, does not
|
|
402
|
+
// hide a real one.
|
|
403
|
+
const match = firstKept(page, GLOBAL[name], SKIP[name], ownDomain);
|
|
404
|
+
if (match) found[name].push({ obs: obs.id, quote: match.quote, ...(match.label ? { label: match.label } : {}), agentFacing: AGENT_FACING.has(obs.role) });
|
|
286
405
|
}
|
|
287
406
|
}
|
|
288
407
|
return found;
|
|
289
408
|
}
|
|
290
409
|
|
|
410
|
+
// The first sentence on the product's own pages saying signup has no inbox step, or null.
|
|
411
|
+
function findNoInbox(doc, bodies) {
|
|
412
|
+
for (const obs of doc.observations) {
|
|
413
|
+
const body = bodies.get(obs.id);
|
|
414
|
+
if (!body || !sameSite(obs.url, doc.target.url)) continue;
|
|
415
|
+
const text = body.html ? htmlToText(body.text) : body.text;
|
|
416
|
+
const m = NO_INBOX.exec(text);
|
|
417
|
+
if (m) return { obs: obs.id, quote: quoteAround(text, m.index) };
|
|
418
|
+
}
|
|
419
|
+
return null;
|
|
420
|
+
}
|
|
421
|
+
|
|
291
422
|
// What testing this pattern end to end requires beyond HTTP requests: an inbox for an
|
|
292
423
|
// email step, a command-line runner, a funded wallet. Whether a deployment has each is
|
|
293
424
|
// decided where the report is shown, not here. Agent identity has no runner support.
|
|
@@ -302,11 +433,18 @@ function needsFor(id, found) {
|
|
|
302
433
|
return needs;
|
|
303
434
|
}
|
|
304
435
|
|
|
436
|
+
// The quotes behind each need, so a step that says the product needs something can show the sentence it came from.
|
|
437
|
+
const NEED_SIGNALS = { inbox: ["emailStep"], cli: ["cliBootstrap"], wallet: ["payPerRequest"], agent_identity: ["agentIdSignIn", "agentIdentity"] };
|
|
438
|
+
function needsEvidenceFor(needs, found) {
|
|
439
|
+
const ordered = (entries) => [...entries.filter((e) => e.agentFacing), ...entries.filter((e) => !e.agentFacing)];
|
|
440
|
+
return Object.fromEntries(needs.map((need) => [need, ordered(NEED_SIGNALS[need].flatMap((name) => found[name] || [])).map(({ obs, quote, label }) => ({ obs, quote, ...(label ? { label } : {}) }))]));
|
|
441
|
+
}
|
|
442
|
+
|
|
305
443
|
function pattern(id, found, signalNames, reason) {
|
|
306
444
|
// The product's agent-facing pages first: a quote from llms.txt or an agent-signup
|
|
307
445
|
// page says more than the same words in homepage navigation.
|
|
308
446
|
const entries = signalNames.flatMap((name) => found[name]);
|
|
309
|
-
const evidence = [...entries.filter((e) => e.agentFacing), ...entries.filter((e) => !e.agentFacing)].map(({ obs, quote }) => ({ obs, quote }));
|
|
447
|
+
const evidence = [...entries.filter((e) => e.agentFacing), ...entries.filter((e) => !e.agentFacing)].map(({ obs, quote, label }) => ({ obs, quote, ...(label ? { label } : {}) }));
|
|
310
448
|
const documented = signalNames.some((name) => found[name].some((entry) => entry.agentFacing));
|
|
311
449
|
return {
|
|
312
450
|
id,
|
|
@@ -316,6 +454,7 @@ function pattern(id, found, signalNames, reason) {
|
|
|
316
454
|
evidence,
|
|
317
455
|
humanBoundary: PATTERNS[id].humanBoundary,
|
|
318
456
|
needs: needsFor(id, found),
|
|
457
|
+
needsEvidence: needsEvidenceFor(needsFor(id, found), found),
|
|
319
458
|
};
|
|
320
459
|
}
|
|
321
460
|
|
|
@@ -342,9 +481,27 @@ function findApiHosts(doc, bodies) {
|
|
|
342
481
|
return [...hosts.values()];
|
|
343
482
|
}
|
|
344
483
|
|
|
484
|
+
// A signup page on another company's site that the product's agent-facing pages send the agent to: AgentID's llms.txt
|
|
485
|
+
// "[Get started](https://console.agentmail.to/sign-up): Create an AgentMail inbox to get an AgentID". That page is not
|
|
486
|
+
// read, so whether an agent can sign up there is unknown, not a gap in this product's docs.
|
|
487
|
+
const SIGNUP_URL = /https:\/\/([a-z0-9.-]+\.[a-z]{2,})(\/[^\s"'`<>)\]]*)?/gi;
|
|
488
|
+
const SIGNUP_PATH = /\/(?:sign-?up|register|get-started)(?:[/?#]|$)/i;
|
|
489
|
+
function findSignupElsewhere(doc, bodies) {
|
|
490
|
+
for (const obs of doc.observations) {
|
|
491
|
+
const body = bodies.get(obs.id);
|
|
492
|
+
if (!body || !obs.ok || !AGENT_FACING.has(obs.role)) continue;
|
|
493
|
+
const text = body.html ? htmlToText(body.text) : body.text;
|
|
494
|
+
for (const match of text.matchAll(SIGNUP_URL)) {
|
|
495
|
+
if (sameSite(`https://${match[1]}/`, doc.target.url) || !SIGNUP_PATH.test(match[2] || "")) continue;
|
|
496
|
+
return { domain: registrableDomain(match[1].toLowerCase()), url: match[0], evidence: [{ obs: obs.id, quote: quoteAround(text, match.index) }] };
|
|
497
|
+
}
|
|
498
|
+
}
|
|
499
|
+
return null;
|
|
500
|
+
}
|
|
501
|
+
|
|
345
502
|
export function detectOnboarding(doc, bodies) {
|
|
346
503
|
const observations = doc.observations;
|
|
347
|
-
const found = findSignals(observations, bodies, doc.target.url);
|
|
504
|
+
const found = findSignals(observations, bodies, doc.target.url, productNames(doc));
|
|
348
505
|
const onIssuerSite = sameSite(AGENTID_ISSUER, doc.target.url);
|
|
349
506
|
const agentId = onIssuerSite ? { entries: [], ownerDocumented: false } : findAgentIdSignIn(doc, bodies);
|
|
350
507
|
found.agentIdSignIn = agentId.entries;
|
|
@@ -352,6 +509,9 @@ export function detectOnboarding(doc, bodies) {
|
|
|
352
509
|
// A signup call documented on a page written for agents is the product saying an agent can sign itself up, even
|
|
353
510
|
// without the words "agent signup": Moltbook's skill.md says "register" and shows POST /api/v1/agents/register.
|
|
354
511
|
found.bootstrap.push(...found.httpBootstrap.filter((e) => e.agentFacing && !found.bootstrap.some((b) => b.obs === e.obs)));
|
|
512
|
+
// The product saying it has no inbox step outweighs any mention of one.
|
|
513
|
+
const noInbox = findNoInbox(doc, bodies);
|
|
514
|
+
if (noInbox) found.emailStep = [];
|
|
355
515
|
const has = (name) => found[name].length > 0;
|
|
356
516
|
const patterns = [];
|
|
357
517
|
|
|
@@ -378,7 +538,9 @@ export function detectOnboarding(doc, bodies) {
|
|
|
378
538
|
patterns.push(p);
|
|
379
539
|
}
|
|
380
540
|
|
|
381
|
-
|
|
541
|
+
// Machine-readable pricing first: when pricing.json lists only free plans, x402 in prose is not a price. Track's docs
|
|
542
|
+
// name x402 as one of the tasks its sister product tests, and that read as Track charging per request.
|
|
543
|
+
if (has("payPerRequest") && !freePlansOnly(doc.pricing)) patterns.push(pattern("pay_per_request", found, ["payPerRequest"], "The docs describe paying per request over HTTP 402, with no account needed first."));
|
|
382
544
|
|
|
383
545
|
// Only the signup side says a person sets access up: a key made in a dashboard, a token a person creates. An auth
|
|
384
546
|
// scheme in the OpenAPI document (Turso's `bearerAuth: http bearer`) says how a request is signed, not who gets the
|
|
@@ -395,6 +557,7 @@ export function detectOnboarding(doc, bodies) {
|
|
|
395
557
|
return {
|
|
396
558
|
patterns,
|
|
397
559
|
apiHosts: findApiHosts(doc, bodies),
|
|
560
|
+
signupElsewhere: findSignupElsewhere(doc, bodies),
|
|
398
561
|
primary: agentFirst?.id || patterns[0]?.id || null,
|
|
399
562
|
reason: patterns.length ? `${patterns.length} way${patterns.length === 1 ? "" : "s"} for an agent to start, from the sources read.` : `No way for an agent to start was found in the sources read.${doc.otherHostsSkipped ? " Links to other hosts were not read." : ""} This is not evidence that none exists.`,
|
|
400
563
|
};
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import { htmlToText, registrableDomain, sameSite } from "./sources.js";
|
|
2
|
-
import { AGENT_FACING, DOMAIN_ACTORS, PROTOCOL_DESCRIPTION, PROTOCOL_SITES, SELLER, paysElsewhere,
|
|
2
|
+
import { AGENT_FACING, DOMAIN_ACTORS, PROTOCOL_DESCRIPTION, PROTOCOL_SITES, SELLER, paysElsewhere, sentenceAround } from "./onboarding.js";
|
|
3
|
+
import { quoteAt } from "./quote.js";
|
|
3
4
|
|
|
4
5
|
// Whether the product's own pages document a way for an agent to buy from it through an API, and whether a person
|
|
5
6
|
// approves first. Rules over page text, like the onboarding signals: each rule is a named pattern, every hit carries
|
|
@@ -239,7 +240,7 @@ export function detectAgentPurchase(doc, bodies) {
|
|
|
239
240
|
const text = body.html ? htmlToText(body.text) : body.text;
|
|
240
241
|
// The product publishes its UCP business profile: the product saying agents can check out with it over UCP.
|
|
241
242
|
if (obs.role === "ucp_profile") {
|
|
242
|
-
if (isUcpProfile(text)) hits.push({ rule: "checkout_protocol_own_api", obs: obs.id, quote:
|
|
243
|
+
if (isUcpProfile(text)) hits.push({ rule: "checkout_protocol_own_api", obs: obs.id, quote: new URL(obs.url).pathname, label: "UCP business profile", approval: false, call: null, score: 2, protocol: "UCP", agentFacing: true });
|
|
243
244
|
continue;
|
|
244
245
|
}
|
|
245
246
|
const sellerPage = SELLER_PAGE.test(new URL(obs.url).pathname);
|
|
@@ -249,7 +250,10 @@ export function detectAgentPurchase(doc, bodies) {
|
|
|
249
250
|
const hit = find(text, ownDomain);
|
|
250
251
|
if (!hit) continue;
|
|
251
252
|
const near = text.slice(Math.max(0, hit.index - APPROVAL_WINDOW), hit.index + APPROVAL_WINDOW);
|
|
252
|
-
|
|
253
|
+
// The sentence or labelled element holding the hit (src/interface/quote.js); null when the words run across two
|
|
254
|
+
// elements of the page, and then the hit has no quote.
|
|
255
|
+
const q = quoteAt(body, text, hit.index, hit.index + 1, obs.role);
|
|
256
|
+
hits.push({ rule, obs: obs.id, quote: q?.quote || null, ...(q?.label ? { label: q.label } : {}), approval: Boolean(hit.approved) || APPROVAL.test(hit.sentence) || APPROVAL.test(near), call: hit.call, score: hit.score, protocol: hit.protocol || protocolIn(hit.sentence, text), agentFacing: AGENT_FACING.has(obs.role) });
|
|
253
257
|
}
|
|
254
258
|
}
|
|
255
259
|
if (!hits.length) return null;
|
|
@@ -265,6 +269,6 @@ export function detectAgentPurchase(doc, bodies) {
|
|
|
265
269
|
approval: hits.some((h) => h.approval) ? "once" : "none_documented",
|
|
266
270
|
call: ruled.find((h) => h.call)?.call || null,
|
|
267
271
|
protocol: ruled.find((h) => h.protocol)?.protocol || null,
|
|
268
|
-
evidence: ordered.filter((h) => h.rule === rule).concat(ordered.filter((h) => h.rule !== rule)).slice(0, EVIDENCE_MAX * 2).map(({ obs, quote }) => ({ obs, quote })),
|
|
272
|
+
evidence: ordered.filter((h) => h.rule === rule).concat(ordered.filter((h) => h.rule !== rule)).slice(0, EVIDENCE_MAX * 2).map(({ obs, quote, label }) => ({ obs, quote, ...(label ? { label } : {}) })),
|
|
269
273
|
};
|
|
270
274
|
}
|
|
@@ -0,0 +1,255 @@
|
|
|
1
|
+
// The quote shown under a step that says the product needs something. Kat reads every one, so a quote is one or two
|
|
2
|
+
// whole sentences from one element of the page (a paragraph, a list item, a table cell), never text run together from
|
|
3
|
+
// separate elements. Resend's pricing page flattened to "scale as you grow. Are you an AI agent? See pricing Add-ons if
|
|
4
|
+
// you send from a": a subtitle, a link and a heading read as one sentence. When the words sit in an element with no
|
|
5
|
+
// sentence (a heading, a table cell, a button, a line of code), the quote is that element's text with a label saying
|
|
6
|
+
// what it is: Heading: "Domains - $20 / mo".
|
|
7
|
+
|
|
8
|
+
// Elements that start a new block. Inline elements (a, span, strong, code) stay inside the sentence they are part of.
|
|
9
|
+
const BLOCK_TAG = /^<\/?\s*(address|article|aside|blockquote|br|button|caption|dd|details|dialog|div|dl|dt|fieldset|figcaption|figure|footer|form|h[1-6]|header|hr|img|input|label|legend|li|main|menu|nav|ol|option|p|pre|section|select|summary|svg|table|tbody|td|template|textarea|tfoot|th|thead|title|tr|ul)\b/i;
|
|
10
|
+
const KIND_OF_TAG = { td: "cell", th: "cell", li: "item", dt: "item", dd: "item", button: "button", option: "button", summary: "button", pre: "code", title: "heading" };
|
|
11
|
+
export const LABEL = { cell: "Table cell", heading: "Heading", item: "List item", button: "Button", code: "Code", line: "Line", paragraph: "Text", phrase: "Phrase" };
|
|
12
|
+
// An element's text with no sentence in it is quoted whole only up to this length; longer is a run of inline elements
|
|
13
|
+
// (a navigation bar, a footer), which is not a quote.
|
|
14
|
+
const PHRASE_MAX = 160;
|
|
15
|
+
// A sentence longer than this is a run of data (a JSON blob in a page, a minified list), not a sentence to quote.
|
|
16
|
+
const SENTENCE_MAX = 320;
|
|
17
|
+
|
|
18
|
+
function decode(s) {
|
|
19
|
+
return s.replace(/<[^>]+>/g, " ").replace(/ /g, " ").replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">").replace(/"/g, '"').replace(/'/g, "'");
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
// Blocks of an HTML page, with offsets into htmlToText(html): the same steps in the same order, split where a block
|
|
23
|
+
// element starts or ends, so the blocks joined with one space are exactly htmlToText's output.
|
|
24
|
+
export function htmlBlocks(html) {
|
|
25
|
+
const src = html.replace(/<script[\s\S]*?<\/script>/gi, " ").replace(/<style[\s\S]*?<\/style>/gi, " ");
|
|
26
|
+
const segments = [];
|
|
27
|
+
let kind = "phrase";
|
|
28
|
+
let last = 0;
|
|
29
|
+
for (const m of src.matchAll(/<[^>]+>/g)) {
|
|
30
|
+
const tag = BLOCK_TAG.exec(m[0]);
|
|
31
|
+
if (!tag) continue;
|
|
32
|
+
segments.push({ kind, raw: src.slice(last, m.index) });
|
|
33
|
+
const name = tag[1].toLowerCase();
|
|
34
|
+
const closing = /^<\s*\//.test(m[0]);
|
|
35
|
+
kind = closing ? "phrase" : /^h[1-6]$/.test(name) ? "heading" : KIND_OF_TAG[name] || "phrase";
|
|
36
|
+
last = m.index + m[0].length;
|
|
37
|
+
}
|
|
38
|
+
segments.push({ kind, raw: src.slice(last) });
|
|
39
|
+
const blocks = [];
|
|
40
|
+
let offset = 0;
|
|
41
|
+
for (const seg of segments) {
|
|
42
|
+
const text = decode(seg.raw).replace(/\s+/g, " ").trim();
|
|
43
|
+
if (!text) continue;
|
|
44
|
+
blocks.push({ start: offset, end: offset + text.length, kind: seg.kind, raw: seg.raw });
|
|
45
|
+
offset += text.length + 1;
|
|
46
|
+
}
|
|
47
|
+
return blocks;
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
const FENCE = /^\s*(?:```|~~~)/;
|
|
51
|
+
const HEADING_LINE = /^\s{0,3}#{1,6}\s/;
|
|
52
|
+
const ITEM_LINE = /^\s*(?:[-*+]|\d+[.)])\s+/;
|
|
53
|
+
const TABLE_LINE = /^\s*\|/;
|
|
54
|
+
// A line that is only an HTML or MDX tag (`<Card title="Verify" href="...">`, `</Card>`) ends a block, like a blank line.
|
|
55
|
+
const TAG_LINE = /^\s*<\/?[A-Za-z][^>]*>\s*$/;
|
|
56
|
+
const INLINE_BLOCK_TAG = /<\/?\s*(?:td|th|tr|table|thead|tbody|p|li|ul|ol|div|h[1-6])\b[^>]*>/i;
|
|
57
|
+
const INLINE_BLOCK_TAG_NAME = /^<(\/)?\s*(td|th|tr|table|thead|tbody|p|li|ul|ol|div|h[1-6])\b/i;
|
|
58
|
+
const TAG_KIND = { td: "cell", th: "cell", li: "item", h1: "heading", h2: "heading", h3: "heading", h4: "heading", h5: "heading", h6: "heading" };
|
|
59
|
+
function pushPiece(blocks, line, start, a, b, kind) {
|
|
60
|
+
while (a < b && /\s/.test(line[a])) a += 1;
|
|
61
|
+
while (b > a && /\s/.test(line[b - 1])) b -= 1;
|
|
62
|
+
if (b > a) blocks.push({ start: start + a, end: start + b, kind });
|
|
63
|
+
}
|
|
64
|
+
// A shell-comment block in a text file (AgentMail's llms.txt): "#" alone is a blank line and "#" followed by two or
|
|
65
|
+
// more spaces is text, not a heading.
|
|
66
|
+
const COMMENT_BLANK = /^\s*#\s*$/;
|
|
67
|
+
const COMMENT_TEXT = /^\s*#\s{2,}\S/;
|
|
68
|
+
|
|
69
|
+
// Blocks of a Markdown or plain-text page, with offsets into the text: a paragraph (lines joined until a blank line),
|
|
70
|
+
// a heading, a list item with its wrapped lines, a table cell, a line of a code block. lineMode makes every line its
|
|
71
|
+
// own block, for YAML and other line-oriented files.
|
|
72
|
+
export function textBlocks(text, { lineMode = false } = {}) {
|
|
73
|
+
const blocks = [];
|
|
74
|
+
let open = null;
|
|
75
|
+
const close = () => {
|
|
76
|
+
if (open) blocks.push(open);
|
|
77
|
+
open = null;
|
|
78
|
+
};
|
|
79
|
+
let inFence = false;
|
|
80
|
+
let at = 0;
|
|
81
|
+
for (const line of text.split("\n")) {
|
|
82
|
+
const start = at;
|
|
83
|
+
const end = at + line.length;
|
|
84
|
+
at = end + 1;
|
|
85
|
+
const blank = !line.trim() || (!inFence && (COMMENT_BLANK.test(line) || TAG_LINE.test(line)));
|
|
86
|
+
if (FENCE.test(line)) {
|
|
87
|
+
close();
|
|
88
|
+
inFence = !inFence;
|
|
89
|
+
continue;
|
|
90
|
+
}
|
|
91
|
+
if (blank) {
|
|
92
|
+
close();
|
|
93
|
+
continue;
|
|
94
|
+
}
|
|
95
|
+
if (inFence || lineMode) {
|
|
96
|
+
close();
|
|
97
|
+
blocks.push({ start, end, kind: inFence ? "code" : "line" });
|
|
98
|
+
continue;
|
|
99
|
+
}
|
|
100
|
+
// HTML or MDX written inline in a text file (AgentID's llms-full.txt has a table as `<td className="...">Can the
|
|
101
|
+
// agent sign up ...</td> <td ...>`): each piece of text between block tags is its own block, as in an HTML page.
|
|
102
|
+
if (INLINE_BLOCK_TAG.test(line)) {
|
|
103
|
+
close();
|
|
104
|
+
let kind = "paragraph";
|
|
105
|
+
let from = 0;
|
|
106
|
+
for (const m of line.matchAll(/<[^>]+>/g)) {
|
|
107
|
+
pushPiece(blocks, line, start, from, m.index, kind);
|
|
108
|
+
const tag = INLINE_BLOCK_TAG_NAME.exec(m[0]);
|
|
109
|
+
if (tag) kind = tag[1] ? "paragraph" : TAG_KIND[tag[2].toLowerCase()] || "paragraph";
|
|
110
|
+
from = m.index + m[0].length;
|
|
111
|
+
}
|
|
112
|
+
pushPiece(blocks, line, start, from, line.length, kind);
|
|
113
|
+
continue;
|
|
114
|
+
}
|
|
115
|
+
if (HEADING_LINE.test(line) && !COMMENT_TEXT.test(line)) {
|
|
116
|
+
close();
|
|
117
|
+
blocks.push({ start, end, kind: "heading" });
|
|
118
|
+
continue;
|
|
119
|
+
}
|
|
120
|
+
if (TABLE_LINE.test(line)) {
|
|
121
|
+
close();
|
|
122
|
+
for (const cell of line.matchAll(/[^|]+/g)) if (cell[0].trim() && !/^[\s:-]+$/.test(cell[0])) blocks.push({ start: start + cell.index, end: start + cell.index + cell[0].length, kind: "cell" });
|
|
123
|
+
continue;
|
|
124
|
+
}
|
|
125
|
+
if (ITEM_LINE.test(line)) {
|
|
126
|
+
close();
|
|
127
|
+
open = { start, end, kind: "item" };
|
|
128
|
+
continue;
|
|
129
|
+
}
|
|
130
|
+
if (open) open.end = end;
|
|
131
|
+
else open = { start, end, kind: "paragraph" };
|
|
132
|
+
}
|
|
133
|
+
close();
|
|
134
|
+
return blocks;
|
|
135
|
+
}
|
|
136
|
+
|
|
137
|
+
const blockCache = new WeakMap();
|
|
138
|
+
// The page's blocks over the text the rules read (htmlToText for HTML, the body itself otherwise).
|
|
139
|
+
export function blocksOf(body, role) {
|
|
140
|
+
if (blockCache.has(body)) return blockCache.get(body);
|
|
141
|
+
const blocks = body.html ? htmlBlocks(body.text) : isJson(body.text) ? null : textBlocks(body.text, { lineMode: role === "openapi" || role === "sitemap" || role === "robots" });
|
|
142
|
+
blockCache.set(body, blocks);
|
|
143
|
+
return blocks;
|
|
144
|
+
}
|
|
145
|
+
|
|
146
|
+
const isJson = (text) => /^\s*[[{]/.test(text);
|
|
147
|
+
|
|
148
|
+
// Where one sentence ends inside a block: terminal punctuation, any closing quote, bracket or Markdown mark, then
|
|
149
|
+
// space and a character that can start a sentence. Not after an abbreviation ("e.g.", "vs.") or an initial.
|
|
150
|
+
const BOUNDARY = /[.!?]["'”’)\]*_`]*(?=\s+[^\sa-z])/g;
|
|
151
|
+
const ABBREVIATION = /\b(?:e\.g|i\.e|etc|vs|cf|approx|incl|Inc|Ltd|Co|Mr|Mrs|Ms|Dr|St|No|U\.S)$/;
|
|
152
|
+
const TERMINAL = /[.!?]["'”’)\]*_`]*$/;
|
|
153
|
+
|
|
154
|
+
function sentencesIn(s) {
|
|
155
|
+
const out = [];
|
|
156
|
+
let from = 0;
|
|
157
|
+
for (const m of s.matchAll(BOUNDARY)) {
|
|
158
|
+
if (m[0][0] === "." && ABBREVIATION.test(s.slice(0, m.index))) continue;
|
|
159
|
+
const end = m.index + m[0].length;
|
|
160
|
+
out.push([from, end]);
|
|
161
|
+
from = end;
|
|
162
|
+
}
|
|
163
|
+
out.push([from, s.length]);
|
|
164
|
+
return out.map(([a, b]) => {
|
|
165
|
+
while (a < b && /\s/.test(s[a])) a += 1;
|
|
166
|
+
while (b > a && /\s/.test(s[b - 1])) b -= 1;
|
|
167
|
+
return [a, b];
|
|
168
|
+
}).filter(([a, b]) => b > a);
|
|
169
|
+
}
|
|
170
|
+
|
|
171
|
+
// Markdown marks at the start of a block are layout, not words: "- ", "## ", "> ", "1. ".
|
|
172
|
+
// A "#" that opens each line of a comment block is layout too.
|
|
173
|
+
const tidy = (s) => s.replace(/^\s*(?:#{1,6}\s+|>\s*|[-*+]\s+|\d+[.)]\s+)/, "").replace(/\n[ \t]*#(?=\s)/g, "\n").replace(/\s+/g, " ").trim();
|
|
174
|
+
|
|
175
|
+
// A div's text with no sentence in it is often several links or buttons side by side ("Start Sending Free View B2B
|
|
176
|
+
// Plans ($14/mo) Read the Docs"). Its phrase is then only the inline elements the words are in: split at every tag
|
|
177
|
+
// but comments, the same way htmlBlocks splits a page, so the pieces line up with the block's text.
|
|
178
|
+
function inlinePiece(raw, start, end) {
|
|
179
|
+
const pieces = [];
|
|
180
|
+
let offset = 0;
|
|
181
|
+
for (const part of raw.split(/<(?!!--)[^>]+>/)) {
|
|
182
|
+
const text = decode(part).replace(/\s+/g, " ").trim();
|
|
183
|
+
if (!text) continue;
|
|
184
|
+
pieces.push({ start: offset, end: offset + text.length, text });
|
|
185
|
+
offset += text.length + 1;
|
|
186
|
+
}
|
|
187
|
+
const hit = pieces.filter((p) => p.end > start && p.start < end);
|
|
188
|
+
return hit.length ? hit.map((p) => p.text).join(" ") : null;
|
|
189
|
+
}
|
|
190
|
+
|
|
191
|
+
// { quote, label? } for the words at [start, end) of a block's text, or null when they cross a sentence pair or the
|
|
192
|
+
// block is a long run with no sentence in it.
|
|
193
|
+
// Sentences are looked for only within WINDOW characters of the match: a 2 MB page on one line would otherwise be split
|
|
194
|
+
// end to end for every match. A sentence that reaches the window's edge is longer than any quote, so it is none.
|
|
195
|
+
const WINDOW = 1000;
|
|
196
|
+
function fromBlock(s, kind, start, end, raw = null) {
|
|
197
|
+
const lo = Math.max(0, start - WINDOW);
|
|
198
|
+
const hi = Math.min(s.length, end + WINDOW);
|
|
199
|
+
const near = s.slice(lo, hi);
|
|
200
|
+
const hit = sentencesIn(near).filter(([a, b]) => b > start - lo && a < end - lo);
|
|
201
|
+
if (!hit.length || hit.length > 2) return null;
|
|
202
|
+
const a = hit[0][0];
|
|
203
|
+
const b = hit.at(-1)[1];
|
|
204
|
+
if ((a === 0 && lo > 0) || (b === near.length && hi < s.length)) return null;
|
|
205
|
+
const piece = tidy(near.slice(a, b));
|
|
206
|
+
if (!piece) return null;
|
|
207
|
+
if (TERMINAL.test(piece) && kind !== "code" && kind !== "cell" && kind !== "heading") return piece.length <= SENTENCE_MAX ? { quote: piece } : null;
|
|
208
|
+
// A Markdown paragraph or list item is one element as its author wrote it, so it is quoted whole even without a full
|
|
209
|
+
// stop (llms.txt lines often have none). A div's text is cut to the inline elements the words are in.
|
|
210
|
+
const phrase = raw !== null && kind === "phrase" ? tidy(inlinePiece(raw, start, end) || "") : piece;
|
|
211
|
+
const max = kind === "paragraph" || kind === "item" ? SENTENCE_MAX : PHRASE_MAX;
|
|
212
|
+
if (!phrase || phrase.length > max) return null;
|
|
213
|
+
return { quote: phrase, label: LABEL[kind] || LABEL.phrase };
|
|
214
|
+
}
|
|
215
|
+
|
|
216
|
+
// A JSON string at [start, end): its text, labelled with its key (description, summary), or for a key itself, "Key"
|
|
217
|
+
// ("OpenAPI path" for an OpenAPI path).
|
|
218
|
+
function fromJson(text, start, end, role) {
|
|
219
|
+
let qs = start;
|
|
220
|
+
while (qs >= 0 && !(text[qs] === '"' && text[qs - 1] !== "\\")) qs -= 1;
|
|
221
|
+
let qe = end;
|
|
222
|
+
while (qe < text.length && !(text[qe] === '"' && text[qe - 1] !== "\\")) qe += 1;
|
|
223
|
+
if (qs < 0 || qe >= text.length || qe - qs > 2000) return null;
|
|
224
|
+
// Inside a string only when an odd number of unescaped quotes come before the opening one plus it.
|
|
225
|
+
const before = text.slice(0, qs + 1).match(/(?<!\\)"/g)?.length || 0;
|
|
226
|
+
if (before % 2 === 0) return null;
|
|
227
|
+
const raw = text.slice(qs + 1, qe).replace(/\\n/g, " ").replace(/\\"/g, '"').replace(/\\\//g, "/").replace(/\\\\/g, "\\");
|
|
228
|
+
const isKey = /^\s*:/.test(text.slice(qe + 1, qe + 20));
|
|
229
|
+
if (isKey) return raw.length <= PHRASE_MAX ? { quote: raw, label: role === "openapi" && raw.startsWith("/") ? "OpenAPI path" : "Key" } : null;
|
|
230
|
+
const key = /"([^"\\]{1,40})"\s*:\s*$/.exec(text.slice(Math.max(0, qs - 60), qs))?.[1] || "Value";
|
|
231
|
+
const inner = fromBlock(raw, "phrase", start - qs - 1, end - qs - 1);
|
|
232
|
+
return inner ? { quote: inner.quote, label: key } : null;
|
|
233
|
+
}
|
|
234
|
+
|
|
235
|
+
// The quote for a match at [start, end) of a page's text, or null when no clean quote holds it: the match runs across
|
|
236
|
+
// two elements of the page, or its element is a long run with no sentence. A caller with a null tries its next match.
|
|
237
|
+
export function quoteAt(body, text, start, end, role) {
|
|
238
|
+
if (!body.html && isJson(body.text)) return fromJson(text, start, end, role);
|
|
239
|
+
const blocks = blocksOf(body, role);
|
|
240
|
+
const block = blocks.find((b) => b.start <= start && end <= b.end);
|
|
241
|
+
if (!block) return null;
|
|
242
|
+
return fromBlock(text.slice(block.start, block.end), block.kind, start - block.start, end - block.start, block.raw ?? null);
|
|
243
|
+
}
|
|
244
|
+
|
|
245
|
+
// The block holding [start, end), and the heading above it on the page, for rules that read what a sentence is about.
|
|
246
|
+
export function blockContext(body, text, start, end, role) {
|
|
247
|
+
if (!body.html && isJson(body.text)) return null;
|
|
248
|
+
const blocks = blocksOf(body, role);
|
|
249
|
+
const i = blocks.findIndex((b) => b.start <= start && end <= b.end);
|
|
250
|
+
if (i < 0) return null;
|
|
251
|
+
let h = i - 1;
|
|
252
|
+
while (h >= 0 && blocks[h].kind !== "heading") h -= 1;
|
|
253
|
+
const slice = (b) => (b ? text.slice(b.start, b.end) : "");
|
|
254
|
+
return { block: slice(blocks[i]), heading: h >= 0 ? slice(blocks[h]) : "", previous: slice(blocks[i - 1]) };
|
|
255
|
+
}
|
package/src/interface/schema.js
CHANGED
|
@@ -23,8 +23,8 @@ export const SCHEMA_DOC = {
|
|
|
23
23
|
authentication_requirement: "{ value: none|required|mixed|not_declared|unknown, reason, basedOn, rule, operations: [{ operation, value, reason, basedOn, rule, source: operation|global|absent, schemeNames }] }; operation security overrides global security; none requires an explicit operation [] or anonymous {} alternative; absent or empty global security is not_declared; unresolved declarations or incomplete operation coverage are unknown",
|
|
24
24
|
pricing: "model, plans, observedPrices, agentCanDetermineCost, ambiguities, agentPurchase. agentPurchase is null or { rule: agent_buys_through_api|payment_required_approval|pay_per_request_own_api|checkout_protocol_own_api, rules, approval: once|none_documented, call, protocol: x402|MPP|ACP|AP2|UCP|null, evidence: [{ obs, quote }] }: a way for an agent to buy from the product through an API, as the product's own pages describe it (an agent buying with a purchase call, a cap or an approval; 402 payment_required with an approval link; x402 or MPP on a call to the product's own API; an ACP or UCP checkout session call on the product's own API, or a UCP business profile at /.well-known/ucp). protocol names the payment or checkout protocol the quoted sentence names, when it names one. approval once means a person approves first and the agent buys after that. A documented claim, never a result.",
|
|
25
25
|
machineAccess: "robots, llms.txt, agent.json facts plus aiCrawlersAllowed, hasAgentReadableIndex and agentAuthDiscoverable",
|
|
26
|
-
onboarding: "{ patterns: [{ id: try_then_claim|limited_until_claimed|agent_is_customer|agent_identity|existing_account|pay_per_request, name, status: documented|inferred, reason, evidence: [{ obs, quote }], humanBoundary, needs: [inbox|cli|wallet|agent_identity], provider?: AgentID }], apiHosts: [{ domain, host, example, evidence }], primary, reason }. apiHosts are API URLs on other domains that the product's agent-facing pages point to. Ways an agent can start, as the public docs describe them; needs lists what testing that way end to end requires beyond HTTP requests, which a given deployment may or may not provide. provider names the identity provider an agent_identity pattern uses, when the docs name one the detector knows (AgentID). A pattern is a documented claim, never a result.",
|
|
27
|
-
funnel: "{ headline, passed, stepsPassing, documented, verified, of, stopsAt, path, paths, steps: [{ id: discover|understand|signup|access|use|pay|manage, name, area: web|onboarding|monetization, question, state: agent_did|agent_can|handoff|needs_person|blocked|not_checked, reason, fix?, basedOn, live?, evidence? }], fixes }. The seven steps an agent takes, from public evidence only. handoff means a person steps in at a documented point and the agent carries on, which counts as passing. agent_did appears only when the latest test run for that step on this product passed; a blocked test run sets blocked. live names that test. evidence lists, newest first, what else set or touched the step: { kind: test|pay|signal|stuck, at, reason, runId?, outcome?, count? }; a test or payment or the product's own signals can decide the state, a stuck report only attaches. documented counts agent_can and handoff steps, verified counts agent_did steps; the headline gives both and never calls a documented step verified. passed counts the passing steps before the first one that does not pass; stepsPassing counts every passing step (documented plus verified), the total the headline gives.",
|
|
26
|
+
onboarding: "{ patterns: [{ id: try_then_claim|limited_until_claimed|agent_is_customer|agent_identity|existing_account|pay_per_request, name, status: documented|inferred, reason, evidence: [{ obs, quote }], humanBoundary, needs: [inbox|cli|wallet|agent_identity], needsEvidence?: { <need>: [{ obs, quote, label? }] }, provider?: AgentID }], apiHosts: [{ domain, host, example, evidence }], signupElsewhere: { domain, url, evidence } | null, primary, reason }. apiHosts are API URLs on other domains that the product's agent-facing pages point to. signupElsewhere is a signup page on another company's site that those pages send the agent to; it is not read. Ways an agent can start, as the public docs describe them; needs lists what testing that way end to end requires beyond HTTP requests, which a given deployment may or may not provide. provider names the identity provider an agent_identity pattern uses, when the docs name one the detector knows (AgentID). A pattern is a documented claim, never a result.",
|
|
27
|
+
funnel: "{ headline, passed, stepsPassing, documented, verified, of, stopsAt, path, paths, steps: [{ id: discover|understand|signup|access|use|pay|manage, name, area: web|onboarding|monetization, question, state: agent_did|agent_can|handoff|needs_person|blocked|not_checked|not_applicable, reason, fix?, basedOn, quotes?, live?, evidence? }], applicable, fixes }. not_applicable: the product's machine-readable files say the step has nothing to do (Pay when every plan in pricing.json costs 0 and no purchase is documented; Manage when that is one plan); it neither passes nor stops the walk, and applicable counts the other steps. quotes: [{ for, obs, quote, label? }], the sentences behind a claim that the product needs or requires something; a need with no quote is not reported. quote is one or two whole sentences from one element of the page; label is set when the quote is an element that is not a sentence (Heading, Table cell, List item, Button, Code, Text, Phrase, OpenAPI path, pricing.json, or a JSON key such as description). The seven steps an agent takes, from public evidence only. handoff means a person steps in at a documented point and the agent carries on, which counts as passing. agent_did appears only when the latest test run for that step on this product passed; a blocked test run sets blocked. live names that test. evidence lists, newest first, what else set or touched the step: { kind: test|pay|signal|stuck, at, reason, runId?, outcome?, count? }; a test or payment or the product's own signals can decide the state, a stuck report only attaches. documented counts agent_can and handoff steps, verified counts agent_did steps; the headline gives both and never calls a documented step verified. passed counts the passing steps before the first one that does not pass; stepsPassing counts every passing step (documented plus verified), the total the headline gives.",
|
|
28
28
|
unknowns: "every field whose verdict is unknown or whose fact is null, with the reason",
|
|
29
29
|
limits: "what this run could not do (no JS rendering, no login, no model interpretation)",
|
|
30
30
|
},
|