@relvehq/agent-ready 0.2.1 → 0.3.0

package/README.md

@@ -8,10 +8,10 @@ they got before.
 - **Serve**: Markdown on `Accept: text/markdown`, on `<page>.md` and on
   `?format=md`, plus a generated [`/llms.txt`](https://llmstxt.org). No AI
   runs per request: it is plain code, in milliseconds.
-- **Refine** (optional, with your own prompt files): an AI model writes a
-  better Markdown file for each page ahead of time, a script checks every
-  number and URL against the page,
-  and unchanged pages are skipped on the next run.
+- **Refine** (optional, with your own OpenAI key): one command writes a
+  better Markdown file for each page ahead of time with Relve's prompt, a
+  check compares every number and URL against the page, and unchanged pages
+  are skipped (free) on the next run.
 - **Proof**: a free checker that scores any site out of 100, and a dashboard
   of which AI agents read which pages.
 
@@ -168,28 +168,49 @@ Most CDNs (CloudFront included) don't vary their cache on `Accept`, so:
   HTML. Safe, just not Markdown. `.md` URLs always work; reliable negotiation
   needs `Accept` in the CDN cache key.
 
-## Refine: better Markdown, written ahead of time
+## Refine: better Markdown with your own OpenAI key
+
+One command, run from your project root with `OPENAI_API_KEY` in `.env.local`,
+`.env` or the shell:
 
 ```
-npx agent-ready refine --site https://your-site.com --prompts ./prompts --sitemap --limit 20 --dry-run
-npx agent-ready refine --site https://your-site.com --prompts ./prompts --site-name "Your Name" --sitemap --limit 20
+npx agent-ready refine --site https://your-site.com --sitemap --dry-run
+npx agent-ready refine --site https://your-site.com --sitemap
 ```
 
-For each page: fetch it, extract its text, links, JSON-LD and a fingerprint;
-skip it if the fingerprint matches the saved file; otherwise fill the prompt,
-call the model once, check the result, retry once with the problems listed,
-and save `<out>/<path>.md` plus `<out>/<path>.json` (default out:
-`.agent-ready/md`). `--dry-run` only reads pages: no model call, nothing written.
-
-| Setting | |
+For each page the command fetches the HTML itself and sends it to the Relve
+API (`relvehq.com`) with your key in a header. Relve's prompt runs on Relve's
+server: it extracts the page, calls OpenAI with your key, checks the result
+(every number and URL must be on the page; one retry), and returns the
+Markdown. The command saves `<out>/<path>.md` plus `<out>/<path>.json`
+(default out: `.agent-ready/md`).
+
+- **Cost:** your own OpenAI usage, one model call per changed page (roughly
+  a few thousand tokens each with the default `gpt-4o-mini`). Relve charges nothing.
+- **Your key:** used for that call only. Relve does not store or log it, and
+  it is never written to the output files.
+- **Re-run when content changes.** Each saved `.json` holds the page's
+  fingerprint; it is sent along, and an unchanged page comes back as
+  `unchanged` with no model call (free). `--force` rewrites everything.
+- **Optional in CI or deploy:** run the command before `next build` (with
+  `OPENAI_API_KEY` as a secret) and commit or ship `.agent-ready/md`.
+
+| Option | |
 |---|---|
-| `OPENAI_API_KEY` | OpenAI, default model `gpt-4o` |
-| `ANTHROPIC_API_KEY` | Claude, default model `claude-opus-5-5` |
-| `AGENT_READY_MODEL` | any other model id, e.g. `gpt-4o-mini` |
+| `--sitemap` / `--paths /a,/b` | which pages (sitemap at `<site>/sitemap.xml`) |
+| `--model` | `gpt-4o-mini` (default, fastest), `gpt-4o`, `gpt-4.1`, `gpt-4.1-mini` |
+| `--site-name` | your brand name, used in the documents |
+| `--out`, `--limit`, `--concurrency` (2), `--force`, `--dry-run` | `--dry-run` lists the pages: no API call, nothing written |
+| `AGENT_READY_REFINE_ENDPOINT` | another API URL (testing) |
+
+The run ends with `written / unchanged / rejected / errors` and the token
+total. Rejected and failed pages are listed with their code (for example
+`too_little_text`) and nothing is saved for them; the exit code is then 1.
+If OpenAI rejects the key or your account is out of quota, the run stops at
+the first page and says so.
 
-Keys are read from `.env.local` and `.env`. Serve the files from the route
-file (`fileStore` is Node only, so keep it out of `agent-ready.config.ts`,
-which the Edge middleware imports):
+Serve the files from the route file (`fileStore` is Node only, so keep it
+out of `agent-ready.config.ts`, which the Edge middleware imports):
 
 ```ts
 // app/%5F%5Fmd/[[...path]]/route.ts
@@ -209,10 +230,14 @@ required section is missing, it has more or fewer than one H1, HTML tags, a
 wrong fingerprint, or it is over the token budget. It cannot catch a reworded
 claim, so read a sample before you switch on.
 
-**The prompt.** The package ships none. Point `refine` at a folder of prompt
-files with `--prompts <dir>` (several: `a,b`) or the `AGENT_READY_PROMPTS`
-variable (`.env.local`, `.env` or the shell); without one it stops before
-fetching anything. A prompt file is Markdown with `## SYSTEM`, `## USER`
+**Advanced: your own prompts.** The package ships no prompt. With
+`--prompts <dir>` (several: `a,b`) or the `AGENT_READY_PROMPTS` variable
+(`.env.local`, `.env` or the shell), `refine` runs your prompt files on your
+machine instead of calling the Relve API, with `OPENAI_API_KEY` (default
+model `gpt-4o`) or `ANTHROPIC_API_KEY` (default `claude-opus-5-5`);
+`AGENT_READY_MODEL` picks another model. The steps are the same: fetch,
+extract, skip unchanged, fill the prompt, one model call, check, one retry,
+save. A prompt file is Markdown with `## SYSTEM`, `## USER`
 (`{{PAGE_URL}}`, `{{PAGE_TEXT}}`, `{{PAGE_LINKS}}`, `{{JSON_LD}}`,
 `{{SOURCE_HASH}}`) and a `## Settings` table (`page_type`, `version`,
 `required_sections`, `token_budget`). A `page_type` of `page` takes every
@@ -327,8 +352,9 @@ if (d.markdown) {
 ```
 agent-ready init   [--stack next|node|wordpress] [--dry-run] [--force] [--cwd <dir>]
 agent-ready check  <url> [--json]
-agent-ready refine --site <origin> (--paths /a,/b | --sitemap) --prompts <dir> [--site-name <name>]
+agent-ready refine --site <origin> (--paths /a,/b | --sitemap) [--model <id>] [--site-name <name>]
                    [--out <dir>] [--limit <n>] [--force] [--concurrency <n>] [--dry-run]
+                   [--prompts <dir>]   (advanced: your own prompts, run locally)
 agent-ready stats  --log <hits.jsonl> [--days <n>] [--json]
 ```
 

package/SETUP.md

@@ -209,7 +209,7 @@ The plugin tells WP Rocket not to cache `.md` and `?format=md` responses, and ma
 
 ### Refined files on WordPress
 
-Relve sends a zip of refined files (`refined-md.zip`). On Settings > Agent Ready, under "Refined files from Relve", choose it and click "Upload zip" (up to 20 MB, 2 MB per file). The page shows how many files are stored. Each upload replaces the whole stored set, so always upload the latest zip Relve sends; "Clear refined files" removes them all. See "Add refined files from Relve" below for how to check it.
+With your own OpenAI key, the plugin can write a better Markdown file for each post itself: see the plugin's settings page (Settings > Agent Ready) and its readme. You can also upload a zip of `.md` files you made with `npx agent-ready refine` (zip the `.agent-ready/md` folder): under the refined files section, choose it and click "Upload zip" (up to 20 MB, 2 MB per file). Each upload replaces the whole stored set; "Clear refined files" removes them all. See "Add refined files" below for how to check it.
 
 ## Test it locally
 
@@ -265,12 +265,29 @@ npx @relvehq/agent-ready check https://your-site.com
 
 Most CDNs do not put `Accept` in the cache key. If the CDN already holds the HTML for a page, an `Accept: text/markdown` request may get that HTML. That is safe, just not Markdown. `.md` URLs and `/llms.txt` always work, because they are their own URLs. For reliable `Accept` negotiation, add the `Accept` header to the CDN cache key.
 
-## Add refined files from Relve (optional)
+## Add refined files: better Markdown with your own OpenAI key (optional)
 
-Relve can write a better Markdown file for each page ahead of time and send you a folder of `.md` and `.json` files.
+One command writes a better Markdown file for each page ahead of time, with Relve's prompt and your own OpenAI key (BYOK).
 
-1. Commit the folder at `.agent-ready/md` in your project root. Example: `.agent-ready/md/index.md`, `.agent-ready/md/pricing.md`.
-2. Tell the package to read it. The store reads files with Node, so put it where only Node code runs.
+1. Put your key in `.env.local`, `.env` or the shell as `OPENAI_API_KEY`, then run from your project root:
+
+```
+npx agent-ready refine --site https://your-site.com --sitemap --dry-run
+npx agent-ready refine --site https://your-site.com --sitemap
+```
+
+`--dry-run` lists the pages and calls nothing. The real run fetches each page's HTML and sends it to the Relve API on relvehq.com with your key. Relve's prompt runs on Relve's server, calls OpenAI with your key, checks the result (every number and URL must be on the page) and returns the Markdown. The files land in `.agent-ready/md`: `index.md`, `pricing.md`, each with a `.json` next to it (fingerprint, model, tokens, date).
+
+- What it costs: your own OpenAI usage, one model call per changed page. Relve charges nothing.
+- Your key: it passes through Relve's server for that call only. Relve never stores or logs it, and it is never written to the output files.
+- Options: `--paths /a,/b` instead of `--sitemap`, `--model gpt-4o` (also `gpt-4.1`, `gpt-4.1-mini`; default `gpt-4o-mini`, the fastest), `--site-name "Your Brand"`, `--limit 20`, `--out <dir>`, `--force`.
+- The run ends with a summary (written, unchanged, rejected, errors, tokens). Rejected or failed pages are listed with a code (for example `too_little_text`) and nothing is saved for them; they keep the plain `convert` Markdown.
+- Re-run it when your content changes. Unchanged pages are skipped with no model call, so they are free. You can add the command to CI or your deploy (before the build, with `OPENAI_API_KEY` as a secret).
+
+Advanced: your own prompts. `--prompts <dir>` (or `AGENT_READY_PROMPTS`) runs your own prompt files on your machine instead of calling the Relve API. See the package README.
+
+2. Commit the folder at `.agent-ready/md` in your project root (or generate it in your deploy).
+3. Tell the package to read it. The store reads files with Node, so put it where only Node code runs.
 
 Next.js: in the route file (`app/%5F%5Fmd/[[...path]]/route.ts`), not in `agent-ready.config.ts`. The config file is also loaded by the middleware, which runs on the Edge runtime and has no file system.
 
@@ -297,7 +314,7 @@ import { fileStore } from "@relvehq/agent-ready/refine/file-store";
 app.use(agentReadyNode(createAgentReady({ site: "https://your-site.com", store: fileStore(".agent-ready/md") })));
 ```
 
-3. Redeploy. Check a refined page:
+4. Redeploy. Check a refined page:
 
 ```
 curl -i https://your-site.com/pricing.md
@@ -305,7 +322,7 @@ curl -i https://your-site.com/pricing.md
 
 The response should carry `x-agent-ready-source: stored`. Pages without a refined file keep working as before (`convert`).
 
-WordPress (Path 5): no folder to commit. Upload `refined-md.zip` on Settings > Agent Ready (see "Refined files on WordPress"), then run the same `curl` on a refined post's `.md` URL. Pages without a refined file keep `x-agent-ready-source: native`.
+WordPress (Path 5): no folder to commit. Use the plugin's OpenAI key setting or upload a zip (see "Refined files on WordPress"), then run the same `curl` on a refined post's `.md` URL. Pages without a refined file keep `x-agent-ready-source: native`.
 
 ## Turn it off
 

package/dist/bin/agent-ready.js

@@ -5,12 +5,14 @@
 //   agent-ready init   [--stack next|node|wordpress] [--dry-run] [--force] [--cwd <dir>] [--pkg <name>]
 //   agent-ready check  <url> [--json]
 //   agent-ready stats  --log <hits.jsonl> [--days <n>] [--json]
-//   agent-ready refine --site <origin> (--paths a,b | --sitemap) --prompts <dir>[,<dir>]
+//   agent-ready refine --site <origin> (--paths a,b | --sitemap) [--model <id>]
 //                      [--site-name <name>] [--out <dir>] [--limit <n>] [--force]
-//                      [--concurrency <n>] [--dry-run]
+//                      [--concurrency <n>] [--dry-run] [--prompts <dir>[,<dir>]]
 //
-// The package ships no prompt. refine reads the folders in --prompts, else in
-// AGENT_READY_PROMPTS, and stops before fetching anything when neither is set.
+// The package ships no prompt. By default refine sends each page to the Relve
+// API (refine/remote.ts) with the caller's OPENAI_API_KEY; Relve's prompt runs
+// there. With --prompts or AGENT_READY_PROMPTS it runs those prompt files
+// locally instead (the original mode).
 //
 // Thin: argument parsing and the real file system; the work is in cli/init.ts
 // and refine/*. NODE ONLY.
@@ -62,16 +64,19 @@ const USAGE = [
     "                     detects Next.js, Node servers (Express, Fastify, Nest, Nitro...) and WordPress;",
     "                     --stack skips detection, --force replaces an installed WordPress plugin",
     `                     guide: ${init_1.ONBOARDING_URL}`,
-    "  agent-ready refine --site <origin> (--paths /a,/b | --sitemap) --prompts <dir>[,<dir>]",
+    "  agent-ready refine --site <origin> (--paths /a,/b | --sitemap) [--model gpt-4o|gpt-4o-mini|gpt-4.1|gpt-4.1-mini]",
     "                     [--site-name <name>] [--out <dir>] [--limit <n>] [--force] [--concurrency <n>] [--dry-run]",
+    "                     better Markdown files in .agent-ready/md, written with Relve's prompt and your own OpenAI key",
     "",
     "  agent-ready check  <url> [--json]                 how agent-ready is a site? (read-only)",
     "  agent-ready stats  --log <hits.jsonl> [--days <n>] [--json]   agent visits summary",
     "",
-    "refine needs a folder of prompt files: --prompts, or AGENT_READY_PROMPTS. The package ships none.",
+    "refine needs OPENAI_API_KEY (read from .env.local / .env or the shell) unless --dry-run. Each page goes to",
+    "relvehq.com with the key; the key is used for that call only and never stored. Unchanged pages are free.",
+    "AGENT_READY_REFINE_ENDPOINT overrides the API URL.",
     "",
-    "refine needs OPENAI_API_KEY or ANTHROPIC_API_KEY (read from .env.local / .env) unless --dry-run.",
-    "AGENT_READY_MODEL picks the model (default gpt-4o with OpenAI, claude-opus-5-5 with Claude).",
+    "Advanced: --prompts <dir>[,<dir>] (or AGENT_READY_PROMPTS) runs your own prompt files locally instead, with",
+    "OPENAI_API_KEY or ANTHROPIC_API_KEY; AGENT_READY_MODEL picks the model (default gpt-4o / claude-opus-5-5).",
 ].join("\n");
 function nodeFs(root) {
     const abs = (p) => (0, node_path_1.resolve)(root, p);
@@ -215,11 +220,13 @@ async function refineCommand(rest, flag, d) {
     // Several folders may be given ("a,b"). Checked before any page is fetched.
     const promptSource = flag("--prompts") ?? env.AGENT_READY_PROMPTS;
     const promptDirs = (promptSource ?? "").split(",").map((p) => p.trim()).filter(Boolean);
-    if (!promptDirs.length) {
-        log("agent-ready refine: no prompts. The package ships none: name a folder of prompt files with --prompts <dir>[,<dir>]" +
-            " or AGENT_READY_PROMPTS (in .env.local, .env or the shell). Each file needs ## SYSTEM, ## USER and ## Settings.");
+    if (rest.includes("--prompts") && !promptDirs.length) {
+        log("agent-ready refine: --prompts needs a folder of prompt files (--prompts <dir>[,<dir>]).");
         return 1;
     }
+    // No prompt folder: Relve's prompt, run on relvehq.com with the caller's key.
+    if (!promptDirs.length)
+        return remoteRefineCommand(rest, flag, site, env, f, log);
     if (!flag("--prompts"))
         log(`agent-ready refine: prompts from AGENT_READY_PROMPTS (${promptDirs.join(", ")})`);
     const siteName = flag("--site-name");
@@ -319,6 +326,93 @@ async function refineCommand(rest, flag, d) {
     log(`model usage: ${report.usage.inputTokens} input + ${report.usage.outputTokens} output tokens`);
     return c.error || c.rejected ? 2 : 0;
 }
+/**
+ * `refine` without --prompts: each page's HTML goes to the Relve API with the
+ * caller's OPENAI_API_KEY; Relve's prompt runs there. Checks happen before any
+ * fetch; the key is never printed.
+ */
+async function remoteRefineCommand(rest, flag, site, env, f, log) {
+    const { refineRemote, stopMessage, savedFingerprint, DEFAULT_REFINE_ENDPOINT, REFINE_MODELS } = await Promise.resolve().then(() => __importStar(require("../refine/remote")));
+    const dryRun = rest.includes("--dry-run");
+    const model = flag("--model");
+    if (model !== undefined && !REFINE_MODELS.includes(model)) {
+        log(`agent-ready refine: --model must be one of ${REFINE_MODELS.join(", ")}`);
+        return 1;
+    }
+    const endpoint = env.AGENT_READY_REFINE_ENDPOINT?.trim() || DEFAULT_REFINE_ENDPOINT;
+    if (!/^https?:\/\//.test(endpoint)) {
+        log("agent-ready refine: AGENT_READY_REFINE_ENDPOINT must be an http(s) URL.");
+        return 1;
+    }
+    const apiKey = env.OPENAI_API_KEY?.trim();
+    if (!apiKey && !dryRun) {
+        log("agent-ready refine: set OPENAI_API_KEY (in .env.local, .env or the shell) to your OpenAI key, or use --dry-run." +
+            " Advanced: --prompts <dir> runs your own prompt files locally.");
+        return 1;
+    }
+    let paths = [];
+    if (flag("--paths"))
+        paths = flag("--paths").split(",").map((p) => p.trim()).filter(Boolean);
+    else if (rest.includes("--sitemap")) {
+        const { sitemapEntries } = await Promise.resolve().then(() => __importStar(require("../core/sitemap")));
+        const entries = await sitemapEntries({ fetch: f, limit: 5000 })({ request: new Request(site + "/llms.txt") });
+        paths = entries.map((e) => e.url);
+    }
+    else {
+        log(USAGE);
+        return 1;
+    }
+    paths = paths.filter((p) => p.startsWith("/"));
+    const limit = Number(flag("--limit") ?? "0");
+    if (limit > 0)
+        paths = paths.slice(0, limit);
+    if (!paths.length) {
+        log("agent-ready refine: no pages (check --paths, or that the site has a /sitemap.xml).");
+        return 1;
+    }
+    const outDir = (0, node_path_1.resolve)(flag("--out") ?? ".agent-ready/md");
+    const force = rest.includes("--force");
+    if (dryRun) {
+        // No API call, no page fetch, nothing written.
+        log(`agent-ready refine --dry-run: ${paths.length} page(s) would go to ${endpoint}${model ? ` (model ${model})` : ""}`);
+        for (const path of paths) {
+            const saved = force ? null : await savedFingerprint(outDir, path);
+            log(`  ${(saved ? "would check" : "would send").padEnd(11)} ${path}${saved ? "  (saved file; free if unchanged)" : ""}`);
+        }
+        if (!apiKey)
+            log("Set OPENAI_API_KEY to run it for real.");
+        return 0;
+    }
+    log("Rewriting with Relve's prompt via relvehq.com using your OpenAI key (the key is used for each call only; Relve does not store it).");
+    const report = await refineRemote({
+        site,
+        paths,
+        apiKey: apiKey,
+        outDir,
+        endpoint,
+        model,
+        siteName: flag("--site-name"),
+        force,
+        concurrency: Number(flag("--concurrency") ?? "2") || 2,
+        fetch: f,
+        onResult: (it) => {
+            const extra = it.status === "ok"
+                ? `  ${it.tokens ?? 0} tokens${it.kind ? `  kind: ${it.kind}` : ""}`
+                : it.status === "rejected"
+                    ? `  ${(it.issues ?? []).slice(0, 5).join("; ")}`
+                    : it.status === "error"
+                        ? `  ${it.code}${it.message ? `: ${it.message}` : ""}`
+                        : "";
+            log(`  ${it.status.padEnd(10)} ${it.path}${extra}`);
+        },
+    });
+    const c = report.counts;
+    if (report.stopped)
+        log(`agent-ready refine: stopped early. ${stopMessage(report.stopped)}`);
+    log(`done: ${c.ok} written, ${c.unchanged} unchanged, ${c.rejected} rejected, ${c.error} errors${c.skipped ? `, ${c.skipped} not run` : ""} -> ${report.dir}`);
+    log(`tokens: ${report.tokens} in the written files (OpenAI bills its usage for each call to your account)`);
+    return c.error || c.rejected || report.stopped ? 1 : 0;
+}
 /**
  * Read `.env.local` then `.env` from the working directory, like Next.js, so
  * the keys a project already has (OPENAI_API_KEY, ANTHROPIC_API_KEY) just

package/dist/refine/remote.d.ts

@@ -0,0 +1,57 @@
+export declare const PACKAGE_VERSION = "0.3.0";
+export declare const DEFAULT_REFINE_ENDPOINT = "https://relvehq.com/api/agent-ready/v1/refine";
+export declare const REFINE_MODELS: readonly ["gpt-4o", "gpt-4o-mini", "gpt-4.1", "gpt-4.1-mini"];
+export declare const USER_AGENT = "agent-ready/0.3.0 (+https://relvehq.com/agent-ready)";
+export type RemoteStatus = "ok" | "unchanged" | "rejected" | "error" | "skipped";
+export interface RemoteItem {
+    path: string;
+    status: RemoteStatus;
+    /** Error code from the contract (e.g. "openai_auth", "too_little_text"), or a local one ("page_fetch", "network", "timeout", "bad_response"). */
+    code?: string;
+    message?: string;
+    /** Check messages on "rejected". */
+    issues?: string[];
+    tokens?: number;
+    kind?: string;
+    model?: string;
+    saved: boolean;
+}
+export interface RemoteReport {
+    items: RemoteItem[];
+    counts: Record<RemoteStatus, number>;
+    tokens: number;
+    /** The error code that stopped the run early, or null. */
+    stopped: string | null;
+    dir: string;
+}
+export interface RemoteOptions {
+    /** Public origin, e.g. "https://your-site.com". */
+    site: string;
+    paths: string[];
+    /** The caller's OpenAI key. Sent in X-OpenAI-Key, nothing else. */
+    apiKey: string;
+    /** Output folder. Default ".agent-ready/md". */
+    outDir?: string;
+    endpoint?: string;
+    model?: string;
+    siteName?: string;
+    /** Ignore saved fingerprints. */
+    force?: boolean;
+    /** Pages at a time. Default 2. */
+    concurrency?: number;
+    fetch?: typeof fetch;
+    /** Relve API timeout. Default 90 s. */
+    timeoutMs?: number;
+    /** Page fetch timeout. Default 30 s. */
+    pageTimeoutMs?: number;
+    /** Wait before retrying a 502/504. Default 2 s. */
+    retryDelayMs?: number;
+    now?: () => Date;
+    sleep?: (ms: number) => Promise<void>;
+    onResult?: (item: RemoteItem) => void;
+}
+/** The saved JSON's fingerprint, else one built from a local run's sourceHash, else null. */
+export declare function savedFingerprint(outDir: string, path: string): Promise<string | null>;
+export declare function refineRemote(opts: RemoteOptions): Promise<RemoteReport>;
+/** What a stop code means, for the CLI. */
+export declare function stopMessage(code: string): string;

package/dist/refine/remote.js

@@ -0,0 +1,238 @@
+"use strict";
+// Refine through the Relve API, with the caller's own OpenAI key (BYOK).
+// changes/agent-ready-byok-refine/contract.md is the wire format.
+//
+//   await refineRemote({ site, paths, apiKey, outDir })
+//
+// For each page: fetch its HTML here, read the saved fingerprint from
+// `<out>/<path>.json`, POST { url, html, siteName?, model?, previousFingerprint? }
+// to the endpoint with the key in `X-OpenAI-Key`, and save `ok` results as
+// `<out>/<path>.md` + `<out>/<path>.json` through fileStore, so the files are
+// served exactly like the ones local refine writes. Relve's prompt runs on
+// Relve's server; it never reaches this machine. The key is sent with each
+// call and is never written, logged or put in an error message here.
+//
+// NODE ONLY (fileStore, node:fs).
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || (function () {
+    var ownKeys = function(o) {
+        ownKeys = Object.getOwnPropertyNames || function (o) {
+            var ar = [];
+            for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
+            return ar;
+        };
+        return ownKeys(o);
+    };
+    return function (mod) {
+        if (mod && mod.__esModule) return mod;
+        var result = {};
+        if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
+        __setModuleDefault(result, mod);
+        return result;
+    };
+})();
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.USER_AGENT = exports.REFINE_MODELS = exports.DEFAULT_REFINE_ENDPOINT = exports.PACKAGE_VERSION = void 0;
+exports.savedFingerprint = savedFingerprint;
+exports.refineRemote = refineRemote;
+exports.stopMessage = stopMessage;
+const file_store_1 = require("./file-store");
+exports.PACKAGE_VERSION = "0.3.0";
+exports.DEFAULT_REFINE_ENDPOINT = "https://relvehq.com/api/agent-ready/v1/refine";
+exports.REFINE_MODELS = ["gpt-4o", "gpt-4o-mini", "gpt-4.1", "gpt-4.1-mini"];
+exports.USER_AGENT = `agent-ready/${exports.PACKAGE_VERSION} (+https://relvehq.com/agent-ready)`;
+/** Errors that mean every later call would fail the same way: the run stops. */
+const FATAL = new Set(["openai_auth", "openai_quota", "missing_key"]);
+/** The saved JSON's fingerprint, else one built from a local run's sourceHash, else null. */
+async function savedFingerprint(outDir, path) {
+    try {
+        const { readFile } = await Promise.resolve().then(() => __importStar(require("node:fs/promises")));
+        const meta = JSON.parse(await readFile(metaFile(outDir, path), "utf8"));
+        if (typeof meta.fingerprint === "string" && meta.fingerprint)
+            return meta.fingerprint;
+        if (typeof meta.sourceHash === "string" && meta.sourceHash)
+            return meta.sourceHash.startsWith("sha256:") ? meta.sourceHash : `sha256:${meta.sourceHash}`;
+        return null;
+    }
+    catch {
+        return null;
+    }
+}
+/** `<out>/<path>.json`, the same name fileStore uses. */
+function metaFile(outDir, path) {
+    const clean = path.replace(/^\/+|\/+$/g, "");
+    return `${outDir.replace(/[\\/]+$/, "")}/${clean || "index"}.json`;
+}
+async function refineRemote(opts) {
+    const site = opts.site.replace(/\/+$/, "");
+    const dir = opts.outDir ?? ".agent-ready/md";
+    const store = (0, file_store_1.fileStore)(dir);
+    const f = opts.fetch ?? fetch;
+    const endpoint = opts.endpoint ?? exports.DEFAULT_REFINE_ENDPOINT;
+    const sleep = opts.sleep ?? ((ms) => new Promise((r) => setTimeout(r, ms)));
+    const now = opts.now ?? (() => new Date());
+    const items = opts.paths.map((path) => ({ path, status: "skipped", saved: false }));
+    let next = 0;
+    let stopped = null;
+    const one = async (path) => {
+        const url = site + path;
+        let html;
+        try {
+            html = await fetchPage(f, url, opts.pageTimeoutMs ?? 30000);
+        }
+        catch (err) {
+            return { path, status: "error", code: "page_fetch", message: err.message, saved: false };
+        }
+        const previousFingerprint = opts.force ? null : await savedFingerprint(dir, path);
+        const body = JSON.stringify({
+            url,
+            html,
+            ...(opts.siteName ? { siteName: opts.siteName } : {}),
+            ...(opts.model ? { model: opts.model } : {}),
+            ...(previousFingerprint ? { previousFingerprint } : {}),
+        });
+        let reply = null;
+        for (let attempt = 0; attempt < 2; attempt++) {
+            reply = await post(f, endpoint, opts.apiKey, body, opts.timeoutMs ?? 90000);
+            if (attempt > 0)
+                break;
+            if (reply.httpStatus === 429 && reply.json?.error === "rate_limited") {
+                const after = Number(reply.json.retryAfter ?? reply.retryAfterHeader ?? 10);
+                await sleep(Math.min(Number.isFinite(after) && after >= 0 ? after : 10, 60) * 1000);
+                continue;
+            }
+            if (reply.httpStatus === 502 || reply.httpStatus === 504) {
+                await sleep(opts.retryDelayMs ?? 2000);
+                continue;
+            }
+            break;
+        }
+        const r = reply;
+        if (r.local)
+            return { path, status: "error", code: r.local.code, message: r.local.message, saved: false };
+        const j = r.json ?? {};
+        if (r.httpStatus === 200 && j.status === "ok" && typeof j.markdown === "string" && typeof j.fingerprint === "string") {
+            const stamp = now().toISOString().replace(/\.\d{3}Z$/, "Z");
+            const tokens = Number(j.tokens) || 0;
+            const doc = {
+                path,
+                url,
+                markdown: j.markdown,
+                // sourceHash keeps local refine's freshness check working on these files.
+                sourceHash: j.fingerprint.replace(/^sha256:/, ""),
+                promptId: String(j.prompt ?? "page-v1"),
+                generatedAt: stamp,
+                tokens,
+                fingerprint: j.fingerprint,
+                model: j.model ?? opts.model ?? null,
+                prompt: j.prompt ?? null,
+                kind: j.kind ?? null,
+                refinedAt: stamp,
+                source: "relve-api",
+            };
+            try {
+                await store.save(doc);
+            }
+            catch (err) {
+                return { path, status: "error", code: "write", message: err.message, saved: false };
+            }
+            return { path, status: "ok", tokens, kind: j.kind ?? undefined, model: j.model ?? undefined, saved: true };
+        }
+        if (r.httpStatus === 200 && j.status === "unchanged")
+            return { path, status: "unchanged", saved: false };
+        if (j.status === "rejected") {
+            return { path, status: "rejected", code: "rejected", issues: Array.isArray(j.issues) ? j.issues.map(String) : [], saved: false };
+        }
+        const code = typeof j.error === "string" ? j.error : r.httpStatus === 200 ? "bad_response" : `http_${r.httpStatus}`;
+        return { path, status: "error", code, message: typeof j.message === "string" ? j.message : undefined, saved: false };
+    };
+    const worker = async () => {
+        while (!stopped && next < opts.paths.length) {
+            const i = next++;
+            const item = await one(opts.paths[i]);
+            items[i] = item;
+            if (item.status === "error" && item.code && FATAL.has(item.code))
+                stopped = item.code;
+            opts.onResult?.(item);
+        }
+    };
+    await Promise.all(Array.from({ length: Math.max(1, Math.min(opts.concurrency ?? 2, opts.paths.length || 1)) }, worker));
+    const counts = { ok: 0, unchanged: 0, rejected: 0, error: 0, skipped: 0 };
+    let tokens = 0;
+    for (const it of items) {
+        counts[it.status]++;
+        tokens += it.tokens ?? 0;
+    }
+    return { items, counts, tokens, stopped, dir: store.dir };
+}
+/** What a stop code means, for the CLI. */
+function stopMessage(code) {
+    if (code === "openai_auth")
+        return "OpenAI rejected the key in OPENAI_API_KEY. Check it at https://platform.openai.com/api-keys and run again.";
+    if (code === "openai_quota")
+        return "Your OpenAI account is out of quota or has no billing set up. Check https://platform.openai.com/settings/organization/billing and run again.";
+    if (code === "missing_key")
+        return "The Relve API got no key. Set OPENAI_API_KEY and run again.";
+    return code;
+}
+async function post(f, endpoint, apiKey, body, timeoutMs) {
+    const ctl = new AbortController();
+    const timer = setTimeout(() => ctl.abort(), timeoutMs);
+    try {
+        const res = await f(endpoint, {
+            method: "POST",
+            headers: { "content-type": "application/json", accept: "application/json", "x-openai-key": apiKey, "user-agent": exports.USER_AGENT },
+            body,
+            signal: ctl.signal,
+        });
+        const text = await res.text();
+        let json = null;
+        try {
+            json = JSON.parse(text);
+        }
+        catch {
+            return { httpStatus: res.status, json: null, retryAfterHeader: null, local: { code: res.ok ? "bad_response" : `http_${res.status}`, message: "the Relve API did not answer with JSON" } };
+        }
+        return { httpStatus: res.status, json, retryAfterHeader: res.headers.get("retry-after") };
+    }
+    catch (err) {
+        const aborted = err?.name === "AbortError" || ctl.signal.aborted;
+        return {
+            httpStatus: 0,
+            json: null,
+            retryAfterHeader: null,
+            local: aborted ? { code: "timeout", message: `no answer from the Relve API in ${Math.round(timeoutMs / 1000)} s` } : { code: "network", message: err?.message ?? "request failed" },
+        };
+    }
+    finally {
+        clearTimeout(timer);
+    }
+}
+async function fetchPage(f, url, timeoutMs) {
+    const ctl = new AbortController();
+    const timer = setTimeout(() => ctl.abort(), timeoutMs);
+    try {
+        const res = await f(url, { headers: { accept: "text/html", "user-agent": exports.USER_AGENT }, redirect: "follow", signal: ctl.signal });
+        if (!res.ok)
+            throw new Error(`HTTP ${res.status}`);
+        return await res.text();
+    }
+    finally {
+        clearTimeout(timer);
+    }
+}