@wrenautomation/read 0.0.0-stage → 0.30.0

package/LICENSE.md

@@ -0,0 +1,61 @@
+# PolyForm Strict License 1.0.0
+
+<https://polyformproject.org/licenses/strict/1.0.0>
+
+## Acceptance
+
+In order to get any license under these terms, you must agree to them as both strict obligations and conditions to all your licenses.
+
+## Copyright License
+
+The licensor grants you a copyright license for the software to do everything you might do with the software that would otherwise infringe the licensor's copyright in it for any permitted purpose, other than distributing the software or making changes or new works based on the software.
+
+## Patent License
+
+The licensor grants you a patent license for the software that covers patent claims the licensor can license, or becomes able to license, that you would infringe by using the software.
+
+## Noncommercial Purposes
+
+Any noncommercial purpose is a permitted purpose.
+
+## Personal Uses
+
+Personal use for research, experiment, and testing for the benefit of public knowledge, personal study, private entertainment, hobby projects, amateur pursuits, or religious observance, without any anticipated commercial application, is use for a permitted purpose.
+
+## Noncommercial Organizations
+
+Use by any charitable organization, educational institution, public research organization, public safety or health organization, environmental protection organization, or government institution is use for a permitted purpose regardless of the source of funding or obligations resulting from the funding.
+
+## Fair Use
+
+You may have "fair use" rights for the software under the law. These terms do not limit them.
+
+## No Other Rights
+
+These terms do not allow you to sublicense or transfer any of your licenses to anyone else, or prevent the licensor from granting licenses to anyone else.  These terms do not imply any other licenses.
+
+## Patent Defense
+
+If you make any written claim that the software infringes or contributes to infringement of any patent, your patent license for the software granted under these terms ends immediately. If your company makes such a claim, your patent license ends immediately for work on behalf of your company.
+
+## Violations
+
+The first time you are notified in writing that you have violated any of these terms, or done anything with the software not covered by your licenses, your licenses can nonetheless continue if you come into full compliance with these terms, and take practical steps to correct past violations, within 32 days of receiving notice.  Otherwise, all your licenses end immediately.
+
+## No Liability
+
+***As far as the law allows, the software comes as is, without any warranty or condition, and the licensor will not be liable to you for any damages arising out of these terms or the use or nature of the software, under any kind of legal claim.***
+
+## Definitions
+
+The **licensor** is the individual or entity offering these terms, and the **software** is the software the licensor makes available under these terms.
+
+**You** refers to the individual or entity agreeing to these terms.
+
+**Your company** is any legal entity, sole proprietorship, or other kind of organization that you work for, plus all organizations that have control over, are under the control of, or are under common control with that organization.  **Control** means ownership of substantially all the assets of an entity, or the power to direct its management and policies by vote, contract, or otherwise.  Control can be direct or indirect.
+
+**Your licenses** are all the licenses granted to you for the software under these terms.
+
+**Use** means anything you do with the software requiring one of your licenses.
+
+Required Notice: Copyright 2026 William Jin (https://wrenautomation.com)

package/README.md

@@ -1,3 +1,37 @@
-# Temporary Holding Version
+# read
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+The web and files in, text out. No database, no accounts, no business rules: a library that
+fetches and reads. Until 0.29.0 this was falcon (files, markup, media) and the fetch and video
+halves of weaver; the research engine on top is [`@wrenautomation/birdwatch`](https://github.com/wrenautomation/engine/tree/main/packages/birdwatch),
+whose `birdwatch` CLI carries these readers (`fetch`, `read`, `list`, `text`, `page`, `watch`).
+
+| Import | Does |
+|---|---|
+| `@wrenautomation/read/fetch` | `PoliteFetcher` (one request at a time per host, a contact in the User-Agent, retries on 429 and 5xx, a byte cap), `robots` in warn mode, `fetchLadder` (plain GET up to a browser through a `LadderClient`: autobrowse's `web GET /fetch`) |
+| `@wrenautomation/read/text` | `htmlText`, `pdfText`, `decodeHtml`, `decodeEntities`; `readPage` (HTML to title, text, links, mailtos, tels) |
+| `@wrenautomation/read/video` | `watchFile` a video file: ffmpeg cuts and frames, an ear for the speech (`cohereSpeech`), an eye for the frames (`cohereEye`), and a video model's read of the whole file when given (`geminiVideo`, or `geminiReader` with no key check, for a proxy that holds the keys); shot labels. Over a URL: `readYoutube` (captions and chapters by yt-dlp, plus a video model's read of the screen in clips), `readYoutubeByUrl` (the video model alone, no download), `readVideo` (any other video: yt-dlp, then `watchFile`), `readAudio` (a podcast by its audio URL, then `hearAudio`), `watch` over a URL or a file, `flatPlaylist` (what a profile, channel or search holds), `download`. Every reader returns the one `VideoRead` shape |
+| `@wrenautomation/read/study` | a study's synthesis reader: the quote gate (`pageQuoted`, `numbersQuoted`), the passages a page is shown by, drafts that cite claims, the markdown report |
+
+Everything is a function of what you pass: a URL, bytes, text, a file path, models, tools. What
+searches, stores, schedules or knows a customer belongs to the caller. Every model call goes
+through `@wrenautomation/models`. `watch` and the readers over a URL need ffmpeg and yt-dlp.
+
+```ts
+import { PoliteFetcher, userAgent } from "@wrenautomation/read/fetch";
+import { readPage } from "@wrenautomation/read/text";
+
+const fetcher = new PoliteFetcher(userAgent("you@example.com"));
+const res = await fetcher.get(url);
+const page = readPage(res.text, res.url);
+```
+
+## Develop
+
+```sh
+pnpm install
+pnpm lint && pnpm typecheck && pnpm test
+```
+
+A `v*` tag publishes to npm (`.github/workflows/release.yml`, trusted publishing: no stored token).
+
+License: PolyForm Strict 1.0.0 (`LICENSE.md`).

package/dist/fetch/fetcher.d.ts

@@ -0,0 +1,84 @@
+/**
+ * A fetch that failed after retries. `status` is the last HTTP status when the
+ * server answered (5xx after retries), null for transport failures: robots handling
+ * needs the difference (a 503 robots.txt means disallow-all; a dead host means the
+ * web's allow default).
+ */
+export declare class FetchError extends Error {
+    name: string;
+    readonly status: number | null;
+    constructor(message: string, status?: number | null);
+}
+/** The declared UA, or a loud failure: contact info is an explicit operator choice, never invented. */
+export declare function userAgent(contact: string | null | undefined): string;
+/** What one GET produced. `url` is the final URL after redirects. */
+export interface FetchResponse {
+    status: number;
+    url: string;
+    text: string;
+    /** The `fetch_tier` the page was kept under, when it was not the caller's own plain fetch. */
+    tier?: string;
+}
+/** The seam crawlers and gates depend on; tests substitute canned pages. */
+export interface Fetcher {
+    readonly userAgent: string;
+    get(url: string): Promise<FetchResponse>;
+}
+export interface DownloadResult {
+    path: string;
+    /** false: already on disk, not re-fetched. */
+    downloaded: boolean;
+    size: number;
+}
+interface PoliteFetcherOptions {
+    /** Seconds between requests to one host. */
+    minInterval?: number;
+    /** Extra uniform(lo, hi) seconds per request so crawls don't tick like a metronome. */
+    jitter?: readonly [number, number] | null;
+    timeout?: number;
+    retries?: number;
+    /** Test seams. */
+    fetch?: typeof globalThis.fetch;
+    sleep?: (ms: number) => Promise<void>;
+    now?: () => number;
+    random?: () => number;
+}
+/**
+ * fetch() wrapper enforcing UA + throttle + retry in one place. The throttle is per
+ * host: a crawl across many small sites is not serialized globally.
+ */
+export declare class PoliteFetcher implements Fetcher {
+    readonly userAgent: string;
+    private readonly minIntervalMs;
+    private readonly jitter;
+    private readonly timeoutMs;
+    private readonly retries;
+    private readonly fetchImpl;
+    private readonly sleep;
+    private readonly now;
+    private readonly random;
+    private readonly lastRequest;
+    /** Requests to one host are serialized so concurrent callers still honor the interval. */
+    private readonly hostQueues;
+    constructor(userAgent: string, opts?: PoliteFetcherOptions);
+    private throttle;
+    private request;
+    /**
+     * One throttled request as the raw `Response`, for a body the caller streams (a file it
+     * filters as it lands) or a method other than GET. `init` wins: its signal replaces the
+     * fetcher's timeout, its headers join the User-Agent. No retries: a half-read stream cannot be
+     * asked again, so the caller decides what a bad status means.
+     */
+    open(url: string, init?: RequestInit): Promise<Response>;
+    /**
+     * GET with throttle and bounded retries. 4xx returns (the caller interprets: a
+     * crawl 404 is data); transport errors and 5xx retry with backoff, then FetchError.
+     */
+    get(url: string): Promise<FetchResponse>;
+    /**
+     * Stream a (possibly large) file to `dest`; skip if already present. Writes to a
+     * .part file and renames, so an interrupted download never masquerades as complete.
+     */
+    download(url: string, dest: string): Promise<DownloadResult>;
+}
+export {};

package/dist/fetch/fetcher.js

@@ -0,0 +1,180 @@
+/**
+ * Polite HTTP fetching, shared by every layer that touches the network: the
+ * discovery ownership gate, the site crawler, bulk downloads. One class holds the
+ * etiquette (declared User-Agent with contact info, per-host throttle, bounded
+ * retries, never re-downloading what is on disk) so it cannot drift between callers.
+ */
+import { createWriteStream } from "node:fs";
+import { mkdir, rename, stat, unlink } from "node:fs/promises";
+import { dirname } from "node:path";
+import { Readable } from "node:stream";
+import { pipeline } from "node:stream/promises";
+const DEFAULT_RETRIES = 3;
+const BACKOFF_MS = 2000;
+/**
+ * A fetch that failed after retries. `status` is the last HTTP status when the
+ * server answered (5xx after retries), null for transport failures: robots handling
+ * needs the difference (a 503 robots.txt means disallow-all; a dead host means the
+ * web's allow default).
+ */
+export class FetchError extends Error {
+    name = "FetchError";
+    status;
+    constructor(message, status = null) {
+        super(message);
+        this.status = status;
+    }
+}
+/** The declared UA, or a loud failure: contact info is an explicit operator choice, never invented. */
+export function userAgent(contact) {
+    if (!contact) {
+        throw new FetchError("outbound fetches need a contact (an email address or URL a remote operator could reach you at)");
+    }
+    return `wrenautomation/0.1 (${contact})`;
+}
+const defaultSleep = (ms) => new Promise((r) => setTimeout(r, ms));
+/**
+ * fetch() wrapper enforcing UA + throttle + retry in one place. The throttle is per
+ * host: a crawl across many small sites is not serialized globally.
+ */
+export class PoliteFetcher {
+    userAgent;
+    minIntervalMs;
+    jitter;
+    timeoutMs;
+    retries;
+    fetchImpl;
+    sleep;
+    now;
+    random;
+    lastRequest = new Map();
+    /** Requests to one host are serialized so concurrent callers still honor the interval. */
+    hostQueues = new Map();
+    constructor(userAgent, opts = {}) {
+        this.userAgent = userAgent;
+        this.minIntervalMs = (opts.minInterval ?? 1.0) * 1000;
+        this.jitter = opts.jitter ?? null;
+        this.timeoutMs = (opts.timeout ?? 30) * 1000;
+        this.retries = Math.max(1, opts.retries ?? DEFAULT_RETRIES);
+        this.fetchImpl = opts.fetch ?? globalThis.fetch;
+        this.sleep = opts.sleep ?? defaultSleep;
+        this.now = opts.now ?? (() => performance.now());
+        this.random = opts.random ?? Math.random;
+    }
+    async throttle(url) {
+        let host = "";
+        try {
+            host = new URL(url).host;
+        }
+        catch {
+            host = "";
+        }
+        const previous = this.hostQueues.get(host) ?? Promise.resolve();
+        const turn = previous.then(async () => {
+            let wait = this.minIntervalMs;
+            if (this.jitter) {
+                const [lo, hi] = this.jitter;
+                wait += (lo + (hi - lo) * this.random()) * 1000;
+            }
+            const elapsed = this.now() - (this.lastRequest.get(host) ?? Number.NEGATIVE_INFINITY);
+            if (elapsed < wait)
+                await this.sleep(wait - elapsed);
+            this.lastRequest.set(host, this.now());
+        });
+        this.hostQueues.set(host, turn.catch(() => { }));
+        await turn;
+    }
+    request(url, init = {}) {
+        const headers = new Headers(init.headers);
+        if (!headers.has("user-agent"))
+            headers.set("User-Agent", this.userAgent);
+        return this.fetchImpl(url, {
+            redirect: "follow",
+            signal: AbortSignal.timeout(this.timeoutMs),
+            ...init,
+            headers,
+        });
+    }
+    /**
+     * One throttled request as the raw `Response`, for a body the caller streams (a file it
+     * filters as it lands) or a method other than GET. `init` wins: its signal replaces the
+     * fetcher's timeout, its headers join the User-Agent. No retries: a half-read stream cannot be
+     * asked again, so the caller decides what a bad status means.
+     */
+    async open(url, init = {}) {
+        await this.throttle(url);
+        return this.request(url, init);
+    }
+    /**
+     * GET with throttle and bounded retries. 4xx returns (the caller interprets: a
+     * crawl 404 is data); transport errors and 5xx retry with backoff, then FetchError.
+     */
+    async get(url) {
+        let lastError = "";
+        let lastStatus = null;
+        for (let attempt = 0; attempt < this.retries; attempt++) {
+            await this.throttle(url);
+            try {
+                const resp = await this.request(url);
+                if (resp.status < 500) {
+                    return { status: resp.status, url: resp.url || url, text: await resp.text() };
+                }
+                lastError = `HTTP ${resp.status}`;
+                lastStatus = resp.status;
+                await resp.body?.cancel().catch(() => { });
+            }
+            catch (err) {
+                lastError = describe(err);
+                lastStatus = null;
+            }
+            if (attempt < this.retries - 1)
+                await this.sleep(BACKOFF_MS * (attempt + 1));
+        }
+        throw new FetchError(`GET ${url} failed after ${this.retries} attempts: ${lastError}`, lastStatus);
+    }
+    /**
+     * Stream a (possibly large) file to `dest`; skip if already present. Writes to a
+     * .part file and renames, so an interrupted download never masquerades as complete.
+     */
+    async download(url, dest) {
+        const existing = await stat(dest).catch(() => null);
+        if (existing)
+            return { path: dest, downloaded: false, size: existing.size };
+        await mkdir(dirname(dest), { recursive: true });
+        const tmp = `${dest}.part`;
+        let lastError = "";
+        for (let attempt = 0; attempt < this.retries; attempt++) {
+            await this.throttle(url);
+            try {
+                const resp = await this.request(url);
+                if (resp.status >= 400) {
+                    lastError = `HTTP ${resp.status}`;
+                    await resp.body?.cancel().catch(() => { });
+                    if (resp.status < 500)
+                        break; // 4xx will not improve on retry
+                    continue;
+                }
+                const body = resp.body ? Readable.fromWeb(resp.body) : Readable.from([]);
+                await pipeline(body, createWriteStream(tmp));
+                await rename(tmp, dest);
+                const size = (await stat(dest)).size;
+                return { path: dest, downloaded: true, size };
+            }
+            catch (err) {
+                lastError = describe(err);
+                if (attempt < this.retries - 1)
+                    await this.sleep(BACKOFF_MS * (attempt + 1));
+            }
+        }
+        await unlink(tmp).catch(() => { });
+        throw new FetchError(`download ${url} failed: ${lastError}`);
+    }
+}
+function describe(err) {
+    if (err instanceof Error) {
+        const cause = err.cause instanceof Error ? `: ${err.cause.message}` : "";
+        return `${err.name}: ${err.message}${cause}`;
+    }
+    return String(err);
+}
+//# sourceMappingURL=fetcher.js.map

package/dist/fetch/fetcher.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"fetcher.js","sourceRoot":"","sources":["../../src/fetch/fetcher.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AACH,OAAO,EAAE,iBAAiB,EAAE,MAAM,SAAS,CAAC;AAC5C,OAAO,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,MAAM,kBAAkB,CAAC;AAC/D,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AACvC,OAAO,EAAE,QAAQ,EAAE,MAAM,sBAAsB,CAAC;AAEhD,MAAM,eAAe,GAAG,CAAC,CAAC;AAC1B,MAAM,UAAU,GAAG,IAAI,CAAC;AAExB;;;;;GAKG;AACH,MAAM,OAAO,UAAW,SAAQ,KAAK;IAC1B,IAAI,GAAG,YAAY,CAAC;IACpB,MAAM,CAAgB;IAC/B,YAAY,OAAe,EAAE,SAAwB,IAAI;QACvD,KAAK,CAAC,OAAO,CAAC,CAAC;QACf,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;IACvB,CAAC;CACF;AAED,uGAAuG;AACvG,MAAM,UAAU,SAAS,CAAC,OAAkC;IAC1D,IAAI,CAAC,OAAO,EAAE,CAAC;QACb,MAAM,IAAI,UAAU,CAClB,gGAAgG,CACjG,CAAC;IACJ,CAAC;IACD,OAAO,uBAAuB,OAAO,GAAG,CAAC;AAC3C,CAAC;AAsCD,MAAM,YAAY,GAAG,CAAC,EAAU,EAAiB,EAAE,CAAC,IAAI,OAAO,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC;AAE1F;;;GAGG;AACH,MAAM,OAAO,aAAa;IAcb;IAbM,aAAa,CAAS;IACtB,MAAM,CAAmC;IACzC,SAAS,CAAS;IAClB,OAAO,CAAS;IAChB,SAAS,CAA0B;IACnC,KAAK,CAAgC;IACrC,GAAG,CAAe;IAClB,MAAM,CAAe;IACrB,WAAW,GAAG,IAAI,GAAG,EAAkB,CAAC;IACzD,0FAA0F;IACzE,UAAU,GAAG,IAAI,GAAG,EAAyB,CAAC;IAE/D,YACW,SAAiB,EAC1B,OAA6B,EAAE;QADtB,cAAS,GAAT,SAAS,CAAQ;QAG1B,IAAI,CAAC,aAAa,GAAG,CAAC,IAAI,CAAC,WAAW,IAAI,GAAG,CAAC,GAAG,IAAI,CAAC;QACtD,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,IAAI,IAAI,CAAC;QAClC,IAAI,CAAC,SAAS,GAAG,CAAC,IAAI,CAAC,OAAO,IAAI,EAAE,CAAC,GAAG,IAAI,CAAC;QAC7C,IAAI,CAAC,OAAO,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,OAAO,IAAI,eAAe,CAAC,CAAC;QAC5D,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC,KAAK,IAAI,UAAU,CAAC,KAAK,CAAC;QAChD,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,IAAI,YAAY,CAAC;QACxC,IAAI,CAAC,GAAG,GAAG,IAAI,CAAC,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC,WAAW,CAAC,GAAG,EAAE,CAAC,CAAC;QACjD,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,IAAI,IAAI,CAAC,MAAM,CAAC;IAC3C,CAAC;IAEO,KAAK,CAAC,QAAQ,CAAC,GAAW;QAChC,IAAI,IAAI,GAAG,EAAE,CAAC;QACd,IAAI,CAAC;YACH,IAAI,GAAG,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC,IAAI,CAAC;QAC3B,CAAC;QAAC,MAAM,CAAC;YACP,IAAI,GAAG,EAAE,CAAC;QACZ,CAAC;QACD,MAAM,QAAQ,GAAG,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,OAAO,CAAC,OAAO,EAAE,CAAC;QAChE,MAAM,IAAI,GAAG,QAAQ,CAAC,IAAI,CAAC,KAAK,IAAI,EAAE;YACpC,IAAI,IAAI,GAAG,IAAI,CAAC,aAAa,CAAC;YAC9B,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBAChB,MAAM,CAAC,EAAE,EAAE,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,CAAC;gBAC7B,IAAI,IAAI,CAAC,EAAE,GAAG,CAAC,EAAE,GAAG,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,GAAG,IAAI,CAAC;YAClD,CAAC;YACD,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,CAAC,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,MAAM,CAAC,iBAAiB,CAAC,CAAC;YACtF,IAAI,OAAO,GAAG,IAAI;gBAAE,MAAM,IAAI,CAAC,KAAK,CAAC,IAAI,GAAG,OAAO,CAAC,CAAC;YACrD,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,IAAI,EAAE,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC;QACzC,CAAC,CAAC,CAAC;QACH,IAAI,CAAC,UAAU,CAAC,GAAG,CACjB,IAAI,EACJ,IAAI,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CACrB,CAAC;QACF,MAAM,IAAI,CAAC;IACb,CAAC;IAEO,OAAO,CAAC,GAAW,EAAE,OAAoB,EAAE;QACjD,MAAM,OAAO,GAAG,IAAI,OAAO,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;QAC1C,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,YAAY,CAAC;YAAE,OAAO,CAAC,GAAG,CAAC,YAAY,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC;QAC1E,OAAO,IAAI,CAAC,SAAS,CAAC,GAAG,EAAE;YACzB,QAAQ,EAAE,QAAQ;YAClB,MAAM,EAAE,WAAW,CAAC,OAAO,CAAC,IAAI,CAAC,SAAS,CAAC;YAC3C,GAAG,IAAI;YACP,OAAO;SACR,CAAC,CAAC;IACL,CAAC;IAED;;;;;OAKG;IACH,KAAK,CAAC,IAAI,CAAC,GAAW,EAAE,OAAoB,EAAE;QAC5C,MAAM,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC;QACzB,OAAO,IAAI,CAAC,OAAO,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;IACjC,CAAC;IAED;;;OAGG;IACH,KAAK,CAAC,GAAG,CAAC,GAAW;QACnB,IAAI,SAAS,GAAG,EAAE,CAAC;QACnB,IAAI,UAAU,GAAkB,IAAI,CAAC;QACrC,KAAK,IAAI,OAAO,GAAG,CAAC,EAAE,OAAO,GAAG,IAAI,CAAC,OAAO,EAAE,OAAO,EAAE,EAAE,CAAC;YACxD,MAAM,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC;YACzB,IAAI,CAAC;gBACH,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;gBACrC,IAAI,IAAI,CAAC,MAAM,GAAG,GAAG,EAAE,CAAC;oBACtB,OAAO,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,GAAG,EAAE,IAAI,CAAC,GAAG,IAAI,GAAG,EAAE,IAAI,EAAE,MAAM,IAAI,CAAC,IAAI,EAAE,EAAE,CAAC;gBAChF,CAAC;gBACD,SAAS,GAAG,QAAQ,IAAI,CAAC,MAAM,EAAE,CAAC;gBAClC,UAAU,GAAG,IAAI,CAAC,MAAM,CAAC;gBACzB,MAAM,IAAI,CAAC,IAAI,EAAE,MAAM,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;YAC5C,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACb,SAAS,GAAG,QAAQ,CAAC,GAAG,CAAC,CAAC;gBAC1B,UAAU,GAAG,IAAI,CAAC;YACpB,CAAC;YACD,IAAI,OAAO,GAAG,IAAI,CAAC,OAAO,GAAG,CAAC;gBAAE,MAAM,IAAI,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,OAAO,GAAG,CAAC,CAAC,CAAC,CAAC;QAC/E,CAAC;QACD,MAAM,IAAI,UAAU,CAClB,OAAO,GAAG,iBAAiB,IAAI,CAAC,OAAO,cAAc,SAAS,EAAE,EAChE,UAAU,CACX,CAAC;IACJ,CAAC;IAED;;;OAGG;IACH,KAAK,CAAC,QAAQ,CAAC,GAAW,EAAE,IAAY;QACtC,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,CAAC;QACpD,IAAI,QAAQ;YAAE,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,UAAU,EAAE,KAAK,EAAE,IAAI,EAAE,QAAQ,CAAC,IAAI,EAAE,CAAC;QAC5E,MAAM,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;QAChD,MAAM,GAAG,GAAG,GAAG,IAAI,OAAO,CAAC;QAC3B,IAAI,SAAS,GAAG,EAAE,CAAC;QACnB,KAAK,IAAI,OAAO,GAAG,CAAC,EAAE,OAAO,GAAG,IAAI,CAAC,OAAO,EAAE,OAAO,EAAE,EAAE,CAAC;YACxD,MAAM,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC;YACzB,IAAI,CAAC;gBACH,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;gBACrC,IAAI,IAAI,CAAC,MAAM,IAAI,GAAG,EAAE,CAAC;oBACvB,SAAS,GAAG,QAAQ,IAAI,CAAC,MAAM,EAAE,CAAC;oBAClC,MAAM,IAAI,CAAC,IAAI,EAAE,MAAM,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;oBAC1C,IAAI,IAAI,CAAC,MAAM,GAAG,GAAG;wBAAE,MAAM,CAAC,gCAAgC;oBAC9D,SAAS;gBACX,CAAC;gBACD,MAAM,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,QAAQ,CAAC,OAAO,CAAC,IAAI,CAAC,IAAa,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;gBAClF,MAAM,QAAQ,CAAC,IAAI,EAAE,iBAAiB,CAAC,GAAG,CAAC,CAAC,CAAC;gBAC7C,MAAM,MAAM,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;gBACxB,MAAM,IAAI,GAAG,CAAC,MAAM,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC;gBACrC,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,UAAU,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;YAChD,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACb,SAAS,GAAG,QAAQ,CAAC,GAAG,CAAC,CAAC;gBAC1B,IAAI,OAAO,GAAG,IAAI,CAAC,OAAO,GAAG,CAAC;oBAAE,MAAM,IAAI,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,OAAO,GAAG,CAAC,CAAC,CAAC,CAAC;YAC/E,CAAC;QACH,CAAC;QACD,MAAM,MAAM,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;QAClC,MAAM,IAAI,UAAU,CAAC,YAAY,GAAG,YAAY,SAAS,EAAE,CAAC,CAAC;IAC/D,CAAC;CACF;AAED,SAAS,QAAQ,CAAC,GAAY;IAC5B,IAAI,GAAG,YAAY,KAAK,EAAE,CAAC;QACzB,MAAM,KAAK,GAAG,GAAG,CAAC,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QACzE,OAAO,GAAG,GAAG,CAAC,IAAI,KAAK,GAAG,CAAC,OAAO,GAAG,KAAK,EAAE,CAAC;IAC/C,CAAC;IACD,OAAO,MAAM,CAAC,GAAG,CAAC,CAAC;AACrB,CAAC"}

package/dist/fetch/index.d.ts

@@ -0,0 +1,9 @@
+/**
+ * The open web by HTTP: a polite fetcher (one request at a time per host, a real User-Agent with a
+ * contact, retries on 429 and 5xx, a byte cap), robots.txt in warn mode, and the fetch ladder
+ * that climbs from a plain GET to a browser (`LadderClient`: autobrowse's `web GET /fetch`).
+ * Turning the page into text is `./text`'s. No database here.
+ */
+export * from "./fetcher.js";
+export * from "./ladder.js";
+export * from "./robots.js";

package/dist/fetch/index.js

@@ -0,0 +1,10 @@
+/**
+ * The open web by HTTP: a polite fetcher (one request at a time per host, a real User-Agent with a
+ * contact, retries on 429 and 5xx, a byte cap), robots.txt in warn mode, and the fetch ladder
+ * that climbs from a plain GET to a browser (`LadderClient`: autobrowse's `web GET /fetch`).
+ * Turning the page into text is `./text`'s. No database here.
+ */
+export * from "./fetcher.js";
+export * from "./ladder.js";
+export * from "./robots.js";
+//# sourceMappingURL=index.js.map

package/dist/fetch/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/fetch/index.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AACH,cAAc,cAAc,CAAC;AAC7B,cAAc,aAAa,CAAC;AAC5B,cAAc,aAAa,CAAC"}

package/dist/fetch/ladder.d.ts

@@ -0,0 +1,44 @@
+/**
+ * The client of autobrowse's fetch ladder, `web GET /fetch`. A caller asks it for a blocked
+ * page; autobrowse climbs `impersonate`, `render` and `exit`. A block is data in the answer.
+ * A failed call is the other result.
+ */
+/** The rungs a ladder climbs, lowest first. */
+export declare const LADDER_TIERS: readonly ["impersonate", "render", "exit"];
+export type LadderTier = (typeof LADDER_TIERS)[number];
+export declare const isLadderTier: (tier: string | null | undefined) => tier is LadderTier;
+/** Whoever answers `web GET /fetch`: autobrowse's site client, or a test double. */
+export interface LadderClient {
+    /** `account`: the credential the call runs as (`reddit@alt`); the worker's default when absent. */
+    call<T>(site: string, method: string, path: string, input?: unknown, o?: {
+        account?: string;
+    }): Promise<T>;
+}
+/** One rung the ladder tried, and what it met there. */
+export interface FetchStep {
+    tier: string;
+    status: number | null;
+    block: string | null;
+}
+/** What `web GET /fetch` answers (`Fetched`): the fields this side reads. */
+interface Fetched {
+    finalUrl: string;
+    status: number | null;
+    html: string;
+    /** The content-type the page answered with, when the worker says. */
+    type?: string | null;
+    /** The rung that answered: the last one tried. */
+    tier: LadderTier;
+    /** null: a usable page; else the signal the last rung still hit. */
+    block: string | null;
+    tries: FetchStep[];
+}
+/** `cap`: the day's `fetch` bucket is spent (429). `offline`: any other failed call. */
+type LadderFailure = "cap" | "offline";
+/** One `web GET /fetch`. A failed call is a result, never a throw. */
+export declare function fetchLadder(sites: LadderClient, url: string, from: LadderTier): Promise<{
+    fetched: Fetched;
+} | {
+    failed: LadderFailure;
+}>;
+export {};

package/dist/fetch/ladder.js

@@ -0,0 +1,18 @@
+/**
+ * The client of autobrowse's fetch ladder, `web GET /fetch`. A caller asks it for a blocked
+ * page; autobrowse climbs `impersonate`, `render` and `exit`. A block is data in the answer.
+ * A failed call is the other result.
+ */
+/** The rungs a ladder climbs, lowest first. */
+export const LADDER_TIERS = ["impersonate", "render", "exit"];
+export const isLadderTier = (tier) => LADDER_TIERS.some((t) => t === tier);
+/** One `web GET /fetch`. A failed call is a result, never a throw. */
+export async function fetchLadder(sites, url, from) {
+    try {
+        return { fetched: await sites.call("web", "GET", "/fetch", { url, from }) };
+    }
+    catch (err) {
+        return { failed: err.status === 429 ? "cap" : "offline" };
+    }
+}
+//# sourceMappingURL=ladder.js.map

package/dist/fetch/ladder.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"ladder.js","sourceRoot":"","sources":["../../src/fetch/ladder.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AACH,+CAA+C;AAC/C,MAAM,CAAC,MAAM,YAAY,GAAG,CAAC,aAAa,EAAE,QAAQ,EAAE,MAAM,CAAU,CAAC;AAEvE,MAAM,CAAC,MAAM,YAAY,GAAG,CAAC,IAA+B,EAAsB,EAAE,CAClF,YAAY,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC;AAsCvC,sEAAsE;AACtE,MAAM,CAAC,KAAK,UAAU,WAAW,CAC/B,KAAmB,EACnB,GAAW,EACX,IAAgB;IAEhB,IAAI,CAAC;QACH,OAAO,EAAE,OAAO,EAAE,MAAM,KAAK,CAAC,IAAI,CAAU,KAAK,EAAE,KAAK,EAAE,QAAQ,EAAE,EAAE,GAAG,EAAE,IAAI,EAAE,CAAC,EAAE,CAAC;IACvF,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,OAAO,EAAE,MAAM,EAAG,GAA4B,CAAC,MAAM,KAAK,GAAG,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,EAAE,CAAC;IACtF,CAAC;AACH,CAAC"}

package/dist/fetch/robots.d.ts

@@ -0,0 +1,14 @@
+import { type Fetcher } from "./fetcher.js";
+export declare class RobotsPolicy {
+    private readonly robot;
+    private readonly disallowAll;
+    private constructor();
+    static allowAll(): RobotsPolicy;
+    static disallowAll(): RobotsPolicy;
+    static parse(text: string): RobotsPolicy;
+    canFetch(userAgent: string, url: string): boolean;
+}
+export type RobotsCache = Map<string, RobotsPolicy>;
+/** The robots policy for this url's scheme+host, fetched once each. */
+export declare function robotsFor(fetcher: Fetcher, url: string, cache: RobotsCache): Promise<RobotsPolicy>;
+export declare function canFetch(fetcher: Fetcher, url: string, cache: RobotsCache): Promise<boolean>;

package/dist/fetch/robots.js

@@ -0,0 +1,92 @@
+/**
+ * robots.txt reading, shared by every page-fetching consumer.
+ *
+ * The policy that binds a fetch is the one on the HOST actually being fetched, over
+ * the SCHEME actually in use: apex and www can serve different files, so policies
+ * are cached per scheme+host and fetched over the URL's own scheme.
+ *
+ * Absence semantics follow RFC 9309: a missing robots.txt (404, or a dead host)
+ * means everything is allowed, but a server that ANSWERED 5xx is saying
+ * "unavailable", which means full disallow.
+ *
+ * Matching is robots-parser's RFC 9309 reading: the group for our product token
+ * (groups naming it merged), else `*`; the longest matching rule wins, Allow on a
+ * tie; `*` and `$` in paths.
+ */
+import parseRobots from "robots-parser";
+import { FetchError } from "./fetcher.js";
+/** robots-parser checks a URL's origin against the file's; ours are read per origin, so one stand-in. */
+const ORIGIN = "https://robots.invalid";
+// A CommonJS module: at runtime the default import is the function its typings
+// declare, which NodeNext resolution types as a namespace instead.
+const robotsParser = parseRobots;
+export class RobotsPolicy {
+    robot;
+    disallowAll;
+    constructor(robot, disallowAll) {
+        this.robot = robot;
+        this.disallowAll = disallowAll;
+    }
+    static allowAll() {
+        return new RobotsPolicy(null, false);
+    }
+    static disallowAll() {
+        return new RobotsPolicy(null, true);
+    }
+    static parse(text) {
+        return new RobotsPolicy(robotsParser(`${ORIGIN}/robots.txt`, text), false);
+    }
+    canFetch(userAgent, url) {
+        if (this.disallowAll)
+            return false;
+        if (!this.robot)
+            return true;
+        return this.robot.isAllowed(onOrigin(url), userAgent) ?? true;
+    }
+}
+/** The url's path and query on the stand-in origin; an unparseable url reads as `/`. */
+function onOrigin(url) {
+    try {
+        const u = new URL(url);
+        return `${ORIGIN}${u.pathname}${u.search}`;
+    }
+    catch {
+        return `${ORIGIN}/`;
+    }
+}
+/** The robots policy for this url's scheme+host, fetched once each. */
+export async function robotsFor(fetcher, url, cache) {
+    let key;
+    try {
+        const u = new URL(url);
+        key = `${u.protocol}//${u.host}`;
+    }
+    catch {
+        key = `https://${url}`;
+    }
+    const hit = cache.get(key);
+    if (hit)
+        return hit;
+    let policy;
+    try {
+        const resp = await fetcher.get(`${key}/robots.txt`);
+        policy = resp.status === 200 ? RobotsPolicy.parse(resp.text) : RobotsPolicy.allowAll();
+    }
+    catch (err) {
+        if (err instanceof FetchError && err.status !== null && err.status >= 500) {
+            policy = RobotsPolicy.disallowAll(); // "unavailable" is not "absent"
+        }
+        else if (err instanceof FetchError) {
+            policy = RobotsPolicy.allowAll(); // unreachable host: the web's allow default
+        }
+        else {
+            throw err;
+        }
+    }
+    cache.set(key, policy);
+    return policy;
+}
+export async function canFetch(fetcher, url, cache) {
+    return (await robotsFor(fetcher, url, cache)).canFetch(fetcher.userAgent, url);
+}
+//# sourceMappingURL=robots.js.map

package/dist/fetch/robots.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"robots.js","sourceRoot":"","sources":["../../src/fetch/robots.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AACH,OAAO,WAAW,MAAM,eAAe,CAAC;AACxC,OAAO,EAAE,UAAU,EAAgB,MAAM,cAAc,CAAC;AAExD,yGAAyG;AACzG,MAAM,MAAM,GAAG,wBAAwB,CAAC;AAKxC,+EAA+E;AAC/E,mEAAmE;AACnE,MAAM,YAAY,GAAG,WAA8D,CAAC;AAEpF,MAAM,OAAO,YAAY;IAEJ;IACA;IAFnB,YACmB,KAAmB,EACnB,WAAoB;QADpB,UAAK,GAAL,KAAK,CAAc;QACnB,gBAAW,GAAX,WAAW,CAAS;IACpC,CAAC;IAEJ,MAAM,CAAC,QAAQ;QACb,OAAO,IAAI,YAAY,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;IACvC,CAAC;IACD,MAAM,CAAC,WAAW;QAChB,OAAO,IAAI,YAAY,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;IACtC,CAAC;IAED,MAAM,CAAC,KAAK,CAAC,IAAY;QACvB,OAAO,IAAI,YAAY,CAAC,YAAY,CAAC,GAAG,MAAM,aAAa,EAAE,IAAI,CAAC,EAAE,KAAK,CAAC,CAAC;IAC7E,CAAC;IAED,QAAQ,CAAC,SAAiB,EAAE,GAAW;QACrC,IAAI,IAAI,CAAC,WAAW;YAAE,OAAO,KAAK,CAAC;QACnC,IAAI,CAAC,IAAI,CAAC,KAAK;YAAE,OAAO,IAAI,CAAC;QAC7B,OAAO,IAAI,CAAC,KAAK,CAAC,SAAS,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,SAAS,CAAC,IAAI,IAAI,CAAC;IAChE,CAAC;CACF;AAED,wFAAwF;AACxF,SAAS,QAAQ,CAAC,GAAW;IAC3B,IAAI,CAAC;QACH,MAAM,CAAC,GAAG,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC;QACvB,OAAO,GAAG,MAAM,GAAG,CAAC,CAAC,QAAQ,GAAG,CAAC,CAAC,MAAM,EAAE,CAAC;IAC7C,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,GAAG,MAAM,GAAG,CAAC;IACtB,CAAC;AACH,CAAC;AAID,uEAAuE;AACvE,MAAM,CAAC,KAAK,UAAU,SAAS,CAC7B,OAAgB,EAChB,GAAW,EACX,KAAkB;IAElB,IAAI,GAAW,CAAC;IAChB,IAAI,CAAC;QACH,MAAM,CAAC,GAAG,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC;QACvB,GAAG,GAAG,GAAG,CAAC,CAAC,QAAQ,KAAK,CAAC,CAAC,IAAI,EAAE,CAAC;IACnC,CAAC;IAAC,MAAM,CAAC;QACP,GAAG,GAAG,WAAW,GAAG,EAAE,CAAC;IACzB,CAAC;IACD,MAAM,GAAG,GAAG,KAAK,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IAC3B,IAAI,GAAG;QAAE,OAAO,GAAG,CAAC;IACpB,IAAI,MAAoB,CAAC;IACzB,IAAI,CAAC;QACH,MAAM,IAAI,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC,GAAG,GAAG,aAAa,CAAC,CAAC;QACpD,MAAM,GAAG,IAAI,CAAC,MAAM,KAAK,GAAG,CAAC,CAAC,CAAC,YAAY,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,YAAY,CAAC,QAAQ,EAAE,CAAC;IACzF,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,IAAI,GAAG,YAAY,UAAU,IAAI,GAAG,CAAC,MAAM,KAAK,IAAI,IAAI,GAAG,CAAC,MAAM,IAAI,GAAG,EAAE,CAAC;YAC1E,MAAM,GAAG,YAAY,CAAC,WAAW,EAAE,CAAC,CAAC,gCAAgC;QACvE,CAAC;aAAM,IAAI,GAAG,YAAY,UAAU,EAAE,CAAC;YACrC,MAAM,GAAG,YAAY,CAAC,QAAQ,EAAE,CAAC,CAAC,4CAA4C;QAChF,CAAC;aAAM,CAAC;YACN,MAAM,GAAG,CAAC;QACZ,CAAC;IACH,CAAC;IACD,KAAK,CAAC,GAAG,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;IACvB,OAAO,MAAM,CAAC;AAChB,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,QAAQ,CAC5B,OAAgB,EAChB,GAAW,EACX,KAAkB;IAElB,OAAO,CAAC,MAAM,SAAS,CAAC,OAAO,EAAE,GAAG,EAAE,KAAK,CAAC,CAAC,CAAC,QAAQ,CAAC,OAAO,CAAC,SAAS,EAAE,GAAG,CAAC,CAAC;AACjF,CAAC"}

package/dist/index.d.ts

@@ -0,0 +1,11 @@
+/**
+ * read: the web and files in, text out. Four doors: `./fetch` (polite HTTP, robots, the fetch
+ * ladder up to a browser), `./text` (HTML and PDF to text, a page to title, text and links),
+ * `./video` (a video or podcast from a URL or a file: captions, `watch`, audio to markdown, what a
+ * channel or search holds) and `./study` (claims that must quote a page, drafts that must cite a
+ * claim, the report). Every model call goes through `@wrenautomation/models`. No database here.
+ */
+export * from "./fetch/index.js";
+export * from "./study/index.js";
+export * from "./text/index.js";
+export * from "./video/index.js";

package/dist/index.js

@@ -0,0 +1,12 @@
+/**
+ * read: the web and files in, text out. Four doors: `./fetch` (polite HTTP, robots, the fetch
+ * ladder up to a browser), `./text` (HTML and PDF to text, a page to title, text and links),
+ * `./video` (a video or podcast from a URL or a file: captions, `watch`, audio to markdown, what a
+ * channel or search holds) and `./study` (claims that must quote a page, drafts that must cite a
+ * claim, the report). Every model call goes through `@wrenautomation/models`. No database here.
+ */
+export * from "./fetch/index.js";
+export * from "./study/index.js";
+export * from "./text/index.js";
+export * from "./video/index.js";
+//# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AACH,cAAc,kBAAkB,CAAC;AACjC,cAAc,kBAAkB,CAAC;AACjC,cAAc,iBAAiB,CAAC;AAChC,cAAc,kBAAkB,CAAC"}

package/dist/study/claims.d.ts

@@ -0,0 +1,19 @@
+/**
+ * A claim as the synthesis reader sees it: one sentence that quotes one page. birdwatch's
+ * `claimExtractor` (`@wrenautomation/birdwatch/document`) draws claims out of a document; a
+ * study's drafts and report (`./drafts`, `./report`) cite them by number. The planner that
+ * lived here (queries per angle, which pages to read, the claim gate per angle) went in 0.5.0:
+ * birdwatch plans and runs.
+ */
+export interface Claim {
+    /** The angle it answers: its index in the study's angles. */
+    angle: number;
+    claim: string;
+    quote: string;
+    source_url: string;
+}
+/**
+ * `{name}` marks filled in one pass: a "{angle}" or "$&" inside a question or
+ * a page stays text.
+ */
+export declare const fill: (template: string, values: Readonly<Record<string, string>>) => string;

package/dist/study/claims.js

@@ -0,0 +1,13 @@
+/**
+ * A claim as the synthesis reader sees it: one sentence that quotes one page. birdwatch's
+ * `claimExtractor` (`@wrenautomation/birdwatch/document`) draws claims out of a document; a
+ * study's drafts and report (`./drafts`, `./report`) cite them by number. The planner that
+ * lived here (queries per angle, which pages to read, the claim gate per angle) went in 0.5.0:
+ * birdwatch plans and runs.
+ */
+/**
+ * `{name}` marks filled in one pass: a "{angle}" or "$&" inside a question or
+ * a page stays text.
+ */
+export const fill = (template, values) => template.replace(/\{(\w+)\}/g, (mark, name) => values[name] ?? mark);
+//# sourceMappingURL=claims.js.map