@mogulmoretti/skrape 0.1.0

package/LICENSE

@@ -0,0 +1,32 @@
+MIT License
+
+Copyright (c) 2026 the skrape contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to
+deal in the Software without restriction, including without limitation the
+rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+sell copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+IN THE SOFTWARE.
+
+---
+
+Additionally, and without limiting the above: this software interacts with
+third-party platforms (including Skool and Loom) on the user's behalf, using
+the user's own authenticated session. It is the user's sole responsibility
+to ensure their use complies with those platforms' terms of service and any
+applicable law. The authors and copyright holders are not responsible for,
+and disclaim all liability arising from, any account action, access
+restriction, or other consequence a user experiences as a result of using
+this software.

package/README.md

@@ -0,0 +1,117 @@
+# skrape
+
+Read Skool course content instead of watching it.
+
+`skrape` pulls the classroom of a Skool community **you already pay for and
+belong to**, and writes clean, readable, timestamped transcripts to disk —
+so you can read a lesson in two minutes instead of watching a 20-minute video.
+
+It works because Skool ships the whole course tree as structured data inside
+each page, and course videos are almost always hosted on Loom, which publishes
+caption tracks. No audio download, no speech-to-text, no scraping of anything
+that isn't already sitting on the page you're allowed to see.
+
+## Before you use this
+
+**This is a personal tool for your own communities.** It signs in as *you*,
+using *your* real browser session, and only ever touches courses your account
+already has access to — it can't and won't bypass Skool's own paywall or
+membership checks.
+
+Automating access to a platform like this generally sits outside most
+platforms' terms of service, even when you're a legitimate paying member and
+even when nothing is being redistributed. Running this is a decision about
+your own account, and it's yours to make — this tool doesn't make it for you,
+and it doesn't hide what it's doing (see [How it works](#how-it-works)).
+Don't use it on communities you don't belong to, don't share or republish
+what it produces, and don't run it in a way designed to look like normal
+browsing if you wouldn't be comfortable explaining what it does.
+
+If a course creator you follow would rather you didn't do this, respect that.
+
+## Install
+
+    npm install -g @mogulmoretti/skrape
+    skrape
+
+That's the whole setup. First run installs the browser skrape needs (a
+one-time, ~150MB download) if it isn't already on your machine, then walks
+you through everything:
+
+1. Checks whether you're signed in, and if not, opens a real Chrome window for
+   you to sign in yourself — nothing is typed on your behalf, and your
+   password never passes through this tool.
+2. Looks up the communities your account belongs to and lets you pick one
+   with the arrow keys (an "enter a slug manually" option is always there
+   too, in case discovery doesn't find one).
+3. Shows you what's about to happen — the community, how many courses were
+   found, where files will land — and waits for you to confirm before any
+   work starts.
+4. Runs the sync with a live progress bar. You can Ctrl+C at any point; it
+   cleans up after itself.
+5. Prints a summary: outcome counts, total words, the output path, and a
+   readable list of anything that wasn't transcribed and why — nothing is
+   silently dropped from the count.
+
+From there you can sync another community, re-sync the same one (it's
+incremental — already-fetched lessons are skipped), open the output folder,
+or quit.
+
+Output lands in `./out/<slug>/transcripts/<course>/NN-lesson.md`.
+
+### Scripting / advanced use
+
+The underlying subcommands work directly too, without the guided flow:
+
+    skrape login                     # once — sign in by hand, session persists
+    skrape sync <slug>                # slug is the part after skool.com/
+    skrape sync <slug> -o ./out -c 4  # custom output dir / concurrency
+
+### Building from source
+
+    git clone <this repo>
+    cd skrape
+    npm install
+    npm run build
+    npm link                        # or: npm install -g .
+
+(Then `node dist/cli.js <command>` also works if you haven't `npm link`ed it.)
+
+## How it works
+
+Skool ships the full course tree as JSON inside each classroom page — no
+clicking through lesson by lesson. `skrape` reads that structure the way any
+browser or crawler would: an authenticated request to a page your account
+can already open.
+
+Course videos are checked against known caption-providing hosts (Loom, at the
+moment). Where a caption track exists, it's downloaded and cleaned into
+readable prose with timestamp anchors. Where one doesn't, the lesson is
+reported as not transcribed rather than silently skipped.
+
+When a page doesn't come back with the data the tool expects — an expired
+session, or Skool changing how a page is built — it falls back to driving an
+actual browser window using your already-authenticated profile, rather than
+guessing or failing silently.
+
+Nothing about this hides its footprint: requests carry a normal browser user
+agent, run at a modest, bounded concurrency, and only ever happen when you
+run the command.
+
+## What this doesn't do
+
+- Doesn't work on communities you're not a member of — there's no bypass for
+  Skool's own access checks, because none of the data is reachable without
+  them.
+- Doesn't redistribute or publish anything. Output is written to your local
+  disk for you to read.
+- Doesn't store your password. Sign-in happens in a real Chrome window that
+  only you interact with.
+- Doesn't transcribe video hosts it doesn't recognize. Those lessons show up
+  in the summary as not transcribed, with a reason.
+
+## License
+
+MIT — see [LICENSE](LICENSE). In short: do what you like with the code, no
+warranty, use it at your own risk and your own judgment about the platforms
+you point it at.

package/dist/auth/session.d.ts

@@ -0,0 +1,26 @@
+export declare function profileDir(): string;
+export declare function dbPath(): string;
+/**
+ * Marker file written once the one-time Chrome/Playwright browser install
+ * has been confirmed (either found already installed, or installed by us).
+ * Its presence lets later runs skip re-probing with a real browser launch.
+ */
+export declare function chromeMarkerPath(): string;
+export declare function ensureRoot(): Promise<void>;
+export declare function isLoggedIn(html: string): boolean;
+/** A handle callers can use to force-close a launched login context, e.g. on Ctrl+C. */
+export interface LoginContextHandle {
+    close(): Promise<void>;
+}
+/**
+ * Opens a real, headed Chrome against a persistent profile and waits for the
+ * user to sign in by hand. Nothing is typed on their behalf and no credentials
+ * are read or stored — the session simply persists in the profile directory
+ * for later runs.
+ *
+ * `onContext`, if given, is invoked with a close() handle as soon as the
+ * context launches — callers that need to guarantee the profile lock is
+ * released on interruption (e.g. Ctrl+C mid-login) should register it there
+ * rather than only closing it at the end of this function.
+ */
+export declare function login(onContext?: (context: LoginContextHandle) => void): Promise<void>;

package/dist/auth/session.js

@@ -0,0 +1,70 @@
+import { homedir } from 'node:os';
+import { join } from 'node:path';
+import { mkdir } from 'node:fs/promises';
+import { extractNextData, PayloadParseError } from '../fetch/nextdata.js';
+const ROOT = join(homedir(), '.skool-skrape');
+export function profileDir() {
+    return join(ROOT, 'chrome-profile');
+}
+export function dbPath() {
+    return join(ROOT, 'skool.db');
+}
+/**
+ * Marker file written once the one-time Chrome/Playwright browser install
+ * has been confirmed (either found already installed, or installed by us).
+ * Its presence lets later runs skip re-probing with a real browser launch.
+ */
+export function chromeMarkerPath() {
+    return join(ROOT, 'chrome-installed');
+}
+export async function ensureRoot() {
+    await mkdir(ROOT, { recursive: true });
+}
+export function isLoggedIn(html) {
+    try {
+        const payload = extractNextData(html);
+        return Boolean(payload?.props?.pageProps?.self);
+    }
+    catch (error) {
+        if (error instanceof PayloadParseError)
+            return false;
+        throw error;
+    }
+}
+/**
+ * Opens a real, headed Chrome against a persistent profile and waits for the
+ * user to sign in by hand. Nothing is typed on their behalf and no credentials
+ * are read or stored — the session simply persists in the profile directory
+ * for later runs.
+ *
+ * `onContext`, if given, is invoked with a close() handle as soon as the
+ * context launches — callers that need to guarantee the profile lock is
+ * released on interruption (e.g. Ctrl+C mid-login) should register it there
+ * rather than only closing it at the end of this function.
+ */
+export async function login(onContext) {
+    await ensureRoot();
+    const { chromium } = await import('playwright');
+    const context = await chromium.launchPersistentContext(profileDir(), {
+        channel: 'chrome',
+        headless: false,
+    });
+    onContext?.(context);
+    const page = context.pages()[0] ?? (await context.newPage());
+    await page.goto('https://www.skool.com/login', { waitUntil: 'domcontentloaded' });
+    console.log('\nA browser window is open. Sign in to Skool there.');
+    console.log('Waiting for you to reach a logged-in page (Ctrl+C to cancel)...\n');
+    await page.waitForFunction(() => {
+        const el = document.getElementById('__NEXT_DATA__');
+        if (!el?.textContent)
+            return false;
+        try {
+            return Boolean(JSON.parse(el.textContent)?.props?.pageProps?.self);
+        }
+        catch {
+            return false;
+        }
+    }, undefined, { timeout: 0 });
+    console.log('Signed in. Session saved — future runs will not need this.');
+    await context.close();
+}

package/dist/cli.d.ts

@@ -0,0 +1,2 @@
+#!/usr/bin/env node
+export {};

package/dist/cli.js

@@ -0,0 +1,123 @@
+#!/usr/bin/env node
+import { Command } from 'commander';
+import { HttpFetcher } from './fetch/http.js';
+import { BrowserFetcher } from './fetch/browser.js';
+import { ResilientFetcher } from './fetch/resilient.js';
+import { openDb } from './store/db.js';
+import { syncClassroom } from './sync.js';
+import { login, profileDir, dbPath, ensureRoot, isLoggedIn } from './auth/session.js';
+const MARKS = {
+    ok: '+', skipped: '=', 'no-video': '.', 'no-access': '-', unavailable: '!', failed: 'x',
+};
+const MAX_CONCURRENCY = 16;
+/** Parse the --concurrency flag to a positive integer, or return null if invalid. */
+function parseConcurrency(raw) {
+    const n = Number(raw);
+    if (!Number.isInteger(n) || n < 1)
+        return null;
+    return Math.min(n, MAX_CONCURRENCY);
+}
+const program = new Command();
+program.name('skrape').description('Read Skool content instead of watching it');
+program
+    .command('login')
+    .description('Sign in to Skool once; the session is reused by later runs')
+    .action(async () => {
+    await login();
+});
+program
+    .command('sync')
+    .argument('<slug>', 'community slug, e.g. demo from skool.com/demo')
+    .option('-o, --out <dir>', 'output directory', './out')
+    .option('-c, --concurrency <n>', 'parallel requests', '4')
+    .description('Pull a community classroom to disk as transcripts')
+    .action(async (slug, options) => {
+    const concurrency = parseConcurrency(options.concurrency);
+    if (concurrency === null) {
+        console.error(`\nInvalid --concurrency value: "${options.concurrency}". ` +
+            `Must be a positive integer (capped at ${MAX_CONCURRENCY}).`);
+        process.exitCode = 1;
+        return;
+    }
+    await ensureRoot();
+    const outDir = `${options.out}/${slug}`;
+    const db = openDb(dbPath());
+    const browser = new BrowserFetcher(profileDir());
+    const fetcher = new ResilientFetcher(new HttpFetcher(), async () => browser, isLoggedIn);
+    try {
+        const summary = await syncClassroom({
+            slug,
+            outDir,
+            db,
+            fetcher,
+            concurrency,
+            onProgress: (event) => {
+                const mark = MARKS[event.outcome] ?? '?';
+                console.log(`${mark} [${event.done}/${event.total}] ${event.course.slice(0, 28).padEnd(28)} ${event.title.slice(0, 50)}`);
+            },
+        });
+        console.log('\n--- summary ---');
+        for (const [outcome, count] of Object.entries(summary.counts)) {
+            if (count > 0)
+                console.log(`  ${outcome.padEnd(14)} ${count}`);
+        }
+        console.log(`  ${'words'.padEnd(14)} ${summary.totalWords.toLocaleString('en-US')}`);
+        if (fetcher.escalatedRoutes.size > 0) {
+            console.log(`\n  escalated to browser: ${[...fetcher.escalatedRoutes].join(', ')}`);
+        }
+        if (summary.problems.length > 0) {
+            console.log('\n--- not transcribed ---');
+            for (const problem of summary.problems) {
+                console.log(`  [${problem.outcome}] ${problem.course} / ${problem.title}: ${problem.reason}`);
+            }
+        }
+        console.log(`\nOutput: ${outDir}`);
+    }
+    catch (error) {
+        console.error(`\nSync failed: ${error.message}`);
+        if (fetcher.escalatedRoutes.size > 0) {
+            console.error(`\n  escalated to browser: ${[...fetcher.escalatedRoutes].join(', ')}`);
+            console.error('If this mentions an unexpected payload, the authenticated browser path was already tried ' +
+                "for the route(s) above, so Skool's page structure most likely changed rather than the " +
+                'session being expired. A fresh `skool login` is still worth trying, but treat it as a ' +
+                'secondary guess.');
+        }
+        else {
+            console.error('If this mentions an unexpected payload, your session may have expired — run: skool login. ' +
+                "It is also possible Skool's page structure changed; if a retry after login fails the same " +
+                'way, that is more likely.');
+        }
+        process.exitCode = 1;
+    }
+    finally {
+        try {
+            await browser.close();
+        }
+        catch (closeError) {
+            // Cosmetic only — do not overwrite a real sync error or change the exit code.
+            console.error(`\nWarning: failed to close browser cleanly: ${closeError.message}`);
+        }
+        finally {
+            db.close();
+        }
+    }
+});
+/**
+ * With no subcommand, launch the guided Ink flow instead of requiring the user
+ * to already know a subcommand and a community slug. `skrape login` and
+ * `skrape sync <slug>` keep working unchanged as scriptable escape hatches.
+ * The TUI module is dynamically imported so plain subcommand invocations
+ * never pay for loading ink/react.
+ */
+async function main() {
+    if (process.argv.length <= 2) {
+        const { runGuidedFlow } = await import('./tui/run.js');
+        await runGuidedFlow();
+        return;
+    }
+    program.parse();
+}
+main().catch((error) => {
+    console.error(error instanceof Error ? error.message : String(error));
+    process.exitCode = 1;
+});

package/dist/discover/communities.d.ts

@@ -0,0 +1,22 @@
+import type { Fetcher } from '../types.js';
+export interface CommunityRef {
+    slug: string;
+    name: string;
+}
+/**
+ * Best-effort scan of the __NEXT_DATA__ payload's `self` object for the signed-in
+ * user's community memberships. Deliberately shape-tolerant rather than hardcoding
+ * one exact path: it walks `self` (bounded depth) looking for the first array whose
+ * every element matches a "group embed" shape (see asCommunityRef above). Returns
+ * null — never a guessed or partial list — when nothing matches, so callers know to
+ * fall back to manual slug entry instead of showing a fabricated list.
+ */
+export declare function extractUserGroups(payload: unknown, maxDepth?: number): CommunityRef[] | null;
+/**
+ * Fetches skool.com/ through the caller's authenticated fetcher stack and looks
+ * for the signed-in user's communities. Returns null (not an empty array) when
+ * the page didn't yield a parseable payload or no group-shaped list was found —
+ * both are treated the same way by callers: fall back to manual slug entry.
+ * A genuine network failure still propagates, distinct from a shape mismatch.
+ */
+export declare function listUserCommunities(fetcher: Fetcher): Promise<CommunityRef[] | null>;

package/dist/discover/communities.js

@@ -0,0 +1,92 @@
+import { extractNextData, PayloadParseError } from '../fetch/nextdata.js';
+function readSelf(payload) {
+    if (!payload || typeof payload !== 'object')
+        return undefined;
+    const props = payload['props'];
+    if (!props || typeof props !== 'object')
+        return undefined;
+    const pageProps = props['pageProps'];
+    if (!pageProps || typeof pageProps !== 'object')
+        return undefined;
+    return pageProps['self'];
+}
+/**
+ * Skool's public, logged-out discovery feed (skool.com/) embeds each community as
+ * `{ group: { name, metadata: { displayName } } }`, where `name` is the URL slug —
+ * confirmed by fetching that page live. Whether the authenticated `self` object
+ * reuses this exact shape for the signed-in user's own memberships could not be
+ * verified without real credentials, so this accepts either that wrapped shape or
+ * an unwrapped `{ name, metadata: { displayName } }` object.
+ */
+function asCommunityRef(candidate) {
+    if (!candidate || typeof candidate !== 'object')
+        return null;
+    const record = candidate;
+    const groupField = record['group'];
+    const inner = groupField && typeof groupField === 'object'
+        ? groupField
+        : record;
+    const slug = inner['name'];
+    if (typeof slug !== 'string' || slug.length === 0)
+        return null;
+    const metadata = inner['metadata'];
+    const displayName = metadata && typeof metadata === 'object'
+        ? metadata['displayName']
+        : undefined;
+    const name = typeof displayName === 'string' && displayName.length > 0 ? displayName : slug;
+    return { slug, name };
+}
+/**
+ * Best-effort scan of the __NEXT_DATA__ payload's `self` object for the signed-in
+ * user's community memberships. Deliberately shape-tolerant rather than hardcoding
+ * one exact path: it walks `self` (bounded depth) looking for the first array whose
+ * every element matches a "group embed" shape (see asCommunityRef above). Returns
+ * null — never a guessed or partial list — when nothing matches, so callers know to
+ * fall back to manual slug entry instead of showing a fabricated list.
+ */
+export function extractUserGroups(payload, maxDepth = 4) {
+    const self = readSelf(payload);
+    if (!self || typeof self !== 'object')
+        return null;
+    const visit = (node, depth) => {
+        if (depth > maxDepth || node === null || typeof node !== 'object')
+            return null;
+        if (Array.isArray(node) && node.length > 0) {
+            const refs = node.map(asCommunityRef);
+            if (refs.every((ref) => ref !== null)) {
+                const seen = new Set();
+                return refs.filter((ref) => {
+                    if (seen.has(ref.slug))
+                        return false;
+                    seen.add(ref.slug);
+                    return true;
+                });
+            }
+        }
+        for (const value of Object.values(node)) {
+            const found = visit(value, depth + 1);
+            if (found)
+                return found;
+        }
+        return null;
+    };
+    return visit(self, 0);
+}
+/**
+ * Fetches skool.com/ through the caller's authenticated fetcher stack and looks
+ * for the signed-in user's communities. Returns null (not an empty array) when
+ * the page didn't yield a parseable payload or no group-shaped list was found —
+ * both are treated the same way by callers: fall back to manual slug entry.
+ * A genuine network failure still propagates, distinct from a shape mismatch.
+ */
+export async function listUserCommunities(fetcher) {
+    const html = await fetcher.getPage('https://www.skool.com/');
+    try {
+        return extractUserGroups(extractNextData(html));
+    }
+    catch (error) {
+        if (error instanceof PayloadParseError)
+            return null;
+        throw error;
+    }
+}

package/dist/discover/skool.d.ts

@@ -0,0 +1,17 @@
+import type { ContentItem, Fetcher } from '../types.js';
+export interface CourseRef {
+    id: string;
+    title: string;
+    hasAccess: boolean;
+}
+export declare function parseCourseList(payload: unknown): CourseRef[];
+export interface Skipped {
+    reason: string;
+}
+export interface CourseTreeResult {
+    items: ContentItem[];
+    skipped: Skipped[];
+}
+export declare function parseCourseTree(payload: unknown, courseTitle: string): CourseTreeResult;
+export declare function listCourses(slug: string, fetcher: Fetcher): Promise<CourseRef[]>;
+export declare function listLessons(slug: string, course: CourseRef, fetcher: Fetcher): Promise<CourseTreeResult>;

package/dist/discover/skool.js

@@ -0,0 +1,91 @@
+import { z } from 'zod';
+import { extractNextData, PayloadParseError } from '../fetch/nextdata.js';
+const MetadataSchema = z.object({
+    title: z.string().optional().catch(undefined),
+    hasAccess: z.number().optional().catch(undefined),
+    videoLink: z.string().optional().catch(undefined),
+    videoLenMs: z.number().optional().catch(undefined),
+});
+const CourseListSchema = z.object({
+    props: z.object({
+        pageProps: z.object({
+            allCourses: z.array(z.object({ id: z.string(), metadata: MetadataSchema })),
+        }),
+    }),
+});
+const TreeSchema = z.object({
+    props: z.object({
+        pageProps: z.object({
+            course: z.object({ children: z.array(z.unknown()).optional() }),
+        }),
+    }),
+});
+export function parseCourseList(payload) {
+    const parsed = CourseListSchema.safeParse(payload);
+    if (!parsed.success)
+        throw new PayloadParseError(`unexpected classroom payload: ${parsed.error.message}`);
+    return parsed.data.props.pageProps.allCourses.map((course) => ({
+        id: course.id,
+        title: course.metadata.title ?? '(untitled course)',
+        // Fail-safe: courses without explicit access flag default to no access, since we must never
+        // probe content the user may not be entitled to. Inside a course (see parseCourseTree),
+        // lessons default to accessible because they're already within an accessible parent.
+        hasAccess: (course.metadata.hasAccess ?? 0) > 0,
+    }));
+}
+export function parseCourseTree(payload, courseTitle) {
+    const parsed = TreeSchema.safeParse(payload);
+    if (!parsed.success)
+        throw new PayloadParseError(`unexpected course payload: ${parsed.error.message}`);
+    const items = [];
+    const skipped = [];
+    let index = 0;
+    const walk = (nodes, section) => {
+        for (const node of nodes ?? []) {
+            // Guard against null or non-object elements that may appear as tombstones/placeholders
+            // in real Skool data. These are visibly accounted for as skipped, not silently dropped.
+            if (node === null || typeof node !== 'object') {
+                skipped.push({ reason: 'node skipped: null or non-object node' });
+                continue;
+            }
+            const metadata = MetadataSchema.safeParse(node.course?.metadata);
+            const meta = metadata.success ? metadata.data : {};
+            const title = meta.title;
+            if (title) {
+                items.push({
+                    nativeId: node.course?.id ?? `${courseTitle}:${index}`,
+                    type: 'lesson',
+                    title,
+                    index: index++,
+                    course: courseTitle,
+                    section,
+                    url: null,
+                    videoUrl: meta.videoLink ?? null,
+                    durationMs: meta.videoLenMs ?? 0,
+                    // Fail-safe: lessons without explicit access flag default to accessible, since we're already
+                    // inside a course the user has access to. Top-level courses default to no access (see parseCourseList).
+                    hasAccess: (meta.hasAccess ?? 1) > 0,
+                    publishedAt: null,
+                    bodyText: null,
+                });
+            }
+            else {
+                // A missing or non-string title (easy to hit, since each field's .catch(undefined) maps a
+                // bad type to undefined) must not make the node vanish uncounted — record it as skipped so
+                // the acceptance-critical lesson count still balances.
+                skipped.push({ reason: 'node skipped: no usable title' });
+            }
+            walk(node.children, title ?? section);
+        }
+    };
+    walk(parsed.data.props.pageProps.course.children, null);
+    return { items, skipped };
+}
+export async function listCourses(slug, fetcher) {
+    const html = await fetcher.getPage(`https://www.skool.com/${slug}/classroom`);
+    return parseCourseList(extractNextData(html));
+}
+export async function listLessons(slug, course, fetcher) {
+    const html = await fetcher.getPage(`https://www.skool.com/${slug}/classroom/${course.id}`);
+    return parseCourseTree(extractNextData(html), course.title);
+}

package/dist/fetch/browser.d.ts

@@ -0,0 +1,16 @@
+import type { Fetcher } from '../types.js';
+/**
+ * Playwright-backed fallback. Playwright is imported dynamically so that users
+ * who never hit the fallback never pay its startup cost — and so the package
+ * still works if the optional dependency is absent.
+ */
+export declare class BrowserFetcher implements Fetcher {
+    private readonly profileDir;
+    private context;
+    private contextPromise;
+    private closed;
+    constructor(profileDir: string);
+    private ensureContext;
+    getPage(url: string): Promise<string>;
+    close(): Promise<void>;
+}

package/dist/fetch/browser.js

@@ -0,0 +1,58 @@
+/**
+ * Playwright-backed fallback. Playwright is imported dynamically so that users
+ * who never hit the fallback never pay its startup cost — and so the package
+ * still works if the optional dependency is absent.
+ */
+export class BrowserFetcher {
+    profileDir;
+    context = null;
+    contextPromise = null;
+    closed = false;
+    constructor(profileDir) {
+        this.profileDir = profileDir;
+    }
+    async ensureContext() {
+        if (this.closed) {
+            throw new Error('BrowserFetcher is closed — cannot open a new context after close()');
+        }
+        if (!this.context) {
+            try {
+                this.contextPromise ??= (async () => {
+                    const { chromium } = await import('playwright');
+                    return await chromium.launchPersistentContext(this.profileDir, {
+                        channel: 'chrome',
+                        headless: true,
+                    });
+                })();
+                this.context = await this.contextPromise;
+            }
+            catch (error) {
+                // Clear the promise memo on rejection to allow retries
+                this.contextPromise = null;
+                throw error;
+            }
+        }
+        return this.context;
+    }
+    async getPage(url) {
+        const context = await this.ensureContext();
+        const page = await context.newPage();
+        try {
+            await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 45_000 });
+            return await page.content();
+        }
+        finally {
+            await page.close();
+        }
+    }
+    async close() {
+        this.closed = true;
+        if (this.context) {
+            await this.context.close();
+        }
+        // Clear both context and its promise memo — no fresh launch is possible after close()
+        // since ensureContext() now rejects once `closed` is set.
+        this.context = null;
+        this.contextPromise = null;
+    }
+}