wellmarked 1.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 WellMarked
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,233 @@
+# wellmarked
+
+[![npm](https://img.shields.io/npm/v/wellmarked.svg)](https://www.npmjs.com/package/wellmarked)
+[![types](https://img.shields.io/npm/types/wellmarked.svg)](https://www.npmjs.com/package/wellmarked)
+
+Official JavaScript/TypeScript SDK for the **[WellMarked](https://wellmarked.io)** API — convert any URL to clean Markdown.
+
+```bash
+npm install wellmarked
+```
+
+Requires Node.js 18+ (uses the built-in `fetch`). Works in any modern runtime with a global `fetch` — Node 18+, Deno, Bun, Cloudflare Workers, Vercel Edge, browsers.
+
+## Quick start
+
+```typescript
+import { WellMarked } from "wellmarked";
+
+const wm = new WellMarked({ apiKey: "wm_..." });
+
+const result = await wm.extract("https://example.com/article");
+console.log(result.markdown);
+console.log(result.metadata.title, "by", result.metadata.author);
+console.log("retrieved at", result.metadata.retrievedAt);
+```
+
+`result.metadata.retrievedAt` is a `Date` (UTC) recording when WellMarked actually fetched the page — distinct from `result.metadata.date` (the article's published date, often `null`). Useful for cache-freshness checks on the caller's side.
+
+The API key can also be picked up from the `WELLMARKED_API_KEY` environment variable, in which case `new WellMarked()` is enough.
+
+Get a key at [wellmarked.io](https://wellmarked.io).
+
+## Pricing
+
+|                       | Free      | Pro                  | Enterprise    |
+|-----------------------|-----------|----------------------|---------------|
+| **Monthly Price**     | $0        | $29/mo               | $199/mo       |
+| **Annual Price**      | —         | $299/yr              | $1,999/yr     |
+| **Included Requests** | 500/mo    | 7,500/mo             | 150,000/mo    |
+| **Bulk Requests**     | ❌         | ✅ (up to 50/request) | ✅ (Unlimited) |
+| **Crawl**             | ❌         | ✅ (depth 5, 1k pages)| ✅ (Unlimited) |
+| **Overage Rate**      | —         | $0.004/req           | $0.002/req    |
+| **JS Rendering**      | ❌         | ✅                    | ✅             |
+| **Priority Queue**    | Standard  | High                 | Highest       |
+
+See additional pricing information at [wellmarked.io/#pricing](https://wellmarked.io/#pricing).
+
+## CommonJS
+
+Both ESM and CommonJS are published. The package's `exports` field routes consumers automatically:
+
+```javascript
+const { WellMarked } = require("wellmarked");
+
+const wm = new WellMarked({ apiKey: "wm_..." });
+const result = await wm.extract("https://example.com/article");
+```
+
+## Bulk extraction
+
+Submit many URLs at once (Pro: up to 50; Enterprise: unlimited). The call returns immediately with a `jobId`. Poll with `getJob` or block until done with `waitForJob`.
+
+```typescript
+let job = await wm.bulk([
+  "https://example.com/article-1",
+  "https://example.com/article-2",
+]);
+job = await wm.waitForJob(job.jobId);        // resolves when status === "done"
+
+for (const item of job.results) {
+  if (item.ok) {
+    console.log(item.metadata!.title);
+  } else {
+    console.log(`${item.url} failed: ${item.error}`);
+  }
+}
+```
+
+`getJob` and `waitForJob` are **polymorphic** — they work for both bulk and crawl `jobId`s. The SDK reads a `kind` discriminator from the API response and returns either a `BulkJob` or a `CrawlJob`. Use the `isCrawlJob(job)` type guard (or check `job.kind === "crawl"`) before reading crawl-specific fields like `job.truncated` or `item.depth`.
+
+```typescript
+import { isCrawlJob } from "wellmarked";
+
+const job = await wm.waitForJob(someJobId);
+if (isCrawlJob(job)) {
+  console.log("truncated:", job.truncated, job.truncatedReason);
+}
+```
+
+## Crawl
+
+Crawl a site BFS-style from a root URL — same-site links only, with per-plan depth and page caps (Pro: depth 5, up to 1,000 pages; Enterprise: unlimited). Like `bulk`, this returns a queued job; poll with `getJob` or block until done with `waitForJob` — the same two methods work on both kinds.
+
+```typescript
+let job = await wm.crawl("https://docs.example.com", { depth: 2 });
+job = await wm.waitForJob(job.jobId);        // works for crawl AND bulk jobIds
+
+for (const page of job.results) {
+  if (page.ok) {
+    console.log(`depth=${page.depth} ${page.metadata!.title}`);
+  } else {
+    console.log(`${page.url} failed: ${page.error}`);
+  }
+}
+
+if (job.kind === "crawl" && job.truncated) {
+  console.log(`crawl stopped early: ${job.truncatedReason}`);
+}
+```
+
+Each successful page consumes one request from your monthly quota — failed pages (timeouts, robots-disallowed, no-content) are not billed. If you run out of quota mid-crawl the job finishes with `truncated: true`, `truncatedReason: "quota_exhausted"`.
+
+## Custom headers
+
+Pass extra HTTP headers on every request — useful for correlation IDs, multi-tenant identifiers, or a custom user-agent suffix:
+
+```typescript
+const wm = new WellMarked({
+  apiKey: "wm_...",
+  headers: { "X-Trace-Id": "req-abc-123", "X-Tenant": "acme" },
+});
+await wm.extract("https://example.com");
+```
+
+Headers can also be added or removed at runtime:
+
+```typescript
+wm.setHeader("X-Run-Id", "run-99");
+await wm.extract(/* ... */);                 // carries X-Run-Id
+wm.removeHeader("X-Run-Id");
+```
+
+`Authorization`, `Content-Type`, and `Accept` are reserved — the SDK manages them itself, and entries passed in `headers:` for those keys are silently ignored. To rotate the bearer token, use `rotateKey()`.
+
+## Usage & rate limits
+
+`getUsage()` is the source of truth for your current-period quota. The quota state belongs on the account, so call `getUsage()` when you want it:
+
+```typescript
+const usage = await wm.getUsage();
+console.log(
+  `${usage.used} / ${usage.limit} used this period (${usage.plan}) — ${usage.remaining} left`,
+);
+```
+
+`GET /usage` itself does not count toward your quota.
+
+## Key rotation
+
+```typescript
+const rotated = await wm.rotateKey();
+console.log("New key:", rotated.apiKey);  // shown once — store it before the program exits
+```
+
+After `rotateKey()` the client automatically switches to the new key for subsequent calls; you still need to persist `rotated.apiKey` somewhere durable, because the previous key stops working immediately and there is no recovery flow.
+
+## Errors
+
+Every non-2xx response is translated into a typed error class. Catch the base class to handle anything, or the specific subclass to handle one failure mode:
+
+```typescript
+import {
+  WellMarked,
+  AuthenticationError,
+  PermissionDeniedError,
+  NotFoundError,
+  UnprocessableEntityError,
+  RateLimitError,
+  APIConnectionError,
+} from "wellmarked";
+
+const wm = new WellMarked();
+try {
+  await wm.extract("https://example.com/paywalled");
+} catch (err) {
+  if (err instanceof RateLimitError) {
+    console.log(`Quota hit. Resets in ${err.retryAfter}s.`);
+  } else if (err instanceof UnprocessableEntityError) {
+    // err.code is one of: no_content, target_timeout, js_rendering_disabled, ...
+    console.log(`Extraction failed (${err.code}): ${err.message}`);
+  } else {
+    throw err;
+  }
+}
+```
+
+| Error class                | HTTP | Typical `code` values                                                                                |
+|----------------------------|------|------------------------------------------------------------------------------------------------------|
+| `AuthenticationError`      | 401  | `missing_api_key`, `invalid_api_key`                                                                 |
+| `PermissionDeniedError`    | 403  | `account_inactive`, `plan_not_supported`, `forbidden`                                                |
+| `NotFoundError`            | 404  | `job_not_found`                                                                                      |
+| `UnprocessableEntityError` | 422  | `no_content`, `target_timeout`, `js_rendering_disabled`, `bulk_cap_exceeded`, `crawl_depth_exceeded` |
+| `RateLimitError`           | 429  | `rate_limit_exceeded`  *(carries `retryAfter` in seconds)*                                           |
+| `InternalServerError`      | 5xx  | —                                                                                                    |
+| `APIConnectionError`       | —    | DNS / TCP / TLS / timeout failures, raised before any HTTP round-trip                                |
+
+All inherit from `WellMarkedError`.
+
+## Configuration
+
+```typescript
+new WellMarked({
+  apiKey: "wm_...",                         // or set WELLMARKED_API_KEY
+  baseUrl: "https://api.wellmarked.io",
+  timeoutMs: 30_000,                        // per request, default 30s
+  fetch: customFetch,                       // optional: bring your own fetch
+  headers: { "X-Trace-Id": "..." },         // optional: extra headers on every request
+});
+```
+
+Passing your own `fetch` is useful for custom proxies, polyfills (e.g. `undici` with a custom dispatcher), or test mocking. Any function with the standard `fetch` signature works.
+
+## TypeScript
+
+Types are bundled. Every public model and error class is exported. The polymorphic `getJob` / `waitForJob` return `BulkJob | CrawlJob` — use the `kind` discriminator or the `isBulkJob`/`isCrawlJob` type guards to narrow.
+
+```typescript
+import type {
+  BulkJob,
+  CrawlJob,
+  ExtractResult,
+  ExtractionMeta,
+  Usage,
+} from "wellmarked";
+```
+
+## For Agents
+
+If you are an agent, feel free to find additional context [here](https://wellmarked.io/llms.txt)!
+
+## License
+
+MIT

package/dist/cjs/client.cjs

@@ -0,0 +1,332 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.APIStatusError = exports.WellMarked = void 0;
+/**
+ * WellMarked client.
+ *
+ * The client is a thin, typed wrapper around the HTTP API. All endpoint
+ * methods are async — there is no separate sync/async split as in the
+ * Python SDK because JavaScript I/O is async by default.
+ *
+ *     import { WellMarked } from "wellmarked";
+ *
+ *     const wm = new WellMarked({ apiKey: "wm_..." });
+ *     const result = await wm.extract("https://example.com/article");
+ *     console.log(result.markdown);
+ *
+ * The API key can also be passed via the `WELLMARKED_API_KEY` environment
+ * variable (Node.js), in which case `new WellMarked()` is enough.
+ */
+const errors_js_1 = require("./errors.cjs");
+Object.defineProperty(exports, "APIStatusError", { enumerable: true, get: function () { return errors_js_1.APIStatusError; } });
+const models_js_1 = require("./models.cjs");
+const version_js_1 = require("./version.cjs");
+const DEFAULT_BASE_URL = "https://api.wellmarked.io";
+const DEFAULT_TIMEOUT_MS = 30000;
+const RESERVED_HEADERS = new Set([
+    "authorization",
+    "content-type",
+    "accept",
+]);
+function resolveApiKey(apiKey) {
+    if (apiKey)
+        return apiKey;
+    const env = typeof process !== "undefined" && process.env
+        ? process.env.WELLMARKED_API_KEY
+        : undefined;
+    if (env)
+        return env;
+    throw new Error("No API key provided. Pass apiKey: ... to the client or set the " +
+        "WELLMARKED_API_KEY environment variable. Generate a key at " +
+        "https://wellmarked.io.");
+}
+function defaultHeaders(apiKey) {
+    return {
+        Authorization: `Bearer ${apiKey}`,
+        "Content-Type": "application/json",
+        Accept: "application/json",
+        "User-Agent": `wellmarked-js/${version_js_1.VERSION}`,
+    };
+}
+function mergeHeaders(apiKey, extra) {
+    const out = defaultHeaders(apiKey);
+    if (!extra)
+        return out;
+    for (const [k, v] of Object.entries(extra)) {
+        if (RESERVED_HEADERS.has(k.toLowerCase()))
+            continue;
+        out[k] = v;
+    }
+    return out;
+}
+function sleep(ms) {
+    return new Promise((resolve) => setTimeout(resolve, ms));
+}
+class WellMarked {
+    constructor(options = {}) {
+        this.apiKey = resolveApiKey(options.apiKey);
+        this.baseUrl = (options.baseUrl ?? DEFAULT_BASE_URL).replace(/\/+$/, "");
+        this.timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
+        const f = options.fetch ?? (typeof fetch !== "undefined" ? fetch : undefined);
+        if (!f) {
+            throw new Error("No fetch implementation available. Pass `fetch:` to the client " +
+                "(undici, node-fetch, etc.) or upgrade to Node 18+.");
+        }
+        // Bind so `this` isn't lost when calling globalThis.fetch.
+        this.fetchImpl = f.bind(globalThis);
+        this.extraHeaders = {};
+        if (options.headers) {
+            for (const [k, v] of Object.entries(options.headers)) {
+                if (RESERVED_HEADERS.has(k.toLowerCase()))
+                    continue;
+                this.extraHeaders[k] = v;
+            }
+        }
+    }
+    // ── Endpoints ──────────────────────────────────────────────────────────────
+    /**
+     * Extract clean Markdown from a single URL.
+     *
+     * Throws:
+     *   - `RateLimitError`            — monthly plan limit reached.
+     *   - `UnprocessableEntityError`  — `no_content`, `target_timeout`, or
+     *                                   `js_rendering_disabled`.
+     *   - `AuthenticationError`       — missing or invalid API key.
+     */
+    async extract(url, options = {}) {
+        const body = await this.request("POST", "/extract", {
+            url,
+            render_js: options.renderJs === true,
+        });
+        return (0, models_js_1.extractResultFromResponse)(body);
+    }
+    /**
+     * Submit a batch of URLs for concurrent extraction.
+     *
+     * Returns immediately with `status="queued"`. Poll with `getJob` or
+     * block with `waitForJob` to collect results.
+     *
+     * Throws:
+     *   - `PermissionDeniedError`     — `plan_not_supported` (Free tier).
+     *   - `UnprocessableEntityError`  — `bulk_cap_exceeded` (50 on Pro).
+     *   - `RateLimitError`            — would exceed remaining monthly quota.
+     */
+    async bulk(urls, options = {}) {
+        const urlList = Array.from(urls);
+        if (urlList.length === 0) {
+            throw new Error("bulk() requires at least one URL.");
+        }
+        const body = await this.request("POST", "/bulk", {
+            urls: urlList,
+            render_js: options.renderJs === true,
+        });
+        return (0, models_js_1.bulkJobFromResponse)(body);
+    }
+    /**
+     * Polymorphic job lookup — works for both bulk and crawl jobs.
+     *
+     * Calls `GET /bulk/{jobId}` first, then inspects the response's `kind`
+     * discriminator field. If the job is actually a crawl, a second request
+     * to `GET /crawl/{jobId}` fetches the full crawl shape (with per-item
+     * depth and the truncated flags). Returns `BulkJob` or `CrawlJob`
+     * accordingly.
+     *
+     * Use `isCrawlJob(job)` (or check `job.kind === "crawl"`) to branch on
+     * crawl-specific behavior. The shared interface (`status`, `completed`,
+     * `total`, `results`, `done`) works on either type.
+     *
+     * Jobs are retained for 6 hours after completion.
+     */
+    async getJob(jobId) {
+        const body = (await this.request("GET", `/bulk/${jobId}`));
+        // /bulk/{id} answers for any jobId today (the endpoint just serializes
+        // results in the bulk shape regardless of stored job_type). The `kind`
+        // field tells us whether we got a bulk-shaped response of a crawl
+        // job; if so, re-fetch via /crawl/{id} for the proper shape.
+        if (body.kind === "crawl") {
+            const crawlBody = (await this.request("GET", `/crawl/${jobId}`));
+            return (0, models_js_1.crawlJobFromResponse)(crawlBody);
+        }
+        return (0, models_js_1.bulkJobFromResponse)(body);
+    }
+    /**
+     * Block until a job reaches `status="done"` (or timeout). Works for both
+     * bulk and crawl jobs.
+     *
+     * The first call uses the polymorphic `getJob` to discover the job's
+     * kind. Subsequent polls go directly to the typed endpoint, so a crawl
+     * job only pays the dispatch round-trip once.
+     *
+     * Throws:
+     *   - `Error` with message "did not finish within ..." — the job didn't
+     *     finish before `timeoutMs` elapsed.
+     */
+    async waitForJob(jobId, options = {}) {
+        const pollIntervalMs = options.pollIntervalMs ?? 2000;
+        const timeoutMs = options.timeoutMs === undefined ? 300000 : options.timeoutMs;
+        const deadline = timeoutMs === null ? null : Date.now() + timeoutMs;
+        let job = await this.getJob(jobId);
+        const isCrawl = job.kind === "crawl";
+        while (!job.done) {
+            if (deadline !== null && Date.now() >= deadline) {
+                throw new Error(`Job ${jobId} did not finish within ${timeoutMs}ms ` +
+                    `(last status: ${job.status}, ${job.completed}/${job.total})`);
+            }
+            await sleep(pollIntervalMs);
+            const path = isCrawl ? `/crawl/${jobId}` : `/bulk/${jobId}`;
+            const body = (await this.request("GET", path));
+            job = isCrawl ? (0, models_js_1.crawlJobFromResponse)(body) : (0, models_js_1.bulkJobFromResponse)(body);
+        }
+        return job;
+    }
+    /**
+     * Crawl a site starting from `url`, BFS to `depth`.
+     *
+     * Returns immediately with `status="queued"`. Use `getJob` to poll, or
+     * `waitForJob` to block until done — both handle crawl and bulk jobIds
+     * transparently.
+     *
+     * Plan caps:
+     *   - Free        → `PermissionDeniedError` (`plan_not_supported`)
+     *   - Pro         → max depth 5, up to 1,000 pages per crawl
+     *   - Enterprise  → unlimited depth and pages
+     *
+     * Throws:
+     *   - `PermissionDeniedError`     — `plan_not_supported` (Free tier).
+     *   - `UnprocessableEntityError`  — `crawl_depth_exceeded`.
+     */
+    async crawl(url, options = {}) {
+        const depth = options.depth ?? 1;
+        if (depth < 0) {
+            throw new Error("depth must be >= 0.");
+        }
+        const body = await this.request("POST", "/crawl", {
+            url,
+            depth,
+            render_js: options.renderJs === true,
+        });
+        return (0, models_js_1.crawlJobFromResponse)(body);
+    }
+    // ── Custom headers ─────────────────────────────────────────────────────────
+    /**
+     * Add or replace a per-request header for the rest of this client's life.
+     *
+     * Authorization / Content-Type / Accept are reserved — calls that try
+     * to set those are silently ignored. To rotate the bearer token, use
+     * `rotateKey()`.
+     */
+    setHeader(name, value) {
+        if (RESERVED_HEADERS.has(name.toLowerCase()))
+            return;
+        this.extraHeaders[name] = value;
+    }
+    /** Remove a header previously added via `headers:` or `setHeader()`. */
+    removeHeader(name) {
+        delete this.extraHeaders[name];
+    }
+    /**
+     * Return your usage for the current billing period.
+     *
+     * Does not count toward your monthly quota.
+     */
+    async getUsage() {
+        const body = await this.request("GET", "/usage");
+        return (0, models_js_1.usageFromResponse)(body);
+    }
+    /**
+     * Mint a new API key. The current key is invalidated immediately.
+     *
+     * The new raw key is in the returned `apiKey` field — store it before
+     * discarding the result. There is no recovery flow.
+     *
+     * The client auto-swaps to the new key for subsequent requests.
+     *
+     * Does not count toward your monthly quota.
+     */
+    async rotateKey() {
+        const body = await this.request("POST", "/keys/rotate");
+        const rotated = (0, models_js_1.rotatedKeyFromResponse)(body);
+        if (rotated.apiKey) {
+            this.apiKey = rotated.apiKey;
+        }
+        return rotated;
+    }
+    /**
+     * Internal: read the current API key. Exposed for tests.
+     * Not part of the public, semver-stable surface.
+     */
+    _getApiKey() {
+        return this.apiKey;
+    }
+    // ── Transport ──────────────────────────────────────────────────────────────
+    async request(method, path, json) {
+        const url = `${this.baseUrl}${path}`;
+        const headers = mergeHeaders(this.apiKey, this.extraHeaders);
+        const init = { method, headers };
+        if (json !== undefined) {
+            init.body = JSON.stringify(json);
+        }
+        let controller = null;
+        let timer = null;
+        if (this.timeoutMs > 0 && typeof AbortController !== "undefined") {
+            controller = new AbortController();
+            init.signal = controller.signal;
+            timer = setTimeout(() => controller.abort(), this.timeoutMs);
+        }
+        let response;
+        try {
+            response = await this.fetchImpl(url, init);
+        }
+        catch (err) {
+            throw new errors_js_1.APIConnectionError(`Could not reach the WellMarked API: ${stringifyError(err)}`, { cause: err });
+        }
+        finally {
+            if (timer !== null)
+                clearTimeout(timer);
+        }
+        let bodyText = "";
+        try {
+            bodyText = await response.text();
+        }
+        catch (err) {
+            throw new errors_js_1.APIConnectionError(`Could not read API response body: ${stringifyError(err)}`, { cause: err });
+        }
+        let body = null;
+        if (bodyText.length > 0) {
+            try {
+                body = JSON.parse(bodyText);
+            }
+            catch {
+                body = null;
+            }
+        }
+        return parseResponse(response.status, body);
+    }
+}
+exports.WellMarked = WellMarked;
+function parseResponse(statusCode, body) {
+    let requestId;
+    if (body && typeof body === "object") {
+        const rid = body.request_id;
+        if (typeof rid === "string")
+            requestId = rid;
+    }
+    if (statusCode >= 200 && statusCode < 300) {
+        if (body === null) {
+            // The API contract says every documented endpoint returns a JSON
+            // body on 2xx. A null body means the server broke that contract
+            // (or a middlebox stripped it); fail loudly rather than letting
+            // downstream parsing crash on `body.foo` of null.
+            throw new errors_js_1.WellMarkedError(`API returned HTTP ${statusCode} with no JSON body. ` +
+                "This is a contract violation — please report it.", { statusCode });
+        }
+        return body;
+    }
+    throw (0, errors_js_1.fromResponse)(statusCode, body, requestId);
+}
+function stringifyError(err) {
+    if (err instanceof Error) {
+        return `${err.name}: ${err.message}`;
+    }
+    return String(err);
+}