minifetch-api 1.2.2 → 1.3.0

package/README.md

@@ -5,20 +5,18 @@
   </a>
 </div>
 
-**Fetch & extract data from web pages.** [Minifetch.com](https://minifetch.com) provides composable extraction APIs for humans and AI Agents-- making web pages simple to access.
+**Pay-per-URL SEO audits. Composable toolkit for AI agents and automation pipelines.** [Minifetch.com](https://minifetch.com) is an SEO toolkit of composable extraction primitives. Run them as a full technical audit or call one at a time for a fraction of the price — and a fraction of the LLM tokens. No subscription.
 
-- ✅ **[Sign up](https://minifetch.com/dashboard) for an account and get your first 125-250 fetches for free. 🎉🎉**
-- ✅ Always pay-as-you-go at competitive prices.
+- ✅ **[Sign up](https://minifetch.com/dashboard) for an account and get free starter credits. 🎉🎉**
+- ✅ Always pay-per-fetch at competitive prices.
 - ⛔ No charge for blocked pages (403 errors).
 
 ---
 **👉 [Dashboard](https://minifetch.com/dashboard) | [Full API docs](https://minifetch.com/docs/api) | [LLMs.txt](https://minifetch.com/llms.txt) | Questions? Join our [Discord](https://discord.gg/EM6ET8Dshm)**
 
-
-
 ---
 **Payments.** Two ways to pay:
-1. Credit card + API key. Get started free - [visit our dashboard to Sign Up](https://minifetch.com/dashboard). Your account will be auto-loaded with 125–250 API calls. Top up with your credit card later.
+1. Credit card + API key. Get started free - [visit our dashboard to Sign Up](https://minifetch.com/dashboard). Your account will be auto-loaded with 25 free technical SEO page audits. Top up with your credit card later.
 2. x402. USDC stablecoin micropayments on Coinbase's Base & Solana networks.
 
 ## Prerequisites
@@ -36,7 +34,8 @@
 ```js
 import Minifetch from "minifetch-api";
 
-// First, initialize the client with your payment choice:
+// First, initialize the client with your payment choice-
+// pick option 1 or 2:
 
 // 1. API Key Payments:
 const client = new Minifetch({
@@ -51,11 +50,11 @@ const client = new Minifetch({
   privateKey: process.env.BASE_PRIVATE_KEY,
 });
 
-// Now you're ready to extract data from a url:
+// Now you're ready to call the API methods:
 try {
   const url = "example.com";
-  const response = await client.checkAndExtractUrlPreview(url);
-  // 200 "ok" responses
+  const response = await client.checkAndRunSeoPageAudit(url);
+  // 200 "ok" responses:
   console.log(response);
 } catch (err) {
   // No charge for errors or 403 blocked urls!
@@ -63,57 +62,78 @@ try {
 }
 ```
 
-### Data Extraction Methods
-
-After the Quick Start, you have the following methods to use. The "checkAndExtract" methods help to avoid paying for blocked URLs. **Price list** & example data [is here.](https://minifetch.com/docs/api#example-data)
+### API Methods
 
-**Wrap** these in a **try/catch** just like in the Quickstart example above. **Code examples** can be also found in the [Github repository](https://github.com/Niche-Networks/minifetch-api/) in the /example- directories.
-
-```js
-// Extracts a light, token-efficient preview of a URL:
-// title, description, and image (only).
-client.checkAndExtractUrlPreview(url);
+After the Quick Start, you have the following methods to use.
 
-// Extracts a clean, token-efficient content summary as markdown from a URL.
-// Removes ads, nav, scripts. More efficient than raw HTML fetches for LLMs.
-// Options: { includeMediaUrls: true } - defaults to false.
-client.checkAndExtractUrlContent(url, options);
+**Wrap** these methods in a **try/catch** just like in the Quickstart example above. **Code examples** can be also found in the [Github repository](https://github.com/Niche-Networks/minifetch-api/) /example- directories.
 
-// Extracts all links from a URL categorized by type (internal/external/anchor).
-// Detects image links, nofollow attributes, analyzes external domain distribution.
-client.checkAndExtractUrlLinks(url);
+The "checkAndExtract" methods check the target URL's `robots.txt` file to ensure its not blocked and tell us your preferred crawl delay (defaults to 1 second between requests to your domain). So fetching 10 URLs takes at least 10 seconds to complete. This is by design, so Minifetch never hammers your server or slows it down for your real users. [Full api docs here.](https://minifetch.com/docs/api)
 
-// Extracts rich structured metadata from a URL:
-// meta tags, json-ld, images, headings, response headers, + more.
-// Setting verbosity to "full" is the drop-in replacement for the
-// [`url-metadata`](https://www.npmjs.com/package/url-metadata) package.
+```js
+await client.checkAndRunSeoPageAudit(url);
+// Price: $0.01
+// Runs a full technical SEO audit on your URL. It combines data from
+// the other API endpoints and runs checks that each return a PASS/
+// WARN/ FAIL result with no black-box scoring — just deterministic,
+// composable signal you can act on or pipe into an agent.
+// Audit rules are documented in the in the audit skill file:
+// https://minifetch.com/skills/seo-page-audit/SKILL.md
+
+await client.checkAndExtractUrlMetadata(url, options);
+// Price: $0.002
+// Fetches & extracts rich structured metadata from your URL: meta tags,
+// hreflang, json-ld, images, headings, response headers, + more.
+// Setting verbosity to "full" is the drop-in replacement for the npm
+// `url-metadata` package.
 // Options:
 // { verbosity: "full" } - defaults to "standard"
 // { includeResponseBody: true } - defaults to false
-client.checkAndExtractUrlMetadata(url, options);
+
+await client.checkAndExtractUrlLinks(url);
+// Price: $0.002
+// Fetches & extracts all links from your URL categorized by type
+// (internal/ external/ anchor). Detects image links, `rel` attributes
+// (nofollow, sponsored, ugc, etc), `title` and `target`, plus image
+// detection. Summary stats include the most-linked-to internal pages
+// (with anchor text variants used for each) and top external domains
+// by link count.
+
+await client.checkAndExtractUrlPreview(url);
+// Price: $0.001
+// For checking how your page unfurls when shared: Extracts the title,
+// meta description, and preview image (only) - the lightweight card
+// social platforms and chat apps render for a link.
+
+await client.checkAndExtractUrlContent(url, options);
+// Price: $0.002
+// For site owners auditing AI readability: returns the clean markdown
+// an LLM extracts from your page after nav, ads, & scripts are stripped.
+// See what survives for AEO and AI search; respects robots.txt.
+// Options: { includeMediaUrls: true } - defaults to false.
 
 // For max control, you can also use the following methods directly.
 // Free - check robots.txt:
-client.preflightCheck(url);
-// Extract data:
-client.extractUrlPreview(url);
-client.extractUrlContent(url, options); // same options as above
-client.extractUrlLinks(url);
-client.extractUrlMetadata(url, options) // same options as above
-
+await client.preflightCheck(url);
+// Paid methods:
+await client.runSeoPageAudit(url);
+await client.extractUrlMetadata(url, options); // same options as above
+await client.extractUrlLinks(url);
+await client.extractUrlPreview(url);
+await client.extractUrlContent(url, options); // same options as above
 ```
 ---
 ### Error Types
-When you wrap the functions above in a try/catch, here are the errors you may encounter. You are never charged for errors.
+When you wrap the functions above in a try/catch, here are the errors you may encounter. You are never charged for URLs that are blocked or error.
 
 - **"RobotsBlockedError: URL is blocked by robots.txt"**
-  - URL is explicitly blocked by the website's robots.txt, cannot be fetched.
+  - Minifetch is explicitly blocked by the website's `robots.txt`, cannot be fetched. If this is your site, read our tutorial [How To Unblock Minifetch](https://minifetch.com/tutorials/unblock-minifetch)
 - **"Network Error: 402 Payment Required"**
   - Check your wallet -- likely you ran out of USDC to pay!
 - **"Network Error: 429 Too Many Requests"**
   - Back off and retry, max 5-10 requests per second.
 - **"Network Error: 502 Bad Gateway"**
-  - URLs that pass their robots.txt check but are blocked anyway via 403 or other tactics may error like this.
+  - URLs that pass their robots.txt check but are blocked anyway via 403 or other tactics may error like this. No charge.
 - **"503 Service Temporarily Unavailable"**
   - Likely encountering upstream timeout errors on the target URL.
 - **"InvalidURLError: Invalid url ${url}"**
@@ -133,7 +153,7 @@ What Minifetch does NOT do *currently* but may offer in the future as an add-on:
 - Javascript execution
 
 ---
-### x402 Best Practices
+### x402 Micropayments Best Practices
 - LLMs & Agents should *never* have direct access to your private key. They *will* expose it!
 - Only keep a small amount of USDC in the account whose private key you use.
 - Keep that account separate from the rest of your onchain funds.

package/dist/esm/client.js

@@ -42,6 +42,26 @@ export class MinifetchClient {
             throw new NetworkError(`Preflight check failed: ${error instanceof Error ? error.message : "Unknown error"}`);
         }
     }
+    /**
+     * Run SEO page audit (paid endpoint)
+     *
+     * @param url
+     * @throws {InvalidUrlError} if URL is invalid
+     * @throws {ExtractionFailedError} various reasons, check README
+     * @throws {PaymentFailedError} if x402 payment fails
+     * @throws {NetworkError} various reasons, check README
+     */
+    async runSeoPageAudit(url) {
+        try {
+            const normalizedUrl = validateAndNormalizeUrl(url);
+            const params = new URLSearchParams({ url: normalizedUrl });
+            const requestUrl = `${this.baseUrl}${this._paidPath("/run/seo-page-audit")}?${params.toString()}`;
+            return await this._makeRequest(requestUrl, normalizedUrl, "Run SEO page audit");
+        }
+        catch (error) {
+            return this._rethrowError(error, url, "Run SEO page audit");
+        }
+    }
     /**
      * Extract URL metadata (paid endpoint)
      *
@@ -62,11 +82,11 @@ export class MinifetchClient {
                 params.set("verbosity", options.verbosity);
             if (options?.includeResponseBody)
                 params.set("includeResponseBody", "true");
-            const requestUrl = `${this.baseUrl}${this._extractPath("url-metadata")}?${params.toString()}`;
+            const requestUrl = `${this.baseUrl}${this._paidPath("/extract/url-metadata")}?${params.toString()}`;
             return await this._makeRequest(requestUrl, normalizedUrl, "Metadata extraction");
         }
         catch (error) {
-            return this._rethrowExtraction(error, url, "Metadata extraction");
+            return this._rethrowError(error, url, "Metadata extraction");
         }
     }
     /**
@@ -81,11 +101,11 @@ export class MinifetchClient {
     async extractUrlLinks(url) {
         try {
             const normalizedUrl = validateAndNormalizeUrl(url);
-            const requestUrl = `${this.baseUrl}${this._extractPath("url-links")}?url=${encodeURIComponent(normalizedUrl)}`;
+            const requestUrl = `${this.baseUrl}${this._paidPath("/extract/url-links")}?url=${encodeURIComponent(normalizedUrl)}`;
             return await this._makeRequest(requestUrl, normalizedUrl, "Links extraction");
         }
         catch (error) {
-            return this._rethrowExtraction(error, url, "Links extraction");
+            return this._rethrowError(error, url, "Links extraction");
         }
     }
     /**
@@ -100,11 +120,11 @@ export class MinifetchClient {
     async extractUrlPreview(url) {
         try {
             const normalizedUrl = validateAndNormalizeUrl(url);
-            const requestUrl = `${this.baseUrl}${this._extractPath("url-preview")}?url=${encodeURIComponent(normalizedUrl)}`;
+            const requestUrl = `${this.baseUrl}${this._paidPath("/extract/url-preview")}?url=${encodeURIComponent(normalizedUrl)}`;
             return await this._makeRequest(requestUrl, normalizedUrl, "Preview extraction");
         }
         catch (error) {
-            return this._rethrowExtraction(error, url, "Preview extraction");
+            return this._rethrowError(error, url, "Preview extraction");
         }
     }
     /**
@@ -124,13 +144,23 @@ export class MinifetchClient {
             const params = new URLSearchParams({ url: normalizedUrl });
             if (options?.includeMediaUrls)
                 params.set("includeMediaUrls", "true");
-            const requestUrl = `${this.baseUrl}${this._extractPath("url-content")}?${params.toString()}`;
+            const requestUrl = `${this.baseUrl}${this._paidPath("/extract/url-content")}?${params.toString()}`;
             return await this._makeRequest(requestUrl, normalizedUrl, "Content extraction");
         }
         catch (error) {
-            return this._rethrowExtraction(error, url, "Content extraction");
+            return this._rethrowError(error, url, "Content extraction");
         }
     }
+    /**
+     * Check URL then run SEO page audit in one call.
+     * Throws RobotsBlockedError if robots.txt blocks the URL.
+     *
+     * @param url
+     */
+    async checkAndRunSeoPageAudit(url) {
+        await this._preflightOrThrow(url);
+        return this.runSeoPageAudit(url);
+    }
     /**
      * Check URL then extract metadata in one call.
      * Throws RobotsBlockedError if robots.txt blocks the URL.
@@ -180,16 +210,16 @@ export class MinifetchClient {
     // Private helpers
     // ---------------------------------------------------------------------------
     /**
-     * Returns the correct extract path segment based on auth mode.
-     * x402 → /api/v1/x402/extract/<endpoint>
-     * apiKey → /api/v1/extract/<endpoint>
+     * Returns the correct paid path segment based on auth mode.
+     * x402 → /api/v1/x402/<endpoint>
+     * apiKey → /api/v1/<endpoint>
      *
      * @param endpoint
      */
-    _extractPath(endpoint) {
+    _paidPath(endpoint) {
         return this.config.authMode === "x402"
-            ? `/api/v1/x402/extract/${endpoint}`
-            : `/api/v1/extract/${endpoint}`;
+            ? `/api/v1/x402${endpoint}`
+            : `/api/v1${endpoint}`;
     }
     /**
      * Dispatch to the correct request handler based on auth mode, then
@@ -237,7 +267,7 @@ export class MinifetchClient {
      * @param url
      * @param label
      */
-    _rethrowExtraction(error, url, label) {
+    _rethrowError(error, url, label) {
         if (error instanceof InvalidUrlError ||
             error instanceof ExtractionFailedError ||
             error instanceof PaymentFailedError ||

package/dist/types/client.d.ts

@@ -21,6 +21,16 @@ export declare class MinifetchClient {
      * @throws {NetworkError} if request fails
      */
     preflightUrlCheck(url: string): Promise<PreflightCheckResponse>;
+    /**
+     * Run SEO page audit (paid endpoint)
+     *
+     * @param url
+     * @throws {InvalidUrlError} if URL is invalid
+     * @throws {ExtractionFailedError} various reasons, check README
+     * @throws {PaymentFailedError} if x402 payment fails
+     * @throws {NetworkError} various reasons, check README
+     */
+    runSeoPageAudit(url: string): Promise<PaidEndpointResponse>;
     /**
      * Extract URL metadata (paid endpoint)
      *
@@ -71,6 +81,13 @@ export declare class MinifetchClient {
     extractUrlContent(url: string, options?: {
         includeMediaUrls?: boolean;
     }): Promise<PaidEndpointResponse>;
+    /**
+     * Check URL then run SEO page audit in one call.
+     * Throws RobotsBlockedError if robots.txt blocks the URL.
+     *
+     * @param url
+     */
+    checkAndRunSeoPageAudit(url: string): Promise<PaidEndpointResponse>;
     /**
      * Check URL then extract metadata in one call.
      * Throws RobotsBlockedError if robots.txt blocks the URL.
@@ -110,13 +127,13 @@ export declare class MinifetchClient {
         includeMediaUrls?: boolean;
     }): Promise<PaidEndpointResponse>;
     /**
-     * Returns the correct extract path segment based on auth mode.
-     * x402 → /api/v1/x402/extract/<endpoint>
-     * apiKey → /api/v1/extract/<endpoint>
+     * Returns the correct paid path segment based on auth mode.
+     * x402 → /api/v1/x402/<endpoint>
+     * apiKey → /api/v1/<endpoint>
      *
      * @param endpoint
      */
-    private _extractPath;
+    private _paidPath;
     /**
      * Dispatch to the correct request handler based on auth mode, then
      * normalize the response into PaidEndpointResponse.
@@ -140,5 +157,5 @@ export declare class MinifetchClient {
      * @param url
      * @param label
      */
-    private _rethrowExtraction;
+    private _rethrowError;
 }

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "minifetch-api",
-  "version": "1.2.2",
-  "description": "Minifetch.com API Client. Fetch & extract data from web pages. For AI Agents & SEO research.",
+  "version": "1.3.0",
+  "description": "Minifetch.com API Client. Pay-per-URL SEO audits. Composable toolkit for AI agents and automation pipelines.",
   "type": "module",
   "main": "./dist/esm/index.js",
   "types": "./dist/types/index.d.ts",
@@ -17,16 +17,37 @@
     "LICENSE"
   ],
   "keywords": [
-    "seo",
+    "SEO",
+    "SEO page audit",
+    "SEO research",
+    "SEO toolkit",
+    "technical SEO",
+    "pay-per-url SEO",
+    "GEO",
+    "AEO",
+    "extract",
+    "content extraction",
+    "content indexing",
+    "html",
+    "html parser",
+    "json-ld",
+    "JSON LD",
+    "link analysis",
+    "link extraction",
     "metadata",
-    "meta-tags",
-    "web-pages",
-    "web-scraping",
-    "scraper",
-    "html-parser",
-    "ai",
-    "ai-agents",
-    "llm",
+    "meta tags",
+    "open graph",
+    "og tags",
+    "AI",
+    "AI Agents",
+    "AI readability",
+    "agent skills",
+    "automation",
+    "monitoring",
+    "LLM",
+    "previews",
+    "twitter cards",
+    "site monitoring",
     "x402",
     "micropayments",
     "usdc"
@@ -37,7 +58,7 @@
   "homepage": "https://minifetch.com",
   "repository": {
     "type": "git",
-    "url": "git@github.com:Niche-Networks/minifetch-api.git"
+    "url": "git+ssh://git@github.com/Niche-Networks/minifetch-api.git"
   },
   "scripts": {
     "dist:clean": "rm -rf dist",