@mambalabsdev/mcp-substack-scraper 0.0.0-stage → 1.0.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mamba Labs
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,122 @@
-# Temporary Holding Version
+# Substack Scraper MCP Server
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+[![Glama score](https://glama.ai/mcp/servers/mambalabsdev/mcp-substack-scraper/badges/score.svg)](https://glama.ai/mcp/servers/mambalabsdev/mcp-substack-scraper) [![MCP Registry](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fregistry.modelcontextprotocol.io%2Fv0%2Fservers%3Fsearch%3Dcom.mambabuilt%252Fmcp-substack-scraper%26limit%3D1&query=%24.servers%5B0%5D._meta%5B%22io.modelcontextprotocol.registry%2Fofficial%22%5D.status&label=mcp%20registry&color=blue)](https://registry.modelcontextprotocol.io/v0/servers?search=com.mambabuilt/mcp-substack-scraper&limit=1) [![npm version](https://img.shields.io/npm/v/@mambalabsdev/mcp-substack-scraper)](https://www.npmjs.com/package/@mambalabsdev/mcp-substack-scraper) [![npm downloads](https://img.shields.io/npm/dm/@mambalabsdev/mcp-substack-scraper)](https://www.npmjs.com/package/@mambalabsdev/mcp-substack-scraper) [![license](https://img.shields.io/github/license/mambalabsdev/mcp-substack-scraper)](https://github.com/mambalabsdev/mcp-substack-scraper/blob/main/LICENSE) [![mcpservers.org](https://img.shields.io/badge/mcpservers.org-listed-blue)](https://mcpservers.org/servers/mambalabsdev/mcp-substack-scraper)
+
+<a href="https://mcprush.com/to/mcp-substack-scraper?ref=badge"><img width="380" height="200" src="https://mcprush.com/badge/mcp-substack-scraper/card.svg" alt="Substack Scraper MCP server" /></a>
+
+An MCP server for the Mamba Labs [Substack Newsletter and Posts Scraper](https://apify.com/mambalabs/substack-scraper) actor on Apify. Give it Substack publications or keywords. It returns publication profiles and posts as flat rows.
+
+## What's Inside
+
+- [What it does](#what-it-does)
+- [Quick start](#quick-start)
+- [Prerequisites](#prerequisites)
+- [Example prompts](#example-prompts)
+- [Tool](#tool)
+- [Inputs](#inputs)
+- [Pricing](#pricing)
+- [Your keys](#your-keys)
+- [Full actor documentation](#full-actor-documentation)
+- [License](#license)
+
+## What it does
+
+- **Publications**: name, author, tagline, subscriber count as Substack shows it (`subscriber_count_text`, and `subscriber_count_min` as a number), subscriber tier text, and monthly, yearly, and founding plan prices.
+- **Posts**: title, URL, date, `audience` (free, paid, or founding), likes, comments, restacks, word count, bylines, and tags. Optional post text with `include_body`.
+- **Search**: keyword search across Substack posts, returned as posts or as the publications behind them.
+
+All of the work runs on Apify. This package is a thin client that starts the actor run, waits for it, and hands back the dataset rows.
+
+## Quick start
+
+You need Node.js 18 or newer and an Apify account with an API token.
+
+Add this to your Claude Desktop config (`claude_desktop_config.json`), Cursor, or any MCP client:
+
+```json
+{
+  "mcpServers": {
+    "mamba-substack-scraper": {
+      "command": "npx",
+      "args": [
+        "-y",
+        "@mambalabsdev/mcp-substack-scraper"
+      ],
+      "env": {
+        "APIFY_TOKEN": "your-apify-token"
+      }
+    }
+  }
+}
+```
+
+Restart the client. The `scrape_substack` tool appears.
+
+## Prerequisites
+
+- Node.js 18 or newer.
+- An Apify API token from [console.apify.com/account/integrations](https://console.apify.com/account/integrations), set as `APIFY_TOKEN`. The server lists its tool without a token; a call needs one.
+
+## Example prompts
+
+- "How many subscribers does Lenny's Newsletter have, and what does a paid plan cost?"
+- "Find Substack newsletters that write about cold email and list their subscriber counts."
+- "What did notboring.co publish in the last 30 days, and which posts were paid only?"
+
+## Tool
+
+`scrape_substack`. One tool. Example arguments:
+
+```json
+{
+  "publications": [
+    "lenny.substack.com"
+  ],
+  "search_queries": [
+    "cold email"
+  ],
+  "max_posts_per_publication": "5",
+  "max_search_results": "5"
+}
+```
+
+## Inputs
+
+Every buyer facing input of the actor, unchanged. Omitted inputs take the actor's own defaults. Numbers and booleans are sent as strings.
+
+| Input | Type | Description |
+|---|---|---|
+| `publications` | array | Substack publications, one per line, up to 100: a URL (https://www.lennysnewsletter.com), a domain (lenny.substack.com), a profile (substack.com/@lenny), or a bare Substack name (lenny). Each one returns a publication row and its newest posts |
+| `search_queries` | array | Keywords to search across Substack posts, one per line, up to 20. Each query returns matching posts, or the publications behind them when Search output is publications |
+| `search_output` | string | What a search query returns Default posts |
+| `max_posts_per_publication` | string | Newest posts returned per publication, 0 to 1000. 0 returns only the publication row Default 20 |
+| `max_search_results` | string | Rows returned per search query, 1 to 1000 Default 50 |
+| `since_days` | string | Optional. Skip posts older than this many days. 0 or empty keeps every post up to the max Default 0 |
+| `include_publication_row` | string | true to return one publication row per listed publication (name, author, subscribers, paid plan prices), billed as a publication row. false returns only post rows Default true |
+| `include_body` | string | true to add the post text (up to 20,000 characters) to each post row of a listed publication. A paid post gives its free preview. Reads one more page per post; no extra charge Default false |
+| `max_charge_usd` | string | Optional. The run stops before it would charge more than this |
+| `proxy_mode` | string | How pages are fetched Default auto |
+
+## Pricing
+
+Pay per event on Apify, billed to your Apify account. Prices in USD.
+
+| Event | Free plan | Bronze | Silver | Gold |
+|---|---|---|---|---|
+| `post-row`, one Substack post returned as a row: from a listed publication or from a search | $0.00115 | $0.001093 | $0.001035 | $0.000977 |
+| `publication-row`, one Substack publication profile returned as a row: name, author, subscriber count as shown, and paid plan prices | $0.002 | $0.0019 | $0.0018 | $0.0017 |
+| `apify-actor-start`, per run start | $0.001057 | $0.001057 | $0.001057 | $0.001057 |
+
+`no_results` and `error` rows are free, including the one row a refused input returns.
+
+## Your keys
+
+The actor needs no key. It reads Substack's public pages and public JSON routes and holds no key of any kind. Proxy traffic adds nothing to your price: Mamba Labs pays the platform costs of the run.
+
+## Full actor documentation
+
+[apify.com/mambalabs/substack-scraper](https://apify.com/mambalabs/substack-scraper)
+
+## License
+
+MIT. See [LICENSE](LICENSE).

package/SECURITY.md

@@ -0,0 +1,31 @@
+# Security Policy
+
+## Supported versions
+
+The latest published `1.x` release of
+`@mambalabsdev/mcp-substack-scraper` receives security updates.
+
+| Version | Supported          |
+| ------- | ------------------ |
+| 1.x     | :white_check_mark: |
+| < 1.0   | :x:                |
+
+## Reporting a vulnerability
+
+Please do not open a public GitHub issue for security reports.
+
+Email security reports to **labs@mambamindsetgrowth.com** with:
+
+- a description of the issue and its impact,
+- steps to reproduce, and
+- any relevant logs or proof of concept.
+
+We aim to acknowledge reports within 3 business days and to provide a remediation
+timeline after triage. Coordinated disclosure is appreciated.
+
+## Handling of credentials
+
+This server reads an `APIFY_TOKEN` from the environment and uses it only to call
+the Apify API on your behalf. The token is never logged or written to disk by this
+package. Treat your `APIFY_TOKEN` as a secret and rotate it at
+https://console.apify.com/account/integrations if you believe it has been exposed.

package/build/index.js

@@ -0,0 +1,199 @@
+#!/usr/bin/env node
+import { readFileSync } from "node:fs";
+import { fileURLToPath } from "node:url";
+import { dirname, join } from "node:path";
+import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
+import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
+import { z } from "zod";
+const here = dirname(fileURLToPath(import.meta.url));
+const pkg = JSON.parse(readFileSync(join(here, "..", "package.json"), "utf8"));
+// Distinctive UA so Apify run meta.userAgent marks MCP-originated runs.
+const USER_AGENT = `mambalabs-mcp ${pkg.name}@${pkg.version}`;
+// Drop undefined values so optional inputs are not sent to the actor.
+function compact(obj) {
+    const out = {};
+    for (const [k, v] of Object.entries(obj)) {
+        if (v !== undefined)
+            out[k] = v;
+    }
+    return out;
+}
+// How long the actor run itself is allowed to take, in seconds. The suite's
+// slowest measured run on 2026-09-22 was 152 s on a seven platform search, so
+// 1800 s is roughly twelve times that: headroom for a slow day without letting a
+// hung run bill indefinitely.
+const ACTOR_RUN_TIMEOUT_SECS = 1800;
+// How long this wrapper waits for that run, in milliseconds. The actor's own
+// timeout plus two minutes, so the run's own TIMED-OUT status is what the
+// caller sees rather than the wrapper giving up first and reporting nothing.
+const WRAPPER_WAIT_MS = (ACTOR_RUN_TIMEOUT_SECS + 120) * 1000;
+const POLL_INTERVAL_MS = 3000;
+const TERMINAL = new Set(["SUCCEEDED", "FAILED", "TIMED-OUT", "ABORTED", "ABORTING"]);
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+// Shared caller. actorPath is the actor's immutable Apify actor ID (a stable key
+// that survives Store renames). The /v2/acts/{id} endpoint accepts it directly,
+// so a Store rename never breaks these calls.
+//
+// START AND POLL, NOT RUN-SYNC. Apify's synchronous endpoints carry a platform
+// ceiling of 300 seconds on the HTTP wait itself and answer 408 past it whatever
+// the timeout parameter says, so a long run reads as a timeout even though the
+// actor goes on to finish. Starting the run, polling it to a terminal status and
+// then reading the dataset is the only way to wait as long as the actor needs.
+//
+// The token is read here rather than at module load, so the tool registers
+// unconditionally and a server started without APIFY_TOKEN still advertises its
+// capabilities instead of reporting none.
+async function runActor(actorPath, actorLabel, input) {
+    const APIFY_TOKEN = process.env.APIFY_TOKEN;
+    if (!APIFY_TOKEN) {
+        return { isError: true, content: [{ type: "text", text: "APIFY_TOKEN is not set. Create a token at https://console.apify.com/account/integrations and set it as the APIFY_TOKEN environment variable." }] };
+    }
+    const headers = {
+        Authorization: `Bearer ${APIFY_TOKEN}`,
+        "Content-Type": "application/json",
+        "User-Agent": USER_AGENT,
+    };
+    const httpError = async (response) => {
+        let detail = "";
+        try {
+            const body = (await response.json());
+            if (body?.error?.message)
+                detail = ` ${body.error.message}`;
+        }
+        catch {
+            detail = "";
+        }
+        switch (response.status) {
+            case 400:
+                return `The ${actorLabel} run was rejected as invalid input.${detail}`;
+            case 401:
+                return "Invalid Apify token. Check your APIFY_TOKEN environment variable.";
+            case 402:
+                return "Insufficient Apify credits. Check your account balance at https://console.apify.com/billing";
+            default:
+                return `Apify request to ${actorLabel} failed with status ${response.status}.${detail}`;
+        }
+    };
+    // 1. Start the run.
+    let started;
+    try {
+        started = await fetch(`https://api.apify.com/v2/acts/${actorPath}/runs?timeout=${ACTOR_RUN_TIMEOUT_SECS}`, { method: "POST", headers, body: JSON.stringify(input) });
+    }
+    catch (err) {
+        const message = err instanceof Error ? err.message : String(err);
+        return { isError: true, content: [{ type: "text", text: `Could not reach the Apify API: ${message}` }] };
+    }
+    if (!started.ok) {
+        return { isError: true, content: [{ type: "text", text: await httpError(started) }] };
+    }
+    let run;
+    try {
+        run = (await started.json()).data ?? {};
+    }
+    catch (err) {
+        const message = err instanceof Error ? err.message : String(err);
+        return { isError: true, content: [{ type: "text", text: `The ${actorLabel} run start returned a response that could not be parsed: ${message}` }] };
+    }
+    const runId = run.id;
+    if (!runId) {
+        return { isError: true, content: [{ type: "text", text: `The ${actorLabel} run start returned no run id, so there is nothing to wait for.` }] };
+    }
+    // 2. Poll to a terminal status.
+    const deadline = Date.now() + WRAPPER_WAIT_MS;
+    let status = run.status ?? "READY";
+    let datasetId = run.defaultDatasetId;
+    while (!TERMINAL.has(status)) {
+        if (Date.now() >= deadline) {
+            return {
+                isError: true,
+                content: [{ type: "text", text: `The ${actorLabel} run ${runId} was still ${status} after ${Math.round(WRAPPER_WAIT_MS / 1000)} seconds and this call stopped waiting. The run itself is still on Apify: read it at https://console.apify.com/actors/runs/${runId}` }],
+            };
+        }
+        await sleep(POLL_INTERVAL_MS);
+        let poll;
+        try {
+            poll = await fetch(`https://api.apify.com/v2/actor-runs/${runId}`, { headers });
+        }
+        catch (err) {
+            const message = err instanceof Error ? err.message : String(err);
+            return { isError: true, content: [{ type: "text", text: `Lost contact with the Apify API while waiting for ${actorLabel} run ${runId}: ${message}` }] };
+        }
+        if (!poll.ok) {
+            return { isError: true, content: [{ type: "text", text: await httpError(poll) }] };
+        }
+        const body = (await poll.json());
+        status = body.data?.status ?? status;
+        datasetId = body.data?.defaultDatasetId ?? datasetId;
+    }
+    // 3. A run that did not succeed is a failure the caller must see, never an
+    // empty success. Surfacing it here is what keeps a crashed run from reading
+    // as "no results found".
+    if (status !== "SUCCEEDED") {
+        return {
+            isError: true,
+            content: [{ type: "text", text: `The ${actorLabel} run did not succeed (run ID: ${runId}, status: ${status}).` }],
+        };
+    }
+    if (!datasetId) {
+        return { isError: true, content: [{ type: "text", text: `The ${actorLabel} run ${runId} succeeded but reported no dataset, so there is nothing to return.` }] };
+    }
+    // 4. Read the dataset.
+    let ds;
+    try {
+        ds = await fetch(`https://api.apify.com/v2/datasets/${datasetId}/items?format=json`, { headers });
+    }
+    catch (err) {
+        const message = err instanceof Error ? err.message : String(err);
+        return { isError: true, content: [{ type: "text", text: `Could not read the ${actorLabel} dataset: ${message}` }] };
+    }
+    if (!ds.ok) {
+        return { isError: true, content: [{ type: "text", text: await httpError(ds) }] };
+    }
+    let items;
+    try {
+        items = await ds.json();
+    }
+    catch (err) {
+        const message = err instanceof Error ? err.message : String(err);
+        return { isError: true, content: [{ type: "text", text: `The ${actorLabel} run returned a response that could not be parsed: ${message}` }] };
+    }
+    if (!Array.isArray(items)) {
+        const asObj = items;
+        const detail = asObj?.error?.message
+            ? `${asObj.error.message}`
+            : JSON.stringify(items);
+        return { isError: true, content: [{ type: "text", text: `The ${actorLabel} run did not return a dataset. ${detail}` }] };
+    }
+    return { content: [{ type: "text", text: JSON.stringify(items, null, 2) }] };
+}
+const server = new McpServer({
+    name: "mamba-substack-scraper",
+    version: pkg.version,
+});
+// Substack Scraper (immutable actor ID UEFsdwiPLmPNVy9xd). Every buyer facing input of the actor
+// is exposed. The actor takes numbers and booleans as strings, so those inputs are strings here too.
+server.registerTool("scrape_substack", {
+    title: "Scrape Substack publications, posts, and search",
+    description: "Read Substack publications and posts, or search Substack by keyword. Give publications as URLs, domains, substack.com profiles, or bare Substack names (for example lenny.substack.com, https://www.notboring.co, lenny). Each publication returns one publication row (name, author, subscriber count as Substack shows it, subscriber tier text, paid plan prices) and its newest posts, one row each with date, free or paid audience, likes, comments, restacks, word count, bylines, and tags. Search queries return matching posts with their publication and subscriber count, or one row per publication when search_output is publications. Input with nothing to run returns one free row whose error_reason starts with input_invalid:. A run that does not succeed is returned as an error with its run ID. Read-only; requires an APIFY_TOKEN; Apify charges per post row and per publication row.",
+    annotations: {
+        title: "Scrape Substack publications, posts, and search",
+        readOnlyHint: true,
+        destructiveHint: false,
+        idempotentHint: true,
+        openWorldHint: true,
+    },
+    inputSchema: {
+        publications: z.array(z.string()).optional().describe("Substack publications, one per line, up to 100: a URL (https://www.lennysnewsletter.com), a domain (lenny.substack.com), a profile (substack.com/@lenny), or a bare Substack name (lenny). Each one returns a publication row and its newest posts."),
+        search_queries: z.array(z.string()).optional().describe("Keywords to search across Substack posts, one per line, up to 20. Each query returns matching posts, or the publications behind them when Search output is publications."),
+        search_output: z.enum(["posts", "publications"]).optional().describe("What a search query returns. Default posts."),
+        max_posts_per_publication: z.string().optional().describe("Newest posts returned per publication, 0 to 1000. 0 returns only the publication row. Default 20."),
+        max_search_results: z.string().optional().describe("Rows returned per search query, 1 to 1000. Default 50."),
+        since_days: z.string().optional().describe("Optional. Skip posts older than this many days. 0 or empty keeps every post up to the max. Default 0."),
+        include_publication_row: z.string().optional().describe("true to return one publication row per listed publication (name, author, subscribers, paid plan prices), billed as a publication row. false returns only post rows. Default true."),
+        include_body: z.string().optional().describe("true to add the post text (up to 20,000 characters) to each post row of a listed publication. A paid post gives its free preview. Reads one more page per post; no extra charge. Default false."),
+        max_charge_usd: z.string().optional().describe("Optional. The run stops before it would charge more than this."),
+        proxy_mode: z.enum(["auto", "direct"]).optional().describe("How pages are fetched. Default auto."),
+    },
+}, async (args) => runActor("UEFsdwiPLmPNVy9xd", "Substack Scraper", compact(args)));
+const transport = new StdioServerTransport();
+await server.connect(transport);

package/package.json

@@ -1,6 +1,50 @@
 {
   "name": "@mambalabsdev/mcp-substack-scraper",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "1.0.1",
+  "description": "MCP server for Substack Scraper. Read Substack publications and posts, or search Substack by keyword: subscriber counts, paid plan prices, and every post with date, paid flag, likes, comments, and restacks. Runs on Apify. Clay-ready output.",
+  "type": "module",
+  "bin": {
+    "mcp-substack-scraper": "./build/index.js"
+  },
+  "files": [
+    "build",
+    "README.md",
+    "LICENSE",
+    "SECURITY.md"
+  ],
+  "scripts": {
+    "build": "tsc && chmod +x build/index.js",
+    "test": "npm run build && node --test test/*.test.js",
+    "prepublishOnly": "npm run build"
+  },
+  "keywords": [
+    "mcp",
+    "model-context-protocol",
+    "substack",
+    "newsletter",
+    "substack-scraper",
+    "creator-economy",
+    "sponsorship",
+    "clay",
+    "apify"
+  ],
+  "author": "Mamba Labs <labs@mambamindsetgrowth.com>",
+  "license": "MIT",
+  "homepage": "https://apify.com/mambalabs/substack-scraper",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/mambalabsdev/mcp-substack-scraper"
+  },
+  "engines": {
+    "node": ">=18"
+  },
+  "dependencies": {
+    "@modelcontextprotocol/sdk": "^1.32.2",
+    "zod": "^4.4.3"
+  },
+  "devDependencies": {
+    "@types/node": "^26.2.0",
+    "typescript": "5.9.3"
+  },
+  "mcpName": "com.mambabuilt/mcp-substack-scraper"
+}