research-powerpack-mcp 3.0.3 → 3.0.5

package/README.md

@@ -1,29 +1,114 @@
-# Research Powerpack MCP
-
-**The ultimate research MCP toolkit** — Reddit mining, web search with CTR aggregation, AI-powered deep research, and intelligent web scraping, all in one modular package.
-
-[![npm version](https://img.shields.io/npm/v/research-powerpack-mcp.svg)](https://www.npmjs.com/package/research-powerpack-mcp)
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+<h1 align="center">🔬 Research Powerpack MCP 🔬</h1>
+<h3 align="center">Stop tab-hopping for research. Start getting god-tier context.</h3>
+
+<p align="center">
+  <strong>
+    <em>The ultimate research toolkit for your AI coding assistant. It searches the web, mines Reddit, scrapes any URL, and synthesizes everything into perfectly structured context your LLM actually understands.</em>
+  </strong>
+</p>
+
+<p align="center">
+  <!-- Package Info -->
+  <a href="https://www.npmjs.com/package/research-powerpack-mcp"><img alt="npm" src="https://img.shields.io/npm/v/research-powerpack-mcp.svg?style=flat-square&color=4D87E6"></a>
+  <a href="#"><img alt="node" src="https://img.shields.io/badge/node-18+-4D87E6.svg?style=flat-square"></a>
+  &nbsp;&nbsp;•&nbsp;&nbsp;
+  <!-- Features -->
+  <a href="https://opensource.org/licenses/MIT"><img alt="license" src="https://img.shields.io/badge/License-MIT-F9A825.svg?style=flat-square"></a>
+  <a href="#"><img alt="platform" src="https://img.shields.io/badge/platform-macOS_|_Linux_|_Windows-2ED573.svg?style=flat-square"></a>
+</p>
+
+<p align="center">
+  <img alt="modular" src="https://img.shields.io/badge/🧩_modular-use_1_tool_or_all_5-2ED573.svg?style=for-the-badge">
+  <img alt="zero crash" src="https://img.shields.io/badge/💪_zero_crash-missing_keys_=_helpful_errors-2ED573.svg?style=for-the-badge">
+</p>
+
+<div align="center">
+
+### 🧭 Quick Navigation
+
+[**⚡ Get Started**](#-get-started-in-60-seconds) •
+[**✨ Key Features**](#-feature-breakdown-the-secret-sauce) •
+[**🎮 Usage & Examples**](#-tool-reference) •
+[**⚙️ API Key Setup**](#-api-key-setup-guides) •
+[**🆚 Why This Slaps**](#-why-this-slaps-other-methods)
+
+</div>
 
 ---
 
-## Why Research Powerpack?
-
-AI coding assistants are only as good as the context they have. This MCP server gives your AI **superpowers for research**:
+**`research-powerpack-mcp`** is the research assistant your AI wishes it had. Stop asking your LLM to guess about things it doesn't know. This MCP server acts like a senior researcher, searching the web, mining Reddit discussions, scraping documentation, and synthesizing everything into perfectly structured context so your AI can actually give you answers worth a damn.
+
+<div align="center">
+<table>
+<tr>
+<td align="center">
+<h3>🔍</h3>
+<b>Batch Web Search</b><br/>
+<sub>100 keywords in parallel</sub>
+</td>
+<td align="center">
+<h3>💬</h3>
+<b>Reddit Mining</b><br/>
+<sub>Real opinions, not marketing</sub>
+</td>
+<td align="center">
+<h3>🌐</h3>
+<b>Universal Scraping</b><br/>
+<sub>JS rendering + geo-targeting</sub>
+</td>
+<td align="center">
+<h3>🧠</h3>
+<b>Deep Research</b><br/>
+<sub>AI synthesis with citations</sub>
+</td>
+</tr>
+</table>
+</div>
+
+How it slaps:
+- **You:** "What's the best database for my use case?"
+- **AI + Powerpack:** Searches Google, mines Reddit threads, scrapes docs, synthesizes findings.
+- **You:** Get an actually informed answer with real community opinions and citations.
+- **Result:** Ship better decisions. Skip the 47 browser tabs.
 
-| Tool | What It Does | Real Value |
-|------|-------------|------------|
-| `web_search` | Batch Google search (up to 100 keywords) with CTR-weighted ranking | Find the most authoritative sources across multiple search angles simultaneously |
-| `search_reddit` | Google-powered Reddit search with advanced operators | Discover real user discussions, opinions, and experiences |
-| `get_reddit_post` | Fetch Reddit posts with smart comment allocation | Extract community wisdom with automatic comment budget distribution |
-| `scrape_links` | Universal URL scraping with automatic fallback | Get full content from any webpage with JS rendering and geo-targeting |
-| `deep_research` | AI-powered batch research with citations | Get comprehensive, evidence-based answers to multiple questions in parallel |
+---
 
-**Modular by design** — use just one tool or all five. Configure only the API keys you need.
+## 💥 Why This Slaps Other Methods
+
+Manually researching is a vibe-killer. `research-powerpack-mcp` makes other methods look ancient.
+
+<table align="center">
+<tr>
+<td align="center"><b>❌ The Old Way (Pain)</b></td>
+<td align="center"><b>✅ The Powerpack Way (Glory)</b></td>
+</tr>
+<tr>
+<td>
+<ol>
+  <li>Open 15 browser tabs.</li>
+  <li>Skim Stack Overflow answers from 2019.</li>
+  <li>Search Reddit, get distracted by drama.</li>
+  <li>Copy-paste random snippets to your AI.</li>
+  <li>Get a mediocre answer from confused context.</li>
+</ol>
+</td>
+<td>
+<ol>
+  <li>Ask your AI to research it.</li>
+  <li>AI searches, scrapes, mines Reddit automatically.</li>
+  <li>Receive synthesized insights with sources.</li>
+  <li>Make an informed decision.</li>
+  <li>Go grab a coffee. ☕</li>
+</ol>
+</td>
+</tr>
+</table>
+
+We're not just fetching random pages. We're building **high-signal, low-noise context** with CTR-weighted ranking, smart comment allocation, and intelligent token distribution that prevents massive responses from breaking your LLM's context window.
 
 ---
 
-## Quick Start
+## 🚀 Get Started in 60 Seconds
 
 ### 1. Install
 
@@ -31,25 +116,23 @@ AI coding assistants are only as good as the context they have. This MCP server
 npm install research-powerpack-mcp
 ```
 
-### 2. Configure (pick what you need)
+### 2. Configure Your MCP Client
 
-Copy `.env.example` to `.env` and add the API keys for the tools you want:
+<div align="center">
 
-```bash
-# Minimal (just web search) - FREE
-SERPER_API_KEY=your_serper_key
+| Client | Config File | Docs |
+|:------:|:-----------:|:----:|
+| 🖥️ **Claude Desktop** | `claude_desktop_config.json` | [Setup](#claude-desktop) |
+| ⌨️ **Claude Code** | `~/.claude.json` or CLI | [Setup](#claude-code-cli) |
+| 🎯 **Cursor** | `.cursor/mcp.json` | [Setup](#cursorwindsurf) |
+| 🏄 **Windsurf** | MCP settings | [Setup](#cursorwindsurf) |
 
-# Full power (all 5 tools)
-SERPER_API_KEY=your_serper_key
-REDDIT_CLIENT_ID=your_reddit_id
-REDDIT_CLIENT_SECRET=your_reddit_secret
-SCRAPEDO_API_KEY=your_scrapedo_key
-OPENROUTER_API_KEY=your_openrouter_key
-```
+</div>
 
-### 3. Add to your MCP client
+#### Claude Desktop
+
+Add to your `claude_desktop_config.json`:
 
-**Claude Desktop** (`claude_desktop_config.json`):
 ```json
 {
   "mcpServers": {
@@ -68,7 +151,47 @@ OPENROUTER_API_KEY=your_openrouter_key
 }
 ```
 
-**Cursor/Windsurf** (`.cursor/mcp.json` or similar):
+#### Claude Code (CLI)
+
+One command to rule them all:
+
+```bash
+claude mcp add research-powerpack npx \
+  --scope user \
+  --env SERPER_API_KEY=your_key \
+  --env REDDIT_CLIENT_ID=your_id \
+  --env REDDIT_CLIENT_SECRET=your_secret \
+  --env OPENROUTER_API_KEY=your_key \
+  --env OPENROUTER_BASE_URL=https://openrouter.ai/api/v1 \
+  --env RESEARCH_MODEL=x-ai/grok-4.1-fast \
+  -- research-powerpack-mcp
+```
+
+Or manually add to `~/.claude.json`:
+
+```json
+{
+  "mcpServers": {
+    "research-powerpack": {
+      "command": "npx",
+      "args": ["research-powerpack-mcp"],
+      "env": {
+        "SERPER_API_KEY": "your_key",
+        "REDDIT_CLIENT_ID": "your_id",
+        "REDDIT_CLIENT_SECRET": "your_secret",
+        "OPENROUTER_API_KEY": "your_key",
+        "OPENROUTER_BASE_URL": "https://openrouter.ai/api/v1",
+        "RESEARCH_MODEL": "x-ai/grok-4.1-fast"
+      }
+    }
+  }
+}
+```
+
+#### Cursor/Windsurf
+
+Add to `.cursor/mcp.json` or equivalent:
+
 ```json
 {
   "mcpServers": {
@@ -83,20 +206,197 @@ OPENROUTER_API_KEY=your_openrouter_key
 }
 ```
 
+> **✨ Zero Crash Promise:** Missing API keys? No problem. The server always starts. Tools just return helpful setup instructions instead of exploding.
+
+---
+
+## ✨ Feature Breakdown: The Secret Sauce
+
+<div align="center">
+
+| Feature | What It Does | Why You Care |
+| :---: | :--- | :--- |
+| **🔍 Batch Search**<br/>`100 keywords parallel` | Search Google for up to 100 queries simultaneously | Cover every angle of a topic in one shot |
+| **📊 CTR Ranking**<br/>`Smart URL scoring` | Identifies URLs that appear across multiple searches | Surfaces high-consensus authoritative sources |
+| **💬 Reddit Mining**<br/>`Real human opinions` | Google-powered Reddit search + native API fetching | Get actual user experiences, not marketing fluff |
+| **🎯 Smart Allocation**<br/>`Token-aware budgets` | 1,000 comment budget distributed across posts | Deep dive on 2 posts or quick scan on 50 |
+| **🌐 Universal Scraping**<br/>`Works on everything` | Auto-fallback: basic → JS render → geo-targeting | Handles SPAs, paywalls, and geo-restricted content |
+| **🧠 Deep Research**<br/>`AI-powered synthesis` | Batch research with web search and citations | Get comprehensive answers to complex questions |
+| **🧩 Modular Design**<br/>`Use what you need` | Each tool works independently | Pay only for the APIs you actually use |
+
+</div>
+
+---
+
+## 🎮 Tool Reference
+
+<div align="center">
+<table>
+<tr>
+<td align="center">
+<h3>🔍</h3>
+<b><code>web_search</code></b><br/>
+<sub>Batch Google search</sub>
+</td>
+<td align="center">
+<h3>💬</h3>
+<b><code>search_reddit</code></b><br/>
+<sub>Find Reddit discussions</sub>
+</td>
+<td align="center">
+<h3>📖</h3>
+<b><code>get_reddit_post</code></b><br/>
+<sub>Fetch posts + comments</sub>
+</td>
+<td align="center">
+<h3>🌐</h3>
+<b><code>scrape_links</code></b><br/>
+<sub>Extract any URL</sub>
+</td>
+<td align="center">
+<h3>🧠</h3>
+<b><code>deep_research</code></b><br/>
+<sub>AI synthesis</sub>
+</td>
+</tr>
+</table>
+</div>
+
+### `web_search`
+
+**Batch web search** using Google via Serper API. Search up to 100 keywords in parallel.
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `keywords` | `string[]` | Yes | Search queries (1-100). Use distinct keywords for maximum coverage. |
+
+**Supports Google operators:** `site:`, `-exclusion`, `"exact phrase"`, `filetype:`
+
+```json
+{
+  "keywords": [
+    "best IDE 2025",
+    "VS Code alternatives",
+    "Cursor vs Windsurf comparison"
+  ]
+}
+```
+
+---
+
+### `search_reddit`
+
+**Search Reddit** via Google with automatic `site:reddit.com` filtering.
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `queries` | `string[]` | Yes | Search queries (max 10) |
+| `date_after` | `string` | No | Filter results after date (YYYY-MM-DD) |
+
+**Search operators:** `intitle:keyword`, `"exact phrase"`, `OR`, `-exclude`
+
+```json
+{
+  "queries": [
+    "best mechanical keyboard 2025",
+    "intitle:keyboard recommendation"
+  ],
+  "date_after": "2024-01-01"
+}
+```
+
+---
+
+### `get_reddit_post`
+
+**Fetch Reddit posts** with smart comment allocation (1,000 comment budget distributed automatically).
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `urls` | `string[]` | Yes | — | Reddit post URLs (2-50) |
+| `fetch_comments` | `boolean` | No | `true` | Whether to fetch comments |
+| `max_comments` | `number` | No | auto | Override comment allocation |
+
+**Smart Allocation:**
+- 2 posts → ~500 comments/post (deep dive)
+- 10 posts → ~100 comments/post
+- 50 posts → ~20 comments/post (quick scan)
+
+```json
+{
+  "urls": [
+    "https://reddit.com/r/programming/comments/abc123/post_title",
+    "https://reddit.com/r/webdev/comments/def456/another_post"
+  ]
+}
+```
+
+---
+
+### `scrape_links`
+
+**Universal URL content extraction** with automatic fallback modes.
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `urls` | `string[]` | Yes | — | URLs to scrape (3-50) |
+| `timeout` | `number` | No | `30` | Timeout per URL (seconds) |
+| `use_llm` | `boolean` | No | `false` | Enable AI extraction |
+| `what_to_extract` | `string` | No | — | Extraction instructions for AI |
+
+**Automatic Fallback:** Basic → JS rendering → JS + US geo-targeting
+
+```json
+{
+  "urls": ["https://example.com/article1", "https://example.com/article2"],
+  "use_llm": true,
+  "what_to_extract": "Extract the main arguments and key statistics"
+}
+```
+
+---
+
+### `deep_research`
+
+**AI-powered batch research** with web search and citations.
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `questions` | `object[]` | Yes | Research questions (2-10) |
+| `questions[].question` | `string` | Yes | The research question |
+| `questions[].file_attachments` | `object[]` | No | Files to include as context |
+
+**Token Allocation:** 32,000 tokens distributed across questions:
+- 2 questions → 16,000 tokens/question (deep dive)
+- 10 questions → 3,200 tokens/question (rapid multi-topic)
+
+```json
+{
+  "questions": [
+    { "question": "What are the current best practices for React Server Components in 2025?" },
+    { "question": "Compare Bun vs Node.js for production workloads with benchmarks." }
+  ]
+}
+```
+
 ---
 
-## Environment Variables & Tool Availability
+## ⚙️ Environment Variables & Tool Availability
 
 Research Powerpack uses a **modular architecture**. Tools are automatically enabled based on which API keys you provide:
 
+<div align="center">
+
 | ENV Variable | Tools Enabled | Free Tier |
-|--------------|---------------|-----------|
-| `SERPER_API_KEY` | `web_search`, `search_reddit` | 2,500 queries |
-| `REDDIT_CLIENT_ID` + `REDDIT_CLIENT_SECRET` | `get_reddit_post` | Unlimited |
-| `SCRAPEDO_API_KEY` | `scrape_links` | 1,000 credits |
-| `OPENROUTER_API_KEY` | `deep_research` + AI extraction in `scrape_links` | Pay-as-you-go |
+|:------------:|:-------------:|:---------:|
+| `SERPER_API_KEY` | `web_search`, `search_reddit` | 2,500 queries/mo |
+| `REDDIT_CLIENT_ID` + `SECRET` | `get_reddit_post` | Unlimited |
+| `SCRAPEDO_API_KEY` | `scrape_links` | 1,000 credits/mo |
+| `OPENROUTER_API_KEY` | `deep_research` + AI in `scrape_links` | Pay-as-you-go |
+| `RESEARCH_MODEL` | Model for `deep_research` | Default: `perplexity/sonar-deep-research` |
+| `LLM_EXTRACTION_MODEL` | Model for AI extraction in `scrape_links` | Default: `openrouter/gpt-oss-120b:nitro` |
 
-**No ENV = No crash.** The server always starts. If you call a tool without the required API key, you get a helpful error message with setup instructions.
+</div>
 
 ### Configuration Examples
 
@@ -109,7 +409,7 @@ SERPER_API_KEY=xxx
 REDDIT_CLIENT_ID=xxx
 REDDIT_CLIENT_SECRET=xxx
 
-# Full research mode (all tools)
+# Full research mode (all 5 tools)
 SERPER_API_KEY=xxx
 REDDIT_CLIENT_ID=xxx
 REDDIT_CLIENT_SECRET=xxx
@@ -119,13 +419,12 @@ OPENROUTER_API_KEY=xxx
 
 ---
 
-## API Key Setup Guides
+## 🔑 API Key Setup Guides
 
 <details>
-<summary><b>🔍 Serper API (Google Search)</b></summary>
+<summary><b>🔍 Serper API (Google Search) — FREE: 2,500 queries/month</b></summary>
 
 ### What you get
-- 2,500 free queries/month
 - Fast Google search results via API
 - Enables `web_search` and `search_reddit` tools
 
@@ -133,23 +432,23 @@ OPENROUTER_API_KEY=xxx
 1. Go to [serper.dev](https://serper.dev)
 2. Click **"Get API Key"** (top right)
 3. Sign up with email or Google
-4. Your API key is displayed on the dashboard
-5. Copy it to your `.env`:
+4. Copy your API key from the dashboard
+5. Add to your config:
    ```
    SERPER_API_KEY=your_key_here
    ```
 
 ### Pricing
 - **Free**: 2,500 queries/month
-- **Paid**: $50/month for 50,000 queries ($0.001/query)
+- **Paid**: $50/month for 50,000 queries
 
 </details>
 
 <details>
-<summary><b>🤖 Reddit OAuth (Reddit API)</b></summary>
+<summary><b>🤖 Reddit OAuth — FREE: Unlimited access</b></summary>
 
 ### What you get
-- Unlimited Reddit API access
+- Full Reddit API access
 - Fetch posts and comments with upvote sorting
 - Enables `get_reddit_post` tool
 
@@ -159,41 +458,33 @@ OPENROUTER_API_KEY=xxx
 3. Fill in:
    - **Name**: `research-powerpack` (or any name)
    - **App type**: Select **"script"** (important!)
-   - **Description**: Optional
-   - **About URL**: Leave blank
-   - **Redirect URI**: `http://localhost:8080` (required but not used)
+   - **Redirect URI**: `http://localhost:8080`
 4. Click **"create app"**
 5. Copy your credentials:
-   - **Client ID**: The string under your app name (e.g., `yuq_M0kWusHp2olglFBnpw`)
+   - **Client ID**: The string under your app name
    - **Client Secret**: The "secret" field
-6. Add to your `.env`:
+6. Add to your config:
    ```
    REDDIT_CLIENT_ID=your_client_id
    REDDIT_CLIENT_SECRET=your_client_secret
    ```
 
-### Tips
-- Script apps have the highest rate limits
-- No user authentication required
-- Works immediately after creation
-
 </details>
 
 <details>
-<summary><b>🌐 Scrape.do (Web Scraping)</b></summary>
+<summary><b>🌐 Scrape.do (Web Scraping) — FREE: 1,000 credits/month</b></summary>
 
 ### What you get
-- 1,000 free scraping credits
 - JavaScript rendering support
 - Geo-targeting and CAPTCHA handling
 - Enables `scrape_links` tool
 
 ### Setup Steps
 1. Go to [scrape.do](https://scrape.do)
-2. Click **"Start Free"** or **"Get Started"**
+2. Click **"Start Free"**
 3. Sign up with email
-4. Your API key is on the dashboard
-5. Add to your `.env`:
+4. Copy your API key from the dashboard
+5. Add to your config:
    ```
    SCRAPEDO_API_KEY=your_key_here
    ```
@@ -203,36 +494,32 @@ OPENROUTER_API_KEY=xxx
 - **JavaScript rendering**: 5 credits
 - **Geo-targeting**: +25 credits
 
-### Pricing
-- **Free**: 1,000 credits (renews monthly)
-- **Starter**: $29/month for 100,000 credits
-
 </details>
 
 <details>
-<summary><b>🧠 OpenRouter (AI Models)</b></summary>
+<summary><b>🧠 OpenRouter (AI Models) — Pay-as-you-go</b></summary>
 
 ### What you get
 - Access to 100+ AI models via one API
 - Enables `deep_research` tool
-- Enables AI extraction in `scrape_links` (`use_llm`, `what_to_extract`)
+- Enables AI extraction in `scrape_links`
 
 ### Setup Steps
 1. Go to [openrouter.ai](https://openrouter.ai)
-2. Click **"Sign In"** → Sign up with Google/GitHub/email
+2. Sign up with Google/GitHub/email
 3. Go to [openrouter.ai/keys](https://openrouter.ai/keys)
 4. Click **"Create Key"**
 5. Copy the key (starts with `sk-or-...`)
-6. Add to your `.env`:
+6. Add to your config:
    ```
    OPENROUTER_API_KEY=sk-or-v1-xxxxx
    ```
 
-### Recommended Models
-The default model is `perplexity/sonar-deep-research` (optimized for research with web search).
-
-Alternative models:
+### Recommended Models for Deep Research
 ```bash
+# Default (optimized for research)
+RESEARCH_MODEL=perplexity/sonar-deep-research
+
 # Fast and capable
 RESEARCH_MODEL=x-ai/grok-4.1-fast
 
@@ -243,207 +530,63 @@ RESEARCH_MODEL=anthropic/claude-3.5-sonnet
 RESEARCH_MODEL=openai/gpt-4o-mini
 ```
 
-### Pricing
-- Pay-as-you-go (no subscription required)
-- Prices vary by model (~$0.001-$0.03 per 1K tokens)
-- `perplexity/sonar-deep-research`: ~$5 per 1M tokens
-
-</details>
-
----
-
-## Tool Reference
-
-### `web_search`
-
-**Batch web search** using Google via Serper API. Search up to 100 keywords in parallel.
-
-| Parameter | Type | Required | Description |
-|-----------|------|----------|-------------|
-| `keywords` | `string[]` | Yes | Search queries (1-100). Use distinct keywords for maximum coverage. |
-
-**Features:**
-- Google search operators: `site:`, `-exclusion`, `"exact phrase"`, `filetype:`
-- CTR-weighted ranking identifies high-consensus URLs
-- Related search suggestions per query
-
-**Example:**
-```json
-{
-  "keywords": [
-    "best IDE 2025",
-    "VS Code alternatives",
-    "Cursor vs Windsurf comparison"
-  ]
-}
-```
-
----
-
-### `search_reddit`
-
-**Search Reddit** via Google with automatic `site:reddit.com` filtering.
-
-| Parameter | Type | Required | Description |
-|-----------|------|----------|-------------|
-| `queries` | `string[]` | Yes | Search queries (max 10). Use distinct queries for multiple perspectives. |
-| `date_after` | `string` | No | Filter results after date (YYYY-MM-DD) |
-
-**Search Operators:**
-- `intitle:keyword` — Match in post title
-- `"exact phrase"` — Exact match
-- `OR` — Match either term
-- `-exclude` — Exclude term
-
-**Example:**
-```json
-{
-  "queries": [
-    "best mechanical keyboard 2025",
-    "intitle:keyboard recommendation",
-    "\"keychron\" OR \"nuphy\" review"
-  ],
-  "date_after": "2024-01-01"
-}
-```
-
----
-
-### `get_reddit_post`
-
-**Fetch Reddit posts** with smart comment allocation (1,000 comment budget distributed automatically).
-
-| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| `urls` | `string[]` | Yes | — | Reddit post URLs (2-50) |
-| `fetch_comments` | `boolean` | No | `true` | Whether to fetch comments |
-| `max_comments` | `number` | No | auto | Override comment allocation |
-
-**Smart Allocation:**
-- 2 posts: ~500 comments/post (deep dive)
-- 10 posts: ~100 comments/post
-- 50 posts: ~20 comments/post (quick scan)
-
-**Example:**
-```json
-{
-  "urls": [
-    "https://reddit.com/r/programming/comments/abc123/post_title",
-    "https://reddit.com/r/webdev/comments/def456/another_post"
-  ],
-  "fetch_comments": true
-}
-```
-
----
-
-### `scrape_links`
-
-**Universal URL content extraction** with automatic fallback modes.
-
-| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| `urls` | `string[]` | Yes | — | URLs to scrape (3-50) |
-| `timeout` | `number` | No | `30` | Timeout per URL (seconds) |
-| `use_llm` | `boolean` | No | `false` | Enable AI extraction (requires `OPENROUTER_API_KEY`) |
-| `what_to_extract` | `string` | No | — | Extraction instructions for AI |
-
-**Automatic Fallback:**
-1. Basic mode (fast)
-2. JavaScript rendering (for SPAs)
-3. JavaScript + US geo-targeting (for restricted content)
+### Recommended Models for AI Extraction (`use_llm` in `scrape_links`)
+```bash
+# Default (fast and cost-effective for extraction)
+LLM_EXTRACTION_MODEL=openrouter/gpt-oss-120b:nitro
 
-**Token Allocation:** 32,000 tokens distributed across URLs:
-- 3 URLs: ~10,666 tokens/URL
-- 10 URLs: ~3,200 tokens/URL
-- 50 URLs: ~640 tokens/URL
+# High quality extraction
+LLM_EXTRACTION_MODEL=anthropic/claude-3.5-sonnet
 
-**Example:**
-```json
-{
-  "urls": [
-    "https://example.com/article1",
-    "https://example.com/article2",
-    "https://example.com/article3"
-  ],
-  "use_llm": true,
-  "what_to_extract": "Extract the main arguments, key statistics, and conclusions"
-}
+# Budget-friendly
+LLM_EXTRACTION_MODEL=openai/gpt-4o-mini
 ```
 
----
-
-### `deep_research`
+> **Note:** `RESEARCH_MODEL` and `LLM_EXTRACTION_MODEL` are independent. You can use a powerful model for deep research and a faster/cheaper model for content extraction, or vice versa.
 
-**AI-powered batch research** with web search and citations.
-
-| Parameter | Type | Required | Description |
-|-----------|------|----------|-------------|
-| `questions` | `object[]` | Yes | Research questions (2-10) |
-| `questions[].question` | `string` | Yes | The research question |
-| `questions[].file_attachments` | `object[]` | No | Files to include as context |
-
-**Token Allocation:** 32,000 tokens distributed across questions:
-- 2 questions: 16,000 tokens/question (deep dive)
-- 5 questions: 6,400 tokens/question (balanced)
-- 10 questions: 3,200 tokens/question (rapid multi-topic)
-
-**Example:**
-```json
-{
-  "questions": [
-    {
-      "question": "What are the current best practices for React Server Components in 2025? Include patterns for data fetching and caching."
-    },
-    {
-      "question": "Compare the performance characteristics of Bun vs Node.js for production workloads. Include benchmarks and real-world case studies."
-    }
-  ]
-}
-```
+</details>
 
 ---
 
-## Recommended Workflows
+## 🔥 Recommended Workflows
 
 ### Research a Technology Decision
 
 ```
-1. web_search: ["React vs Vue 2025", "Next.js vs Nuxt comparison", "frontend framework benchmarks"]
-2. search_reddit: ["best frontend framework 2025", "migrating from React to Vue", "Next.js production experience"]
-3. get_reddit_post: [URLs from step 2]
-4. scrape_links: [Documentation and blog URLs from step 1]
-5. deep_research: [Synthesize findings into specific questions]
+1. web_search → ["React vs Vue 2025", "Next.js vs Nuxt comparison"]
+2. search_reddit → ["best frontend framework 2025", "Next.js production experience"]
+3. get_reddit_post → [URLs from step 2]
+4. scrape_links → [Documentation and blog URLs from step 1]
+5. deep_research → [Synthesize findings into specific questions]
 ```
 
 ### Competitive Analysis
 
 ```
-1. web_search: ["competitor name review", "competitor vs alternatives", "competitor pricing"]
-2. scrape_links: [Competitor websites, review sites, comparison pages]
-3. search_reddit: ["competitor name experience", "switching from competitor"]
-4. get_reddit_post: [URLs from step 3]
+1. web_search → ["competitor name review", "competitor vs alternatives"]
+2. scrape_links → [Competitor websites, review sites]
+3. search_reddit → ["competitor name experience", "switching from competitor"]
+4. get_reddit_post → [URLs from step 3]
 ```
 
 ### Debug an Obscure Error
 
 ```
-1. web_search: ["exact error message", "error message + framework name"]
-2. search_reddit: ["error message", "framework + error type"]
-3. get_reddit_post: [URLs with solutions]
-4. scrape_links: [Stack Overflow answers, GitHub issues]
+1. web_search → ["exact error message", "error + framework name"]
+2. search_reddit → ["error message", "framework + error type"]
+3. get_reddit_post → [URLs with solutions]
+4. scrape_links → [Stack Overflow answers, GitHub issues]
 ```
 
 ---
 
-## Enable Full Power Mode
+## 🔥 Enable Full Power Mode
 
 For the best research experience, configure all four API keys:
 
 ```bash
-# .env
 SERPER_API_KEY=your_serper_key       # Free: 2,500 queries/month
-REDDIT_CLIENT_ID=your_reddit_id       # Free: Nearly unlimited
+REDDIT_CLIENT_ID=your_reddit_id       # Free: Unlimited
 REDDIT_CLIENT_SECRET=your_reddit_secret
 SCRAPEDO_API_KEY=your_scrapedo_key   # Free: 1,000 credits/month
 OPENROUTER_API_KEY=your_openrouter_key # Pay-as-you-go
@@ -455,11 +598,11 @@ This unlocks:
 - **Deep research with web search** and citations
 - **Complete Reddit mining** (search → fetch → analyze)
 
-Total setup time: ~10 minutes. Total free tier value: ~$50/month equivalent.
+**Total setup time:** ~10 minutes. **Total free tier value:** ~$50/month equivalent.
 
 ---
 
-## Development
+## 🛠️ Development
 
 ```bash
 # Clone
@@ -481,6 +624,27 @@ npm run typecheck
 
 ---
 
-## License
+## 🔥 Common Issues & Quick Fixes
+
+<details>
+<summary><b>Expand for troubleshooting tips</b></summary>
+
+| Problem | Solution |
+| :--- | :--- |
+| **Tool returns "API key not configured"** | Add the required ENV variable to your MCP config. The error message tells you exactly which key is missing. |
+| **Reddit posts returning empty** | Check your `REDDIT_CLIENT_ID` and `REDDIT_CLIENT_SECRET`. Make sure you created a "script" type app. |
+| **Scraping fails on JavaScript sites** | This is expected for first attempt. The tool auto-retries with JS rendering. If still failing, the site may be blocking scrapers. |
+| **Deep research taking too long** | Use a faster model like `x-ai/grok-4.1-fast` instead of `perplexity/sonar-deep-research`. |
+| **Token limit errors** | Reduce the number of URLs/questions per request. The tool distributes a fixed token budget. |
+
+</details>
+
+---
+
+<div align="center">
+
+**Built with 🔥 because manually researching for your AI is a soul-crushing waste of time.**
 
 MIT © [Yiğit Konur](https://github.com/yigitkonur)
+
+</div>

package/dist/config/index.d.ts

@@ -54,6 +54,6 @@ export declare const REDDIT: {
 export declare const CTR_WEIGHTS: Record<number, number>;
 export declare const LLM_EXTRACTION: {
     readonly MODEL: string;
-    readonly MAX_TOKENS: 4000;
+    readonly MAX_TOKENS: 8000;
 };
 //# sourceMappingURL=index.d.ts.map

package/dist/config/index.js

@@ -76,7 +76,7 @@ export const REDDIT = {
     RETRY_DELAYS: [2000, 4000, 8000, 16000, 32000],
 };
 // ============================================================================
-// CTR Weights for URL Ranking
+// CTR Weights for URL Ranking (inspired fro CTR thing)
 // ============================================================================
 export const CTR_WEIGHTS = {
     1: 100.00,
@@ -94,7 +94,7 @@ export const CTR_WEIGHTS = {
 // LLM Extraction Model (uses OPENROUTER for scrape_links AI extraction)
 // ============================================================================
 export const LLM_EXTRACTION = {
-    MODEL: process.env.LLM_EXTRACTION_MODEL || 'anthropic/claude-3.5-sonnet',
-    MAX_TOKENS: 4000,
+    MODEL: process.env.LLM_EXTRACTION_MODEL || 'openrouter/gpt-oss-120b:nitro',
+    MAX_TOKENS: 8000,
 };
 //# sourceMappingURL=index.js.map

package/dist/config/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/config/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAaH,MAAM,UAAU,QAAQ;IACtB,OAAO;QACL,eAAe,EAAE,OAAO,CAAC,GAAG,CAAC,gBAAgB,IAAI,EAAE;QACnD,cAAc,EAAE,OAAO,CAAC,GAAG,CAAC,cAAc,IAAI,SAAS;QACvD,gBAAgB,EAAE,OAAO,CAAC,GAAG,CAAC,gBAAgB,IAAI,SAAS;QAC3D,oBAAoB,EAAE,OAAO,CAAC,GAAG,CAAC,oBAAoB,IAAI,SAAS;KACpE,CAAC;AACJ,CAAC;AAED,+EAA+E;AAC/E,6BAA6B;AAC7B,+EAA+E;AAE/E,MAAM,CAAC,MAAM,QAAQ,GAAG;IACtB,QAAQ,EAAE,OAAO,CAAC,GAAG,CAAC,mBAAmB,IAAI,8BAA8B;IAC3E,KAAK,EAAE,OAAO,CAAC,GAAG,CAAC,cAAc,IAAI,gCAAgC;IACrE,OAAO,EAAE,OAAO,CAAC,GAAG,CAAC,kBAAkB,IAAI,EAAE;IAC7C,UAAU,EAAE,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,cAAc,IAAI,SAAS,EAAE,EAAE,CAAC;IACjE,gBAAgB,EAAG,OAAO,CAAC,GAAG,CAAC,wBAAsD,IAAI,MAAM;IAC/F,QAAQ,EAAE,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,gBAAgB,IAAI,KAAK,EAAE,EAAE,CAAC;CACrD,CAAC;AAEX,+EAA+E;AAC/E,2BAA2B;AAC3B,+EAA+E;AAE/E,MAAM,CAAC,MAAM,MAAM,GAAG;IACpB,IAAI,EAAE,wBAAwB;IAC9B,OAAO,EAAE,OAAO;IAChB,WAAW,EAAE,6DAA6D;CAClE,CAAC;AAcX,MAAM,UAAU,eAAe;IAC7B,MAAM,GAAG,GAAG,QAAQ,EAAE,CAAC;IACvB,OAAO;QACL,MAAM,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,gBAAgB,IAAI,GAAG,CAAC,oBAAoB,CAAC;QAC5D,MAAM,EAAE,CAAC,CAAC,GAAG,CAAC,cAAc;QAC5B,QAAQ,EAAE,CAAC,CAAC,GAAG,CAAC,eAAe;QAC/B,YAAY,EAAE,CAAC,CAAC,QAAQ,CAAC,OAAO;QAChC,aAAa,EAAE,CAAC,CAAC,QAAQ,CAAC,OAAO,EAAE,uCAAuC;KAC3E,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,oBAAoB,CAAC,UAA8B;IACjE,MAAM,QAAQ,GAAuC;QACnD,MAAM,EAAE,qLAAqL;QAC7L,MAAM,EAAE,gKAAgK;QACxK,QAAQ,EAAE,mJAAmJ;QAC7J,YAAY,EAAE,gJAAgJ;QAC9J,aAAa,EAAE,4KAA4K;KAC5L,CAAC;IACF,OAAO,QAAQ,CAAC,UAAU,CAAC,CAAC;AAC9B,CAAC;AAED,+EAA+E;AAC/E,mDAAmD;AACnD,+EAA+E;AAE/E,MAAM,CAAC,MAAM,OAAO,GAAG;IACrB,cAAc,EAAE,EAAE;IAClB,UAAU,EAAE,EAAE;IACd,iBAAiB,EAAE,KAAK;IACxB,QAAQ,EAAE,CAAC;IACX,QAAQ,EAAE,EAAE;IACZ,WAAW,EAAE,CAAC;IACd,YAAY,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAU;IACzC,iBAAiB,EAAE,sMAAsM;CACjN,CAAC;AAEX,+EAA+E;AAC/E,uBAAuB;AACvB,+EAA+E;AAE/E,MAAM,CAAC,MAAM,MAAM,GAAG;IACpB,cAAc,EAAE,EAAE;IAClB,UAAU,EAAE,EAAE;IACd,kBAAkB,EAAE,IAAI;IACxB,qBAAqB,EAAE,GAAG;IAC1B,SAAS,EAAE,CAAC;IACZ,SAAS,EAAE,EAAE;IACb,WAAW,EAAE,CAAC;IACd,YAAY,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,KAAK,CAAU;CAC/C,CAAC;AAEX,+EAA+E;AAC/E,8BAA8B;AAC9B,+EAA+E;AAE/E,MAAM,CAAC,MAAM,WAAW,GAA2B;IACjD,CAAC,EAAE,MAAM;IACT,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,EAAE,EAAE,KAAK;CACD,CAAC;AAEX,+EAA+E;AAC/E,wEAAwE;AACxE,+EAA+E;AAE/E,MAAM,CAAC,MAAM,cAAc,GAAG;IAC5B,KAAK,EAAE,OAAO,CAAC,GAAG,CAAC,oBAAoB,IAAI,6BAA6B;IACxE,UAAU,EAAE,IAAI;CACR,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/config/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAaH,MAAM,UAAU,QAAQ;IACtB,OAAO;QACL,eAAe,EAAE,OAAO,CAAC,GAAG,CAAC,gBAAgB,IAAI,EAAE;QACnD,cAAc,EAAE,OAAO,CAAC,GAAG,CAAC,cAAc,IAAI,SAAS;QACvD,gBAAgB,EAAE,OAAO,CAAC,GAAG,CAAC,gBAAgB,IAAI,SAAS;QAC3D,oBAAoB,EAAE,OAAO,CAAC,GAAG,CAAC,oBAAoB,IAAI,SAAS;KACpE,CAAC;AACJ,CAAC;AAED,+EAA+E;AAC/E,6BAA6B;AAC7B,+EAA+E;AAE/E,MAAM,CAAC,MAAM,QAAQ,GAAG;IACtB,QAAQ,EAAE,OAAO,CAAC,GAAG,CAAC,mBAAmB,IAAI,8BAA8B;IAC3E,KAAK,EAAE,OAAO,CAAC,GAAG,CAAC,cAAc,IAAI,gCAAgC;IACrE,OAAO,EAAE,OAAO,CAAC,GAAG,CAAC,kBAAkB,IAAI,EAAE;IAC7C,UAAU,EAAE,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,cAAc,IAAI,SAAS,EAAE,EAAE,CAAC;IACjE,gBAAgB,EAAG,OAAO,CAAC,GAAG,CAAC,wBAAsD,IAAI,MAAM;IAC/F,QAAQ,EAAE,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,gBAAgB,IAAI,KAAK,EAAE,EAAE,CAAC;CACrD,CAAC;AAEX,+EAA+E;AAC/E,2BAA2B;AAC3B,+EAA+E;AAE/E,MAAM,CAAC,MAAM,MAAM,GAAG;IACpB,IAAI,EAAE,wBAAwB;IAC9B,OAAO,EAAE,OAAO;IAChB,WAAW,EAAE,6DAA6D;CAClE,CAAC;AAcX,MAAM,UAAU,eAAe;IAC7B,MAAM,GAAG,GAAG,QAAQ,EAAE,CAAC;IACvB,OAAO;QACL,MAAM,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,gBAAgB,IAAI,GAAG,CAAC,oBAAoB,CAAC;QAC5D,MAAM,EAAE,CAAC,CAAC,GAAG,CAAC,cAAc;QAC5B,QAAQ,EAAE,CAAC,CAAC,GAAG,CAAC,eAAe;QAC/B,YAAY,EAAE,CAAC,CAAC,QAAQ,CAAC,OAAO;QAChC,aAAa,EAAE,CAAC,CAAC,QAAQ,CAAC,OAAO,EAAE,uCAAuC;KAC3E,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,oBAAoB,CAAC,UAA8B;IACjE,MAAM,QAAQ,GAAuC;QACnD,MAAM,EAAE,qLAAqL;QAC7L,MAAM,EAAE,gKAAgK;QACxK,QAAQ,EAAE,mJAAmJ;QAC7J,YAAY,EAAE,gJAAgJ;QAC9J,aAAa,EAAE,4KAA4K;KAC5L,CAAC;IACF,OAAO,QAAQ,CAAC,UAAU,CAAC,CAAC;AAC9B,CAAC;AAED,+EAA+E;AAC/E,mDAAmD;AACnD,+EAA+E;AAE/E,MAAM,CAAC,MAAM,OAAO,GAAG;IACrB,cAAc,EAAE,EAAE;IAClB,UAAU,EAAE,EAAE;IACd,iBAAiB,EAAE,KAAK;IACxB,QAAQ,EAAE,CAAC;IACX,QAAQ,EAAE,EAAE;IACZ,WAAW,EAAE,CAAC;IACd,YAAY,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAU;IACzC,iBAAiB,EAAE,sMAAsM;CACjN,CAAC;AAEX,+EAA+E;AAC/E,uBAAuB;AACvB,+EAA+E;AAE/E,MAAM,CAAC,MAAM,MAAM,GAAG;IACpB,cAAc,EAAE,EAAE;IAClB,UAAU,EAAE,EAAE;IACd,kBAAkB,EAAE,IAAI;IACxB,qBAAqB,EAAE,GAAG;IAC1B,SAAS,EAAE,CAAC;IACZ,SAAS,EAAE,EAAE;IACb,WAAW,EAAE,CAAC;IACd,YAAY,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,KAAK,CAAU;CAC/C,CAAC;AAEX,+EAA+E;AAC/E,uDAAuD;AACvD,+EAA+E;AAE/E,MAAM,CAAC,MAAM,WAAW,GAA2B;IACjD,CAAC,EAAE,MAAM;IACT,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,CAAC,EAAE,KAAK;IACR,EAAE,EAAE,KAAK;CACD,CAAC;AAEX,+EAA+E;AAC/E,wEAAwE;AACxE,+EAA+E;AAE/E,MAAM,CAAC,MAAM,cAAc,GAAG;IAC5B,KAAK,EAAE,OAAO,CAAC,GAAG,CAAC,oBAAoB,IAAI,+BAA+B;IAC1E,UAAU,EAAE,IAAI;CACR,CAAC"}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "research-powerpack-mcp",
-  "version": "3.0.3",
+  "version": "3.0.5",
   "description": "The ultimate research MCP toolkit: Reddit mining, web search with CTR aggregation, AI-powered deep research, and intelligent web scraping - all in one modular package",
   "type": "module",
   "main": "dist/index.js",