iflow-mcp_bashirk-inbound-mcp 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,348 @@
1
+ Metadata-Version: 2.4
2
+ Name: iflow-mcp_bashirk-inbound-mcp
3
+ Version: 0.1.0
4
+ Summary: A production-grade lead generation MCP server using FastMCP, Crawl4AI and external enrichment services
5
+ Author-email: bashirk <hi@kobotai.co>
6
+ License: Apache-2.0
7
+ Project-URL: Homepage, https://github.com/bashirk/inbound-mcp
8
+ Project-URL: Repository, https://github.com/bashirk/inbound-mcp
9
+ Project-URL: Issues, https://github.com/bashirk/inbound-mcp/issues
10
+ Classifier: Development Status :: 4 - Beta
11
+ Classifier: Intended Audience :: Developers
12
+ Classifier: License :: OSI Approved :: Apache Software License
13
+ Classifier: Programming Language :: Python :: 3
14
+ Classifier: Programming Language :: Python :: 3.10
15
+ Classifier: Programming Language :: Python :: 3.11
16
+ Classifier: Programming Language :: Python :: 3.12
17
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
18
+ Requires-Python: >=3.10
19
+ Description-Content-Type: text/markdown
20
+ Requires-Dist: mcp>=1.26.0
21
+ Requires-Dist: crawl4ai>=0.4.3
22
+ Requires-Dist: aiocache>=0.12.0
23
+ Requires-Dist: aiohttp>=3.9.0
24
+ Requires-Dist: uvloop>=0.19.0
25
+ Requires-Dist: python-dotenv>=1.0.0
26
+ Requires-Dist: pydantic>=2.5.0
27
+ Requires-Dist: playwright>=1.40.0
28
+ Provides-Extra: dev
29
+ Requires-Dist: pytest>=7.4.0; extra == "dev"
30
+ Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
31
+ Requires-Dist: pytest-benchmark>=4.0.0; extra == "dev"
32
+ Requires-Dist: black>=23.9.0; extra == "dev"
33
+ Requires-Dist: mypy>=1.5.0; extra == "dev"
34
+ Requires-Dist: isort>=5.12.0; extra == "dev"
35
+ Provides-Extra: monitoring
36
+ Requires-Dist: prometheus-client>=0.17.0; extra == "monitoring"
37
+ Requires-Dist: sentry-sdk>=1.30.0; extra == "monitoring"
38
+ Provides-Extra: server
39
+ Requires-Dist: gunicorn>=21.2.0; extra == "server"
40
+ Requires-Dist: uvicorn>=0.23.0; extra == "server"
41
+
42
+ # Lead Generation Server Documentation
43
+
44
+ ![MCP SDK](https://img.shields.io/badge/MCP_SDK-2.1.0-blue)
45
+ ![Crawl4AI](https://img.shields.io/badge/Crawl4AI-0.4.3bx-green)
46
+ ![Python](https://img.shields.io/badge/Python-3.10%2B-yellow)
47
+
48
+ ## Table of Contents
49
+ 1. [Overview](#overview)
50
+ 2. [Features](#features)
51
+ 3. [Architecture](#architecture)
52
+ 4. [Prerequisites](#prerequisites)
53
+ 5. [Installation](#installation)
54
+ 6. [Configuration](#configuration)
55
+ 7. [Running the Server](#running-the-server)
56
+ 8. [API Documentation](#api-documentation)
57
+ 9. [Examples](#examples)
58
+ 10. [Advanced Configuration](#advanced-configuration)
59
+ 11. [Troubleshooting](#troubleshooting)
60
+ 12. [Contributing](#contributing)
61
+ 13. [License](#license)
62
+ 14. [Roadmap](#roadmap)
63
+ 15. [Support](#support)
64
+
65
+ ---
66
+
67
+ ## Overview <a name="overview"></a>
68
+ A production-grade lead generation system built on:
69
+ - **MCP Python SDK** for protocol-compliant AI services
70
+ - **Crawl4AI** for intelligent web crawling
71
+ - **AsyncIO** for high-concurrency operations
72
+
73
+ Implements a full lead lifecycle from discovery to enrichment with:
74
+ - UUID-based lead tracking
75
+ - Multi-source data aggregation
76
+ - Smart caching strategies
77
+ - Enterprise-grade error handling
78
+
79
+ ---
80
+
81
+ ## Features <a name="features"></a>
82
+ | Feature | Tech Stack | Throughput |
83
+ |---------|------------|------------|
84
+ | Lead Generation | Google CSE, Crawl4AI | 120 req/min |
85
+ | Data Enrichment | Hunter.io, Clearbit [Hubspot Breeze] | 80 req/min |
86
+ | LinkedIn Scraping | Playwright, Stealth Mode | 40 req/min |
87
+ | Caching | aiocache, Redis | 10K ops/sec |
88
+ | Monitoring | Prometheus, Custom Metrics | Real-time |
89
+
90
+ ---
91
+
92
+ ## Architecture <a name="architecture"></a>
93
+ ```mermaid
94
+ graph TD
95
+ A[Client] --> B[MCP Server]
96
+ B --> C{Lead Manager}
97
+ C --> D[Google CSE]
98
+ C --> E[Crawl4AI]
99
+ C --> F[Hunter.io]
100
+ C --> G[Clearbit]
101
+ C --> H[LinkedIn Scraper]
102
+ C --> I[(Redis Cache)]
103
+ C --> J[Lead Store]
104
+ ```
105
+
106
+ ---
107
+
108
+ ## Prerequisites <a name="prerequisites"></a>
109
+ - Python 3.10+
110
+ - API Keys:
111
+ ```bash
112
+ export HUNTER_API_KEY="your_key"
113
+ export CLEARBIT_API_KEY="your_key"
114
+ export GOOGLE_CSE_ID="your_id"
115
+ export GOOGLE_API_KEY="your_key"
116
+ ```
117
+ - LinkedIn Session Cookie (for scraping)
118
+ - 4GB+ RAM (8GB recommended for heavy scraping)
119
+
120
+ ---
121
+
122
+ ## Installation <a name="installation"></a>
123
+
124
+ ### Production Setup
125
+ ```bash
126
+ # Create virtual environment
127
+ python -m venv .venv && source .venv/bin/activate
128
+
129
+ # Install with production dependencies
130
+ pip install mcp crawl4ai[all] aiocache aiohttp uvloop
131
+
132
+ # Set up browser dependencies
133
+ python -m playwright install chromium
134
+ ```
135
+
136
+ ### Docker Deployment
137
+ ```dockerfile
138
+ FROM python:3.10-slim
139
+
140
+ RUN apt-get update && apt-get install -y \
141
+ gcc \
142
+ libpython3-dev \
143
+ chromium \
144
+ && rm -rf /var/lib/apt/lists/*
145
+
146
+ COPY . /app
147
+ WORKDIR /app
148
+
149
+ RUN pip install --no-cache-dir -r requirements.txt
150
+ CMD ["python", "-m", "mcp", "run", "lead_server.py"]
151
+ ```
152
+
153
+ ---
154
+
155
+ ## Configuration <a name="configuration"></a>
156
+ `config.yaml`
157
+ ```yaml
158
+ services:
159
+ hunter:
160
+ api_key: ${HUNTER_API_KEY}
161
+ rate_limit: 50/60s
162
+
163
+ clearbit:
164
+ api_key: ${CLEARBIT_API_KEY}
165
+ cache_ttl: 86400
166
+
167
+ scraping:
168
+ stealth_mode: true
169
+ headless: true
170
+ timeout: 30
171
+ max_retries: 3
172
+
173
+ cache:
174
+ backend: redis://localhost:6379/0
175
+ default_ttl: 3600
176
+ ```
177
+
178
+ ---
179
+
180
+ ## Running the Server <a name="running-the-server"></a>
181
+
182
+ ### Development Mode
183
+ ```bash
184
+ mcp dev lead_server.py --reload --port 8080
185
+ ```
186
+
187
+ ### Production
188
+ ```bash
189
+ gunicorn -w 4 -k uvicorn.workers.UvicornWorker lead_server:app
190
+ ```
191
+
192
+ ### Docker
193
+ ```bash
194
+ docker build -t lead-server .
195
+ docker run -p 8080:8080 -e HUNTER_API_KEY=your_key lead-server
196
+ ```
197
+
198
+ ---
199
+
200
+ ## API Documentation <a name="api-documentation"></a>
201
+
202
+ ### 1. Generate Lead
203
+ ```http
204
+ POST /tools/lead_generation
205
+ Content-Type: application/json
206
+
207
+ {
208
+ "search_terms": "OpenAI"
209
+ }
210
+ ```
211
+
212
+ Response:
213
+ ```json
214
+ {
215
+ "lead_id": "550e8400-e29b-41d4-a716-446655440000",
216
+ "status": "pending",
217
+ "estimated_time": 15
218
+ }
219
+ ```
220
+
221
+ ### 2. Enrich Lead
222
+ ```http
223
+ POST /tools/data_enrichment
224
+ Content-Type: application/json
225
+
226
+ {
227
+ "lead_id": "550e8400-e29b-41d4-a716-446655440000"
228
+ }
229
+ ```
230
+
231
+ ### 3. Monitor Leads
232
+ ```http
233
+ GET /tools/lead_maintenance
234
+ ```
235
+
236
+ ---
237
+
238
+ ## Examples <a name="examples"></a>
239
+
240
+ ### Python Client
241
+ ```python
242
+ from mcp.client import Client
243
+
244
+ async with Client() as client:
245
+ # Generate lead
246
+ lead = await client.call_tool(
247
+ "lead_generation",
248
+ {"search_terms": "Anthropic"}
249
+ )
250
+
251
+ # Enrich with all services
252
+ enriched = await client.call_tool(
253
+ "data_enrichment",
254
+ {"lead_id": lead['lead_id']}
255
+ )
256
+
257
+ # Get full lead data
258
+ status = await client.call_tool(
259
+ "lead_status",
260
+ {"lead_id": lead['lead_id']}
261
+ )
262
+ ```
263
+
264
+ ### cURL
265
+ ```bash
266
+ # Generate lead
267
+ curl -X POST http://localhost:8080/tools/lead_generation \
268
+ -H "Content-Type: application/json" \
269
+ -d '{"search_terms": "Cohere AI"}'
270
+ ```
271
+
272
+ ---
273
+
274
+ ## Advanced Configuration <a name="advanced-configuration"></a>
275
+
276
+ ### Caching Strategies
277
+ ```python
278
+ from aiocache import Cache
279
+
280
+ # Configure Redis cluster
281
+ Cache.from_url(
282
+ "redis://cluster-node1:6379/0",
283
+ timeout=10,
284
+ retry=True,
285
+ retry_timeout=2
286
+ )
287
+ ```
288
+
289
+ ### Rate Limiting
290
+ ```python
291
+ from mcp.server.middleware import RateLimiter
292
+
293
+ mcp.add_middleware(
294
+ RateLimiter(
295
+ rules={
296
+ "lead_generation": "100/1m",
297
+ "data_enrichment": "50/1m"
298
+ }
299
+ )
300
+ )
301
+ ```
302
+
303
+ ---
304
+
305
+ ## Troubleshooting <a name="troubleshooting"></a>
306
+
307
+ | Error | Solution |
308
+ |-------|----------|
309
+ | `403 Forbidden` from Google | Rotate IPs or use official CSE API |
310
+ | `429 Too Many Requests` | Implement exponential backoff |
311
+ | `Playwright Timeout` | Increase `scraping.timeout` in config |
312
+ | `Cache Miss` | Verify Redis connection and TTL settings |
313
+
314
+ ---
315
+
316
+ ## Contributing <a name="contributing"></a>
317
+ 1. Fork the repository
318
+ 2. Create feature branch: `git checkout -b feature/new-enrichment`
319
+ 3. Commit changes: `git commit -am 'Add Clearbit alternative'`
320
+ 4. Push to branch: `git push origin feature/new-enrichment`
321
+ 5. Submit pull request
322
+
323
+ ---
324
+
325
+ ## License <a name="license"></a>
326
+ Apache 2.0 - See [LICENSE](LICENSE) for details.
327
+
328
+ ---
329
+
330
+ ## Roadmap <a name="roadmap"></a>
331
+ - [ ] **Q2 2025**: AI-powered lead scoring
332
+ - [ ] **Q3 2025**: Distributed crawling cluster support
333
+
334
+ ---
335
+
336
+ ## Support <a name="support"></a>
337
+ For enterprise support and custom integrations:
338
+ 📧 Email: [hi@kobotai.co](mailto:support@leadgen.ai)
339
+ 🐦 Twitter: [@KobotAIco](https://x.com/KobotAIco)
340
+
341
+ ---
342
+
343
+ ```bash
344
+ # Run benchmark tests
345
+ pytest tests/ --benchmark-json=results.json
346
+ ```
347
+
348
+ ![Benchmark Results](https://via.placeholder.com/800x400.png?text=Performance+Metrics)
@@ -0,0 +1,307 @@
1
+ # Lead Generation Server Documentation
2
+
3
+ ![MCP SDK](https://img.shields.io/badge/MCP_SDK-2.1.0-blue)
4
+ ![Crawl4AI](https://img.shields.io/badge/Crawl4AI-0.4.3bx-green)
5
+ ![Python](https://img.shields.io/badge/Python-3.10%2B-yellow)
6
+
7
+ ## Table of Contents
8
+ 1. [Overview](#overview)
9
+ 2. [Features](#features)
10
+ 3. [Architecture](#architecture)
11
+ 4. [Prerequisites](#prerequisites)
12
+ 5. [Installation](#installation)
13
+ 6. [Configuration](#configuration)
14
+ 7. [Running the Server](#running-the-server)
15
+ 8. [API Documentation](#api-documentation)
16
+ 9. [Examples](#examples)
17
+ 10. [Advanced Configuration](#advanced-configuration)
18
+ 11. [Troubleshooting](#troubleshooting)
19
+ 12. [Contributing](#contributing)
20
+ 13. [License](#license)
21
+ 14. [Roadmap](#roadmap)
22
+ 15. [Support](#support)
23
+
24
+ ---
25
+
26
+ ## Overview <a name="overview"></a>
27
+ A production-grade lead generation system built on:
28
+ - **MCP Python SDK** for protocol-compliant AI services
29
+ - **Crawl4AI** for intelligent web crawling
30
+ - **AsyncIO** for high-concurrency operations
31
+
32
+ Implements a full lead lifecycle from discovery to enrichment with:
33
+ - UUID-based lead tracking
34
+ - Multi-source data aggregation
35
+ - Smart caching strategies
36
+ - Enterprise-grade error handling
37
+
38
+ ---
39
+
40
+ ## Features <a name="features"></a>
41
+ | Feature | Tech Stack | Throughput |
42
+ |---------|------------|------------|
43
+ | Lead Generation | Google CSE, Crawl4AI | 120 req/min |
44
+ | Data Enrichment | Hunter.io, Clearbit [Hubspot Breeze] | 80 req/min |
45
+ | LinkedIn Scraping | Playwright, Stealth Mode | 40 req/min |
46
+ | Caching | aiocache, Redis | 10K ops/sec |
47
+ | Monitoring | Prometheus, Custom Metrics | Real-time |
48
+
49
+ ---
50
+
51
+ ## Architecture <a name="architecture"></a>
52
+ ```mermaid
53
+ graph TD
54
+ A[Client] --> B[MCP Server]
55
+ B --> C{Lead Manager}
56
+ C --> D[Google CSE]
57
+ C --> E[Crawl4AI]
58
+ C --> F[Hunter.io]
59
+ C --> G[Clearbit]
60
+ C --> H[LinkedIn Scraper]
61
+ C --> I[(Redis Cache)]
62
+ C --> J[Lead Store]
63
+ ```
64
+
65
+ ---
66
+
67
+ ## Prerequisites <a name="prerequisites"></a>
68
+ - Python 3.10+
69
+ - API Keys:
70
+ ```bash
71
+ export HUNTER_API_KEY="your_key"
72
+ export CLEARBIT_API_KEY="your_key"
73
+ export GOOGLE_CSE_ID="your_id"
74
+ export GOOGLE_API_KEY="your_key"
75
+ ```
76
+ - LinkedIn Session Cookie (for scraping)
77
+ - 4GB+ RAM (8GB recommended for heavy scraping)
78
+
79
+ ---
80
+
81
+ ## Installation <a name="installation"></a>
82
+
83
+ ### Production Setup
84
+ ```bash
85
+ # Create virtual environment
86
+ python -m venv .venv && source .venv/bin/activate
87
+
88
+ # Install with production dependencies
89
+ pip install mcp crawl4ai[all] aiocache aiohttp uvloop
90
+
91
+ # Set up browser dependencies
92
+ python -m playwright install chromium
93
+ ```
94
+
95
+ ### Docker Deployment
96
+ ```dockerfile
97
+ FROM python:3.10-slim
98
+
99
+ RUN apt-get update && apt-get install -y \
100
+ gcc \
101
+ libpython3-dev \
102
+ chromium \
103
+ && rm -rf /var/lib/apt/lists/*
104
+
105
+ COPY . /app
106
+ WORKDIR /app
107
+
108
+ RUN pip install --no-cache-dir -r requirements.txt
109
+ CMD ["python", "-m", "mcp", "run", "lead_server.py"]
110
+ ```
111
+
112
+ ---
113
+
114
+ ## Configuration <a name="configuration"></a>
115
+ `config.yaml`
116
+ ```yaml
117
+ services:
118
+ hunter:
119
+ api_key: ${HUNTER_API_KEY}
120
+ rate_limit: 50/60s
121
+
122
+ clearbit:
123
+ api_key: ${CLEARBIT_API_KEY}
124
+ cache_ttl: 86400
125
+
126
+ scraping:
127
+ stealth_mode: true
128
+ headless: true
129
+ timeout: 30
130
+ max_retries: 3
131
+
132
+ cache:
133
+ backend: redis://localhost:6379/0
134
+ default_ttl: 3600
135
+ ```
136
+
137
+ ---
138
+
139
+ ## Running the Server <a name="running-the-server"></a>
140
+
141
+ ### Development Mode
142
+ ```bash
143
+ mcp dev lead_server.py --reload --port 8080
144
+ ```
145
+
146
+ ### Production
147
+ ```bash
148
+ gunicorn -w 4 -k uvicorn.workers.UvicornWorker lead_server:app
149
+ ```
150
+
151
+ ### Docker
152
+ ```bash
153
+ docker build -t lead-server .
154
+ docker run -p 8080:8080 -e HUNTER_API_KEY=your_key lead-server
155
+ ```
156
+
157
+ ---
158
+
159
+ ## API Documentation <a name="api-documentation"></a>
160
+
161
+ ### 1. Generate Lead
162
+ ```http
163
+ POST /tools/lead_generation
164
+ Content-Type: application/json
165
+
166
+ {
167
+ "search_terms": "OpenAI"
168
+ }
169
+ ```
170
+
171
+ Response:
172
+ ```json
173
+ {
174
+ "lead_id": "550e8400-e29b-41d4-a716-446655440000",
175
+ "status": "pending",
176
+ "estimated_time": 15
177
+ }
178
+ ```
179
+
180
+ ### 2. Enrich Lead
181
+ ```http
182
+ POST /tools/data_enrichment
183
+ Content-Type: application/json
184
+
185
+ {
186
+ "lead_id": "550e8400-e29b-41d4-a716-446655440000"
187
+ }
188
+ ```
189
+
190
+ ### 3. Monitor Leads
191
+ ```http
192
+ GET /tools/lead_maintenance
193
+ ```
194
+
195
+ ---
196
+
197
+ ## Examples <a name="examples"></a>
198
+
199
+ ### Python Client
200
+ ```python
201
+ from mcp.client import Client
202
+
203
+ async with Client() as client:
204
+ # Generate lead
205
+ lead = await client.call_tool(
206
+ "lead_generation",
207
+ {"search_terms": "Anthropic"}
208
+ )
209
+
210
+ # Enrich with all services
211
+ enriched = await client.call_tool(
212
+ "data_enrichment",
213
+ {"lead_id": lead['lead_id']}
214
+ )
215
+
216
+ # Get full lead data
217
+ status = await client.call_tool(
218
+ "lead_status",
219
+ {"lead_id": lead['lead_id']}
220
+ )
221
+ ```
222
+
223
+ ### cURL
224
+ ```bash
225
+ # Generate lead
226
+ curl -X POST http://localhost:8080/tools/lead_generation \
227
+ -H "Content-Type: application/json" \
228
+ -d '{"search_terms": "Cohere AI"}'
229
+ ```
230
+
231
+ ---
232
+
233
+ ## Advanced Configuration <a name="advanced-configuration"></a>
234
+
235
+ ### Caching Strategies
236
+ ```python
237
+ from aiocache import Cache
238
+
239
+ # Configure Redis cluster
240
+ Cache.from_url(
241
+ "redis://cluster-node1:6379/0",
242
+ timeout=10,
243
+ retry=True,
244
+ retry_timeout=2
245
+ )
246
+ ```
247
+
248
+ ### Rate Limiting
249
+ ```python
250
+ from mcp.server.middleware import RateLimiter
251
+
252
+ mcp.add_middleware(
253
+ RateLimiter(
254
+ rules={
255
+ "lead_generation": "100/1m",
256
+ "data_enrichment": "50/1m"
257
+ }
258
+ )
259
+ )
260
+ ```
261
+
262
+ ---
263
+
264
+ ## Troubleshooting <a name="troubleshooting"></a>
265
+
266
+ | Error | Solution |
267
+ |-------|----------|
268
+ | `403 Forbidden` from Google | Rotate IPs or use official CSE API |
269
+ | `429 Too Many Requests` | Implement exponential backoff |
270
+ | `Playwright Timeout` | Increase `scraping.timeout` in config |
271
+ | `Cache Miss` | Verify Redis connection and TTL settings |
272
+
273
+ ---
274
+
275
+ ## Contributing <a name="contributing"></a>
276
+ 1. Fork the repository
277
+ 2. Create feature branch: `git checkout -b feature/new-enrichment`
278
+ 3. Commit changes: `git commit -am 'Add Clearbit alternative'`
279
+ 4. Push to branch: `git push origin feature/new-enrichment`
280
+ 5. Submit pull request
281
+
282
+ ---
283
+
284
+ ## License <a name="license"></a>
285
+ Apache 2.0 - See [LICENSE](LICENSE) for details.
286
+
287
+ ---
288
+
289
+ ## Roadmap <a name="roadmap"></a>
290
+ - [ ] **Q2 2025**: AI-powered lead scoring
291
+ - [ ] **Q3 2025**: Distributed crawling cluster support
292
+
293
+ ---
294
+
295
+ ## Support <a name="support"></a>
296
+ For enterprise support and custom integrations:
297
+ 📧 Email: [hi@kobotai.co](mailto:support@leadgen.ai)
298
+ 🐦 Twitter: [@KobotAIco](https://x.com/KobotAIco)
299
+
300
+ ---
301
+
302
+ ```bash
303
+ # Run benchmark tests
304
+ pytest tests/ --benchmark-json=results.json
305
+ ```
306
+
307
+ ![Benchmark Results](https://via.placeholder.com/800x400.png?text=Performance+Metrics)