pse-mcp 0.1.0

package/MCP Documents/mcp-windows.txt

@@ -0,0 +1,161 @@
+# Windows-Specific MCP Implementation Guide
+
+## Common Issues with stdio on Windows
+
+### 1. Line Ending Differences
+- Windows uses CRLF (`\r\n`) for line endings
+- Unix-based systems use LF (`\n`)
+- This difference can cause serious problems in stdio communication with MCP servers
+- JSON-RPC 2.0 (which MCP uses) expects consistent message framing
+
+### 2. Text vs. Binary Mode
+- Windows default text mode automatically translates line endings
+- This automatic translation can corrupt protocol data
+- Local MCP servers must not log messages to stdout
+- All protocol communication must use binary mode
+
+### 3. Console Buffering Issues
+- Windows console applications have different buffering behavior
+- Can cause timing issues or message truncation in stdio transport
+- Requires proper buffer handling in both client and server
+
+### 4. Environment Variable Inheritance
+- MCP servers inherit only a subset of environment variables automatically
+- Windows has different environment variable handling
+- May require additional configuration for proper environment setup
+
+## Best Practices for Windows Implementation
+
+### 1. Use Binary Mode for stdio Streams
+For Python implementations on Windows:
+```python
+import sys
+import os
+
+# Force binary mode for stdin/stdout
+if sys.platform == 'win32':
+    import msvcrt
+    msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY)
+    msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
+```
+
+For TypeScript/Node.js implementations:
+```typescript
+if (process.platform === 'win32') {
+    process.stdin.setEncoding('binary');
+    process.stdout.setDefaultEncoding('binary');
+}
+```
+
+### 2. Leverage Existing SDKs
+- Use official SDKs when possible
+- Python SDK includes proper Windows stdio handling
+- TypeScript SDK handles cross-platform differences
+- Avoid implementing custom stdio handling
+
+### 3. Console Output Best Practices
+- Use stderr for logging and debugging
+- Never write to stdout in server implementations
+- Log messages are automatically captured by host applications
+- Keep protocol communication separate from logging
+
+### 4. Path Handling
+- Windows uses backslashes (`\`) as path separators
+- Forward slashes (`/`) also work on Windows
+- Escape backslashes in string literals
+- Consider using path.join() for cross-platform compatibility
+
+### 5. Testing Guidelines
+- Test explicitly on Windows
+- Don't assume Unix-working code will work on Windows
+- Test with different Windows versions
+- Verify stdio behavior in various scenarios
+
+### 6. Error Handling
+- Implement platform-specific error handling
+- Handle Windows-specific exceptions
+- Add retry logic for transient issues
+- Provide detailed error messages
+
+### 7. Transport Alternatives
+- Consider HTTP+SSE for more consistent behavior
+- Use WebSocket transport when available
+- Implement fallback mechanisms
+- Document transport limitations
+
+## Implementation Examples
+
+### Basic Python MCP Server
+```python
+from mcp import Server, StdioTransport
+import sys
+import os
+
+# Windows-specific setup
+if sys.platform == 'win32':
+    import msvcrt
+    msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY)
+    msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
+
+# Create server with proper error handling
+server = Server(
+    name="windows-example",
+    version="1.0.0",
+    transport=StdioTransport()
+)
+
+# Log to stderr, not stdout
+def log_error(msg):
+    print(f"Error: {msg}", file=sys.stderr)
+
+try:
+    server.start()
+except Exception as e:
+    log_error(f"Server error: {e}")
+    sys.exit(1)
+```
+
+### Basic TypeScript MCP Server
+```typescript
+import { McpServer } from "@modelcontextprotocol/sdk/server";
+import { StdioServerTransport } from "@modelcontextprotocol/sdk/transport/stdio";
+
+// Windows-specific setup
+if (process.platform === 'win32') {
+    process.stdin.setEncoding('binary');
+    process.stdout.setDefaultEncoding('binary');
+}
+
+const server = new McpServer({
+    name: "windows-example",
+    version: "1.0.0"
+});
+
+// Error handling for Windows
+process.on('uncaughtException', (error) => {
+    console.error(`Uncaught Exception: ${error.message}`);
+    process.exit(1);
+});
+
+const transport = new StdioServerTransport();
+server.listen(transport).catch((error) => {
+    console.error(`Server Error: ${error.message}`);
+    process.exit(1);
+});
+```
+
+## Best Practices Summary
+
+1. **Always Use Binary Mode**: Prevent line ending translation issues
+2. **Proper Error Handling**: Account for Windows-specific exceptions
+3. **Path Management**: Handle Windows path separators correctly
+4. **Testing**: Verify functionality specifically on Windows
+5. **SDK Usage**: Leverage official SDKs for platform compatibility
+6. **Logging**: Use stderr for debugging, keep stdout clean
+7. **Transport Options**: Consider alternatives to stdio when needed
+
+## Additional Resources
+
+- [Official MCP Windows Documentation](https://modelcontextprotocol.io/docs/windows)
+- [Windows-Specific Examples](https://github.com/modelcontextprotocol/examples/windows)
+- [Troubleshooting Guide](https://modelcontextprotocol.io/docs/troubleshooting#windows)

package/QWEN.md

@@ -0,0 +1,207 @@
+# Google Search MCP Server - Project Context
+
+## Project Overview
+
+The Google Search MCP Server is a Model Context Protocol (MCP) server that provides Google search capabilities and webpage content analysis tools. This server enables AI models to perform Google searches and analyze webpage content programmatically through a standardized MCP interface.
+
+### Key Features
+- Google Custom Search integration
+- Advanced search features (filters, sorting, pagination, categorization)
+- Webpage content analysis in multiple formats (markdown, HTML, plain text)
+- Batch webpage analysis
+- Result categorization and classification
+- Content summarization
+- Optimized, human-readable responses
+- MCP-compliant interface
+
+### Technology Stack
+- **Language**: TypeScript
+- **Framework**: Node.js
+- **Core Dependencies**:
+  - `@modelcontextprotocol/sdk`: MCP server SDK
+  - `googleapis`: Google API integration
+  - `@mozilla/readability`: Content extraction
+  - `cheerio`: HTML parsing
+  - `jsdom`: DOM manipulation
+  - `turndown`: HTML to Markdown conversion
+  - `axios`: HTTP client
+  - `express`: Web framework (if needed)
+
+## Project Structure
+
+```
+D:\ai\pse-mcp\
+├── dist/                 # Compiled JavaScript files
+├── dist-package/         # Distribution package
+├── MCP Documents/        # MCP protocol documentation
+├── src/                  # Source TypeScript files
+│   ├── services/         # Service implementations
+│   │   ├── google-search.service.ts
+│   │   └── content-extractor.service.ts
+│   ├── google-search.ts  # Main server entry point
+│   ├── mcp.d.ts          # MCP type definitions
+│   └── types.ts          # Type definitions
+├── package.json          # Project dependencies and scripts
+├── tsconfig.json         # TypeScript configuration
+├── README.md             # Project documentation
+└── ...
+```
+
+## Building and Running
+
+### Prerequisites
+- Node.js (v16 or higher)
+- Google Cloud Platform account
+- Custom Search Engine ID
+- Google API Key
+
+### Setup Commands
+1. Install dependencies:
+   ```bash
+   npm install
+   ```
+
+2. Build the TypeScript code:
+   ```bash
+   npm run build
+   ```
+
+3. Run the server:
+   ```bash
+   npm run start
+   ```
+
+### Environment Variables
+Required environment variables:
+- `GOOGLE_API_KEY`: Your Google API key
+- `GOOGLE_SEARCH_ENGINE_ID`: Your Custom Search Engine ID
+
+### Development Scripts
+- `npm run build`: Compiles TypeScript to JavaScript
+- `npm run start`: Runs the compiled server
+- `npm run dev`: Watches for changes and recompiles (tsc -w)
+
+## Architecture and Components
+
+### Main Components
+1. **GoogleSearchService**: Handles Google API interactions for search functionality
+2. **ContentExtractor**: Manages webpage content analysis and extraction
+3. **Main Server (google-search.ts)**: MCP server implementation with tool handlers
+
+### Services
+
+#### GoogleSearchService
+- Integrates with Google Custom Search API
+- Provides caching mechanism (5-minute TTL, max 100 entries)
+- Implements advanced search filtering and pagination
+- Categorizes search results by content type
+- Handles error management and response formatting
+
+#### ContentExtractor
+- Extracts webpage content using Mozilla Readability
+- Converts content to markdown, HTML, or plain text formats
+- Implements content caching (30-minute TTL, max 50 entries)
+- Generates content summaries and statistics
+- Handles batch processing of multiple webpages
+
+### Available Tools
+
+#### 1. google_search
+Searches Google and returns relevant results with advanced filtering options:
+- Query string (required)
+- Number of results (default: 5, max: 10)
+- Site filtering
+- Language filtering (ISO 639-1 codes)
+- Date restrictions
+- Exact phrase matching
+- Result type (news, images, videos)
+- Pagination support
+- Sorting (relevance or date)
+
+#### 2. extract_webpage_content
+Extracts and analyzes content from a single webpage:
+- URL (required)
+- Output format (markdown, html, text)
+- Removes ads, navigation, and clutter
+- Returns title, description, content stats, and summary
+
+#### 3. extract_multiple_webpages
+Extracts content from multiple webpages in a single request:
+- Array of URLs (max 5 per request)
+- Output format (markdown, html, text)
+- Batch processing capability
+
+## Configuration
+
+The server configuration needs to be added to the MCP settings file (typically located at `%APPDATA%/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json`):
+
+```json
+{
+  "mcpServers": {
+    "google-search": {
+      "autoApprove": [
+        "google_search",
+        "extract_webpage_content",
+        "extract_multiple_webpages"
+      ],
+      "disabled": false,
+      "timeout": 60,
+      "command": "node",
+      "args": [
+        "/path/to/google-search-mcp-server/dist/google-search.js"
+      ],
+      "env": {
+        "GOOGLE_API_KEY": "your-google-api-key",
+        "GOOGLE_SEARCH_ENGINE_ID": "your-custom-search-engine-id"
+      },
+      "transportType": "stdio"
+    }
+  }
+}
+```
+
+## Development Conventions
+
+### Coding Standards
+- TypeScript with strict mode enabled
+- Use of async/await for asynchronous operations
+- Proper error handling with descriptive messages
+- Input validation for all tool arguments
+- Caching strategies for performance optimization
+
+### Type Safety
+- Comprehensive type definitions in `types.ts`
+- Strict typing for all function parameters and return values
+- MCP request/response schema validation
+
+### Caching Strategy
+- Search results: 5-minute TTL with max 100 entries
+- Webpage content: 30-minute TTL with max 50 entries
+- Cache keys generated from request parameters
+- Automatic cleanup of oldest entries when limits exceeded
+
+## MCP Protocol Integration
+
+The server implements the Model Context Protocol with:
+- Standardized tool listing and calling
+- Input schema validation
+- Error response formatting
+- Stdio transport for communication with MCP clients
+
+## Testing and Verification
+
+To verify the server is working:
+1. Build with `npm run build`
+2. Start with `npm run start`
+3. Use MCP client to call the available tools
+4. Verify search results and content extraction work as expected
+
+## Deployment
+
+For distribution, the project includes a process to create a compiled distribution package:
+1. Build the TypeScript code
+2. Create a distribution package with only necessary files
+3. Include production dependencies only
+4. Simplified package.json for end users
+
+The distribution approach allows for shipping compiled JavaScript without exposing source code while maintaining functionality.

package/README.md

@@ -0,0 +1,220 @@
+# Version 2.0 is here
+
+# Google Search MCP Server
+An MCP (Model Context Protocol) server that provides Google search capabilities and webpage content analysis tools. This server enables AI models to perform Google searches and analyze webpage content programmatically.
+
+## Features
+
+- Google Custom Search integration
+- Advanced search features (filters, sorting, pagination, categorization)
+- Webpage content analysis in multiple formats (markdown, HTML, plain text)
+- Batch webpage analysis
+- Result categorization and classification
+- Content summarization
+- Optimized, human-readable responses
+- MCP-compliant interface
+
+## Prerequisites
+
+- Node.js (v16 or higher)
+- Google Cloud Platform account
+- Custom Search Engine ID
+- Google API Key
+
+## Installation
+
+1. Clone the repository
+2. Install Node.js dependencies:
+```bash
+npm install
+```
+3. Build the TypeScript code:
+```bash
+npm run build
+```
+
+## Configuration
+
+1. Set up environment variables for your Google API credentials:
+
+You can either set these as system environment variables or configure them in your MCP settings file.
+
+Required environment variables:
+- `GOOGLE_API_KEY`: Your Google API key
+- `GOOGLE_SEARCH_ENGINE_ID`: Your Custom Search Engine ID
+
+2. Add the server configuration to your MCP settings file (typically located at `%APPDATA%/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json`):
+```json
+{
+  "mcpServers": {
+    "google-search": {
+      "autoApprove": [
+        "google_search",
+        "extract_webpage_content",
+        "extract_multiple_webpages"
+      ],
+      "disabled": false,
+      "timeout": 60,
+      "command": "node",
+      "args": [
+        "/path/to/google-search-mcp-server/dist/google-search.js"
+      ],
+      "env": {
+        "GOOGLE_API_KEY": "your-google-api-key",
+        "GOOGLE_SEARCH_ENGINE_ID": "your-custom-search-engine-id"
+      },
+      "transportType": "stdio"
+    }
+  }
+}
+```
+
+## Running
+
+Start the MCP server:
+```bash
+npm run start
+```
+
+## Available Tools
+
+### 1. google_search
+Search Google and return relevant results from the web. This tool finds web pages, articles, and information on specific topics using Google's search engine.
+
+```typescript
+{
+  "name": "google_search",
+  "arguments": {
+    "query": "your search query",
+    "num_results": 5, // optional, default: 5
+    "site": "example.com", // optional, limit results to specific website
+    "language": "en", // optional, filter by language (ISO 639-1 code)
+    "dateRestrict": "m6", // optional, filter by date (e.g., "m6" for last 6 months)
+    "exactTerms": "exact phrase", // optional, search for exact phrase
+    "resultType": "news", // optional, specify type (news, images, videos)
+    "page": 2, // optional, page number for pagination (starts at 1)
+    "resultsPerPage": 10, // optional, results per page (max: 10)
+    "sort": "date" // optional, sort by "date" or "relevance" (default)
+  }
+}
+```
+
+Response includes:
+- Search results with title, link, snippet in a readable format
+- Pagination information (current page, total results, etc.)
+- Categories of results (automatically detected)
+- Navigation hints for pagination
+
+### 2. extract_webpage_content
+Extract and analyze content from a webpage, converting it to readable text. This tool fetches the main content while removing ads, navigation elements, and other clutter.
+
+```typescript
+{
+  "name": "extract_webpage_content",
+  "arguments": {
+    "url": "https://example.com",
+    "format": "markdown" // optional, format options: "markdown" (default), "html", or "text"
+  }
+}
+```
+
+Response includes:
+- Title and description of the webpage
+- Content statistics (word count, character count)
+- Content summary
+- Content preview (first 500 characters)
+
+### 3. extract_multiple_webpages
+Extract and analyze content from multiple webpages in a single request. Ideal for comparing information across different sources or gathering comprehensive information on a topic.
+
+```typescript
+{
+  "name": "extract_multiple_webpages",
+  "arguments": {
+    "urls": [
+      "https://example1.com",
+      "https://example2.com"
+    ],
+    "format": "html" // optional, format options: "markdown" (default), "html", or "text"
+  }
+}
+```
+
+Response includes:
+- Title and description of each webpage
+- Content statistics for each webpage
+- Content summary for each webpage
+- Content preview for each webpage (first 150 characters)
+
+## Getting Google API Credentials
+
+1. Go to the [Google Cloud Console](https://console.cloud.google.com/)
+2. Create a new project or select an existing one
+3. Enable the Custom Search API
+4. Create API credentials (API Key)
+5. Go to the [Custom Search Engine](https://programmablesearchengine.google.com/about/) page
+6. Create a new search engine and get your Search Engine ID
+7. Add these credentials to your MCP settings file or set them as environment variables
+
+## Error Handling
+
+The server provides detailed error messages for:
+- Missing or invalid API credentials
+- Failed search requests
+- Invalid webpage URLs
+- Network connectivity issues
+
+## Architecture
+
+The server is built with TypeScript and uses the MCP SDK to provide a standardized interface for AI models to interact with Google Search and webpage content analysis tools. It consists of two main services:
+
+1. **GoogleSearchService**: Handles Google API interactions for search functionality
+2. **ContentExtractor**: Manages webpage content analysis and extraction
+
+The server uses caching mechanisms to improve performance and reduce API calls.
+
+## Distributing the Built Version
+
+If you prefer to distribute only the built version of this tool rather than the source code, you can follow these steps:
+
+1. Build the TypeScript code:
+```bash
+npm run build
+```
+
+2. Create a distribution package with only the necessary files:
+```bash
+# Create a distribution directory
+mkdir -p dist-package
+
+# Copy the compiled JavaScript files
+cp -r dist dist-package/
+
+# Copy package files (without dev dependencies)
+cp package.json dist-package/
+cp README.md dist-package/
+
+# Create a simplified package.json for distribution
+node -e "const pkg = require('./package.json'); delete pkg.devDependencies; delete pkg.scripts.build; delete pkg.scripts.dev; pkg.scripts.start = 'node dist/google-search.js'; require('fs').writeFileSync('dist-package/package.json', JSON.stringify(pkg, null, 2));"
+```
+
+3. Users can then install and run the built version:
+```bash
+# Install production dependencies only
+npm install --production
+
+# Start the server
+npm start
+```
+
+This approach allows you to distribute the compiled JavaScript files without exposing the TypeScript source code. Users will still need to:
+
+1. Configure their Google API credentials as environment variables
+2. Add the server configuration to their MCP settings file
+3. Install the production dependencies
+
+Note that the package.json in the distribution will only include production dependencies and a simplified set of scripts.
+
+## License
+
+MIT

package/dist/content-fetcher.js

@@ -0,0 +1,36 @@
+import axios from 'axios';
+export class ContentFetcher {
+    constructor(port = 5001) {
+        this.baseUrl = `http://localhost:${port}`;
+    }
+    async fetchContent(url) {
+        try {
+            const response = await axios.post(`${this.baseUrl}/analyze`, { url });
+            return response.data;
+        }
+        catch (error) {
+            if (axios.isAxiosError(error)) {
+                throw new Error(`Failed to fetch content: ${error.response?.data?.error || error.message}`);
+            }
+            if (error instanceof Error) {
+                throw new Error(`Failed to fetch content: ${error.message}`);
+            }
+            throw new Error('Failed to fetch content: Unknown error');
+        }
+    }
+    async batchFetchContent(urls) {
+        try {
+            const response = await axios.post(`${this.baseUrl}/batch_analyze`, { urls });
+            return response.data;
+        }
+        catch (error) {
+            if (axios.isAxiosError(error)) {
+                throw new Error(`Failed to batch fetch content: ${error.response?.data?.error || error.message}`);
+            }
+            if (error instanceof Error) {
+                throw new Error(`Failed to batch fetch content: ${error.message}`);
+            }
+            throw new Error('Failed to batch fetch content: Unknown error');
+        }
+    }
+}