github-to-mcp-monorepo 1.0.0

package/mkdocs/docs/concepts/classification.md

@@ -0,0 +1,254 @@
+# Repository Classification
+
+How GitHub to MCP analyzes and categorizes repositories.
+
+## Why Classification Matters
+
+Different repository types require different extraction strategies:
+
+- **API SDKs** → Extract from OpenAPI specs
+- **MCP Servers** → Parse existing tool definitions
+- **CLI Tools** → Extract command structure
+- **Libraries** → Default to universal tools
+
+## Classification Types
+
+### mcp-server
+
+An existing MCP server implementation.
+
+**Indicators:**
+- `@modelcontextprotocol/sdk` in dependencies
+- `mcp.json` configuration file
+- `server.tool()` patterns in code
+- "MCP" in README/description
+
+**Extraction Strategy:**
+Parse existing tool definitions and re-export them.
+
+---
+
+### api-sdk
+
+A client library for REST or GraphQL APIs.
+
+**Indicators:**
+- `openapi.json`, `swagger.yaml` files
+- `schema.graphql` file
+- Package name contains `-sdk`, `-client`, `-api`
+- REST client patterns in code
+
+**Extraction Strategy:**
+Parse OpenAPI/GraphQL specs, generate typed API tools.
+
+---
+
+### cli-tool
+
+A command-line interface application.
+
+**Indicators:**
+- `bin` entry in package.json
+- `argparse`, `commander`, `yargs` dependencies
+- `click`, `typer` for Python
+- CLI documentation in README
+
+**Extraction Strategy:**
+Parse CLI help output or code to extract commands.
+
+---
+
+### library
+
+A general-purpose programming library.
+
+**Indicators:**
+- Published to npm/PyPI
+- Exports functions/classes
+- No specific API or CLI patterns
+- Generic package name
+
+**Extraction Strategy:**
+Universal tools only, plus any detected patterns.
+
+---
+
+### documentation
+
+A documentation-only repository.
+
+**Indicators:**
+- Primarily markdown/MDX files
+- MkDocs, Docusaurus, Nextra config
+- No source code directories
+- "docs" in repo name
+
+**Extraction Strategy:**
+Universal tools optimized for documentation browsing.
+
+---
+
+### data
+
+A data or configuration repository.
+
+**Indicators:**
+- JSON, YAML, CSV files
+- No source code
+- "data", "config", "assets" in name
+- Large binary files
+
+**Extraction Strategy:**
+Universal tools for data access.
+
+---
+
+### unknown
+
+Repository type couldn't be determined.
+
+**Extraction Strategy:**
+Universal tools only.
+
+---
+
+## Classification Logic
+
+```typescript
+function classifyRepository(repo: RepositoryInfo): RepositoryType {
+  const { readme, files, packageJson } = repo;
+  
+  // Priority 1: Check for MCP server
+  if (isMcpServer(packageJson, files)) {
+    return 'mcp-server';
+  }
+  
+  // Priority 2: Check for API patterns
+  if (hasApiSpecs(files)) {
+    return 'api-sdk';
+  }
+  
+  // Priority 3: Check for CLI
+  if (isCliTool(packageJson, files)) {
+    return 'cli-tool';
+  }
+  
+  // Priority 4: Check for documentation
+  if (isDocumentation(files, readme)) {
+    return 'documentation';
+  }
+  
+  // Priority 5: Check for data repo
+  if (isDataRepo(files)) {
+    return 'data';
+  }
+  
+  // Priority 6: Check for library
+  if (isLibrary(packageJson)) {
+    return 'library';
+  }
+  
+  return 'unknown';
+}
+```
+
+## Detection Functions
+
+### isMcpServer
+
+```typescript
+function isMcpServer(pkg: PackageJson, files: FileTree): boolean {
+  // Check dependencies
+  const deps = { ...pkg.dependencies, ...pkg.devDependencies };
+  if (deps['@modelcontextprotocol/sdk']) return true;
+  
+  // Check for MCP config
+  if (files.includes('mcp.json')) return true;
+  
+  // Check for MCP patterns in source
+  const sourceFiles = files.filter(f => f.endsWith('.ts') || f.endsWith('.js'));
+  for (const file of sourceFiles) {
+    const content = await readFile(file);
+    if (content.includes('server.tool(') || 
+        content.includes('McpServer')) {
+      return true;
+    }
+  }
+  
+  return false;
+}
+```
+
+### hasApiSpecs
+
+```typescript
+function hasApiSpecs(files: FileTree): boolean {
+  const specPatterns = [
+    'openapi.json', 'openapi.yaml', 'openapi.yml',
+    'swagger.json', 'swagger.yaml', 'swagger.yml',
+    'api-spec.json', 'api-spec.yaml',
+    'schema.graphql', 'schema.gql',
+  ];
+  
+  return specPatterns.some(pattern => 
+    files.some(f => f.endsWith(pattern))
+  );
+}
+```
+
+### isCliTool
+
+```typescript
+function isCliTool(pkg: PackageJson, files: FileTree): boolean {
+  // Check for bin entry
+  if (pkg.bin) return true;
+  
+  // Check for CLI dependencies
+  const cliDeps = ['commander', 'yargs', 'meow', 'cac', 'argparse'];
+  const deps = Object.keys({ ...pkg.dependencies, ...pkg.devDependencies });
+  if (cliDeps.some(d => deps.includes(d))) return true;
+  
+  // Check for cli directory
+  if (files.some(f => f.startsWith('cli/') || f.startsWith('bin/'))) {
+    return true;
+  }
+  
+  return false;
+}
+```
+
+## Manual Override
+
+Classification can be overridden via options:
+
+```typescript
+const result = await generateFromGithub(url, {
+  forceType: 'api-sdk'  // Override automatic classification
+});
+```
+
+Or via CLI:
+
+```bash
+github-to-mcp <url> --type api-sdk
+```
+
+## Classification Impact
+
+| Type | Universal Tools | OpenAPI | GraphQL | Code Analysis |
+|------|-----------------|---------|---------|---------------|
+| mcp-server | ✅ | ❌ | ❌ | ✅ (MCP patterns) |
+| api-sdk | ✅ | ✅ | ✅ | ❌ |
+| cli-tool | ✅ | ❌ | ❌ | ✅ (CLI patterns) |
+| library | ✅ | ❌ | ❌ | Limited |
+| documentation | ✅ | ❌ | ❌ | ❌ |
+| data | ✅ | ❌ | ❌ | ❌ |
+| unknown | ✅ | ❌ | ❌ | ❌ |
+
+---
+
+## See Also
+
+- [How It Works](how-it-works.md) - Full extraction flow
+- [Tool Types](tool-types.md) - Understanding tools
+- [Configuration](../getting-started/configuration.md) - Override options

package/mkdocs/docs/concepts/how-it-works.md

@@ -0,0 +1,299 @@
+# How It Works
+
+A deep dive into the GitHub to MCP conversion process.
+
+## High-Level Flow
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                        User Input (GitHub URL)                   │
+└─────────────────────────────────────────────────────────────────┘
+                                ↓
+┌─────────────────────────────────────────────────────────────────┐
+│  1. FETCH       │  Download repo metadata, README, file tree    │
+└─────────────────────────────────────────────────────────────────┘
+                                ↓
+┌─────────────────────────────────────────────────────────────────┐
+│  2. CLASSIFY    │  Determine repo type (API, CLI, Library...)   │
+└─────────────────────────────────────────────────────────────────┘
+                                ↓
+┌─────────────────────────────────────────────────────────────────┐
+│  3. EXTRACT     │  Find tools from OpenAPI, GraphQL, code, etc. │
+└─────────────────────────────────────────────────────────────────┘
+                                ↓
+┌─────────────────────────────────────────────────────────────────┐
+│  4. GENERATE    │  Create MCP server code and tool impls        │
+└─────────────────────────────────────────────────────────────────┘
+                                ↓
+┌─────────────────────────────────────────────────────────────────┐
+│  5. PACKAGE     │  Bundle with dependencies and config          │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+## Step 1: Fetch Repository
+
+The process starts by fetching information from GitHub:
+
+```typescript
+// Pseudocode
+const repoInfo = await githubClient.getRepository(url);
+const readme = await githubClient.getReadme(url);
+const fileTree = await githubClient.getFileTree(url, { depth: 3 });
+```
+
+### What's Fetched
+
+| Data | Purpose |
+|------|---------|
+| Repository metadata | Name, description, stars, language |
+| README content | Context for classification |
+| File tree | Find specs, source files |
+| Package files | package.json, pyproject.toml |
+| Spec files | openapi.json, schema.graphql |
+
+### Rate Limiting
+
+GitHub API limits are handled automatically:
+
+- **Anonymous**: 60 requests/hour (very limited)
+- **Authenticated**: 5,000 requests/hour (recommended)
+
+## Step 2: Classify Repository
+
+The repo is analyzed to determine its type:
+
+```typescript
+function classifyRepository(repoInfo): RepositoryType {
+  // Check for MCP server indicators
+  if (hasMcpDependency(repoInfo) || hasMcpServerFile(repoInfo)) {
+    return 'mcp-server';
+  }
+  
+  // Check for API patterns
+  if (hasOpenApiSpec(repoInfo) || hasGraphqlSchema(repoInfo)) {
+    return 'api-sdk';
+  }
+  
+  // Check for CLI patterns
+  if (hasCliPackage(repoInfo) || hasBinEntry(repoInfo)) {
+    return 'cli-tool';
+  }
+  
+  // ... more checks
+}
+```
+
+### Classification Types
+
+| Type | Indicators | Tool Extraction |
+|------|------------|-----------------|
+| `mcp-server` | `@modelcontextprotocol/sdk` dependency | Parse existing tools |
+| `api-sdk` | OpenAPI spec, REST client patterns | Extract API endpoints |
+| `cli-tool` | `bin` in package.json, argparse | Extract CLI commands |
+| `library` | Generic npm/pip package | Universal tools only |
+| `documentation` | Mostly markdown files | Universal tools only |
+
+## Step 3: Extract Tools
+
+Tools are extracted from multiple sources:
+
+### Universal Tools
+
+Always included (unless disabled):
+
+```typescript
+const universalTools = [
+  { name: 'get_readme', source: 'universal' },
+  { name: 'list_files', source: 'universal' },
+  { name: 'read_file', source: 'universal' },
+  { name: 'search_code', source: 'universal' },
+];
+```
+
+### OpenAPI Extraction
+
+```typescript
+// Find and parse OpenAPI specs
+const specs = findOpenApiSpecs(fileTree);
+
+for (const spec of specs) {
+  const parsed = await parseOpenApi(spec.content);
+  
+  for (const path of Object.keys(parsed.paths)) {
+    for (const method of ['get', 'post', 'put', 'delete']) {
+      const operation = parsed.paths[path][method];
+      if (operation) {
+        tools.push(createToolFromOperation(operation, path, method));
+      }
+    }
+  }
+}
+```
+
+### GraphQL Extraction
+
+```typescript
+// Parse GraphQL schema
+const schema = parseGraphqlSchema(schemaContent);
+
+// Extract queries as tools
+for (const query of schema.queries) {
+  tools.push({
+    name: query.name,
+    description: query.description,
+    inputSchema: convertGraphqlArgsToJsonSchema(query.args),
+    source: 'graphql',
+  });
+}
+
+// Extract mutations as tools
+for (const mutation of schema.mutations) {
+  tools.push({
+    name: mutation.name,
+    // ...
+  });
+}
+```
+
+### Code Analysis
+
+For MCP servers and Python packages:
+
+```typescript
+// Find @mcp.tool decorators
+const mcpTools = findMcpDecorators(sourceFiles);
+
+// Find FastAPI/Flask routes
+const apiRoutes = findApiRoutes(sourceFiles);
+
+// Find CLI commands
+const cliCommands = findCliCommands(sourceFiles);
+```
+
+## Step 4: Generate Code
+
+The extracted tools are converted to working MCP server code:
+
+### Server Template
+
+```typescript
+const serverCode = `
+import { Server } from '@modelcontextprotocol/sdk/server/index.js';
+import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
+
+const server = new Server({
+  name: '${repoName}-mcp',
+  version: '1.0.0',
+}, {
+  capabilities: {
+    tools: {},
+  },
+});
+
+// Register tools
+${tools.map(tool => generateToolRegistration(tool)).join('\n')}
+
+// Start server
+const transport = new StdioServerTransport();
+await server.connect(transport);
+`;
+```
+
+### Tool Implementation
+
+Each tool gets an implementation:
+
+```typescript
+function generateToolImplementation(tool: Tool): string {
+  if (tool.source === 'universal') {
+    return universalImplementations[tool.name];
+  }
+  
+  if (tool.source === 'openapi') {
+    return `
+async function ${tool.name}(params) {
+  const response = await fetch('${tool.endpoint}', {
+    method: '${tool.method}',
+    headers: { 'Content-Type': 'application/json' },
+    body: JSON.stringify(params),
+  });
+  return response.json();
+}`;
+  }
+  
+  // ... other sources
+}
+```
+
+## Step 5: Package Output
+
+Finally, everything is bundled:
+
+### Output Structure
+
+```
+output/
+├── package.json          # Dependencies
+├── tsconfig.json         # TypeScript config
+├── server.ts             # Main server (source)
+├── server.mjs            # Compiled server
+├── tools/
+│   ├── index.ts
+│   ├── universal/
+│   │   ├── read_file.ts
+│   │   └── ...
+│   └── extracted/
+│       ├── create_customer.ts
+│       └── ...
+└── README.md             # Usage instructions
+```
+
+### Generated package.json
+
+```json
+{
+  "name": "repo-name-mcp",
+  "version": "1.0.0",
+  "type": "module",
+  "main": "server.mjs",
+  "scripts": {
+    "start": "node server.mjs",
+    "build": "tsc"
+  },
+  "dependencies": {
+    "@modelcontextprotocol/sdk": "^1.0.0"
+  }
+}
+```
+
+---
+
+## Performance Considerations
+
+### Caching
+
+- Repository metadata is cached for 5 minutes
+- File contents are cached per session
+- OpenAPI parsing results are memoized
+
+### Parallel Processing
+
+- Multiple files are fetched in parallel
+- Tool extraction runs concurrently
+- Limited by GitHub API rate limits
+
+### Size Limits
+
+| Limit | Default | Purpose |
+|-------|---------|---------|
+| Max depth | 3 | Prevent scanning huge repos |
+| Max file size | 100KB | Skip large binary files |
+| Max files | 1000 | Reasonable processing time |
+
+---
+
+## See Also
+
+- [Tool Types](tool-types.md) - Understanding extracted tools
+- [Classification](classification.md) - Repository classification
+- [Architecture](../contributing/architecture.md) - Code architecture

package/mkdocs/docs/concepts/index.md

@@ -0,0 +1,77 @@
+# Concepts
+
+Understand how GitHub to MCP works under the hood.
+
+## Core Concepts
+
+<div class="feature-grid" markdown>
+
+<div class="feature-card" markdown>
+
+### [How It Works](how-it-works.md)
+
+The end-to-end flow from GitHub URL to MCP server.
+
+</div>
+
+<div class="feature-card" markdown>
+
+### [Tool Types](tool-types.md)
+
+Understanding universal and extracted tools.
+
+</div>
+
+<div class="feature-card" markdown>
+
+### [Repository Classification](classification.md)
+
+How repos are analyzed and categorized.
+
+</div>
+
+<div class="feature-card" markdown>
+
+### [MCP Protocol](mcp-protocol.md)
+
+Understanding the Model Context Protocol.
+
+</div>
+
+</div>
+
+## Quick Overview
+
+### What is MCP?
+
+The **Model Context Protocol (MCP)** is an open standard that enables AI assistants to interact with external tools and data sources. It provides a standardized way for AI models to:
+
+- Call external functions (tools)
+- Access external data (resources)
+- Use predefined conversation templates (prompts)
+
+### What GitHub to MCP Does
+
+1. **Fetches** a GitHub repository
+2. **Analyzes** its structure and contents
+3. **Extracts** tool definitions from various sources
+4. **Generates** a complete MCP server
+5. **Packages** everything for easy deployment
+
+### Why This Matters
+
+Without MCP, AI assistants are limited to their training data. With MCP:
+
+- **Claude** can read your actual codebase
+- **Cursor** can call real API endpoints
+- **ChatGPT** can search your documentation
+
+## Key Terms
+
+| Term | Definition |
+|------|------------|
+| **MCP Server** | A program that exposes tools via the MCP protocol |
+| **Tool** | A function that an AI can call with parameters |
+| **Resource** | Static data that an AI can read |
+| **Transport** | How the AI communicates with the server (stdio, HTTP) |
+| **Schema** | JSON Schema describing a tool's input/output |