@imenam/mcp-http-gateway 2.0.0

package/README.md

@@ -0,0 +1,198 @@
+# MCP HTTP Gateway
+
+Serveur HTTP qui expose n'importe quel serveur MCP (Model Context Protocol) en tant qu'API REST, sans modifier les MCPs eux-mêmes. Conçu pour permettre à des agents ou applications ne supportant pas nativement le protocole MCP de communiquer avec l'écosystème MCP via de simples requêtes HTTP.
+
+## Fonctionnement
+
+Chaque MCP tourne en tant que processus enfant (transport stdio). Le gateway s'y connecte via le SDK MCP officiel et expose ses outils sur des routes HTTP configurables.
+
+```
+Agent / App externe
+       │
+       │  HTTP
+       ▼
+MCP HTTP Gateway (port 3500)
+       │
+       │  stdio (SDK MCP)
+       ├──▶ mcp-documentor  (/documentor)
+       ├──▶ mcp-raindrop     (/raindrop)
+       └──▶ ...
+```
+
+## Installation
+
+**En tant que CLI globale :**
+
+```bash
+npm install -g @imenam/mcp-http-gateway
+```
+
+**Via npx (sans installation) :**
+
+```bash
+npx @imenam/mcp-http-gateway
+```
+
+**En dépendance locale d'un projet :**
+
+```bash
+npm install @imenam/mcp-http-gateway
+```
+
+## Configuration
+
+### `.env`
+
+```env
+GATEWAY_PORT=3500
+```
+
+### `config/mcp-servers.json`
+
+À créer dans le **répertoire courant** depuis lequel tu lances le gateway (pas dans le package lui-même). Un fichier `config/mcp-servers.example.json` est disponible comme point de départ.
+
+Même format que la configuration `mcpServers` de Cursor. Ajouter `MCP_SERVER_ROUTE` dans `env` pour définir la route HTTP du MCP, et `description` pour la documentation.
+
+```json
+{
+  "mcpServers": {
+    "mon-mcp": {
+      "command": "node",
+      "args": ["/chemin/vers/dist/index.js"],
+      "description": "Description courte du MCP",
+      "env": {
+        "MA_VARIABLE": "valeur",
+        "MCP_SERVER_ROUTE": "/ma-route"
+      }
+    }
+  }
+}
+```
+
+> Un MCP sans `MCP_SERVER_ROUTE` est ignoré au démarrage.
+
+## Démarrage
+
+```bash
+npm start
+```
+
+Le gateway connecte chaque MCP configuré et démarre le serveur HTTP. Les MCPs en échec au démarrage sont ignorés sans bloquer les autres.
+
+## Routes
+
+### `GET /mcps`
+
+Retourne la liste des MCPs connectés avec leur route, description et instructions d'utilisation (format Markdown).
+
+```bash
+curl http://localhost:3500/mcps
+```
+
+---
+
+### `GET /{route}/tools`
+
+Liste les outils disponibles d'un MCP avec leur nom, description et schéma d'arguments.
+
+```bash
+curl http://localhost:3500/documentor/tools
+```
+
+---
+
+### `POST /{route}/tools/:toolName`
+
+Appelle un outil du MCP. Les arguments sont passés dans le body JSON.
+
+```bash
+curl -X POST http://localhost:3500/documentor/tools/list_projects -H "Content-Type: application/json" -d '{}'
+```
+
+```bash
+curl -X POST http://localhost:3500/documentor/tools/get_doc -H "Content-Type: application/json" -d '{"id":"mon-doc-id"}'
+```
+
+#### Réponse
+
+La réponse est le format brut du protocole MCP — un tableau `content` de blocs typés :
+
+```json
+{
+  "content": [
+    { "type": "text", "text": "Résultat de l'outil..." }
+  ]
+}
+```
+
+Pour extraire le texte directement avec `jq` :
+
+```bash
+curl -s -X POST http://localhost:3500/documentor/tools/list_projects -H "Content-Type: application/json" -d '{}' | jq -r '.content[0].text'
+```
+
+## Mode MCP
+
+Le gateway peut également démarrer en tant que **serveur MCP** (transport stdio). Dans ce mode, il se connecte à une instance du gateway HTTP et expose ses outils comme des outils MCP natifs.
+
+### Cas d'usage : agent dans un conteneur Docker
+
+Le gateway HTTP tourne **hors du conteneur**, avec accès aux MCPs et leurs clés sensibles. L'agent dans le conteneur reçoit uniquement le gateway en mode MCP — aucune clé ne lui est injectée.
+
+```
+[Agent dans conteneur]
+        │  stdio (MCP)
+        ▼
+[mcp-http-gateway --mcp]  ←── GATEWAY_URL
+        │  HTTP
+        ▼
+[mcp-http-gateway hors conteneur]
+        │  stdio
+        ▼
+[MCPs réels (clés sensibles)]
+```
+
+### Démarrage
+
+```bash
+GATEWAY_URL=http://host.docker.internal:3500 npx @imenam/mcp-http-gateway --mcp
+```
+
+### Configuration dans le conteneur
+
+```json
+{
+  "mcpServers": {
+    "gateway": {
+      "command": "npx",
+      "args": ["@imenam/mcp-http-gateway", "--mcp"],
+      "env": {
+        "GATEWAY_URL": "http://host.docker.internal:3500"
+      }
+    }
+  }
+}
+```
+
+> `host.docker.internal` est l'adresse de l'hôte depuis un conteneur Docker.
+
+### Outils exposés
+
+Le mode MCP expose 3 outils qui reflètent directement les routes du gateway :
+
+| Outil | Description |
+|---|---|
+| `list_mcps` | Liste les MCPs disponibles avec leur route et description |
+| `list_tools` | Liste les outils d'un MCP donné (`route` requis) |
+| `call_tool` | Appelle un outil (`route`, `tool_name`, `arguments`) |
+
+L'agent découvre d'abord les MCPs disponibles, explore les outils d'un MCP si besoin, puis appelle l'outil voulu.
+
+---
+
+## Ajouter un MCP
+
+1. Ajouter l'entrée dans `config/mcp-servers.json` avec `MCP_SERVER_ROUTE` dans `env`
+2. Redémarrer le gateway
+
+Aucune modification du MCP n'est nécessaire.

package/embeddings/db.js

@@ -0,0 +1,58 @@
+import crypto from 'crypto';
+import fs from 'fs';
+import path from 'path';
+import Database from 'better-sqlite3';
+
+let database;
+
+export function initDb(dbPath) {
+  fs.mkdirSync(path.dirname(dbPath), { recursive: true });
+  database = new Database(dbPath);
+  database.pragma('journal_mode = WAL');
+  database.exec(`
+    CREATE TABLE IF NOT EXISTS tool_embeddings (
+      cache_key   TEXT PRIMARY KEY,
+      desc_hash   TEXT NOT NULL,
+      mcp_name    TEXT NOT NULL,
+      tool_name   TEXT NOT NULL,
+      description TEXT NOT NULL,
+      embedding   BLOB NOT NULL
+    );
+  `);
+}
+
+export async function getOrComputeToolEmbedding(mcpName, tool, getEmbeddingFn) {
+  const cacheKey = `${mcpName}::${tool.name}`;
+  const descHash = crypto.createHash('md5').update(tool.description).digest('hex');
+
+  const existing = database
+    .prepare('SELECT desc_hash FROM tool_embeddings WHERE cache_key = ?')
+    .get(cacheKey);
+
+  if (existing?.desc_hash === descHash) return;
+
+  const embedding = await getEmbeddingFn(tool.description);
+  database.prepare(`
+    INSERT INTO tool_embeddings (cache_key, desc_hash, mcp_name, tool_name, description, embedding)
+    VALUES (?, ?, ?, ?, ?, ?)
+    ON CONFLICT(cache_key) DO UPDATE SET
+      desc_hash = excluded.desc_hash,
+      description = excluded.description,
+      embedding = excluded.embedding
+  `).run(
+    cacheKey, descHash, mcpName, tool.name, tool.description,
+    Buffer.from(new Float32Array(embedding).buffer),
+  );
+}
+
+export function getAllToolEmbeddings() {
+  return database
+    .prepare('SELECT mcp_name, tool_name, description, embedding FROM tool_embeddings')
+    .all()
+    .map(row => ({
+      mcp_name: row.mcp_name,
+      tool_name: row.tool_name,
+      description: row.description,
+      embedding: Array.from(new Float32Array(row.embedding.buffer, row.embedding.byteOffset, row.embedding.byteLength / 4)),
+    }));
+}

package/embeddings/vectorizer.js

@@ -0,0 +1,25 @@
+import OpenAI from 'openai';
+
+let openaiClient;
+
+function getClient() {
+  if (!openaiClient) openaiClient = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
+  return openaiClient;
+}
+
+export async function getEmbedding(text) {
+  const model = process.env.OPENAI_EMBEDDING_MODEL ?? 'text-embedding-3-small';
+  const response = await getClient().embeddings.create({ model, input: text });
+  return response.data[0].embedding;
+}
+
+export function cosineSimilarity(vecA, vecB) {
+  let dot = 0, normA = 0, normB = 0;
+  for (let i = 0; i < vecA.length; i++) {
+    dot += vecA[i] * vecB[i];
+    normA += vecA[i] ** 2;
+    normB += vecB[i] ** 2;
+  }
+  const mag = Math.sqrt(normA) * Math.sqrt(normB);
+  return mag === 0 ? 0 : dot / mag;
+}

package/mcp-mode.js

@@ -0,0 +1,99 @@
+#!/usr/bin/env node
+import 'dotenv/config';
+import { Server } from '@modelcontextprotocol/sdk/server/index.js';
+import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
+import {
+  ListToolsRequestSchema,
+  CallToolRequestSchema,
+} from '@modelcontextprotocol/sdk/types.js';
+
+const GATEWAY_URL = process.env.GATEWAY_URL?.replace(/\/$/, '');
+if (!GATEWAY_URL) {
+  console.error('[mcp-mode] GATEWAY_URL environment variable is required');
+  process.exit(1);
+}
+
+// --- MCP Server ---
+
+const server = new Server(
+  { name: 'mcp-http-gateway-mcp', version: '1.0.0' },
+  { capabilities: { tools: {} } }
+);
+
+const tools = [
+  {
+    name: 'list_mcps',
+    description: 'List all MCPs available on the gateway with their route and description.',
+    inputSchema: { type: 'object', properties: {} },
+  },
+  {
+    name: 'list_tools',
+    description: 'List all tools available for a given MCP route.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        route: { type: 'string', description: 'MCP route as returned by list_mcps (e.g. "/documentor")' },
+      },
+      required: ['route'],
+    },
+  },
+  {
+    name: 'call_tool',
+    description: 'Call a tool on a given MCP route.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        route:     { type: 'string', description: 'MCP route (e.g. "/documentor")' },
+        tool_name: { type: 'string', description: 'Tool name to call' },
+        arguments: { type: 'object', description: 'Arguments to pass to the tool' },
+      },
+      required: ['route', 'tool_name'],
+    },
+  },
+];
+
+server.setRequestHandler(ListToolsRequestSchema, async () => ({ tools }));
+
+server.setRequestHandler(CallToolRequestSchema, async (request) => {
+  const { name, arguments: args = {} } = request.params;
+
+  try {
+    if (name === 'list_mcps') {
+      const res = await fetch(`${GATEWAY_URL}/mcp-definitions`);
+      if (!res.ok) throw new Error(`HTTP ${res.status}`);
+      const definitions = await res.json();
+      const text = definitions
+        .map(m => `- ${m.id} (route: ${m.route})${m.description ? ` — ${m.description}` : ''}`)
+        .join('\n');
+      return { content: [{ type: 'text', text }] };
+    }
+
+    if (name === 'list_tools') {
+      const res = await fetch(`${GATEWAY_URL}${args.route}/tools`);
+      if (!res.ok) throw new Error(`HTTP ${res.status}`);
+      const toolsList = await res.json();
+      const text = JSON.stringify(toolsList, null, 2);
+      return { content: [{ type: 'text', text }] };
+    }
+
+    if (name === 'call_tool') {
+      const res = await fetch(`${GATEWAY_URL}${args.route}/tools/${args.tool_name}`, {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify(args.arguments ?? {}),
+      });
+      if (!res.ok) {
+        const text = await res.text();
+        return { content: [{ type: 'text', text: `Gateway error ${res.status}: ${text}` }], isError: true };
+      }
+      return await res.json();
+    }
+
+    return { content: [{ type: 'text', text: `Unknown tool: ${name}` }], isError: true };
+  } catch (err) {
+    return { content: [{ type: 'text', text: `Request failed: ${err.message}` }], isError: true };
+  }
+});
+
+const transport = new StdioServerTransport();
+await server.connect(transport);

package/package.json

@@ -0,0 +1,50 @@
+{
+  "name": "@imenam/mcp-http-gateway",
+  "version": "2.0.0",
+  "type": "module",
+  "description": "HTTP gateway that exposes MCP servers as REST routes — bridge between MCP and any HTTP-capable agent",
+  "main": "server.js",
+  "bin": {
+    "mcp-http-gateway": "server.js"
+  },
+  "files": [
+    "server.js",
+    "mcp-mode.js",
+    "README.md",
+    "embeddings/"
+  ],
+  "scripts": {
+    "start": "node server.js",
+    "release": "npm version patch && npm publish --access public && git push origin master --follow-tags"
+  },
+  "keywords": [
+    "mcp",
+    "model-context-protocol",
+    "gateway",
+    "http",
+    "ai",
+    "agent",
+    "llm",
+    "bridge"
+  ],
+  "author": "Matthieu Pesnot-Pin <matthieupin@gmail.com>",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/Matthieu-Pesnot-Pin/mcp-http-gateway.git"
+  },
+  "bugs": {
+    "url": "https://github.com/Matthieu-Pesnot-Pin/mcp-http-gateway/issues"
+  },
+  "homepage": "https://github.com/Matthieu-Pesnot-Pin/mcp-http-gateway#readme",
+  "license": "ISC",
+  "engines": {
+    "node": ">=18"
+  },
+  "dependencies": {
+    "@modelcontextprotocol/sdk": "^1.27.1",
+    "better-sqlite3": "^12.8.0",
+    "dotenv": "^17.3.1",
+    "express": "^5.2.1",
+    "openai": "^6.31.0"
+  }
+}

package/server.js

@@ -0,0 +1,185 @@
+#!/usr/bin/env node
+import 'dotenv/config';
+
+if (process.argv.includes('--mcp')) {
+  await import('./mcp-mode.js');
+  // Process stays alive: the MCP transport keeps stdin open until the client disconnects
+}
+
+import { readFileSync } from 'fs';
+import path from 'path';
+import express from 'express';
+import { Client } from '@modelcontextprotocol/sdk/client/index.js';
+import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';
+import { initDb, getOrComputeToolEmbedding, getAllToolEmbeddings } from './embeddings/db.js';
+import { getEmbedding, cosineSimilarity } from './embeddings/vectorizer.js';
+
+// Normalise les différents formats de config rencontrés dans la nature :
+// - `command` string + `args` array  (format Cursor)
+// - `command` array                  (format OpenCode/autres)
+// - `env` ou `environment`           (selon l'outil)
+function normalizeCfg(cfg) {
+  const env = cfg.env ?? cfg.environment ?? {};
+  const command = Array.isArray(cfg.command) ? cfg.command[0] : cfg.command;
+  const args = Array.isArray(cfg.command) ? cfg.command.slice(1) : (cfg.args ?? []);
+  return { ...cfg, command, args, env };
+}
+
+// --- MCP client ---
+
+async function connectMcp(name, cfg) {
+  const transport = new StdioClientTransport({
+    command: cfg.command,
+    args: cfg.args ?? [],
+    env: { ...process.env, ...(cfg.env ?? {}) },
+  });
+  const client = new Client({ name: 'mcp-gateway', version: '1.0.0' }, { capabilities: {} });
+  await client.connect(transport);
+  console.log(`[gateway] Connected: ${name} → ${cfg.env?.MCP_SERVER_ROUTE}`);
+  return client;
+}
+
+// --- Routes ---
+
+function registerMcpRoutes(app, route, client, name) {
+  app.get(`${route}/tools`, async (req, res) => {
+    try {
+      const { tools } = await client.listTools();
+      res.json(tools);
+    } catch (err) {
+      console.error(`[${name}] listTools error:`, err.message);
+      res.status(500).json({ error: err.message });
+    }
+  });
+
+  app.post(`${route}/tools/:toolName`, async (req, res) => {
+    try {
+      const result = await client.callTool({
+        name: req.params.toolName,
+        arguments: req.body ?? {},
+      });
+      res.json(result);
+    } catch (err) {
+      console.error(`[${name}] callTool(${req.params.toolName}) error:`, err.message);
+      res.status(500).json({ error: err.message });
+    }
+  });
+}
+
+// --- Bootstrap ---
+
+async function main() {
+  const GATEWAY_PORT = parseInt(process.env.GATEWAY_PORT ?? '3500', 10);
+
+  // GATEWAY_CONFIG overrides the default path (config/mcp-servers.json in cwd).
+  const configPath = process.env.GATEWAY_CONFIG
+    ? path.resolve(process.env.GATEWAY_CONFIG)
+    : path.join(process.cwd(), 'config', 'mcp-servers.json');
+  const config = JSON.parse(readFileSync(configPath, 'utf-8'));
+  const servers = config.mcpServers ?? {};
+
+  const app = express();
+  app.use(express.json({ type: '*/*' }));
+
+  const dataDir = process.env.GATEWAY_DATA_DIR ?? path.join(process.cwd(), 'data');
+  const dbPath = path.join(dataDir, 'embeddings', 'embeddings.db');
+  initDb(dbPath);
+
+  const clients = [];
+  const registry = []; // { name, route, description, tools }
+
+  for (const [name, cfg] of Object.entries(servers)) {
+    const route = cfg.env?.MCP_SERVER_ROUTE;
+    if (!route) {
+      console.warn(`[gateway] Skipping "${name}": no MCP_SERVER_ROUTE defined`);
+      continue;
+    }
+
+    try {
+      const client = await connectMcp(name, cfg);
+      const { tools } = await client.listTools();
+      clients.push(client);
+      registry.push({ name, route, description: cfg.description ?? null, tools });
+      registerMcpRoutes(app, route, client, name);
+      for (const tool of tools) {
+        if (!tool.description) continue;
+        getOrComputeToolEmbedding(name, tool, getEmbedding).catch(err =>
+          console.error(`[gateway] Embedding error for ${name}::${tool.name}:`, err.message)
+        );
+      }
+    } catch (err) {
+      console.error(`[gateway] Failed to connect "${name}":`, err.message);
+    }
+  }
+
+  app.get('/tools/search', async (req, res) => {
+    const q = req.query.q?.trim();
+    const limit = Math.min(parseInt(req.query.limit ?? '5', 10), 20);
+    if (!q) return res.status(400).json({ error: 'Missing query parameter: q' });
+    try {
+      const queryEmbedding = await getEmbedding(q);
+      const results = getAllToolEmbeddings()
+        .map(row => ({ ...row, score: cosineSimilarity(queryEmbedding, row.embedding) }))
+        .sort((a, b) => b.score - a.score)
+        .slice(0, limit)
+        .map(({ mcp_name, tool_name, description, score }) => ({
+          mcp_name,
+          mcp_description: registry.find(r => r.name === mcp_name)?.description ?? null,
+          tool_name,
+          description,
+          score,
+        }));
+      res.json(results);
+    } catch (err) {
+      console.error('[gateway] /tools/search error:', err.message);
+      res.status(500).json({ error: err.message });
+    }
+  });
+
+  app.get('/mcp-definitions', (req, res) => {
+    res.json(
+      registry.map(({ name, route, description, tools }) => ({
+        id: name,
+        route,
+        description,
+        tools,
+      }))
+    );
+  });
+
+  app.get('/mcps', (req, res) => {
+    const lines = [
+      '# MCP Gateway — Serveurs disponibles',
+      '',
+      'Les outils de chaque serveur sont accessibles via `POST /{route}/tools/{toolName}` avec les arguments dans le body JSON.',
+      'Pour lister les outils disponibles d\'un serveur, appelez `GET /{route}/tools`.',
+      '',
+      '## Serveurs',
+      '',
+      ...registry.map(({ name, route, description }) =>
+        `### ${name}\n- **Route** : \`${route}\`\n- **Description** : ${description ?? 'Non renseignée'}`
+      ),
+    ];
+    res.type('text/plain').send(lines.join('\n'));
+  });
+
+  app.listen(GATEWAY_PORT, () => {
+    console.log(`[gateway] Listening on http://localhost:${GATEWAY_PORT}`);
+  });
+
+  const shutdown = async () => {
+    console.log('[gateway] Shutting down...');
+    await Promise.allSettled(clients.map(c => c.close()));
+    process.exit(0);
+  };
+
+  process.on('SIGTERM', shutdown);
+  process.on('SIGINT', shutdown);
+}
+
+if (!process.argv.includes('--mcp')) {
+  main().catch(err => {
+    console.error('[gateway] Fatal error:', err.message);
+    process.exit(1);
+  });
+}