@techwavedev/agi-agent-kit 1.6.0 → 1.6.1

package/templates/skills/extended/devops/capi-mcp-gateway/SKILL.md

@@ -0,0 +1,343 @@
+---
+name: capi-mcp-gateway
+description: "Deploy, configure, and interact with CAPI Gateway's MCP (Model Context Protocol) endpoint. Turns any REST API into an MCP tool without backend code changes — LLM agents discover and invoke tools via JSON-RPC 2.0 over Streamable HTTP. Use when: (1) exposing existing REST services as MCP tools for AI agents, (2) setting up a CAPI MCP Gateway (Docker, JAR, or Helm), (3) registering services in Consul with MCP metadata, (4) connecting Claude Desktop, Cursor, or custom agents to a CAPI /mcp endpoint, (5) building Python/TypeScript agent loops that use CAPI-discovered tools, (6) debugging MCP sessions, tool routing, or JSON-RPC errors."
+---
+
+# CAPI MCP Gateway Skill
+
+Deploy and operate the [CAPI Gateway](https://github.com/surisoft-io/capi-core) MCP endpoint — a REST-to-MCP bridge that lets LLM agents discover and invoke any Consul-registered REST API as MCP tools, with zero code changes on backends.
+
+---
+
+## Quick Start
+
+### 1. Start CAPI with MCP Enabled
+
+**Docker (fastest):**
+
+```bash
+docker run -p 8380:8380 -p 8381:8381 -p 8383:8383 \
+  -v $(pwd)/config.yaml:/capi/config/config.yaml \
+  -e CAPI_CONFIG_FILE=/capi/config/config.yaml \
+  surisoft/capi-core
+```
+
+**JAR:**
+
+```bash
+CAPI_CONFIG_FILE=config.yaml java -jar capi-core.jar
+```
+
+**Helm (Kubernetes):**
+
+```bash
+helm install capi-core helm/capi-core -f my-values.yaml
+```
+
+### 2. Minimal `config.yaml` with MCP
+
+```yaml
+capi:
+  version: 1.0.0
+  instanceName: default
+  publicEndpoint: http://localhost:8380/api/
+  runningMode: full
+  adminPort: 8381
+
+  rest:
+    enabled: true
+    port: 8380
+
+  mcp:
+    enabled: true
+    port: 8383
+    sessionTtl: 1800000       # 30 min
+    toolCallTimeout: 30000    # 30 sec
+
+  consulHosts:
+    - endpoint: http://localhost:8500
+```
+
+### 3. Register a Service in Consul
+
+```json
+{
+  "ID": "order-service-1",
+  "Name": "order-service",
+  "Address": "10.0.1.50",
+  "Port": 8080,
+  "Meta": {
+    "group": "v1",
+    "scheme": "http",
+    "root-context": "/api",
+    "mcp-enabled": "true",
+    "mcp-toolPrefix": "orders",
+    "mcp-tools": "get,create,search",
+    "mcp-tools-get-description": "Get an order by ID",
+    "mcp-tools-get-inputSchema": "{\"type\":\"object\",\"properties\":{\"orderId\":{\"type\":\"string\"}},\"required\":[\"orderId\"]}",
+    "mcp-tools-create-description": "Create a new order",
+    "mcp-tools-create-inputSchema": "{\"type\":\"object\",\"properties\":{\"product\":{\"type\":\"string\"},\"quantity\":{\"type\":\"integer\"}}}",
+    "mcp-tools-search-description": "Search orders by criteria",
+    "mcp-tools-search-inputSchema": "{\"type\":\"object\",\"properties\":{\"query\":{\"type\":\"string\"}}}",
+    "mcp-category": "commerce",
+    "mcp-timeout": "10000",
+    "mcp-streaming": "search"
+  }
+}
+```
+
+Register via Consul HTTP API:
+
+```bash
+curl -X PUT http://localhost:8500/v1/agent/service/register \
+  -H "Content-Type: application/json" \
+  -d @service.json
+```
+
+### 4. Verify Tools
+
+```bash
+curl http://localhost:8381/info/mcp/tools
+```
+
+---
+
+## MCP Wire Protocol (JSON-RPC 2.0)
+
+All interactions go to a single `POST /mcp` endpoint on port `8383`.
+
+| Method | Description |
+|--------|-------------|
+| `initialize` | Create session; returns `Mcp-Session-Id` header |
+| `tools/list` | Aggregated tool catalog from all MCP-enabled services |
+| `tools/call` | Invoke a tool — CAPI routes to the correct backend |
+| `ping` | Health check |
+
+### curl Walkthrough
+
+```bash
+# 1. Initialize session
+curl -s -D- -X POST http://localhost:8383/mcp \
+  -H "Content-Type: application/json" \
+  -d '{"jsonrpc":"2.0","method":"initialize","id":1}'
+# → Mcp-Session-Id header in response
+
+# 2. List tools
+curl -s -X POST http://localhost:8383/mcp \
+  -H "Content-Type: application/json" \
+  -H "Mcp-Session-Id: <session-id>" \
+  -d '{"jsonrpc":"2.0","method":"tools/list","id":2}'
+
+# 3. Call a tool
+curl -s -X POST http://localhost:8383/mcp \
+  -H "Content-Type: application/json" \
+  -H "Mcp-Session-Id: <session-id>" \
+  -d '{"jsonrpc":"2.0","method":"tools/call","id":3,"params":{"name":"orders.get","arguments":{"orderId":"12345"}}}'
+
+# 4. Streaming tool call (SSE)
+curl -N -X POST http://localhost:8383/mcp \
+  -H "Content-Type: application/json" \
+  -H "Accept: text/event-stream" \
+  -H "Mcp-Session-Id: <session-id>" \
+  -d '{"jsonrpc":"2.0","method":"tools/call","id":4,"params":{"name":"orders.search","arguments":{"query":"pending"}}}'
+
+# 5. Health
+curl http://localhost:8383/mcp/health
+```
+
+---
+
+## Client Integration
+
+### Claude Desktop
+
+Use `mcp-remote` to bridge stdio → HTTP:
+
+```json
+{
+  "mcpServers": {
+    "capi": {
+      "command": "npx",
+      "args": ["-y", "mcp-remote", "http://localhost:8383/mcp"]
+    }
+  }
+}
+```
+
+> If OAuth2 is enabled, pass Bearer token via `mcp-remote` flags.
+
+### Cursor / IDE
+
+Same `mcp-remote` bridge applies — configure in your IDE's MCP settings.
+
+### Python Agent
+
+```python
+import requests
+
+BASE = "http://localhost:8383/mcp"
+HEADERS = {"Content-Type": "application/json"}
+
+def jsonrpc(method, params=None, req_id=1, extra_headers=None):
+    body = {"jsonrpc": "2.0", "method": method, "id": req_id}
+    if params:
+        body["params"] = params
+    h = {**HEADERS, **(extra_headers or {})}
+    return requests.post(BASE, json=body, headers=h)
+
+# Initialize
+resp = jsonrpc("initialize")
+session_id = resp.headers["Mcp-Session-Id"]
+sh = {"Mcp-Session-Id": session_id}
+
+# List tools
+tools = jsonrpc("tools/list", req_id=2, extra_headers=sh).json()["result"]["tools"]
+
+# Call tool
+result = jsonrpc("tools/call", req_id=3, extra_headers=sh,
+    params={"name": "orders.get", "arguments": {"orderId": "12345"}})
+print(result.json()["result"]["content"][0]["text"])
+```
+
+See [references/agent_loop.md](references/agent_loop.md) for the full LLM agent loop pattern.
+
+---
+
+## Consul MCP Metadata Reference
+
+| Key | Description |
+|-----|-------------|
+| `mcp-enabled` | `"true"` to expose service as MCP tools |
+| `mcp-tools` | Comma-separated tool names (e.g. `"get,create,search"`) |
+| `mcp-tools-{name}-description` | Human-readable description for LLM tool selection |
+| `mcp-tools-{name}-inputSchema` | JSON Schema defining tool input parameters |
+| `mcp-toolPrefix` | Namespace prefix (e.g. `"orders"` → `orders.get`) |
+| `mcp-streaming` | Comma-separated tool names that support SSE streaming |
+| `mcp-category` | Semantic classification; used as OPA input |
+| `mcp-timeout` | Per-service timeout in ms |
+
+> If a service exposes an OpenAPI definition and omits `inputSchema`, CAPI derives schemas from the spec.
+
+---
+
+## Admin Endpoints
+
+| Endpoint | Description |
+|----------|-------------|
+| `GET /info/mcp` | MCP status: enabled, port, tool count, sessions |
+| `GET /info/mcp/tools` | Full tool catalog as JSON |
+| `GET /info/mcp/sessions` | Active session count |
+| `GET /info/health` | Gateway health (`8381`) |
+| `GET /info/routes` | All managed REST routes |
+
+---
+
+## Session Management
+
+- Sessions created via `initialize` JSON-RPC method
+- Identified by `Mcp-Session-Id` HTTP header
+- TTL-based with sliding expiration (default 30 min)
+- Stored in Hazelcast distributed cache
+- Client can close via a `close` notification
+
+---
+
+## Error Codes
+
+| Code | Meaning |
+|------|---------|
+| `-32700` | Parse error — invalid JSON |
+| `-32600` | Invalid request — missing `jsonrpc` or `method` |
+| `-32601` | Method not found |
+| `-32602` | Invalid params — tool not found |
+| `-32603` | Internal error — backend failure |
+| `-32000` | Auth error — missing/invalid token or OPA deny |
+
+---
+
+## Security (OAuth2 + OPA)
+
+When `oauth2.enabled: true` in config, clients must include a Bearer token:
+
+```bash
+curl -X POST http://localhost:8383/mcp \
+  -H "Authorization: Bearer $TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"jsonrpc":"2.0","method":"initialize","id":1}'
+```
+
+For fine-grained control, enable OPA. MCP extends the OPA input with:
+- MCP client identity
+- Session metadata
+- Tool name and category
+
+See [references/security.md](references/security.md) for OPA Rego policy examples.
+
+---
+
+## Ports Summary
+
+| Port | Purpose | Config Key |
+|------|---------|------------|
+| 8380 | REST gateway | `capi.rest.port` |
+| 8381 | Admin API | `capi.adminPort` |
+| 8382 | WebSocket/SSE | `capi.websocket.port` |
+| 8383 | **MCP Gateway** | `capi.mcp.port` |
+| 8384 | gRPC Gateway | `capi.grpc.port` |
+
+---
+
+## Troubleshooting
+
+| Issue | Likely Cause | Fix |
+|-------|-------------|-----|
+| No tools in `tools/list` | Service missing `mcp-enabled` metadata | Add `"mcp-enabled": "true"` to Consul Meta |
+| `Tool not found` error | Wrong prefix or name | Check `mcp-toolPrefix` + `mcp-tools` values |
+| Session rejected | OAuth2 enabled but no token | Send `Authorization: Bearer <token>` header |
+| Backend timeout | Slow service | Increase `mcp-timeout` or `toolCallTimeout` |
+| Connection refused on 8383 | MCP not enabled | Set `capi.mcp.enabled: true` in config |
+
+---
+
+## Reference Files
+
+- [Agent Loop Pattern](references/agent_loop.md) — Full LLM agent loop with CAPI MCP
+- [Security & OPA](references/security.md) — Auth config, OPA Rego examples
+- [Configuration Reference](references/configuration.md) — Complete `config.yaml` reference
+
+---
+
+## AGI Framework Integration
+
+### Hybrid Memory Integration (Qdrant + BM25)
+
+Before executing complex tasks with this skill:
+```bash
+python3 execution/memory_manager.py auto --query "<task summary>"
+```
+
+**Decision Tree:**
+- **Cache hit?** Use cached response directly — no need to re-process.
+- **Memory match?** Inject `context_chunks` into your reasoning.
+- **No match?** Proceed normally, then store results:
+
+```bash
+python3 execution/memory_manager.py store \
+  --content "Description of what was decided/solved" \
+  --type decision \
+  --tags capi-mcp-gateway <relevant-tags>
+```
+
+> **Note:** Storing automatically updates both Vector (Qdrant) and Keyword (BM25) indices.
+
+### Agent Team Collaboration
+
+- **Strategy**: This skill communicates via the shared memory system.
+- **Orchestration**: Invoked by `orchestrator` via intelligent routing.
+- **Context Sharing**: Always read previous agent outputs from memory before starting.
+
+### Local LLM Support
+
+When available, use local Ollama models for embedding and lightweight inference:
+- Embeddings: `nomic-embed-text` via Qdrant memory system
+- Lightweight analysis: Local models reduce API costs for repetitive patterns

package/templates/skills/extended/devops/capi-mcp-gateway/references/agent_loop.md

@@ -0,0 +1,237 @@
+# LLM Agent Loop with CAPI MCP Gateway
+
+Pattern for building an LLM agent that discovers and invokes tools via CAPI's MCP endpoint.
+
+---
+
+## Full Python Implementation
+
+```python
+import requests
+import json
+
+BASE = "http://localhost:8383/mcp"
+HEADERS = {"Content-Type": "application/json"}
+
+class CapiMcpClient:
+    """Client for CAPI MCP Gateway (JSON-RPC 2.0 over Streamable HTTP)."""
+
+    def __init__(self, base_url=BASE, auth_token=None):
+        self.base_url = base_url
+        self.session_id = None
+        self._req_id = 0
+        self._extra = {}
+        if auth_token:
+            self._extra["Authorization"] = f"Bearer {auth_token}"
+
+    def _next_id(self):
+        self._req_id += 1
+        return self._req_id
+
+    def _call(self, method, params=None):
+        body = {"jsonrpc": "2.0", "method": method, "id": self._next_id()}
+        if params:
+            body["params"] = params
+        headers = {**HEADERS, **self._extra}
+        if self.session_id:
+            headers["Mcp-Session-Id"] = self.session_id
+        resp = requests.post(self.base_url, json=body, headers=headers)
+        resp.raise_for_status()
+        if "Mcp-Session-Id" in resp.headers:
+            self.session_id = resp.headers["Mcp-Session-Id"]
+        data = resp.json()
+        if "error" in data:
+            raise McpError(data["error"]["code"], data["error"]["message"])
+        return data["result"]
+
+    def initialize(self):
+        """Start session. Must be called first."""
+        return self._call("initialize")
+
+    def list_tools(self):
+        """Return aggregated tool catalog."""
+        return self._call("tools/list")["tools"]
+
+    def call_tool(self, name, arguments):
+        """Invoke a tool by name with arguments dict."""
+        result = self._call("tools/call", {"name": name, "arguments": arguments})
+        return result["content"][0]["text"]
+
+    def ping(self):
+        return self._call("ping")
+
+
+class McpError(Exception):
+    def __init__(self, code, message):
+        self.code = code
+        self.message = message
+        super().__init__(f"MCP Error {code}: {message}")
+```
+
+---
+
+## Agent Loop Pattern
+
+Bridge LLM function-calling with CAPI MCP tool invocation:
+
+```python
+def agent_loop(user_message, mcp_client, llm):
+    """
+    1. Discover tools from CAPI
+    2. Present as LLM function definitions
+    3. Forward tool calls back to CAPI
+    4. Feed results to LLM for final answer
+    """
+    # 1. Discover tools
+    tools = mcp_client.list_tools()
+
+    # 2. Present to LLM as function definitions
+    llm_response = llm.chat(
+        messages=[{"role": "user", "content": user_message}],
+        tools=[{
+            "type": "function",
+            "function": {
+                "name": t["name"],
+                "description": t["description"],
+                "parameters": t["inputSchema"]
+            }
+        } for t in tools]
+    )
+
+    # 3. If no tool call, return direct answer
+    if not llm_response.tool_calls:
+        return llm_response.content
+
+    # 4. Execute tool calls via CAPI
+    messages = [
+        {"role": "user", "content": user_message},
+        {"role": "assistant", "tool_calls": llm_response.tool_calls}
+    ]
+    for call in llm_response.tool_calls:
+        result = mcp_client.call_tool(call.function.name,
+                                       json.loads(call.function.arguments))
+        messages.append({
+            "role": "tool",
+            "content": result,
+            "tool_call_id": call.id
+        })
+
+    # 5. Final LLM response with tool results
+    return llm.chat(messages=messages).content
+```
+
+### Usage
+
+```python
+from openai import OpenAI
+
+llm = OpenAI()
+mcp = CapiMcpClient(auth_token="<your-token>")
+mcp.initialize()
+
+answer = agent_loop("What is the status of order 12345?", mcp, llm)
+print(answer)
+```
+
+---
+
+## Multi-Turn Agent
+
+For multi-turn conversations, cache the tool list and reuse the session:
+
+```python
+class CapiAgent:
+    def __init__(self, mcp_url, auth_token, llm):
+        self.mcp = CapiMcpClient(mcp_url, auth_token)
+        self.mcp.initialize()
+        self.tools = self.mcp.list_tools()
+        self.llm = llm
+        self.history = []
+
+    def chat(self, user_message):
+        self.history.append({"role": "user", "content": user_message})
+        response = self.llm.chat(
+            messages=self.history,
+            tools=[{
+                "type": "function",
+                "function": {
+                    "name": t["name"],
+                    "description": t["description"],
+                    "parameters": t["inputSchema"]
+                }
+            } for t in self.tools]
+        )
+
+        while response.tool_calls:
+            self.history.append({"role": "assistant", "tool_calls": response.tool_calls})
+            for call in response.tool_calls:
+                result = self.mcp.call_tool(
+                    call.function.name,
+                    json.loads(call.function.arguments)
+                )
+                self.history.append({
+                    "role": "tool",
+                    "content": result,
+                    "tool_call_id": call.id
+                })
+            response = self.llm.chat(messages=self.history, tools=[...])
+
+        self.history.append({"role": "assistant", "content": response.content})
+        return response.content
+```
+
+---
+
+## SSE Streaming
+
+For tools that declare streaming (via `mcp-streaming` metadata):
+
+```python
+import sseclient
+
+def call_streaming_tool(mcp_client, tool_name, arguments):
+    """Invoke a streaming tool and yield SSE events."""
+    body = {
+        "jsonrpc": "2.0",
+        "method": "tools/call",
+        "id": mcp_client._next_id(),
+        "params": {"name": tool_name, "arguments": arguments}
+    }
+    headers = {
+        **HEADERS,
+        "Accept": "text/event-stream",
+        "Mcp-Session-Id": mcp_client.session_id
+    }
+    resp = requests.post(mcp_client.base_url, json=body,
+                         headers=headers, stream=True)
+    client = sseclient.SSEClient(resp)
+    for event in client.events():
+        yield json.loads(event.data)
+```
+
+---
+
+## Error Handling
+
+```python
+try:
+    result = mcp.call_tool("orders.get", {"orderId": "123"})
+except McpError as e:
+    if e.code == -32602:
+        print(f"Tool not found: {e.message}")
+    elif e.code == -32000:
+        print(f"Auth error: {e.message}")
+    elif e.code == -32603:
+        print(f"Backend error: {e.message}")
+    else:
+        print(f"MCP error ({e.code}): {e.message}")
+```
+
+| Code | Meaning | Recovery |
+|------|---------|----------|
+| `-32700` | Parse error | Fix JSON body |
+| `-32600` | Invalid request | Check `jsonrpc` and `method` fields |
+| `-32601` | Method not found | Use `initialize`, `tools/list`, `tools/call`, or `ping` |
+| `-32602` | Invalid params | Verify tool name exists in `tools/list` |
+| `-32603` | Internal error | Backend timeout — retry or increase `mcp-timeout` |
+| `-32000` | Auth error | Refresh token or check OPA policy |