@biggora/claude-plugins 1.1.0 → 1.2.0

package/gemini-cli-workspace/iteration-1/eval-2-mcp-server-config/with_skill/outputs/response.md

@@ -0,0 +1,212 @@
+# Connecting a Custom Python MCP Server to Gemini CLI
+
+You need to configure your server in `settings.json`, pass the API key via the `env` field, and use `includeTools` to restrict which tools are exposed. Here is the full walkthrough.
+
+---
+
+## 1. Configure the MCP Server in settings.json
+
+Edit (or create) `~/.gemini/settings.json` for a global configuration, or `<your-project>/.gemini/settings.json` for project-scoped configuration. Add your server under the `mcpServers` key:
+
+```json
+{
+  "mcpServers": {
+    "myPythonServer": {
+      "command": "python",
+      "args": ["-m", "my_mcp_server"],
+      "cwd": "./mcp_tools",
+      "env": {
+        "MY_API_KEY": "$MY_API_KEY"
+      },
+      "timeout": 15000,
+      "trust": false,
+      "includeTools": ["search_records", "get_status"]
+    }
+  }
+}
+```
+
+### What each field does
+
+| Field | Type | Description |
+|---|---|---|
+| `command` | string | The executable to run. Use `"python"` (or `"python3"` depending on your system). |
+| `args` | string[] | Arguments passed to the command. `["-m", "my_mcp_server"]` runs your package as a module. You can also use a script path like `["./server.py"]`. |
+| `cwd` | string | Working directory for the server process. Optional -- set this if your server needs to run from a specific directory. |
+| `env` | object | Environment variables passed to the server process. Use the `"$VAR_NAME"` syntax to reference variables from your shell environment. |
+| `timeout` | number | Connection timeout in milliseconds. Default is 10000 (10 seconds). Increase if your server takes longer to start. |
+| `trust` | boolean | When `true`, skips all tool confirmation dialogs. Keep this `false` unless you fully control and trust the server. |
+| `includeTools` | string[] | **Whitelist** -- only these tools will be exposed to Gemini. If set, all other tools from the server are hidden. |
+| `excludeTools` | string[] | **Blacklist** -- these specific tools will be hidden. All others remain available. |
+
+**Important:** `includeTools` and `excludeTools` are mutually exclusive approaches. Use one or the other:
+- Use `includeTools` when your server exposes many tools but you only want a few available.
+- Use `excludeTools` when you want most tools but need to block specific dangerous ones.
+
+---
+
+## 2. Pass the API Key Securely
+
+Never hardcode secrets directly in `settings.json`. Instead, use the `"$VAR_NAME"` pattern to reference environment variables from your shell:
+
+```json
+"env": {
+  "MY_API_KEY": "$MY_API_KEY"
+}
+```
+
+Then set the variable in your shell before running Gemini:
+
+```bash
+export MY_API_KEY="sk-your-actual-api-key-here"
+gemini
+```
+
+Alternatively, you can store the variable in a `.env` file that Gemini loads automatically:
+
+- `~/.gemini/.env` -- global env file
+- `./.gemini/.env` -- project-level env file
+
+Example `.gemini/.env`:
+
+```
+MY_API_KEY=sk-your-actual-api-key-here
+```
+
+**Security note:** Gemini CLI automatically redacts sensitive env vars (names containing `TOKEN`, `SECRET`, `KEY`, etc.) from MCP processes unless you explicitly list them in the `env` field. So you must include the variable in `env` for it to reach your server.
+
+---
+
+## 3. Full Working Example
+
+Suppose you have a Python MCP server at `~/mcp-servers/analytics_server.py` that exposes five tools (`query_data`, `generate_report`, `delete_records`, `export_csv`, `run_migration`), and you only want `query_data` and `generate_report` available. The server needs a `ANALYTICS_API_KEY`.
+
+### settings.json
+
+```json
+{
+  "mcpServers": {
+    "analytics": {
+      "command": "python3",
+      "args": ["analytics_server.py"],
+      "cwd": "/home/user/mcp-servers",
+      "env": {
+        "ANALYTICS_API_KEY": "$ANALYTICS_API_KEY"
+      },
+      "timeout": 15000,
+      "trust": false,
+      "includeTools": ["query_data", "generate_report"]
+    }
+  }
+}
+```
+
+### Shell setup
+
+```bash
+export ANALYTICS_API_KEY="your-key-here"
+gemini
+```
+
+### Using the server in a Gemini session
+
+Once inside the interactive session, you can verify the server connected:
+
+```
+/mcp list
+/mcp status
+```
+
+Then invoke the tools by addressing the server:
+
+```
+> @analytics Query monthly active users for Q4 2025
+> @analytics Generate a report on user retention trends
+```
+
+---
+
+## 4. Alternative: Use excludeTools Instead
+
+If you want all tools available except specific dangerous ones:
+
+```json
+{
+  "mcpServers": {
+    "analytics": {
+      "command": "python3",
+      "args": ["analytics_server.py"],
+      "cwd": "/home/user/mcp-servers",
+      "env": {
+        "ANALYTICS_API_KEY": "$ANALYTICS_API_KEY"
+      },
+      "timeout": 15000,
+      "trust": false,
+      "excludeTools": ["delete_records", "run_migration"]
+    }
+  }
+}
+```
+
+This exposes everything except `delete_records` and `run_migration`.
+
+---
+
+## 5. Adding the Server via CLI (Alternative to Manual Editing)
+
+You can also add MCP servers from the command line:
+
+```bash
+gemini mcp add analytics --command "python3 analytics_server.py"
+```
+
+However, the CLI `add` command provides less control over fields like `env`, `includeTools`, and `cwd`. For full configuration, editing `settings.json` directly is recommended.
+
+To manage servers later:
+
+```bash
+gemini mcp list           # list all configured servers
+gemini mcp remove analytics  # remove a server
+```
+
+Or from within an interactive session:
+
+```
+/mcp list
+/mcp enable analytics
+/mcp disable analytics
+```
+
+---
+
+## 6. Minimal Python MCP Server Skeleton
+
+For reference, here is the basic structure of a Python MCP server that your configuration would launch:
+
+```python
+# analytics_server.py
+import os
+import json
+import sys
+
+# Read the API key passed through the env config
+api_key = os.environ.get("ANALYTICS_API_KEY")
+if not api_key:
+    print("Error: ANALYTICS_API_KEY not set", file=sys.stderr)
+    sys.exit(1)
+
+# Your MCP server implementation here
+# The server communicates over stdio using the MCP protocol
+# Tools like query_data, generate_report, etc. are registered
+# and handled according to the MCP specification.
+```
+
+---
+
+## Summary
+
+1. Add your server to `mcpServers` in `~/.gemini/settings.json` (or project-level).
+2. Pass the API key using `"env": { "MY_API_KEY": "$MY_API_KEY" }` -- never hardcode it.
+3. Restrict tools with `"includeTools": ["tool1", "tool2"]` (whitelist) or `"excludeTools": ["bad_tool"]` (blacklist).
+4. Set the actual environment variable in your shell or in a `.gemini/.env` file.
+5. Verify the connection with `/mcp list` and `/mcp status` inside an interactive session.

package/gemini-cli-workspace/iteration-1/eval-2-mcp-server-config/with_skill/timing.json

@@ -0,0 +1,5 @@
+{
+  "total_tokens": 67196,
+  "duration_ms": 64105,
+  "total_duration_seconds": 64.1
+}

package/gemini-cli-workspace/iteration-1/eval-2-mcp-server-config/without_skill/grading.json

@@ -0,0 +1,37 @@
+{
+  "eval_id": 2,
+  "eval_name": "mcp-server-config",
+  "configuration": "without_skill",
+  "expectations": [
+    {
+      "text": "Points to ~/.gemini/settings.json or project-level settings.json",
+      "passed": true,
+      "evidence": "Mentions both `~/.gemini/settings.json` and `.gemini/settings.json` in project root"
+    },
+    {
+      "text": "Shows mcpServers config with command, args fields",
+      "passed": true,
+      "evidence": "Shows mcpServers config with command, args, cwd, env fields"
+    },
+    {
+      "text": "Shows $VAR pattern for environment variable references in env field",
+      "passed": false,
+      "evidence": "Hardcodes API key directly in env (`\"MY_API_KEY\": \"sk-abc123your-actual-key\"`). Uses `${MY_API_KEY}` syntax but adds caveat that it might not be supported. Does not show the correct `$VAR` pattern."
+    },
+    {
+      "text": "Explains includeTools and/or excludeTools for restricting exposed tools",
+      "passed": false,
+      "evidence": "Does not mention includeTools or excludeTools. Instead suggests server-side tool filtering and uses non-existent `toolFilter` field with `allow` sub-field."
+    },
+    {
+      "text": "Mentions security considerations (trust field, not hardcoding keys, etc.)",
+      "passed": false,
+      "evidence": "Does not mention the trust field. Actually recommends hardcoding as Option A. Does not mention env var redaction behavior."
+    },
+    {
+      "text": "Uses correct Python command (python/python3) in the command field",
+      "passed": true,
+      "evidence": "Uses `python` in the command field correctly"
+    }
+  ]
+}

package/gemini-cli-workspace/iteration-1/eval-2-mcp-server-config/without_skill/outputs/response.md

@@ -0,0 +1,427 @@
+# Connecting a Custom Python MCP Server to Gemini CLI
+
+## Overview
+
+Gemini CLI supports MCP (Model Context Protocol) servers through its settings configuration. You can connect a custom Python MCP server, pass environment variables (like API keys), and restrict which tools are exposed.
+
+## 1. Build Your Python MCP Server
+
+First, make sure your Python MCP server uses the official MCP SDK. Here is a minimal example:
+
+```python
+# my_mcp_server.py
+import os
+from mcp.server.fastmcp import FastMCP
+
+# Read the API key from environment
+API_KEY = os.environ.get("MY_API_KEY")
+if not API_KEY:
+    raise ValueError("MY_API_KEY environment variable is required")
+
+mcp = FastMCP("my-custom-server")
+
+@mcp.tool()
+def fetch_data(query: str) -> str:
+    """Fetch data from the external API using the configured API key."""
+    # Use API_KEY here for your requests
+    return f"Results for '{query}' using key ending in ...{API_KEY[-4:]}"
+
+@mcp.tool()
+def analyze_data(data: str) -> str:
+    """Analyze the provided data."""
+    return f"Analysis of: {data}"
+
+@mcp.tool()
+def internal_debug(info: str) -> str:
+    """Internal debug tool - you may want to hide this from Gemini."""
+    return f"Debug: {info}"
+
+if __name__ == "__main__":
+    mcp.run()
+```
+
+Install the dependency:
+
+```bash
+pip install mcp
+```
+
+## 2. Configure Gemini CLI to Use Your MCP Server
+
+Gemini CLI reads MCP server configuration from a `settings.json` file. There are two levels:
+
+- **Global (user-level):** `~/.gemini/settings.json`
+- **Project-level:** `.gemini/settings.json` in your project root
+
+### Basic Configuration
+
+Create or edit `~/.gemini/settings.json` (or `.gemini/settings.json` in your project):
+
+```json
+{
+  "mcpServers": {
+    "my-custom-server": {
+      "command": "python",
+      "args": ["my_mcp_server.py"],
+      "cwd": "/absolute/path/to/your/server/directory",
+      "env": {
+        "MY_API_KEY": "your-api-key-value-here"
+      }
+    }
+  }
+}
+```
+
+### Configuration Fields Explained
+
+| Field | Description |
+|-------|-------------|
+| `command` | The executable to run (e.g., `python`, `python3`, `node`, `npx`) |
+| `args` | Array of arguments passed to the command |
+| `cwd` | Working directory for the server process (optional but recommended) |
+| `env` | Environment variables passed to the server process |
+
+### Using a Virtual Environment
+
+If your MCP server has its own virtual environment:
+
+```json
+{
+  "mcpServers": {
+    "my-custom-server": {
+      "command": "/path/to/venv/bin/python",
+      "args": ["/absolute/path/to/my_mcp_server.py"],
+      "env": {
+        "MY_API_KEY": "your-api-key-value-here"
+      }
+    }
+  }
+}
+```
+
+On Windows, use the Windows path to the venv Python:
+
+```json
+{
+  "mcpServers": {
+    "my-custom-server": {
+      "command": "C:\\path\\to\\venv\\Scripts\\python.exe",
+      "args": ["C:\\path\\to\\my_mcp_server.py"],
+      "env": {
+        "MY_API_KEY": "your-api-key-value-here"
+      }
+    }
+  }
+}
+```
+
+## 3. Passing the API Key Securely
+
+There are several approaches to handle the API key:
+
+### Option A: Inline in settings.json (simplest, but key is in plaintext)
+
+```json
+{
+  "mcpServers": {
+    "my-custom-server": {
+      "command": "python",
+      "args": ["my_mcp_server.py"],
+      "env": {
+        "MY_API_KEY": "sk-abc123your-actual-key"
+      }
+    }
+  }
+}
+```
+
+### Option B: Reference a shell variable at launch time
+
+Set the environment variable in your shell profile (`.bashrc`, `.zshrc`, etc.):
+
+```bash
+export MY_API_KEY="sk-abc123your-actual-key"
+```
+
+Then in `settings.json`, you can omit the `env` block entirely if the variable is already in your shell environment. The child process inherits the parent environment. However, to be explicit:
+
+```json
+{
+  "mcpServers": {
+    "my-custom-server": {
+      "command": "python",
+      "args": ["my_mcp_server.py"],
+      "env": {
+        "MY_API_KEY": "${MY_API_KEY}"
+      }
+    }
+  }
+}
+```
+
+> **Note:** Whether `${MY_API_KEY}` variable interpolation is supported depends on the Gemini CLI version. If it is not supported, you may need to use a wrapper script approach (see Option C).
+
+### Option C: Wrapper script (most flexible)
+
+Create a wrapper script `run_server.sh`:
+
+```bash
+#!/bin/bash
+# run_server.sh
+# Load API key from a secure location
+source ~/.secrets/my_api_keys.env
+export MY_API_KEY
+exec python /path/to/my_mcp_server.py "$@"
+```
+
+Then configure:
+
+```json
+{
+  "mcpServers": {
+    "my-custom-server": {
+      "command": "bash",
+      "args": ["/path/to/run_server.sh"]
+    }
+  }
+}
+```
+
+## 4. Restricting Which Tools Are Exposed
+
+There are two approaches to restricting tools: server-side and client-side.
+
+### Approach A: Server-Side Restriction (Recommended)
+
+Only define the tools you want exposed in your MCP server. Simply remove or comment out any tools you do not want Gemini to access:
+
+```python
+# my_mcp_server.py - only expose the tools you want
+from mcp.server.fastmcp import FastMCP
+import os
+
+API_KEY = os.environ.get("MY_API_KEY")
+mcp = FastMCP("my-custom-server")
+
+@mcp.tool()
+def fetch_data(query: str) -> str:
+    """Fetch data from the external API."""
+    return f"Results for '{query}'"
+
+@mcp.tool()
+def analyze_data(data: str) -> str:
+    """Analyze the provided data."""
+    return f"Analysis of: {data}"
+
+# internal_debug is NOT registered - Gemini will never see it
+
+if __name__ == "__main__":
+    mcp.run()
+```
+
+### Approach B: Conditional Tool Registration
+
+Use an environment variable or config file to control which tools are registered:
+
+```python
+# my_mcp_server.py - conditionally register tools
+from mcp.server.fastmcp import FastMCP
+import os
+import json
+
+API_KEY = os.environ.get("MY_API_KEY")
+ALLOWED_TOOLS = os.environ.get("ALLOWED_TOOLS", "").split(",")
+
+mcp = FastMCP("my-custom-server")
+
+def register_if_allowed(tool_name):
+    """Decorator factory that only registers a tool if it's in the allowed list."""
+    def decorator(func):
+        if not ALLOWED_TOOLS or not ALLOWED_TOOLS[0]:
+            # No restriction set, register all tools
+            return mcp.tool()(func)
+        elif tool_name in ALLOWED_TOOLS:
+            return mcp.tool()(func)
+        return func
+    return decorator
+
+@register_if_allowed("fetch_data")
+def fetch_data(query: str) -> str:
+    """Fetch data from the external API."""
+    return f"Results for '{query}'"
+
+@register_if_allowed("analyze_data")
+def analyze_data(data: str) -> str:
+    """Analyze the provided data."""
+    return f"Analysis of: {data}"
+
+@register_if_allowed("internal_debug")
+def internal_debug(info: str) -> str:
+    """Internal debug tool."""
+    return f"Debug: {info}"
+
+if __name__ == "__main__":
+    mcp.run()
+```
+
+Then in your `settings.json`, pass the allowed tools list:
+
+```json
+{
+  "mcpServers": {
+    "my-custom-server": {
+      "command": "python",
+      "args": ["my_mcp_server.py"],
+      "env": {
+        "MY_API_KEY": "your-api-key-here",
+        "ALLOWED_TOOLS": "fetch_data,analyze_data"
+      }
+    }
+  }
+}
+```
+
+This way, `internal_debug` is never registered and Gemini cannot call it.
+
+### Approach C: Gemini CLI Tool Filtering (if supported)
+
+Some MCP clients support an `allowedTools` or `toolFilter` field in the server configuration. If Gemini CLI supports this, the configuration would look like:
+
+```json
+{
+  "mcpServers": {
+    "my-custom-server": {
+      "command": "python",
+      "args": ["my_mcp_server.py"],
+      "env": {
+        "MY_API_KEY": "your-api-key-here"
+      },
+      "toolFilter": {
+        "allow": ["fetch_data", "analyze_data"]
+      }
+    }
+  }
+}
+```
+
+> **Note:** The exact field name and syntax for client-side tool filtering varies by MCP client implementation. Check the Gemini CLI documentation for the exact syntax. If Gemini CLI does not support this field, use the server-side approaches (A or B) described above.
+
+## 5. Complete Working Example
+
+### Directory Structure
+
+```
+my-mcp-project/
+  my_mcp_server.py
+  requirements.txt
+  .gemini/
+    settings.json
+```
+
+### requirements.txt
+
+```
+mcp>=1.0.0
+```
+
+### my_mcp_server.py
+
+```python
+import os
+from mcp.server.fastmcp import FastMCP
+
+API_KEY = os.environ.get("MY_API_KEY")
+if not API_KEY:
+    raise ValueError("MY_API_KEY environment variable must be set")
+
+ALLOWED_TOOLS = [t.strip() for t in os.environ.get("ALLOWED_TOOLS", "").split(",") if t.strip()]
+
+mcp = FastMCP("my-custom-server")
+
+
+def expose(tool_name):
+    """Only register the tool if it passes the allowlist filter."""
+    def decorator(func):
+        if not ALLOWED_TOOLS or tool_name in ALLOWED_TOOLS:
+            return mcp.tool()(func)
+        return func
+    return decorator
+
+
+@expose("fetch_data")
+def fetch_data(query: str) -> str:
+    """Fetch data from the external API using the configured API key."""
+    # Your real implementation here
+    return f"Fetched results for '{query}'"
+
+
+@expose("analyze_data")
+def analyze_data(data: str) -> str:
+    """Analyze the provided data and return insights."""
+    return f"Analysis complete for: {data}"
+
+
+@expose("dangerous_reset")
+def dangerous_reset() -> str:
+    """Reset all data - restricted tool."""
+    return "Data reset complete"
+
+
+if __name__ == "__main__":
+    mcp.run()
+```
+
+### .gemini/settings.json
+
+```json
+{
+  "mcpServers": {
+    "my-custom-server": {
+      "command": "python",
+      "args": ["my_mcp_server.py"],
+      "cwd": "/absolute/path/to/my-mcp-project",
+      "env": {
+        "MY_API_KEY": "sk-your-api-key-here",
+        "ALLOWED_TOOLS": "fetch_data,analyze_data"
+      }
+    }
+  }
+}
+```
+
+With this configuration:
+- Gemini CLI launches your Python MCP server as a subprocess
+- The `MY_API_KEY` environment variable is passed securely to the server process
+- Only `fetch_data` and `analyze_data` are registered as tools; `dangerous_reset` is excluded
+- Gemini can discover and invoke the allowed tools during conversation
+
+## 6. Testing Your Server
+
+Before connecting to Gemini CLI, test your server standalone:
+
+```bash
+# Test that the server starts correctly
+MY_API_KEY="test-key" python my_mcp_server.py
+
+# Or with tool restrictions
+MY_API_KEY="test-key" ALLOWED_TOOLS="fetch_data" python my_mcp_server.py
+```
+
+You can also use the MCP Inspector tool to verify your server works:
+
+```bash
+npx @modelcontextprotocol/inspector python my_mcp_server.py
+```
+
+Then launch Gemini CLI in your project directory. It will automatically pick up the `.gemini/settings.json` and start your MCP server.
+
+## 7. Troubleshooting
+
+| Issue | Solution |
+|-------|----------|
+| Server not found | Use absolute paths in `command` and `args` |
+| API key not available | Verify the `env` block in settings.json; check with a print/log statement |
+| Tools not showing up | Confirm the tool functions are decorated with `@mcp.tool()` and not filtered out |
+| Permission errors | Ensure the Python executable and script have correct permissions |
+| Server crashes on start | Run the server manually from the terminal first to see error output |
+| Wrong Python version | Use the full path to the correct Python interpreter in `command` |

package/gemini-cli-workspace/iteration-1/eval-2-mcp-server-config/without_skill/timing.json

@@ -0,0 +1,5 @@
+{
+  "total_tokens": 59258,
+  "duration_ms": 65428,
+  "total_duration_seconds": 65.4
+}