@biggora/claude-plugins 1.2.0 → 1.3.0

package/codex-cli-workspace/iteration-1/eval-2-mcp-server-config/with_skill/outputs/response.md

@@ -1,86 +0,0 @@
-# Adding a Custom MCP Server to Codex CLI and Running Codex as an MCP Server
-
-## Adding Your Python-based MCP Server with GITHUB_TOKEN
-
-Add the following to your Codex configuration file at `~/.codex/config.toml`:
-
-```toml
-[mcp_servers.my-python-server]
-command = "python"
-args = ["/path/to/your/mcp_server.py"]
-env = { GITHUB_TOKEN = "$GITHUB_TOKEN" }
-```
-
-Key details:
-
-- **`command`** -- the executable to launch your server (e.g., `python`, `python3`, or an absolute path to your Python interpreter).
-- **`args`** -- an array of arguments passed to the command. Point this at your MCP server script or module entry point.
-- **`env`** -- a table of environment variables forwarded to the server process. Using `"$GITHUB_TOKEN"` tells Codex to pull the value from your shell environment at runtime, so you do not hardcode the token in the config file. Make sure `GITHUB_TOKEN` is exported in your shell (e.g., via `export GITHUB_TOKEN="ghp_..."` in your `.bashrc`/`.zshrc` or a `.env` loader).
-
-If your Python server is packaged as a module you run with `python -m`, the config would look like:
-
-```toml
-[mcp_servers.my-python-server]
-command = "python"
-args = ["-m", "my_mcp_server"]
-env = { GITHUB_TOKEN = "$GITHUB_TOKEN" }
-```
-
-If your MCP server exposes an HTTP endpoint instead of running as a stdio subprocess, use the `url` form:
-
-```toml
-[mcp_servers.my-python-server]
-url = "https://localhost:8080/mcp"
-```
-
-In that case you would manage the `GITHUB_TOKEN` env var in whatever process starts the HTTP server, not in the Codex config.
-
-### Verifying it works
-
-After adding the config, launch Codex and use the `/mcp` slash command in the TUI to list all available MCP tools. Your server's tools should appear in the list.
-
-### Per-project config
-
-You can also place MCP server config in a project-level `.codex/config.toml` (in your project root) if the server is only relevant to a specific project. Project config is only loaded for trusted projects.
-
----
-
-## Running Codex Itself as an MCP Server
-
-To expose Codex CLI as an MCP server that other agents can consume, run:
-
-```bash
-codex mcp
-```
-
-This starts Codex in MCP server mode, making its capabilities (code reading, editing, execution) available as MCP tools to any MCP-compatible client or agent. Other agents can then connect to Codex the same way Codex connects to third-party MCP servers -- either via stdio or by pointing at the running process.
-
----
-
-## Full Config Example Combining Both
-
-Here is a complete `~/.codex/config.toml` that sets up your Python MCP server, a GitHub MCP server (for reference), and general Codex settings:
-
-```toml
-model = "gpt-5-codex"
-approval_policy = "on-request"
-
-[sandbox_workspace_write]
-network_access = false
-
-[mcp_servers.github]
-command = "npx"
-args = ["-y", "@modelcontextprotocol/server-github"]
-env = { GITHUB_TOKEN = "$GITHUB_TOKEN" }
-
-[mcp_servers.my-python-server]
-command = "python"
-args = ["/path/to/your/mcp_server.py"]
-env = { GITHUB_TOKEN = "$GITHUB_TOKEN" }
-```
-
-If you need network access for your MCP server to make API calls, set `network_access = true` under `[sandbox_workspace_write]`, or pass it as a one-off flag:
-
-```bash
-codex -c 'sandbox_workspace_write.network_access=true' "Use my-python-server to fetch data"
-```

package/codex-cli-workspace/iteration-1/eval-2-mcp-server-config/with_skill/timing.json

@@ -1,5 +0,0 @@
-{
-  "total_tokens": 67753,
-  "duration_ms": 38647,
-  "total_duration_seconds": 38.6
-}

package/codex-cli-workspace/iteration-1/eval-2-mcp-server-config/without_skill/grading.json

@@ -1,58 +0,0 @@
-{
-  "expectations": [
-    {
-      "text": "Points to ~/.codex/config.toml for MCP server configuration",
-      "passed": false,
-      "evidence": "Uses '~/.codex/config.json' (line 11) — wrong format. Codex CLI uses TOML (config.toml), not JSON."
-    },
-    {
-      "text": "Shows [mcp_servers.name] TOML section with command, args fields",
-      "passed": false,
-      "evidence": "Uses JSON 'mcpServers' key (line 20) instead of TOML [mcp_servers.name] sections. Entire config is in JSON format."
-    },
-    {
-      "text": "Shows $GITHUB_TOKEN or $ENV_VAR pattern for environment variable references",
-      "passed": false,
-      "evidence": "Hardcodes 'your-github-token-here' (line 25) instead of using the $GITHUB_TOKEN reference pattern that Codex supports for pulling from shell environment."
-    },
-    {
-      "text": "Shows 'codex mcp' command for running codex as an MCP server",
-      "passed": false,
-      "evidence": "Uses hallucinated 'codex --mcp-server' flag (line 98). The correct command is 'codex mcp' (a subcommand, not a flag)."
-    },
-    {
-      "text": "Uses correct Python command (python/python3) in the MCP server command field",
-      "passed": true,
-      "evidence": "Line 22: '\"command\": \"python\"'"
-    },
-    {
-      "text": "Does not invent non-existent config keys or MCP server fields",
-      "passed": false,
-      "evidence": "Hallucinated: JSON config format (mcpServers), --mcp-server flag, config.json file. Real format is TOML with [mcp_servers.name] sections."
-    }
-  ],
-  "summary": {
-    "passed": 1,
-    "failed": 5,
-    "total": 6,
-    "pass_rate": 0.167
-  },
-  "claims": [
-    {
-      "claim": "Codex uses config.json for configuration",
-      "type": "factual",
-      "verified": false,
-      "evidence": "Codex CLI uses ~/.codex/config.toml (TOML format)"
-    },
-    {
-      "claim": "codex --mcp-server starts MCP server mode",
-      "type": "factual",
-      "verified": false,
-      "evidence": "Correct command is 'codex mcp' (subcommand)"
-    }
-  ],
-  "eval_feedback": {
-    "suggestions": [],
-    "overall": "Baseline heavily hallucinated — wrong config format, wrong MCP command, hardcoded secrets"
-  }
-}

package/codex-cli-workspace/iteration-1/eval-2-mcp-server-config/without_skill/outputs/response.md

@@ -1,164 +0,0 @@
-# Adding a Custom MCP Server to Codex CLI & Running Codex as an MCP Server
-
-## Part 1: Adding a Custom Python MCP Server with GITHUB_TOKEN
-
-Codex CLI supports MCP (Model Context Protocol) servers that extend its capabilities with custom tools. You configure them in your Codex CLI configuration file.
-
-### Configuration File Location
-
-Codex CLI reads its configuration from a JSON file. The typical location is:
-
-- **Linux/macOS**: `~/.codex/config.json` (or potentially `~/.config/codex/config.json`)
-- You can also use a project-local configuration
-
-### Adding Your Python MCP Server
-
-In your Codex CLI configuration file, add an `mcpServers` section. For a Python-based MCP server that needs the `GITHUB_TOKEN` environment variable:
-
-```json
-{
-  "mcpServers": {
-    "my-github-server": {
-      "command": "python",
-      "args": ["/path/to/your/mcp_server.py"],
-      "env": {
-        "GITHUB_TOKEN": "your-github-token-here"
-      }
-    }
-  }
-}
-```
-
-### Configuration Fields
-
-| Field | Description |
-|-------|-------------|
-| `command` | The executable to run (e.g., `python`, `python3`, `node`) |
-| `args` | Array of arguments passed to the command |
-| `env` | Object of environment variables to set for the server process |
-
-### Using a Virtual Environment
-
-If your Python MCP server has dependencies managed via a virtual environment, point to the Python binary inside the venv:
-
-```json
-{
-  "mcpServers": {
-    "my-github-server": {
-      "command": "/path/to/venv/bin/python",
-      "args": ["/path/to/your/mcp_server.py"],
-      "env": {
-        "GITHUB_TOKEN": "your-github-token-here"
-      }
-    }
-  }
-}
-```
-
-### Using uvx or pipx
-
-If your MCP server is distributed as a Python package, you can use `uvx` (from the `uv` tool) or `pipx`:
-
-```json
-{
-  "mcpServers": {
-    "my-github-server": {
-      "command": "uvx",
-      "args": ["my-github-mcp-server"],
-      "env": {
-        "GITHUB_TOKEN": "your-github-token-here"
-      }
-    }
-  }
-}
-```
-
-### Security Tip for the Token
-
-Rather than hardcoding the token in the config file, you can reference an environment variable that is already set in your shell session. However, the `env` field in the MCP config explicitly sets environment variables for the child process. To avoid storing secrets in config files, consider:
-
-1. Setting `GITHUB_TOKEN` in your shell profile (`.bashrc`, `.zshrc`) so it is inherited by child processes automatically. In that case you may not need the `env` field at all.
-2. Using a secrets manager or `.env` file that your MCP server reads directly.
-
-### Verifying the Server is Connected
-
-After configuring the MCP server, start Codex CLI. It should automatically launch and connect to your MCP server via stdio. The tools provided by your server will be available in the Codex session. You can verify by asking Codex to list available tools or by using one of the tools your server exposes.
-
----
-
-## Part 2: Running Codex CLI as an MCP Server
-
-Codex CLI can itself be run as an MCP server, allowing other agents or tools that support MCP to consume it as a tool provider.
-
-### Starting Codex as an MCP Server
-
-Use the `--mcp-server` flag (or similar) to start Codex in MCP server mode:
-
-```bash
-codex --mcp-server
-```
-
-This starts Codex CLI listening on **stdio** (stdin/stdout) using the MCP protocol, making it available as a tool for other MCP-compatible clients.
-
-### Consuming Codex from Another Agent
-
-In the consuming agent's MCP configuration, add Codex as a server:
-
-```json
-{
-  "mcpServers": {
-    "codex": {
-      "command": "codex",
-      "args": ["--mcp-server"],
-      "env": {
-        "OPENAI_API_KEY": "your-openai-api-key"
-      }
-    }
-  }
-}
-```
-
-### Example: Adding Codex as an MCP Server in Claude Desktop
-
-If you are using Claude Desktop (which also supports MCP), you can add Codex to the Claude Desktop MCP config (typically `claude_desktop_config.json`):
-
-```json
-{
-  "mcpServers": {
-    "codex": {
-      "command": "npx",
-      "args": ["@openai/codex", "--mcp-server"],
-      "env": {
-        "OPENAI_API_KEY": "your-openai-api-key"
-      }
-    }
-  }
-}
-```
-
-### What Codex Exposes as an MCP Server
-
-When running as an MCP server, Codex typically exposes tools that allow the consuming agent to:
-
-- Execute coding tasks via Codex's AI capabilities
-- Run shell commands in a sandboxed environment
-- Perform file operations (read, write, edit)
-- Leverage Codex's code generation and reasoning abilities
-
----
-
-## Summary
-
-| Goal | Configuration |
-|------|---------------|
-| Add custom MCP server to Codex | Add entry to `mcpServers` in Codex config with `command`, `args`, and `env` |
-| Pass `GITHUB_TOKEN` to MCP server | Use the `env` field in the server config |
-| Run Codex as an MCP server | Start with `codex --mcp-server` flag |
-| Consume Codex from another agent | Add Codex as an MCP server entry in the consuming agent's config |
-
-### Key Points
-
-- MCP servers communicate over **stdio** by default (stdin/stdout JSON-RPC messages).
-- The `env` field lets you pass environment variables like `GITHUB_TOKEN` to the server process.
-- Codex CLI can both **consume** MCP servers (as a client) and **be consumed** as an MCP server (acting as a server for other agents).
-- Make sure your Python MCP server implements the MCP protocol correctly (handles `initialize`, `tools/list`, `tools/call` methods over stdio).

package/codex-cli-workspace/iteration-1/eval-2-mcp-server-config/without_skill/timing.json

@@ -1,5 +0,0 @@
-{
-  "total_tokens": 64950,
-  "duration_ms": 41699,
-  "total_duration_seconds": 41.7
-}

package/codex-cli-workspace/iteration-1/eval-3-profiles-troubleshooting/eval_metadata.json

@@ -1,13 +0,0 @@
-{
-  "eval_id": 3,
-  "eval_name": "profiles-troubleshooting",
-  "prompt": "I'm getting Landlock errors on WSL2 when running codex. Also, I need to set up two profiles in config.toml — one for safe daily dev and one for CI with no approvals. How do I fix the Landlock issue and configure the profiles?",
-  "assertions": [
-    {"id": "landlock-wsl-fix", "text": "Mentions updating WSL2 kernel or using --dangerously-bypass-approvals-and-sandbox (--yolo) as workarounds"},
-    {"id": "profiles-toml-syntax", "text": "Shows correct [profiles.name] TOML syntax for defining profiles"},
-    {"id": "ci-profile-no-approval", "text": "CI profile uses approval_policy = 'never' for fully automated mode"},
-    {"id": "dev-profile-safe", "text": "Dev profile uses a safe approval_policy like 'on-request' or 'untrusted'"},
-    {"id": "profile-usage-flag", "text": "Shows --profile flag for selecting a profile at runtime"},
-    {"id": "no-hallucinated-fixes", "text": "Does not invent non-existent troubleshooting steps or config options"}
-  ]
-}

package/codex-cli-workspace/iteration-1/eval-3-profiles-troubleshooting/with_skill/grading.json

@@ -1,52 +0,0 @@
-{
-  "expectations": [
-    {
-      "text": "Mentions updating WSL2 kernel or using --dangerously-bypass-approvals-and-sandbox (--yolo) as workarounds",
-      "passed": true,
-      "evidence": "Option 1: 'wsl --update' to update kernel. Option 2: '--dangerously-bypass-approvals-and-sandbox' with '(aliased as --yolo)'"
-    },
-    {
-      "text": "Shows correct [profiles.name] TOML syntax for defining profiles",
-      "passed": true,
-      "evidence": "Lines 66-79: '[profiles.dev]' and '[profiles.ci]' sections in config.toml"
-    },
-    {
-      "text": "CI profile uses approval_policy = 'never' for fully automated mode",
-      "passed": true,
-      "evidence": "Line 77: 'approval_policy = \"never\"'"
-    },
-    {
-      "text": "Dev profile uses a safe approval_policy like 'on-request' or 'untrusted'",
-      "passed": true,
-      "evidence": "Line 68: 'approval_policy = \"on-request\"'"
-    },
-    {
-      "text": "Shows --profile flag for selecting a profile at runtime",
-      "passed": true,
-      "evidence": "Lines 85-93: 'codex --profile dev' and 'codex -p dev'"
-    },
-    {
-      "text": "Does not invent non-existent troubleshooting steps or config options",
-      "passed": true,
-      "evidence": "All flags (--dangerously-bypass-approvals-and-sandbox, --yolo, -s danger-full-access, --profile, -p) and config keys (approval_policy, model, profile) match the skill documentation"
-    }
-  ],
-  "summary": {
-    "passed": 6,
-    "failed": 0,
-    "total": 6,
-    "pass_rate": 1.0
-  },
-  "claims": [
-    {
-      "claim": "danger-full-access sandbox mode avoids Landlock syscall",
-      "type": "factual",
-      "verified": true,
-      "evidence": "Reasonable inference — danger-full-access provides unrestricted access per the skill doc"
-    }
-  ],
-  "eval_feedback": {
-    "suggestions": [],
-    "overall": "No suggestions, evals look solid"
-  }
-}

package/codex-cli-workspace/iteration-1/eval-3-profiles-troubleshooting/with_skill/outputs/response.md

@@ -1,130 +0,0 @@
-# Fixing Landlock Errors on WSL2 and Setting Up Codex Profiles
-
-## Fixing the Landlock Error
-
-Landlock is a Linux kernel security feature that Codex uses for filesystem sandboxing. WSL2 kernels often ship with an older version that does not fully support Landlock, which causes Codex to fail when it tries to enforce sandbox restrictions.
-
-### Options to fix it
-
-**Option 1: Update your WSL2 kernel** (recommended long-term fix)
-
-Update WSL2 to get a kernel version that supports Landlock (5.13+, ideally 6.1+):
-
-```bash
-# From a Windows PowerShell (admin):
-wsl --update
-
-# Then restart WSL:
-wsl --shutdown
-```
-
-After restarting, verify the kernel version inside WSL:
-
-```bash
-uname -r
-```
-
-If the version is 5.13 or higher, Landlock should work and the error should be resolved.
-
-**Option 2: Bypass the sandbox** (for isolated/trusted environments only)
-
-If updating the kernel is not an option, you can bypass the sandbox entirely using the `--dangerously-bypass-approvals-and-sandbox` flag (aliased as `--yolo`):
-
-```bash
-codex --dangerously-bypass-approvals-and-sandbox "your task here"
-```
-
-Only use this in isolated environments (containers, throwaway VMs, CI runners) since it removes all filesystem and approval restrictions.
-
-**Option 3: Use `danger-full-access` sandbox mode**
-
-A slightly less drastic alternative is to use the unrestricted sandbox mode while keeping approval prompts active:
-
-```bash
-codex -s danger-full-access "your task here"
-```
-
-This avoids the Landlock syscall entirely while still requiring approval for sensitive actions.
-
----
-
-## Setting Up Two Profiles in config.toml
-
-Edit (or create) `~/.codex/config.toml` and add the following:
-
-```toml
-# Default settings (used when no profile is specified)
-model = "gpt-5-codex"
-approval_policy = "on-request"
-
-[sandbox_workspace_write]
-network_access = false
-
-# ----------------------------------------------------------
-# Profile: safe daily development
-# ----------------------------------------------------------
-[profiles.dev]
-model = "gpt-5-codex"
-approval_policy = "on-request"
-# This is equivalent to --full-auto: it will ask before
-# sensitive actions but auto-approve reads and workspace writes.
-
-# ----------------------------------------------------------
-# Profile: CI with no approvals
-# ----------------------------------------------------------
-[profiles.ci]
-model = "gpt-5-codex"
-approval_policy = "never"
-# "never" means Codex will not pause for any approval prompts,
-# making it suitable for fully automated CI pipelines.
-```
-
-### Using the profiles
-
-```bash
-# Daily development (safe, asks before sensitive actions)
-codex --profile dev "Refactor the auth module"
-
-# CI pipeline (no approvals, fully automated)
-codex --profile ci "Run tests and fix failures"
-
-# Short form
-codex -p dev "task"
-codex -p ci "task"
-```
-
-### Setting a default profile
-
-If you want `dev` to be your default without typing `--profile` every time, add this at the top level of your config:
-
-```toml
-profile = "dev"
-```
-
-Then you only need to explicitly pass `--profile ci` when running in CI.
-
-### GitHub Actions example using the CI profile
-
-```yaml
-- name: Run Codex
-  run: |
-    npm i -g @openai/codex
-    codex exec --profile ci "Update CHANGELOG for this release"
-  env:
-    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
-```
-
-### Config precedence reminder
-
-CLI flags always override profile settings, which override user config defaults:
-
-1. CLI flags (`--model`, `-a`, `-c key=value`)
-2. Project config (`.codex/config.toml` in project root)
-3. User config (`~/.codex/config.toml`, including profiles)
-4. Built-in defaults
-
-So even with a profile active, you can override individual settings on the command line, for example:
-
-```bash
-codex --profile ci --model gpt-5-pro "task"
-```

package/codex-cli-workspace/iteration-1/eval-3-profiles-troubleshooting/with_skill/timing.json

@@ -1,5 +0,0 @@
-{
-  "total_tokens": 67804,
-  "duration_ms": 38914,
-  "total_duration_seconds": 38.9
-}

package/codex-cli-workspace/iteration-1/eval-3-profiles-troubleshooting/without_skill/grading.json

@@ -1,64 +0,0 @@
-{
-  "expectations": [
-    {
-      "text": "Mentions updating WSL2 kernel or using --dangerously-bypass-approvals-and-sandbox (--yolo) as workarounds",
-      "passed": false,
-      "evidence": "Mentions 'wsl --update' (correct) but does NOT mention --dangerously-bypass-approvals-and-sandbox or --yolo. Instead invents '--no-sandbox' flag and 'CODEX_SANDBOX_TYPE=none' env var, neither of which exist."
-    },
-    {
-      "text": "Shows correct [profiles.name] TOML syntax for defining profiles",
-      "passed": false,
-      "evidence": "Uses '[profile.dev]' and '[profile.ci]' (singular 'profile') instead of the correct '[profiles.dev]' and '[profiles.ci]' (plural 'profiles')."
-    },
-    {
-      "text": "CI profile uses approval_policy = 'never' for fully automated mode",
-      "passed": false,
-      "evidence": "Uses 'approval_mode = \"full-auto\"' — wrong key name (should be approval_policy) and wrong value (should be 'never'). 'full-auto' is a CLI shortcut flag, not an approval_policy value."
-    },
-    {
-      "text": "Dev profile uses a safe approval_policy like 'on-request' or 'untrusted'",
-      "passed": false,
-      "evidence": "Uses 'approval_mode = \"suggest\"' — wrong key name (should be approval_policy) and 'suggest' is not a valid approval policy value. Valid values are: untrusted, on-request, never, reject."
-    },
-    {
-      "text": "Shows --profile flag for selecting a profile at runtime",
-      "passed": true,
-      "evidence": "Line 157: 'codex --profile dev' and 'codex --profile ci'"
-    },
-    {
-      "text": "Does not invent non-existent troubleshooting steps or config options",
-      "passed": false,
-      "evidence": "Hallucinated: '--no-sandbox' flag, 'CODEX_SANDBOX_TYPE=none' env var, 'sandbox = none/permissive' config keys, 'approval_mode' key (should be approval_policy), 'suggest'/'auto-edit'/'full-auto' as config values, '[profile.name]' syntax (should be [profiles.name])."
-    }
-  ],
-  "summary": {
-    "passed": 1,
-    "failed": 5,
-    "total": 6,
-    "pass_rate": 0.167
-  },
-  "claims": [
-    {
-      "claim": "Approval modes are suggest, auto-edit, full-auto",
-      "type": "factual",
-      "verified": false,
-      "evidence": "Real approval_policy values: untrusted, on-request, never, reject"
-    },
-    {
-      "claim": "--no-sandbox disables sandboxing",
-      "type": "factual",
-      "verified": false,
-      "evidence": "No --no-sandbox flag exists. The bypass flag is --dangerously-bypass-approvals-and-sandbox (--yolo)"
-    },
-    {
-      "claim": "Profile sections use [profile.name] syntax",
-      "type": "factual",
-      "verified": false,
-      "evidence": "Correct syntax is [profiles.name] (plural)"
-    }
-  ],
-  "eval_feedback": {
-    "suggestions": [],
-    "overall": "Baseline massively hallucinated — wrong config keys, wrong values, invented flags and env vars"
-  }
-}