@gaffer-sh/mcp 0.4.0 → 0.4.1

package/README.md

@@ -4,11 +4,12 @@ MCP (Model Context Protocol) server for [Gaffer](https://gaffer.sh) - give your
 
 ## What is this?
 
-This MCP server connects AI coding assistants like Claude Code and Cursor to your Gaffer test history. It allows AI to:
+This MCP server connects AI coding assistants like Claude Code and Cursor to your Gaffer test history and coverage data. It allows AI to:
 
 - Check your project's test health (pass rate, flaky tests, trends)
 - Look up the history of specific tests to understand stability
 - Get context about test failures when debugging
+- Analyze code coverage and identify untested areas
 - Browse all your projects (with user API Keys)
 - Access test report files (HTML reports, coverage, etc.)
 
@@ -17,20 +18,6 @@ This MCP server connects AI coding assistants like Claude Code and Cursor to you
 1. A [Gaffer](https://gaffer.sh) account with test results uploaded
 2. An API Key from Account Settings > API Keys
 
-## Authentication
-
-The MCP server supports two types of authentication:
-
-### User API Keys (Recommended)
-
-User API Keys (`gaf_` prefix) provide read-only access to all projects across your organizations. This is the recommended approach as it allows your AI assistant to work across multiple projects.
-
-Get your API Key from: **Account Settings > API Keys**
-
-### Project Upload Tokens (Legacy)
-
-Project Upload Tokens (`gfr_` prefix) are designed for uploading test results and only provide access to a single project. While still supported for backward compatibility, user API Keys are preferred for the MCP server.
-
 ## Setup
 
 ### Claude Code
@@ -71,144 +58,197 @@ Add to `.cursor/mcp.json` in your project:
 
 ## Available Tools
 
-### `list_projects`
+### Project & Test Run Tools
 
-List all projects you have access to. **Requires a user API Key (`gaf_`).**
+| Tool | Description |
+|------|-------------|
+| `list_projects` | List all projects you have access to |
+| `get_project_health` | Get health metrics (pass rate, flaky count, trends) |
+| `list_test_runs` | List recent test runs with optional filtering |
+| `get_test_run_details` | Get parsed test results for a specific test run |
+| `get_report` | Get report file URLs for a test run |
+| `get_report_browser_url` | Get a browser-navigable URL for viewing reports |
 
-**Input:**
-- `organizationId` (optional): Filter by organization
-- `limit` (optional): Max results (default: 50)
+### Test Analysis Tools
 
-**Returns:**
-- List of projects with IDs, names, and organization info
+| Tool | Description |
+|------|-------------|
+| `get_test_history` | Get pass/fail history for a specific test |
+| `get_flaky_tests` | Get tests with high flip rates (pass↔fail) |
+| `get_slowest_tests` | Get slowest tests by P95 duration |
+| `compare_test_metrics` | Compare test performance between commits |
 
-**Example prompt:** "What projects do I have in Gaffer?"
+### Coverage Tools
 
-### `get_project_health`
+| Tool | Description |
+|------|-------------|
+| `get_coverage_summary` | Get overall coverage metrics and trends |
+| `get_coverage_for_file` | Get coverage for specific files or paths |
+| `get_untested_files` | Get files below a coverage threshold |
+| `find_uncovered_failure_areas` | Find files with low coverage AND test failures |
 
-Get the health metrics for a project.
+## Tool Details
 
-**Input:**
-- `projectId` (required with user API Keys): Project ID from `list_projects`
-- `days` (optional): Number of days to analyze (default: 30)
+### `list_projects`
 
-**Returns:**
-- Health score (0-100)
-- Pass rate percentage
-- Number of test runs
-- Flaky test count
-- Trend (up/down/stable)
+List all projects you have access to.
 
-**Example prompt:** "What's the health of my test suite?"
+- **Input:** `organizationId` (optional), `limit` (optional, default: 50)
+- **Returns:** List of projects with IDs, names, and organization info
+- **Example:** "What projects do I have in Gaffer?"
 
-### `get_test_history`
+### `get_project_health`
 
-Get the pass/fail history for a specific test.
+Get the health metrics for a project.
 
-**Input:**
-- `projectId` (required with user API Keys): Project ID from `list_projects`
-- `testName`: Exact test name to search for (one of testName or filePath required)
-- `filePath`: File path containing the test (one of testName or filePath required)
-- `limit` (optional): Max results (default: 20)
+- **Input:** `projectId` (required), `days` (optional, default: 30)
+- **Returns:** Health score (0-100), pass rate, test run count, flaky test count, trend
+- **Example:** "What's the health of my test suite?"
+
+### `get_test_history`
 
-**Returns:**
-- History of test runs with pass/fail status
-- Duration of each run
-- Branch and commit info
-- Error messages for failures
-- Summary statistics
+Get the pass/fail history for a specific test.
 
-**Example prompts:**
-- "Is the login test flaky? Check its history"
-- "Show me the history for tests in auth.test.ts"
+- **Input:** `projectId` (required), `testName` or `filePath` (one required), `limit` (optional)
+- **Returns:** History of runs with status, duration, branch, commit, errors
+- **Example:** "Is the login test flaky? Check its history"
 
 ### `get_flaky_tests`
 
 Get the list of flaky tests in a project.
 
-**Input:**
-- `projectId` (required with user API Keys): Project ID from `list_projects`
-- `threshold` (optional): Minimum flip rate to be considered flaky (0-1, default: 0.1)
-- `limit` (optional): Max results (default: 50)
-- `days` (optional): Analysis period in days (default: 30)
-
-**Returns:**
-- List of flaky tests with flip rates
-- Number of status transitions
-- Total runs analyzed
-- When each test last ran
-
-**Example prompts:**
-- "Which tests are flaky in my project?"
-- "Show me the most unstable tests"
+- **Input:** `projectId` (required), `threshold` (optional, default: 0.1), `days` (optional), `limit` (optional)
+- **Returns:** List of flaky tests with flip rates, transition counts, run counts
+- **Example:** "Which tests are flaky in my project?"
 
 ### `list_test_runs`
 
 List recent test runs with optional filtering.
 
-**Input:**
-- `projectId` (required with user API Keys): Project ID from `list_projects`
-- `commitSha` (optional): Filter by commit SHA (supports prefix matching)
-- `branch` (optional): Filter by branch name
-- `status` (optional): Filter by "passed" (no failures) or "failed" (has failures)
-- `limit` (optional): Max results (default: 20)
+- **Input:** `projectId` (required), `commitSha` (optional), `branch` (optional), `status` (optional), `limit` (optional)
+- **Returns:** List of test runs with pass/fail/skip counts, commit and branch info
+- **Example:** "What tests failed in the last commit?"
+
+### `get_test_run_details`
 
-**Returns:**
-- List of test runs with pass/fail/skip counts
-- Commit SHA and branch info
-- Pagination info
+Get parsed test results for a specific test run.
 
-**Example prompts:**
-- "What tests failed in the last commit?"
-- "Show me test runs on the main branch"
-- "Did any tests fail on my feature branch?"
+- **Input:** `testRunId` (required), `projectId` (required), `status` (optional filter), `limit` (optional)
+- **Returns:** Individual test results with name, status, duration, file path, errors
+- **Example:** "Show me all failed tests from this test run"
 
 ### `get_report`
 
-Get the report files for a specific test run. **Requires a user API Key (`gaf_`).**
+Get URLs for report files uploaded with a test run.
 
-**Input:**
-- `testRunId` (required): Test run ID from `list_test_runs`
+- **Input:** `testRunId` (required)
+- **Returns:** List of files with filename, size, content type, download URL
+- **Example:** "Get the Playwright report for the latest test run"
 
-**Returns:**
-- Test run ID and project info
-- Framework used (e.g., playwright, vitest)
-- List of files with:
-  - Filename (e.g., "report.html", "coverage/index.html")
-  - File size in bytes
-  - Content type (e.g., "text/html")
-  - Download URL
+### `get_report_browser_url`
 
-**Example prompts:**
-- "Get the Playwright report for the latest test run"
-- "What files were uploaded with this test run?"
-- "Show me the coverage report"
+Get a browser-navigable URL for viewing a test report.
+
+- **Input:** `projectId` (required), `testRunId` (required), `filename` (optional)
+- **Returns:** Signed URL valid for 30 minutes
+- **Example:** "Give me a link to view the test report"
 
 ### `get_slowest_tests`
 
-Get the slowest tests in a project, sorted by P95 duration. **Requires a user API Key (`gaf_`).**
-
-**Input:**
-- `projectId` (required): Project ID from `list_projects`
-- `days` (optional): Analysis period in days (default: 30, max: 365)
-- `limit` (optional): Max tests to return (default: 20, max: 100)
-- `framework` (optional): Filter by framework (e.g., "playwright", "vitest")
-
-**Returns:**
-- List of slowest tests with:
-  - name: Short test name
-  - fullName: Full test name including describe blocks
-  - filePath: Test file path (if available)
-  - framework: Test framework used
-  - avgDurationMs: Average duration in milliseconds
-  - p95DurationMs: 95th percentile duration
-  - runCount: Number of runs in the period
-- Summary with project info
-
-**Example prompts:**
-- "Which tests are slowing down my CI pipeline?"
-- "Find the slowest Playwright tests"
-- "Show me e2e tests that need optimization"
+Get the slowest tests in a project, sorted by P95 duration.
+
+- **Input:** `projectId` (required), `days` (optional), `limit` (optional), `framework` (optional), `branch` (optional)
+- **Returns:** List of tests with average and P95 duration, run count
+- **Example:** "Which tests are slowing down my CI pipeline?"
+
+### `compare_test_metrics`
+
+Compare test metrics between two commits or test runs.
+
+- **Input:** `projectId` (required), `testName` (required), `beforeCommit`/`afterCommit` OR `beforeRunId`/`afterRunId`
+- **Returns:** Before/after metrics with duration change and percentage
+- **Example:** "Did my fix make this test faster?"
+
+### `get_coverage_summary`
+
+Get the coverage metrics summary for a project.
+
+- **Input:** `projectId` (required), `days` (optional, default: 30)
+- **Returns:** Line/branch/function coverage percentages, trend, report count, lowest coverage files
+- **Example:** "What's our test coverage?"
+
+### `get_coverage_for_file`
+
+Get coverage metrics for specific files or paths.
+
+- **Input:** `projectId` (required), `filePath` (required - exact or partial match)
+- **Returns:** List of matching files with line/branch/function coverage
+- **Example:** "What's the coverage for our API routes?"
+
+### `get_untested_files`
+
+Get files with little or no test coverage.
+
+- **Input:** `projectId` (required), `maxCoverage` (optional, default: 10%), `limit` (optional)
+- **Returns:** List of files below threshold sorted by coverage (lowest first)
+- **Example:** "Which files have no tests?"
+
+### `find_uncovered_failure_areas`
+
+Find code areas with both low coverage AND test failures (high risk).
+
+- **Input:** `projectId` (required), `days` (optional), `coverageThreshold` (optional, default: 80%)
+- **Returns:** Risk areas ranked by score, with file path, coverage %, failure count
+- **Example:** "Where should we focus our testing efforts?"
+
+## Prioritizing Coverage Improvements
+
+When using coverage tools to improve your test suite, combine coverage data with codebase exploration for best results:
+
+### 1. Understand Code Utilization
+
+Before targeting files purely by coverage percentage, explore which code is actually critical:
+
+- **Find entry points:** Look for route definitions, event handlers, exported functions - these reveal what code actually executes in production
+- **Find heavily-imported files:** Files imported by many others are high-value targets
+- **Identify critical business logic:** Look for files handling auth, payments, data mutations, or core domain logic
+
+### 2. Prioritize by Impact
+
+Low coverage alone doesn't indicate priority. Consider:
+
+- **High utilization + low coverage = highest priority** - Code that runs frequently but lacks tests
+- **Large files with 0% coverage** - More uncovered lines means bigger impact on overall coverage
+- **Files with both failures and low coverage** - Use `find_uncovered_failure_areas` for this
+
+### 3. Use Path-Based Queries
+
+The `get_untested_files` tool may return many frontend components. For backend or specific areas:
+
+```
+# Query specific paths with get_coverage_for_file
+get_coverage_for_file(filePath="server/services")
+get_coverage_for_file(filePath="src/api")
+get_coverage_for_file(filePath="lib/core")
+```
+
+### 4. Iterative Improvement
+
+1. Get baseline with `get_coverage_summary`
+2. Identify targets with `get_coverage_for_file` on critical paths
+3. Write tests for highest-impact files
+4. Re-check coverage after CI uploads new results
+5. Repeat
+
+## Authentication
+
+### User API Keys (Recommended)
+
+User API Keys (`gaf_` prefix) provide read-only access to all projects across your organizations. Get your API Key from: **Account Settings > API Keys**
+
+### Project Upload Tokens (Legacy)
+
+Project Upload Tokens (`gfr_` prefix) are designed for uploading test results and only provide access to a single project. User API Keys are preferred for the MCP server.
 
 ## Environment Variables
 

package/dist/index.js

@@ -668,7 +668,16 @@ Returns:
 - Branch coverage (covered/total/percentage)
 - Function coverage (covered/total/percentage)
 
-Use this to check coverage for specific files or find files by path pattern.`
+This is the preferred tool for targeted coverage analysis. Use path prefixes to focus on
+specific areas of the codebase:
+- "server/services" - Backend service layer
+- "server/utils" - Backend utilities
+- "src/api" - API routes
+- "lib/core" - Core business logic
+
+Before querying, explore the codebase to identify critical paths - entry points,
+heavily-imported files, and code handling auth/payments/data mutations.
+Prioritize: high utilization + low coverage = highest impact.`
 };
 
 // src/tools/get-coverage-summary.ts
@@ -726,7 +735,11 @@ Returns:
 - Latest report date
 - Top 5 files with lowest coverage
 
-Use this to understand your project's overall test coverage health.`
+Use this to understand your project's overall test coverage health.
+
+After getting the summary, use get_coverage_for_file with path prefixes to drill into
+specific areas (e.g., "server/services", "src/api", "lib/core"). This helps identify
+high-value targets in critical code paths rather than just the files with lowest coverage.`
 };
 
 // src/tools/get-flaky-tests.ts
@@ -1280,7 +1293,13 @@ Returns:
 - Each file includes line/branch/function coverage metrics
 - Total count of files matching the criteria
 
-Use this to find files that need tests added.`
+IMPORTANT: Results may be dominated by certain file types (e.g., UI components) that are
+numerous but not necessarily the highest priority. For targeted analysis of specific code
+areas (backend, services, utilities), use get_coverage_for_file with path prefixes instead.
+
+To prioritize effectively, explore the codebase to understand which code is heavily utilized
+(entry points, frequently-imported files, critical business logic) and then query coverage
+for those specific paths.`
 };
 
 // src/tools/list-projects.ts
@@ -1446,10 +1465,33 @@ async function main() {
     process.exit(1);
   }
   const client = GafferApiClient.fromEnv();
-  const server = new McpServer({
-    name: "gaffer",
-    version: "0.1.0"
-  });
+  const server = new McpServer(
+    {
+      name: "gaffer",
+      version: "0.1.0"
+    },
+    {
+      instructions: `Gaffer provides test analytics and coverage data for your projects.
+
+## Coverage Analysis Best Practices
+
+When helping users improve test coverage, combine coverage data with codebase exploration:
+
+1. **Understand code utilization first**: Before targeting files by coverage percentage, explore which code is critical:
+   - Find entry points (route definitions, event handlers, exported functions)
+   - Find heavily-imported files (files imported by many others are high-value targets)
+   - Identify critical business logic (auth, payments, data mutations)
+
+2. **Prioritize by impact**: Low coverage alone doesn't indicate priority. Consider:
+   - High utilization + low coverage = highest priority
+   - Large files with 0% coverage have bigger impact than small files
+   - Use find_uncovered_failure_areas for files with both low coverage AND test failures
+
+3. **Use path-based queries**: The get_untested_files tool may return many files of a certain type (e.g., UI components). For targeted analysis, use get_coverage_for_file with path prefixes to focus on specific areas of the codebase.
+
+4. **Iterate**: Get baseline \u2192 identify targets \u2192 write tests \u2192 re-check coverage after CI uploads new results.`
+    }
+  );
   server.registerTool(
     getProjectHealthMetadata.name,
     {

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@gaffer-sh/mcp",
   "type": "module",
-  "version": "0.4.0",
+  "version": "0.4.1",
   "description": "MCP server for Gaffer test history - give your AI assistant memory of your tests",
   "license": "MIT",
   "author": "Gaffer <hello@gaffer.sh>",