simplecov-mcp 0.3.0 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/LICENSE +21 -0
- data/README.md +173 -356
- data/docs/ADVANCED_USAGE.md +967 -0
- data/docs/ARCHITECTURE.md +79 -0
- data/docs/BRANCH_ONLY_COVERAGE.md +81 -0
- data/docs/CLI_USAGE.md +637 -0
- data/docs/DEVELOPMENT.md +82 -0
- data/docs/ERROR_HANDLING.md +93 -0
- data/docs/EXAMPLES.md +430 -0
- data/docs/INSTALLATION.md +352 -0
- data/docs/LIBRARY_API.md +635 -0
- data/docs/MCP_INTEGRATION.md +488 -0
- data/docs/TROUBLESHOOTING.md +276 -0
- data/docs/arch-decisions/001-x-arch-decision.md +93 -0
- data/docs/arch-decisions/002-x-arch-decision.md +157 -0
- data/docs/arch-decisions/003-x-arch-decision.md +163 -0
- data/docs/arch-decisions/004-x-arch-decision.md +199 -0
- data/docs/arch-decisions/005-x-arch-decision.md +187 -0
- data/docs/arch-decisions/README.md +60 -0
- data/docs/presentations/simplecov-mcp-presentation.md +249 -0
- data/exe/simplecov-mcp +4 -4
- data/lib/simplecov_mcp/app_context.rb +26 -0
- data/lib/simplecov_mcp/base_tool.rb +74 -0
- data/lib/simplecov_mcp/cli.rb +234 -0
- data/lib/simplecov_mcp/cli_config.rb +56 -0
- data/lib/simplecov_mcp/commands/base_command.rb +78 -0
- data/lib/simplecov_mcp/commands/command_factory.rb +39 -0
- data/lib/simplecov_mcp/commands/detailed_command.rb +24 -0
- data/lib/simplecov_mcp/commands/list_command.rb +13 -0
- data/lib/simplecov_mcp/commands/raw_command.rb +22 -0
- data/lib/simplecov_mcp/commands/summary_command.rb +24 -0
- data/lib/simplecov_mcp/commands/uncovered_command.rb +26 -0
- data/lib/simplecov_mcp/commands/version_command.rb +18 -0
- data/lib/simplecov_mcp/constants.rb +22 -0
- data/lib/simplecov_mcp/error_handler.rb +124 -0
- data/lib/simplecov_mcp/error_handler_factory.rb +31 -0
- data/lib/simplecov_mcp/errors.rb +179 -0
- data/lib/simplecov_mcp/formatters/source_formatter.rb +148 -0
- data/lib/simplecov_mcp/mcp_server.rb +40 -0
- data/lib/simplecov_mcp/mode_detector.rb +55 -0
- data/lib/simplecov_mcp/model.rb +300 -0
- data/lib/simplecov_mcp/option_normalizers.rb +92 -0
- data/lib/simplecov_mcp/option_parser_builder.rb +134 -0
- data/lib/simplecov_mcp/option_parsers/env_options_parser.rb +50 -0
- data/lib/simplecov_mcp/option_parsers/error_helper.rb +109 -0
- data/lib/simplecov_mcp/path_relativizer.rb +61 -0
- data/lib/simplecov_mcp/presenters/base_coverage_presenter.rb +44 -0
- data/lib/simplecov_mcp/presenters/coverage_detailed_presenter.rb +16 -0
- data/lib/simplecov_mcp/presenters/coverage_raw_presenter.rb +16 -0
- data/lib/simplecov_mcp/presenters/coverage_summary_presenter.rb +16 -0
- data/lib/simplecov_mcp/presenters/coverage_uncovered_presenter.rb +16 -0
- data/lib/simplecov_mcp/presenters/project_coverage_presenter.rb +52 -0
- data/lib/simplecov_mcp/resolvers/coverage_line_resolver.rb +126 -0
- data/lib/simplecov_mcp/resolvers/resolver_factory.rb +28 -0
- data/lib/simplecov_mcp/resolvers/resultset_path_resolver.rb +78 -0
- data/lib/simplecov_mcp/resultset_loader.rb +136 -0
- data/lib/simplecov_mcp/staleness_checker.rb +243 -0
- data/lib/{simple_cov_mcp → simplecov_mcp}/tools/all_files_coverage_tool.rb +31 -13
- data/lib/{simple_cov_mcp → simplecov_mcp}/tools/coverage_detailed_tool.rb +7 -7
- data/lib/{simple_cov_mcp → simplecov_mcp}/tools/coverage_raw_tool.rb +7 -7
- data/lib/{simple_cov_mcp → simplecov_mcp}/tools/coverage_summary_tool.rb +7 -7
- data/lib/simplecov_mcp/tools/coverage_table_tool.rb +90 -0
- data/lib/{simple_cov_mcp → simplecov_mcp}/tools/help_tool.rb +13 -4
- data/lib/{simple_cov_mcp → simplecov_mcp}/tools/uncovered_lines_tool.rb +7 -7
- data/lib/{simple_cov_mcp → simplecov_mcp}/tools/version_tool.rb +11 -3
- data/lib/simplecov_mcp/util.rb +82 -0
- data/lib/{simple_cov_mcp → simplecov_mcp}/version.rb +1 -1
- data/lib/simplecov_mcp.rb +144 -2
- data/spec/MCP_INTEGRATION_TESTS_README.md +111 -0
- data/spec/TIMESTAMPS.md +48 -0
- data/spec/all_files_coverage_tool_spec.rb +29 -25
- data/spec/base_tool_spec.rb +11 -10
- data/spec/cli/show_default_report_spec.rb +33 -0
- data/spec/cli_config_spec.rb +137 -0
- data/spec/cli_enumerated_options_spec.rb +68 -0
- data/spec/cli_error_spec.rb +105 -47
- data/spec/cli_source_spec.rb +82 -23
- data/spec/cli_spec.rb +140 -5
- data/spec/cli_success_predicate_spec.rb +141 -0
- data/spec/cli_table_spec.rb +1 -1
- data/spec/cli_usage_spec.rb +10 -26
- data/spec/commands/base_command_spec.rb +187 -0
- data/spec/commands/command_factory_spec.rb +72 -0
- data/spec/commands/detailed_command_spec.rb +48 -0
- data/spec/commands/raw_command_spec.rb +46 -0
- data/spec/commands/summary_command_spec.rb +47 -0
- data/spec/commands/uncovered_command_spec.rb +49 -0
- data/spec/constants_spec.rb +61 -0
- data/spec/coverage_table_tool_spec.rb +17 -33
- data/spec/error_handler_spec.rb +22 -13
- data/spec/error_mode_spec.rb +143 -0
- data/spec/errors_edge_cases_spec.rb +239 -0
- data/spec/errors_stale_spec.rb +2 -2
- data/spec/file_based_mcp_tools_spec.rb +99 -0
- data/spec/fixtures/project1/lib/bar.rb +0 -1
- data/spec/fixtures/project1/lib/foo.rb +0 -1
- data/spec/help_tool_spec.rb +11 -17
- data/spec/integration_spec.rb +845 -0
- data/spec/logging_fallback_spec.rb +128 -0
- data/spec/mcp_logging_spec.rb +44 -0
- data/spec/mcp_server_integration_spec.rb +23 -0
- data/spec/mcp_server_spec.rb +15 -4
- data/spec/mode_detector_spec.rb +148 -0
- data/spec/model_error_handling_spec.rb +210 -0
- data/spec/model_staleness_spec.rb +40 -10
- data/spec/option_normalizers_spec.rb +204 -0
- data/spec/option_parsers/env_options_parser_spec.rb +233 -0
- data/spec/option_parsers/error_helper_spec.rb +222 -0
- data/spec/path_relativizer_spec.rb +83 -0
- data/spec/presenters/coverage_detailed_presenter_spec.rb +19 -0
- data/spec/presenters/coverage_raw_presenter_spec.rb +15 -0
- data/spec/presenters/coverage_summary_presenter_spec.rb +15 -0
- data/spec/presenters/coverage_uncovered_presenter_spec.rb +16 -0
- data/spec/presenters/project_coverage_presenter_spec.rb +86 -0
- data/spec/resolvers/coverage_line_resolver_spec.rb +57 -0
- data/spec/resolvers/resolver_factory_spec.rb +61 -0
- data/spec/resolvers/resultset_path_resolver_spec.rb +55 -0
- data/spec/resultset_loader_spec.rb +167 -0
- data/spec/shared_examples/README.md +115 -0
- data/spec/shared_examples/coverage_presenter_examples.rb +66 -0
- data/spec/shared_examples/file_based_mcp_tools.rb +174 -0
- data/spec/shared_examples/mcp_tool_text_json_response.rb +16 -0
- data/spec/simple_cov_mcp_module_spec.rb +16 -0
- data/spec/simplecov_mcp_model_spec.rb +340 -9
- data/spec/simplecov_mcp_opts_spec.rb +182 -0
- data/spec/spec_helper.rb +147 -4
- data/spec/staleness_checker_spec.rb +373 -0
- data/spec/staleness_more_spec.rb +16 -13
- data/spec/support/mcp_runner.rb +64 -0
- data/spec/tools_error_handling_spec.rb +144 -0
- data/spec/util_spec.rb +109 -34
- data/spec/version_spec.rb +117 -9
- data/spec/version_tool_spec.rb +131 -10
- metadata +120 -63
- data/lib/simple_cov/mcp.rb +0 -9
- data/lib/simple_cov_mcp/base_tool.rb +0 -70
- data/lib/simple_cov_mcp/cli.rb +0 -390
- data/lib/simple_cov_mcp/error_handler.rb +0 -131
- data/lib/simple_cov_mcp/error_handler_factory.rb +0 -38
- data/lib/simple_cov_mcp/errors.rb +0 -176
- data/lib/simple_cov_mcp/mcp_server.rb +0 -30
- data/lib/simple_cov_mcp/model.rb +0 -104
- data/lib/simple_cov_mcp/staleness_checker.rb +0 -125
- data/lib/simple_cov_mcp/tools/coverage_table_tool.rb +0 -61
- data/lib/simple_cov_mcp/util.rb +0 -122
- data/lib/simple_cov_mcp.rb +0 -102
- data/spec/coverage_detailed_tool_spec.rb +0 -36
- data/spec/coverage_raw_tool_spec.rb +0 -32
- data/spec/coverage_summary_tool_spec.rb +0 -39
- data/spec/legacy_shim_spec.rb +0 -13
- data/spec/uncovered_lines_tool_spec.rb +0 -33
@@ -0,0 +1,276 @@
|
|
1
|
+
# Troubleshooting Guide
|
2
|
+
|
3
|
+
This guide helps you diagnose and fix common issues with simplecov-mcp. Issues are organized by symptom for quick lookup.
|
4
|
+
|
5
|
+
## Table of Contents
|
6
|
+
|
7
|
+
- [Installation Issues](#installation-issues)
|
8
|
+
- [Running Issues](#running-issues)
|
9
|
+
- [Coverage Data Issues](#coverage-data-issues)
|
10
|
+
- [MCP Server Issues](#mcp-server-issues)
|
11
|
+
- [Development Issues](#development-issues)
|
12
|
+
- [Diagnostic Commands](#diagnostic-commands)
|
13
|
+
|
14
|
+
## Installation Issues
|
15
|
+
|
16
|
+
### "command not found: simplecov-mcp"
|
17
|
+
|
18
|
+
**Symptom:** Shell can't find the simplecov-mcp executable.
|
19
|
+
|
20
|
+
**Solutions:**
|
21
|
+
|
22
|
+
1. **Verify installation:**
|
23
|
+
```bash
|
24
|
+
gem list simplecov-mcp
|
25
|
+
```
|
26
|
+
If not listed, install it: `gem install simplecov-mcp`
|
27
|
+
|
28
|
+
2. **Check PATH:**
|
29
|
+
```bash
|
30
|
+
which simplecov-mcp
|
31
|
+
echo $PATH | grep -o "$(gem env gemdir)/bin"
|
32
|
+
```
|
33
|
+
|
34
|
+
3. **Rehash version manager shims:**
|
35
|
+
```bash
|
36
|
+
# rbenv
|
37
|
+
rbenv rehash
|
38
|
+
|
39
|
+
# asdf
|
40
|
+
asdf reshim ruby
|
41
|
+
|
42
|
+
# RVM
|
43
|
+
rvm reload
|
44
|
+
```
|
45
|
+
|
46
|
+
4. **Use bundler as workaround:**
|
47
|
+
```bash
|
48
|
+
bundle exec simplecov-mcp
|
49
|
+
```
|
50
|
+
|
51
|
+
### "cannot load such file -- mcp"
|
52
|
+
|
53
|
+
**Symptom:** Ruby can't load the mcp gem dependency.
|
54
|
+
|
55
|
+
**Cause:** Ruby version is too old (< 3.2).
|
56
|
+
|
57
|
+
**Solution:**
|
58
|
+
|
59
|
+
```bash
|
60
|
+
# Check your Ruby version
|
61
|
+
ruby -v # Should be 3.2.0 or higher
|
62
|
+
|
63
|
+
# Upgrade Ruby, then reinstall
|
64
|
+
gem install simplecov-mcp
|
65
|
+
```
|
66
|
+
|
67
|
+
### Multiple versions causing conflicts
|
68
|
+
|
69
|
+
**Symptom:** "wrong number of arguments" or other unexpected errors.
|
70
|
+
|
71
|
+
**Solution:** Clean up and reinstall:
|
72
|
+
|
73
|
+
```bash
|
74
|
+
gem uninstall simplecov-mcp
|
75
|
+
# Select "All versions" if prompted
|
76
|
+
gem install simplecov-mcp
|
77
|
+
```
|
78
|
+
|
79
|
+
## Running Issues
|
80
|
+
|
81
|
+
### Running the Test Suite with RVM (Codex macOS)
|
82
|
+
|
83
|
+
Codex's macOS sandbox forbids `/bin/ps`; RVM shells need it. When you run `bundle exec rspec` there, the shell falls back to macOS Ruby 2.6 and Bundler dies with `Gem::Resolver::APISet::GemParser` errors.
|
84
|
+
|
85
|
+
**Workarounds:**
|
86
|
+
|
87
|
+
- Run outside the macOS sandbox (Codex on Ubuntu, Gemini, Claude Code, local shells) or use a version manager that does not invoke `ps`.
|
88
|
+
- Or execute RSpec with explicit RVM paths:
|
89
|
+
```bash
|
90
|
+
PATH="$HOME/.rvm/gems/ruby-3.4.5@simplecov-mcp/bin:$HOME/.rvm/rubies/ruby-3.4.5/bin:$PATH" \
|
91
|
+
GEM_HOME="$HOME/.rvm/gems/ruby-3.4.5@simplecov-mcp" \
|
92
|
+
GEM_PATH="$HOME/.rvm/gems/ruby-3.4.5@simplecov-mcp:$HOME/.rvm/gems/ruby-3.4.5@global" \
|
93
|
+
$HOME/.rvm/rubies/ruby-3.4.5/bin/bundle exec rspec
|
94
|
+
```
|
95
|
+
- Use a different AI coding agent and/or operating system.
|
96
|
+
|
97
|
+
## Coverage Data Issues
|
98
|
+
|
99
|
+
### Missing `coverage/.resultset.json`
|
100
|
+
|
101
|
+
`simplecov-mcp` only reads coverage data; it never generates it. If you see "Could not find .resultset.json":
|
102
|
+
|
103
|
+
1. Run the test suite with SimpleCov enabled (default project setup already enables it).
|
104
|
+
```bash
|
105
|
+
bundle exec rspec
|
106
|
+
ls coverage/.resultset.json
|
107
|
+
```
|
108
|
+
2. If your coverage lives elsewhere, point the tools at it:
|
109
|
+
```bash
|
110
|
+
simplecov-mcp --resultset build/coverage/.resultset.json
|
111
|
+
# or
|
112
|
+
export SIMPLECOV_MCP_OPTS="--resultset build/coverage"
|
113
|
+
```
|
114
|
+
|
115
|
+
### Stale Coverage Errors
|
116
|
+
|
117
|
+
`--stale error` (or `staleness: 'error'`) compares file mtimes and line counts to the coverage snapshot. When it fails:
|
118
|
+
|
119
|
+
- Regenerate coverage (`bundle exec rspec`) so the snapshot matches current sources.
|
120
|
+
- Or drop back to warning-only behaviour using `--stale off` / `staleness: 'off'`.
|
121
|
+
|
122
|
+
If you only care about a subset of files, supply `--tracked-globs` (CLI) or `tracked_globs:` (API) so new files outside those globs do not trigger staleness.
|
123
|
+
|
124
|
+
### "No coverage data found for file"
|
125
|
+
|
126
|
+
The model looks up files by absolute path, then cwd-relative path, then basename. If you still hit this error:
|
127
|
+
|
128
|
+
1. Verify the file is listed in the coverage table (`simplecov-mcp list | grep model.rb`).
|
129
|
+
2. Use the exact project-relative path that SimpleCov recorded (case-sensitive, no symlinks).
|
130
|
+
3. If the file truly never executes under tests, add coverage or exclude it from your workflow.
|
131
|
+
|
132
|
+
## MCP Server Issues
|
133
|
+
|
134
|
+
### MCP Server Won't Start
|
135
|
+
|
136
|
+
**Symptom:** AI assistant reports "Could not connect to MCP server".
|
137
|
+
|
138
|
+
**Diagnostic steps:**
|
139
|
+
|
140
|
+
1. **Verify executable exists and works:**
|
141
|
+
```bash
|
142
|
+
which simplecov-mcp
|
143
|
+
simplecov-mcp version
|
144
|
+
```
|
145
|
+
|
146
|
+
2. **Test MCP server mode manually:**
|
147
|
+
```bash
|
148
|
+
echo '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"version_tool","arguments":{}}}' | simplecov-mcp
|
149
|
+
```
|
150
|
+
Should return JSON-RPC response.
|
151
|
+
|
152
|
+
3. **Check Ruby version in MCP context:**
|
153
|
+
```bash
|
154
|
+
# The version your MCP client sees might differ from your shell
|
155
|
+
$(which simplecov-mcp) --version
|
156
|
+
```
|
157
|
+
|
158
|
+
4. **Use absolute path in MCP config:**
|
159
|
+
```bash
|
160
|
+
# Find absolute path
|
161
|
+
which simplecov-mcp
|
162
|
+
|
163
|
+
# Update your MCP client config to use this path
|
164
|
+
```
|
165
|
+
|
166
|
+
### MCP Tools Not Available in AI Assistant
|
167
|
+
|
168
|
+
**Symptom:** AI says "I don't have access to simplecov-mcp tools".
|
169
|
+
|
170
|
+
**Solutions:**
|
171
|
+
|
172
|
+
1. **Restart AI assistant** - Config changes often require restart
|
173
|
+
|
174
|
+
2. **Verify MCP server is configured:**
|
175
|
+
```bash
|
176
|
+
# For Claude Code
|
177
|
+
claude mcp list
|
178
|
+
|
179
|
+
# Check logs
|
180
|
+
tail -f ~/simplecov_mcp.log
|
181
|
+
```
|
182
|
+
|
183
|
+
3. **Try CLI fallback:**
|
184
|
+
```bash
|
185
|
+
# If MCP isn't working, use CLI with --json
|
186
|
+
simplecov-mcp list --json
|
187
|
+
simplecov-mcp summary lib/file.rb --json
|
188
|
+
```
|
189
|
+
|
190
|
+
### Path Issues with Version Managers
|
191
|
+
|
192
|
+
**Symptom:** Works in terminal but not in MCP client.
|
193
|
+
|
194
|
+
**Cause:** MCP client doesn't have your shell environment (PATH, RVM, etc.).
|
195
|
+
|
196
|
+
**Solution:** Use absolute paths in MCP configuration:
|
197
|
+
|
198
|
+
```bash
|
199
|
+
# For rbenv/asdf
|
200
|
+
which simplecov-mcp
|
201
|
+
# Use this path in MCP config
|
202
|
+
|
203
|
+
# For RVM, create wrapper
|
204
|
+
rvm wrapper ruby-3.3.8 simplecov-mcp simplecov-mcp
|
205
|
+
# Use: ~/.rvm/wrappers/ruby-3.3.8/simplecov-mcp
|
206
|
+
```
|
207
|
+
|
208
|
+
## Development Issues
|
209
|
+
|
210
|
+
### Test Suite Failures
|
211
|
+
|
212
|
+
**Symptom:** `bundle exec rspec` fails with coverage errors.
|
213
|
+
|
214
|
+
**Common causes:**
|
215
|
+
|
216
|
+
1. **Stale coverage data** - Delete `coverage/` directory and re-run
|
217
|
+
2. **SimpleCov not loaded** - Check `spec/spec_helper.rb` requires SimpleCov
|
218
|
+
3. **Wrong Ruby version** - Verify Ruby >= 3.2
|
219
|
+
|
220
|
+
## Diagnostic Commands
|
221
|
+
|
222
|
+
Before reporting an issue, run these diagnostic commands and include the output:
|
223
|
+
|
224
|
+
```bash
|
225
|
+
# System info
|
226
|
+
ruby -v
|
227
|
+
gem -v
|
228
|
+
bundle -v
|
229
|
+
|
230
|
+
# simplecov-mcp info
|
231
|
+
gem list simplecov-mcp
|
232
|
+
which simplecov-mcp
|
233
|
+
simplecov-mcp version
|
234
|
+
|
235
|
+
# Test basic functionality
|
236
|
+
simplecov-mcp --help
|
237
|
+
simplecov-mcp list 2>&1
|
238
|
+
|
239
|
+
# Check coverage data
|
240
|
+
ls -la coverage/.resultset.json
|
241
|
+
head -20 coverage/.resultset.json
|
242
|
+
|
243
|
+
# Test MCP mode
|
244
|
+
echo '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"version_tool","arguments":{}}}' | simplecov-mcp 2>&1
|
245
|
+
```
|
246
|
+
|
247
|
+
## Getting More Help
|
248
|
+
|
249
|
+
If the above doesn't solve your problem:
|
250
|
+
|
251
|
+
1. **Check error mode** - Run with `--error-mode trace` for full stack traces:
|
252
|
+
```bash
|
253
|
+
simplecov-mcp --error-mode trace summary lib/file.rb
|
254
|
+
```
|
255
|
+
|
256
|
+
2. **Check logs:**
|
257
|
+
```bash
|
258
|
+
# MCP server logs
|
259
|
+
tail -50 simplecov_mcp.log
|
260
|
+
|
261
|
+
# Or specify custom log location
|
262
|
+
simplecov-mcp --log-file /tmp/debug.log summary lib/file.rb
|
263
|
+
```
|
264
|
+
|
265
|
+
3. **Search existing issues:**
|
266
|
+
https://github.com/keithrbennett/simplecov-mcp/issues
|
267
|
+
|
268
|
+
4. **Report a bug:**
|
269
|
+
Include output from [Diagnostic Commands](#diagnostic-commands) above
|
270
|
+
|
271
|
+
## Related Documentation
|
272
|
+
|
273
|
+
- [Installation Guide](INSTALLATION.md) - Setup and PATH configuration
|
274
|
+
- [CLI Usage](CLI_USAGE.md) - Command-line options and examples
|
275
|
+
- [MCP Integration](MCP_INTEGRATION.md) - MCP server configuration
|
276
|
+
- [Error Handling](ERROR_HANDLING.md) - Understanding error modes
|
@@ -0,0 +1,93 @@
|
|
1
|
+
# ADR 001: Dual-Mode Operation (CLI and MCP Server)
|
2
|
+
|
3
|
+
## Status
|
4
|
+
|
5
|
+
Accepted
|
6
|
+
|
7
|
+
## Context
|
8
|
+
|
9
|
+
SimpleCov MCP needed to serve two distinct use cases:
|
10
|
+
|
11
|
+
1. **Human users** wanting a command-line tool to inspect coverage reports in their terminal
|
12
|
+
2. **AI agents and MCP clients** needing programmatic access to coverage data via the Model Context Protocol (MCP) over JSON-RPC
|
13
|
+
|
14
|
+
We considered three approaches:
|
15
|
+
|
16
|
+
1. **Separate binaries/gems**: Create `simplecov-cli` and `simplecov-mcp` as separate projects
|
17
|
+
2. **Single binary with explicit mode flags**: Require users to pass `--mcp` or `--cli` to select mode
|
18
|
+
3. **Automatic mode detection**: Single binary that automatically detects the operating mode based on input
|
19
|
+
|
20
|
+
### Key Constraints
|
21
|
+
|
22
|
+
- MCP servers communicate via JSON-RPC over stdin/stdout, so any human-readable output would corrupt the protocol
|
23
|
+
- CLI users expect immediate, readable output without ceremony
|
24
|
+
- The gem should be simple to install and use for both audiences
|
25
|
+
- Mode detection must be reliable and unambiguous
|
26
|
+
|
27
|
+
## Decision
|
28
|
+
|
29
|
+
We implemented **automatic mode detection** via a single entry point (`SimpleCovMcp.run`) that routes to either CLI or MCP server mode based on the execution context.
|
30
|
+
|
31
|
+
### Mode Detection Algorithm
|
32
|
+
|
33
|
+
The `ModeDetector` class (lib/simplecov_mcp/mode_detector.rb:6) implements a priority-based detection strategy:
|
34
|
+
|
35
|
+
1. **Explicit CLI flags** (`--force-cli`, `-h`, `--help`, `--version`) → CLI mode
|
36
|
+
2. **Presence of subcommands** (non-option arguments like `summary`, `list`) → CLI mode
|
37
|
+
3. **TTY detection** fallback: `stdin.tty?` returns true → CLI mode, false → MCP server mode
|
38
|
+
|
39
|
+
The implementation is in `lib/simplecov_mcp.rb:34-52`:
|
40
|
+
|
41
|
+
```ruby
|
42
|
+
def run(argv)
|
43
|
+
env_opts = parse_env_opts_for_mode_detection
|
44
|
+
full_argv = env_opts + argv
|
45
|
+
|
46
|
+
if ModeDetector.cli_mode?(full_argv)
|
47
|
+
CoverageCLI.new.run(argv)
|
48
|
+
else
|
49
|
+
SimpleCovMcp.default_log_file = parse_log_file(full_argv)
|
50
|
+
MCPServer.new.run
|
51
|
+
end
|
52
|
+
end
|
53
|
+
```
|
54
|
+
|
55
|
+
### Why This Works
|
56
|
+
|
57
|
+
- **MCP clients** pipe JSON-RPC to stdin (not a TTY) and don't pass subcommands → routes to MCP server
|
58
|
+
- **CLI users** run from an interactive terminal (TTY) or pass explicit subcommands → routes to CLI
|
59
|
+
- **Edge cases** are covered by explicit flags (`--force-cli` for testing MCP mode from a TTY)
|
60
|
+
|
61
|
+
## Consequences
|
62
|
+
|
63
|
+
### Positive
|
64
|
+
|
65
|
+
1. **User convenience**: Single gem to install (`gem install simplecov-mcp`), single executable (`simplecov-mcp`)
|
66
|
+
2. **No ceremony**: Users don't need to remember mode flags or understand the MCP/CLI distinction
|
67
|
+
3. **Testable**: The `ModeDetector` class is a pure function that can be tested in isolation
|
68
|
+
4. **Clear separation**: CLI and MCP server implementations remain completely separate after routing
|
69
|
+
|
70
|
+
### Negative
|
71
|
+
|
72
|
+
1. **Complexity**: Requires maintaining the mode detection logic and keeping it accurate
|
73
|
+
2. **Potential ambiguity**: In unusual environments (non-TTY CLI execution without subcommands), users must understand `--force-cli`
|
74
|
+
3. **Shared dependencies**: Some components (error handling, coverage model) must work correctly in both modes
|
75
|
+
|
76
|
+
### Trade-offs
|
77
|
+
|
78
|
+
- **Versus separate gems**: More initial complexity, but better DX (single installation, no confusion about which gem to use)
|
79
|
+
- **Versus explicit mode flags**: Slightly more "magical", but eliminates user error and reduces boilerplate
|
80
|
+
|
81
|
+
### Future Constraints
|
82
|
+
|
83
|
+
- Mode detection logic must remain stable and backward-compatible
|
84
|
+
- Any new CLI subcommands must be registered in `ModeDetector::SUBCOMMANDS`
|
85
|
+
- Shared components (like `CoverageModel`) must never output to stdout/stderr in ways that differ by mode
|
86
|
+
|
87
|
+
## References
|
88
|
+
|
89
|
+
- Implementation: `lib/simplecov_mcp.rb:34-52`
|
90
|
+
- Mode detection: `lib/simplecov_mcp/mode_detector.rb:6-63`
|
91
|
+
- CLI implementation: `lib/simplecov_mcp/cli.rb`
|
92
|
+
- MCP server implementation: `lib/simplecov_mcp/mcp_server.rb`
|
93
|
+
- Related ADR: [002: Context-Aware Error Handling](002-x-arch-decision.md)
|
@@ -0,0 +1,157 @@
|
|
1
|
+
# ADR 002: Context-Aware Error Handling Strategy
|
2
|
+
|
3
|
+
## Status
|
4
|
+
|
5
|
+
Accepted
|
6
|
+
|
7
|
+
## Context
|
8
|
+
|
9
|
+
SimpleCov MCP operates in three distinct contexts, each with different error handling requirements:
|
10
|
+
|
11
|
+
1. **CLI mode**: Human users expect friendly error messages, exit codes, and optional debug traces
|
12
|
+
2. **MCP server mode**: AI agents/clients need structured error responses that don't crash the server
|
13
|
+
3. **Library mode**: Embedding applications need exceptions they can catch and handle programmatically
|
14
|
+
|
15
|
+
Initially, we considered uniform error handling across all modes, but this created poor user experiences:
|
16
|
+
|
17
|
+
- CLI users saw raw exceptions with stack traces (scary and unhelpful)
|
18
|
+
- MCP servers crashed on errors instead of returning error responses
|
19
|
+
- Library users got friendly messages logged to stderr (unwanted side effects in their applications)
|
20
|
+
|
21
|
+
### Key Requirements
|
22
|
+
|
23
|
+
- **CLI**: User-friendly messages, meaningful exit codes, optional stack traces for debugging
|
24
|
+
- **MCP Server**: Logged errors (to file, not stdout), structured JSON-RPC error responses, no server crashes
|
25
|
+
- **Library**: Raise custom exceptions with no logging, allowing consumers to handle errors as needed
|
26
|
+
- **Consistency**: Same underlying error types, but different presentation strategies
|
27
|
+
|
28
|
+
## Decision
|
29
|
+
|
30
|
+
We implemented a **context-aware error handling strategy** using three components:
|
31
|
+
|
32
|
+
### 1. Custom Exception Hierarchy
|
33
|
+
|
34
|
+
All errors inherit from `SimpleCovMcp::Error` (lib/simplecov_mcp/errors.rb) with a `user_friendly_message` method:
|
35
|
+
|
36
|
+
```ruby
|
37
|
+
class Error < StandardError
|
38
|
+
def user_friendly_message
|
39
|
+
message # Can be overridden in subclasses
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
class FileNotFoundError < FileError; end
|
44
|
+
class CoverageDataError < Error; end
|
45
|
+
class ResultsetNotFoundError < CoverageDataError; end
|
46
|
+
# ... etc
|
47
|
+
```
|
48
|
+
|
49
|
+
This provides a unified interface for presenting errors to users while preserving exception types for programmatic handling.
|
50
|
+
|
51
|
+
### 2. ErrorHandler Class
|
52
|
+
|
53
|
+
The `ErrorHandler` class (lib/simplecov_mcp/error_handler.rb:7) provides configurable error handling behavior:
|
54
|
+
|
55
|
+
```ruby
|
56
|
+
class ErrorHandler
|
57
|
+
attr_accessor :error_mode, :logger
|
58
|
+
|
59
|
+
VALID_ERROR_MODES = [:off, :on, :trace].freeze
|
60
|
+
|
61
|
+
def initialize(error_mode: :on, logger: nil)
|
62
|
+
@error_mode = error_mode
|
63
|
+
@logger = logger
|
64
|
+
end
|
65
|
+
|
66
|
+
def handle_error(error, context: nil, reraise: true)
|
67
|
+
log_error(error, context)
|
68
|
+
if reraise
|
69
|
+
raise error.is_a?(SimpleCovMcp::Error) ? error : convert_standard_error(error)
|
70
|
+
end
|
71
|
+
end
|
72
|
+
end
|
73
|
+
```
|
74
|
+
|
75
|
+
The `convert_standard_error` method (lib/simplecov_mcp/error_handler.rb:37) transforms Ruby's standard errors into user-friendly custom exceptions:
|
76
|
+
|
77
|
+
- `Errno::ENOENT` → `FileNotFoundError`
|
78
|
+
- `JSON::ParserError` → `CoverageDataError`
|
79
|
+
- `Errno::EACCES` → `FilePermissionError`
|
80
|
+
|
81
|
+
### 3. ErrorHandlerFactory
|
82
|
+
|
83
|
+
The `ErrorHandlerFactory` (lib/simplecov_mcp/error_handler_factory.rb:4) creates mode-specific handlers:
|
84
|
+
|
85
|
+
```ruby
|
86
|
+
module ErrorHandlerFactory
|
87
|
+
def self.for_cli(error_mode: :on)
|
88
|
+
ErrorHandler.new(error_mode: error_mode)
|
89
|
+
end
|
90
|
+
|
91
|
+
def self.for_library(error_mode: :off)
|
92
|
+
ErrorHandler.new(error_mode: :off) # No logging
|
93
|
+
end
|
94
|
+
|
95
|
+
def self.for_mcp_server(error_mode: :on)
|
96
|
+
ErrorHandler.new(error_mode: :on) # Logs to file
|
97
|
+
end
|
98
|
+
end
|
99
|
+
```
|
100
|
+
|
101
|
+
### Error Flow by Mode
|
102
|
+
|
103
|
+
**CLI Mode** (lib/simplecov_mcp/cli.rb):
|
104
|
+
1. Catches all exceptions in the main run loop
|
105
|
+
2. Uses `for_cli` handler to log errors if debug mode is enabled
|
106
|
+
3. Displays `user_friendly_message` to the user
|
107
|
+
4. Exits with appropriate code (1 for errors, 2 for usage errors)
|
108
|
+
|
109
|
+
**MCP Server Mode** (lib/simplecov_mcp/base_tool.rb:46):
|
110
|
+
1. Each tool wraps execution in a rescue block
|
111
|
+
2. Uses `for_mcp_server` handler to log errors to `~/simplecov_mcp.log`
|
112
|
+
3. Returns structured JSON-RPC error response
|
113
|
+
4. Server continues running (no crashes)
|
114
|
+
|
115
|
+
**Library Mode** (lib/simplecov_mcp.rb:75):
|
116
|
+
1. Uses `for_library` handler with `error_mode: :off` (no logging)
|
117
|
+
2. Raises custom exceptions directly
|
118
|
+
3. Consumers catch and handle `SimpleCovMcp::Error` subclasses
|
119
|
+
|
120
|
+
## Consequences
|
121
|
+
|
122
|
+
### Positive
|
123
|
+
|
124
|
+
1. **Excellent UX**: Each context gets appropriate error handling behavior
|
125
|
+
2. **Robustness**: MCP server never crashes on tool errors
|
126
|
+
3. **Debuggability**: CLI users can enable stack traces with error modes, MCP errors are logged
|
127
|
+
4. **Clean library API**: No unwanted side effects (logging, stderr output) when used as a library
|
128
|
+
5. **Type safety**: Custom exceptions allow programmatic error handling by type
|
129
|
+
|
130
|
+
### Negative
|
131
|
+
|
132
|
+
1. **Complexity**: Three error handling paths to maintain and test
|
133
|
+
2. **Coordination required**: All error types must implement `user_friendly_message` consistently
|
134
|
+
3. **Error conversion overhead**: Standard errors must be converted to custom exceptions
|
135
|
+
|
136
|
+
### Trade-offs
|
137
|
+
|
138
|
+
- **Versus uniform error handling**: More code complexity, but dramatically better UX in each context
|
139
|
+
- **Versus separate error classes per mode**: Single error hierarchy is simpler, factory pattern adds mode-specific behavior
|
140
|
+
|
141
|
+
### Implementation Notes
|
142
|
+
|
143
|
+
The `ErrorHandler.convert_standard_error` method (lib/simplecov_mcp/error_handler.rb:37) uses pattern matching on exception types and error messages to provide helpful, context-aware error messages. This includes:
|
144
|
+
|
145
|
+
- Extracting filenames from system error messages
|
146
|
+
- Detecting SimpleCov-specific error patterns
|
147
|
+
- Providing actionable suggestions ("please run your tests first")
|
148
|
+
|
149
|
+
## References
|
150
|
+
|
151
|
+
- Custom exceptions: `lib/simplecov_mcp/errors.rb`
|
152
|
+
- ErrorHandler implementation: `lib/simplecov_mcp/error_handler.rb:7-124`
|
153
|
+
- ErrorHandlerFactory: `lib/simplecov_mcp/error_handler_factory.rb:4-29`
|
154
|
+
- CLI error handling: `lib/simplecov_mcp/cli.rb` (rescue block in run method)
|
155
|
+
- MCP tool error handling: `lib/simplecov_mcp/base_tool.rb:46-54`
|
156
|
+
- Library mode: `lib/simplecov_mcp.rb:75-86`
|
157
|
+
- Related ADR: [001: Dual-Mode Operation](001-x-arch-decision.md)
|
@@ -0,0 +1,163 @@
|
|
1
|
+
# ADR 003: Coverage Staleness Detection
|
2
|
+
|
3
|
+
## Status
|
4
|
+
|
5
|
+
Accepted
|
6
|
+
|
7
|
+
## Context
|
8
|
+
|
9
|
+
Coverage data can become outdated when source files are modified after tests run. This creates misleading results:
|
10
|
+
|
11
|
+
- Coverage percentages appear lower/higher than reality
|
12
|
+
- Line numbers in coverage reports don't match the current source
|
13
|
+
- AI agents and users may make decisions based on stale data
|
14
|
+
|
15
|
+
We needed a staleness detection system that could:
|
16
|
+
|
17
|
+
1. Detect when source files have been modified since coverage was collected
|
18
|
+
2. Detect when source files have different line counts than coverage data
|
19
|
+
3. Handle edge cases (deleted files, files without trailing newlines)
|
20
|
+
4. Support both file-level and project-level checks
|
21
|
+
5. Allow users to control whether staleness is reported or causes errors
|
22
|
+
|
23
|
+
### Alternative Approaches Considered
|
24
|
+
|
25
|
+
1. **No staleness checking**: Simple, but leads to confusing/incorrect reports
|
26
|
+
2. **Single timestamp check**: Fast, but misses line count mismatches (files edited and reverted)
|
27
|
+
3. **Content hashing**: Accurate, but expensive for large projects
|
28
|
+
4. **Multi-type detection with modes**: More complex, but provides accurate detection with user control
|
29
|
+
|
30
|
+
## Decision
|
31
|
+
|
32
|
+
We implemented a **three-type staleness detection system** with configurable error modes.
|
33
|
+
|
34
|
+
### Three Staleness Types
|
35
|
+
|
36
|
+
The `StalenessChecker` class (lib/simplecov_mcp/staleness_checker.rb:8) detects three distinct types of staleness:
|
37
|
+
|
38
|
+
1. **Type 'M' (Missing)**: The source file exists in coverage but is now deleted/missing
|
39
|
+
- Returned by `stale_for_file?` when `File.file?(file_abs)` returns false
|
40
|
+
- Example: File was deleted after tests ran
|
41
|
+
|
42
|
+
2. **Type 'T' (Timestamp)**: The source file's mtime is newer than the coverage timestamp
|
43
|
+
- Detected by comparing `File.mtime(file_abs)` with coverage timestamp
|
44
|
+
- Example: File was edited after tests ran
|
45
|
+
|
46
|
+
3. **Type 'L' (Length)**: The source file line count doesn't match the coverage lines array length
|
47
|
+
- Detected by comparing `File.foreach(path).count` with `coverage_lines.length`
|
48
|
+
- Handles edge case: Files without trailing newlines (adjusts count by 1)
|
49
|
+
- Example: Lines were added/removed without changing mtime (rare but possible with version control)
|
50
|
+
|
51
|
+
### Implementation Details
|
52
|
+
|
53
|
+
The core algorithm is in `compute_file_staleness_details` (lib/simplecov_mcp/staleness_checker.rb:137):
|
54
|
+
|
55
|
+
```ruby
|
56
|
+
def compute_file_staleness_details(file_abs, coverage_lines)
|
57
|
+
coverage_ts = coverage_timestamp
|
58
|
+
exists = File.file?(file_abs)
|
59
|
+
file_mtime = exists ? File.mtime(file_abs) : nil
|
60
|
+
|
61
|
+
cov_len = coverage_lines.respond_to?(:length) ? coverage_lines.length : 0
|
62
|
+
src_len = exists ? safe_count_lines(file_abs) : 0
|
63
|
+
|
64
|
+
newer = !!(file_mtime && file_mtime.to_i > coverage_ts.to_i)
|
65
|
+
|
66
|
+
# Adjust for missing trailing newline edge case
|
67
|
+
adjusted_src_len = src_len
|
68
|
+
if exists && cov_len.positive? && src_len == cov_len + 1 && missing_trailing_newline?(file_abs)
|
69
|
+
adjusted_src_len -= 1
|
70
|
+
end
|
71
|
+
|
72
|
+
len_mismatch = (cov_len.positive? && adjusted_src_len != cov_len)
|
73
|
+
newer &&= !len_mismatch # Prioritize length mismatch over timestamp
|
74
|
+
|
75
|
+
{
|
76
|
+
exists: exists,
|
77
|
+
file_mtime: file_mtime,
|
78
|
+
coverage_timestamp: coverage_ts,
|
79
|
+
cov_len: cov_len,
|
80
|
+
src_len: src_len,
|
81
|
+
newer: newer,
|
82
|
+
len_mismatch: len_mismatch
|
83
|
+
}
|
84
|
+
end
|
85
|
+
```
|
86
|
+
|
87
|
+
### Staleness Modes
|
88
|
+
|
89
|
+
The checker supports two modes (lib/simplecov_mcp/staleness_checker.rb:9):
|
90
|
+
|
91
|
+
- **`:off`** (default): Staleness is detected but only reported in responses, never raises errors
|
92
|
+
- **`:error`**: Staleness raises `CoverageDataStaleError` or `CoverageDataProjectStaleError`
|
93
|
+
|
94
|
+
This allows:
|
95
|
+
- Interactive tools to show warnings without crashing
|
96
|
+
- CI systems to fail builds on stale coverage
|
97
|
+
- AI agents to decide how to handle staleness based on their goals
|
98
|
+
|
99
|
+
### File-Level vs Project-Level Checks
|
100
|
+
|
101
|
+
**File-level** (`check_file!` and `stale_for_file?`, lib/simplecov_mcp/staleness_checker.rb:25,49):
|
102
|
+
- Checks a single file's staleness
|
103
|
+
- Returns `false` or staleness type character ('M', 'T', 'L')
|
104
|
+
- Used by single-file tools (summary, detailed, uncovered)
|
105
|
+
|
106
|
+
**Project-level** (`check_project!`, lib/simplecov_mcp/staleness_checker.rb:59):
|
107
|
+
- Checks all covered files plus optionally tracked files
|
108
|
+
- Detects:
|
109
|
+
- Files newer than coverage timestamp
|
110
|
+
- Files deleted since coverage was collected
|
111
|
+
- Tracked files missing from coverage (newly added files)
|
112
|
+
- Raises `CoverageDataProjectStaleError` with lists of problematic files
|
113
|
+
- Used by `all_files_coverage_tool` and `coverage_table_tool`
|
114
|
+
|
115
|
+
### Tracked Globs Feature
|
116
|
+
|
117
|
+
The project-level check supports `tracked_globs` parameter to detect newly added files:
|
118
|
+
|
119
|
+
```ruby
|
120
|
+
# Detects if lib/**/*.rb files exist that have no coverage data
|
121
|
+
checker.check_project!(coverage_map) # with tracked_globs: ['lib/**/*.rb']
|
122
|
+
```
|
123
|
+
|
124
|
+
This helps teams ensure new files are included in test runs.
|
125
|
+
|
126
|
+
## Consequences
|
127
|
+
|
128
|
+
### Positive
|
129
|
+
|
130
|
+
1. **Accurate detection**: Three types catch different staleness scenarios comprehensively
|
131
|
+
2. **Edge case handling**: Missing trailing newlines handled correctly
|
132
|
+
3. **User control**: Modes allow errors or warnings based on use case
|
133
|
+
4. **Detailed information**: Staleness errors include specific file lists and timestamps
|
134
|
+
5. **Project awareness**: Can detect newly added files that lack coverage
|
135
|
+
|
136
|
+
### Negative
|
137
|
+
|
138
|
+
1. **Complexity**: Three staleness types are harder to understand than a single timestamp check
|
139
|
+
2. **Performance**: Line counting and mtime checks for every file add overhead
|
140
|
+
3. **Maintenance burden**: Edge case logic (trailing newlines) requires careful testing
|
141
|
+
4. **Ambiguity**: When multiple staleness types apply, prioritization logic (length > timestamp) may surprise users
|
142
|
+
|
143
|
+
### Trade-offs
|
144
|
+
|
145
|
+
- **Versus timestamp-only**: More accurate but slower and more complex
|
146
|
+
- **Versus content hashing**: Fast enough for most projects, but can't detect "edit then revert" scenarios
|
147
|
+
- **Versus no checking**: Essential for reliable coverage reporting, worth the complexity
|
148
|
+
|
149
|
+
### Edge Cases Handled
|
150
|
+
|
151
|
+
1. **Missing trailing newline**: Files without `\n` at EOF have `line_count == coverage_length + 1`, checker adjusts for this
|
152
|
+
2. **Deleted files**: Appear as 'M' (missing) type staleness
|
153
|
+
3. **Empty files**: `cov_len.positive?` guard prevents false positives
|
154
|
+
4. **No coverage timestamp**: Defaults to 0, effectively disabling timestamp checks
|
155
|
+
|
156
|
+
## References
|
157
|
+
|
158
|
+
- Implementation: `lib/simplecov_mcp/staleness_checker.rb:8-168`
|
159
|
+
- File-level checking: `lib/simplecov_mcp/staleness_checker.rb:25-55`
|
160
|
+
- Project-level checking: `lib/simplecov_mcp/staleness_checker.rb:59-95`
|
161
|
+
- Staleness detail computation: `lib/simplecov_mcp/staleness_checker.rb:137-166`
|
162
|
+
- Error types: `lib/simplecov_mcp/errors.rb` (CoverageDataStaleError, CoverageDataProjectStaleError)
|
163
|
+
- Usage in tools: `lib/simplecov_mcp/tools/all_files_coverage_tool.rb`, `lib/simplecov_mcp/model.rb`
|