coatepec 0.7.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop.yml +5 -0
  3. data/CHANGELOG.md +140 -0
  4. data/README.md +153 -50
  5. data/ROADMAP.md +31 -64
  6. data/lib/coatepec/introspection/bounded_options.rb +22 -0
  7. data/lib/coatepec/introspection/controller.rb +19 -28
  8. data/lib/coatepec/introspection/model.rb +33 -43
  9. data/lib/coatepec/introspection/model_resolver.rb +32 -0
  10. data/lib/coatepec/introspection/route_entries.rb +77 -0
  11. data/lib/coatepec/introspection/routes.rb +70 -20
  12. data/lib/coatepec/mcp/defaults.rb +25 -0
  13. data/lib/coatepec/mcp/response.rb +16 -9
  14. data/lib/coatepec/mcp/tools.rb +100 -93
  15. data/lib/coatepec/mcp.rb +3 -2
  16. data/lib/coatepec/project.rb +11 -1
  17. data/lib/coatepec/project_config.rb +40 -1
  18. data/lib/coatepec/spec/failure_collapser.rb +111 -0
  19. data/lib/coatepec/spec/flaky_checker.rb +3 -1
  20. data/lib/coatepec/spec/fork_strategy.rb +5 -12
  21. data/lib/coatepec/spec/guarded_fork_strategy.rb +20 -18
  22. data/lib/coatepec/spec/path_policy.rb +45 -12
  23. data/lib/coatepec/spec/process_strategy.rb +28 -17
  24. data/lib/coatepec/spec/result.rb +61 -19
  25. data/lib/coatepec/spec/rspec_adapter.rb +58 -0
  26. data/lib/coatepec/spec/runner.rb +33 -27
  27. data/lib/coatepec/spec/spawn_strategy.rb +9 -8
  28. data/lib/coatepec/test_unit/adapter.rb +132 -0
  29. data/lib/coatepec/test_unit/child_entry.rb +33 -0
  30. data/lib/coatepec/test_unit/json_reporter.rb +107 -0
  31. data/lib/coatepec/test_unit/line_filtering.rb +46 -0
  32. data/lib/coatepec/version.rb +1 -1
  33. data/lib/coatepec/worker/rails_runtime.rb +10 -1
  34. data/lib/coatepec/worker/server.rb +9 -2
  35. data/lib/coatepec/worker_manager.rb +8 -6
  36. data/lib/coatepec.rb +5 -0
  37. metadata +17 -5
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9e0e730ee65e046252297e502039ca8db8b17b4d1774564f79cb708dbe4500b2
4
- data.tar.gz: 66d1afbd32ef8ba95de436e046d5b8b90452578a5a1ee0af2a442f7a14f0054d
3
+ metadata.gz: 72a7dae3c61c36718da0a815ac037c9a50b1a2ae76e282e67d6533131d21c28f
4
+ data.tar.gz: 99c4c1bd193cffe81e49d711f6bedfebd5cb1552ab91f4455ab7b4d31fbabda5
5
5
  SHA512:
6
- metadata.gz: b5168b16140e5199c41a11c4e9a73efee643d2acf80405090a7b1ff0ba32ab94432b1e02b396ca5456ba879960025ef640434fcc265ca19147633e00d0b47a88
7
- data.tar.gz: 209b126ecc16dadfacad123098e06bcb9d53649f7f3a8b2eca7555a6d098d35f7307cc81b38b015ea0891b49ef088bbe55c3f33ea2de243e14e0212322e68ded
6
+ metadata.gz: 845750211ab96287bccd73b1108451357453925cd0113e92d4251dacca836437783cf78288fa8ca63de5fdbaa772e1ed1f1fc8ce2afbcb346ff9e2f55c1be85a
7
+ data.tar.gz: '04698525bb87f82b589a6d50a61a18edbbd4b8e1520fcdf41d3c9e682b75af664480e4dd5adc2cea54be1f8c6afbe71defc4d8157949e954e90c63d60d9e935c'
data/.rubocop.yml CHANGED
@@ -15,6 +15,11 @@ Style/StringLiterals:
15
15
  Style/StringLiteralsInInterpolation:
16
16
  EnforcedStyle: double_quotes
17
17
 
18
+ Metrics/ParameterLists:
19
+ # Keyword-only signatures here mirror the MCP tools' input schemas, and a tool
20
+ # gaining an input shouldn't force a refactor; positional lists stay capped at 5.
21
+ CountKeywordArgs: false
22
+
18
23
  Metrics/BlockLength:
19
24
  # RSpec's describe/context/it DSL naturally produces long blocks; this is the
20
25
  # standard exclusion used by rubocop-rspec's own default config. inherit_mode
data/CHANGELOG.md CHANGED
@@ -1,5 +1,145 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.9.0
4
+
5
+ - `rails_spec_run` results always carry `execution_mode`: `fork` (Linux, or
6
+ a macOS guarded fork that ran), `spawn` (macOS with `macos_fork: false`),
7
+ `spawn_fallback` (the guard declined to fork) or `spawn_after_crash` (the
8
+ forked child crashed and the run was retried). The key used to appear
9
+ only when the guarded fork was in play.
10
+ - Behaviour change on macOS: projects that never set `macos_fork: true` now
11
+ fork the warm worker by default, the way it always has on Linux, using
12
+ the guarded fork shipped in 0.7.0 (thread-count and gem-denylist checks,
13
+ a spawn retry if the child crashes). Set `macos_fork: false` in
14
+ `.coatepec.yml` to keep a fresh spawn per call. Measured on a Rails
15
+ 8.2.0.alpha app: 966 ms saved per run, no fallbacks across nine runs.
16
+ `OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES` is now a recommendation for
17
+ projects that see `spawn_after_crash`, not a requirement.
18
+ - `.coatepec.yml` gains a `defaults` map: `spec_run.include_passing`,
19
+ `spec_run.include_stdout`, `spec_run.timeout_seconds` and `routes.engines`
20
+ set project-wide defaults for those inputs. A call argument always wins;
21
+ an omitted key means the built-in default; an unknown key or an
22
+ out-of-range value fails the call with `invalid_config` naming it. The
23
+ file is re-read on every call.
24
+ - `rails_runtime_status` reports `spec_strategy` (`fork`, `guarded_fork` or
25
+ `spawn`), `fallbacks` (how many guarded-fork runs fell back to spawn in
26
+ this worker's lifetime; `null` unless the guarded fork is in use) and
27
+ `defaults` (the effective `rails_spec_run` and `rails_routes` defaults
28
+ after `.coatepec.yml`), so an agent can see every default in one call.
29
+ `fallbacks` also counts `rails_spec_flaky_check` rounds, which run through
30
+ the same strategy.
31
+
32
+ ## 0.8.0
33
+
34
+ - Tool responses are now compact JSON rather than pretty-printed (22-32%
35
+ smaller on measured payloads); set `COATEPEC_PRETTY=1` on the server
36
+ process to restore indentation.
37
+ - `rails_spec_run` now omits passing examples from `examples` by default --
38
+ a 49-test green run drops from ~3,700 tokens to ~160 -- and `summary` gains
39
+ `pending_count`. Pass the new `include_passing: true` input to get the full
40
+ roster back (still capped at 500 examples). `rails_spec_flaky_check` is
41
+ unaffected: it still sees every example.
42
+ - `examples[].description` (and `flaky_examples[].description`) is omitted
43
+ when it would only repeat `id`, which is always the case for Minitest.
44
+ - `rails_routes` (with `engines: "include"`) and `rails_controller` now
45
+ include the routes of engines mounted in the application, expanded one
46
+ level deep (an engine mounted inside another engine stays an opaque mount
47
+ route, the same boundary `bin/rails routes` draws). Paths carry the mount
48
+ point, so `/widget_admin/audits` is what both tools report, and a
49
+ controller living inside an engine no longer has all of its actions listed
50
+ under `unroutable_actions`.
51
+ - `rails_routes` gains an `engine` column and `rails_controller`'s
52
+ `actions[].routes` entries an `engine` field: `null` for an application
53
+ route, the engine's class name otherwise. `rails_routes`' `query` matches
54
+ against it like every other column, so `query: "Avo::Engine"` with
55
+ `engines: "include"` or `"only"` returns exactly that engine's routes.
56
+ Application routes are listed before engine routes.
57
+ - An engine route's `name` (`route_name` in `rails_controller`) is relative
58
+ to its engine: it is reached through the mount's helper,
59
+ `<mount name>.<name>_path`, where the mount name is the `name` of the mount
60
+ route itself -- never as a top-level url helper.
61
+ - `rails_routes` and `rails_controller` now omit routes Rails marks
62
+ `internal` -- its own `/rails/info` and friends -- matching
63
+ `bin/rails routes`. This is a behaviour change for callers that relied on
64
+ seeing them.
65
+ - Add Minitest support to `rails_spec_run` and `rails_spec_flaky_check`:
66
+ selectors under a `test/` root ending in `_test.rb` run through Rails'
67
+ Minitest runner in the same warm worker, with the same
68
+ `example`/`seed`/`fail_fast`/`timeout_seconds` inputs and the same result
69
+ shape (`id` is `ClassName#test_method`; a skip is `"pending"`, an error is
70
+ `"failed"`). The framework is chosen per call from the selector paths --
71
+ an app that has both `spec/` and `test/` works without configuration; one
72
+ call may not mix the two (`mixed_test_frameworks`).
73
+ - The Minitest child sets `PARALLEL_WORKERS=1`, so a selection above Rails'
74
+ parallelization threshold runs serially under the call's single timeout
75
+ instead of forking a worker tree.
76
+ - `file:LINE` selection is implemented by Coatepec itself rather than relying
77
+ on Rails' line filtering, which is absent in apps generated with
78
+ `--skip-test` and targets a method Minitest 6 no longer calls on Rails 7.1.
79
+ - Path validation now runs before the framework is required, so an invalid
80
+ path on a Minitest-only app reports `invalid_spec_path` instead of
81
+ `unsupported_test_framework`. `invalid_spec_path` keeps its code for both
82
+ frameworks.
83
+ - `rails_model` collapses validators with identical name, attributes and
84
+ options into one entry (a concern and the model body declaring the same
85
+ validation used to appear twice); the 200-item cap now counts distinct
86
+ validators.
87
+ - `rails_routes` defaults to 100 routes per page (was 50 -- engine expansion
88
+ roughly doubled route counts when engines are included) and returns
89
+ `next_offset` for the follow-up call, `null` on the last page.
90
+ - `rails_spec_run`: when several tests fail with the same error text (a
91
+ broken layout erroring every controller test, say), `stdout` keeps the
92
+ first failure block and replaces each repeat with one roll-up line
93
+ naming the other tests -- a 5-error controller run drops from ~4,850 B to
94
+ roughly a third of that. Backtrace frames and RSpec's `Failure/Error:`
95
+ source line are ignored when deciding that two blocks match.
96
+ - `rails_spec_run` gains `include_stdout` (`failures`, the default; `always`;
97
+ `never`). A passing run's `stdout` is `null` by default -- its progress
98
+ dots and summary line only repeat `summary` -- and a failing run's is
99
+ returned; `always` and `never` override that either way. The key stays so
100
+ the result shape is uniform. `stderr` is always returned.
101
+ - `rails_routes` returns application routes only by default and gains
102
+ `engines` to change that: `include` lists the routes of mounted engines
103
+ too -- the routes 0.8.0 added by expanding engines -- and `only` lists just
104
+ those; the default matches pre-0.8.0 output (application routes, the mount
105
+ route included). Every response carries `engines` (the filter applied) and
106
+ `engines_excluded` (how many routes matching `query` were withheld), so the
107
+ omission is stated, never silent. The filter applies after `query` and
108
+ before paging, so `matched` and `next_offset` describe the kept set; an
109
+ engine's mount route counts as an application route. Measured on an app
110
+ with a mounted admin engine, six typical route queries cost 57% less
111
+ context with engine routes withheld.
112
+ - `rails_model` cuts an array-valued validator option longer than 20 entries
113
+ to its first 20 and adds `<option>_count` (the full length) and
114
+ `<option>_truncated: true` beside it -- a 249-code `inclusion` list no
115
+ longer costs 1.4 KB per model. Shorter lists are unchanged and carry no
116
+ sibling keys.
117
+ - Every tool response's `meta` is now just `duration_ms`; `project_root` is
118
+ reported once by `rails_runtime_status` (beside `environment`) instead of
119
+ on every call.
120
+ - `rails_spec_run` results carry `child_pid`, `signaled`, `termsig`,
121
+ `stopsig` and `coredump` only when the child did not exit normally (a
122
+ timeout kill or a crash); a normal run reports `status`, `exit_code` and
123
+ the output fields alongside the usual `summary` and `examples`.
124
+ - `rails_model` returns `name`, `table_name`, `primary_key`, `abstract_class`
125
+ and `counts` (the size of each of its four lists, after validator
126
+ de-duplication and the 200-item caps) by default, and gains `fields`, an
127
+ array of `columns`/`associations`/`validators`/`enums` naming the lists to
128
+ include; omitted or `[]` means counts only. Callers that read the lists
129
+ must now ask for them -- the default response is roughly an order of
130
+ magnitude smaller than the full one on a typical model.
131
+ - `rails_routes` returns `columns` (`name`, `verb`, `path`, `controller`,
132
+ `action`, `engine`) and `rows` instead of one object per route -- the six
133
+ key names were a third of every item's bytes -- and both `rails_routes`
134
+ and `rails_controller` report paths without the `(.:format)` suffix Rails
135
+ appends to most routes.
136
+ - `rails_spec_run`'s `summary` gains `error_count` (how many of
137
+ `failure_count` were errors rather than assertion failures) and
138
+ `assertion_count` (Minitest's assertion total), so a green run answers
139
+ "how many assertions ran?" without `stdout` and a failing run's counts
140
+ match the `0 failures, 5 errors` line beside them. Both are `null` for
141
+ RSpec, which reports neither.
142
+
3
143
  ## 0.7.0
4
144
 
5
145
  - Add `rails_controller`: reports an `ActionController` controller's
data/README.md CHANGED
@@ -57,9 +57,9 @@ MCP client
57
57
  v
58
58
  Coatepec parent (Rails-free)
59
59
  `-- private NDJSON --> test worker (Rails "test", booted lazily, kept warm)
60
- |-- Linux: Process.fork --> isolated RSpec child
61
- `-- macOS: Process.spawn --> fresh RSpec process (default)
62
- Process.fork, guarded --> opt-in, see Configuration
60
+ |-- Linux: Process.fork --> isolated RSpec/Minitest child
61
+ `-- macOS: Process.fork, guarded --> isolated RSpec/Minitest child (default)
62
+ Process.spawn --> fresh process, see Configuration (macos_fork: false)
63
63
  ```
64
64
 
65
65
  ## Configuration
@@ -68,23 +68,43 @@ An optional `.coatepec.yml` at the target Rails app's root enables
68
68
  per-project settings:
69
69
 
70
70
  ```yaml
71
- macos_fork: true # opt into forking on macOS (see below)
71
+ macos_fork: false # opt out of forking on macOS (default true; see below)
72
72
  macos_fork_unsafe_gems: [some_gem] # extends the built-in fork-unsafe denylist
73
+
74
+ defaults:
75
+ spec_run:
76
+ include_passing: true # true or false (default false)
77
+ include_stdout: always # failures (default), always, never
78
+ timeout_seconds: 300 # 1..900 (default 120)
79
+ routes:
80
+ engines: include # include, exclude (default), only
73
81
  ```
74
82
 
75
83
  A missing file means every setting takes its default -- this file is never
76
84
  required.
77
85
 
78
- ### macOS fork (experimental, opt-in)
86
+ `defaults` sets project-wide values for those four inputs; a call argument
87
+ always wins, an omitted key falls back to the built-in default, and an
88
+ unknown key or value fails the call with `invalid_config` naming it. The
89
+ file is re-read on every call, so edits apply immediately.
90
+ `defaults.spec_run` applies to `rails_spec_run` only; `rails_spec_flaky_check`
91
+ keeps its own per-round budget.
92
+
93
+ Tool responses are compact JSON -- no indentation, nothing downstream reads
94
+ it. Set `COATEPEC_PRETTY=1` in the MCP server's `env` (the same place as
95
+ `OBJC_DISABLE_INITIALIZE_FORK_SAFETY` in the JSON example below) to
96
+ pretty-print them when you are reading the sidecar by hand.
97
+
98
+ ### macOS fork (default since 0.9.0)
79
99
 
80
- On macOS, `rails_spec_run` normally spawns a fresh `bundle exec rspec`
81
- process per call, re-booting Rails every time -- the warm-worker speedup
82
- described above only applies on Linux by default. Setting `macos_fork:
83
- true` lets Coatepec attempt `Process.fork` on macOS too, reusing the warm
84
- boot the way Linux does.
100
+ On macOS, `rails_spec_run` forks the warm worker for each call, reusing the
101
+ boot the way it always has on Linux. Setting `macos_fork: false` in
102
+ `.coatepec.yml` opts the project out: every call then spawns a fresh
103
+ `bundle exec rspec` process and re-boots Rails, so only the Linux lane
104
+ gets the warm-worker speedup described above.
85
105
 
86
- This is opt-in because forking a process with native extensions loaded
87
- isn't universally safe. Before each fork, Coatepec checks the worker's
106
+ The fork is guarded, because forking a process with native extensions
107
+ loaded isn't universally safe. Before each fork, Coatepec checks the worker's
88
108
  live thread count against its post-boot baseline and its loaded gems
89
109
  against a denylist, falling back to a fresh spawn for that one call if
90
110
  either check looks risky. The built-in denylist ships empty -- no single
@@ -93,17 +113,17 @@ thread-count-only until a project adds its own
93
113
  `macos_fork_unsafe_gems`. If a fork is attempted and the child crashes
94
114
  anyway, Coatepec transparently retries via spawn and returns that result
95
115
  -- fork stays enabled for later calls. Every `rails_spec_run` result
96
- includes an `execution_mode` field (`fork`, `spawn_fallback`, or
97
- `spawn_after_crash`) so you can see which path actually ran for a given
116
+ includes an `execution_mode` field (`fork`, `spawn`, `spawn_fallback`,
117
+ or `spawn_after_crash`) so you can see which path actually ran for a given
98
118
  call; `spawn_after_crash` results also carry the crashed fork's own stderr
99
119
  under `crashed_fork_stderr` so the crash can be diagnosed.
100
120
 
101
- `macos_fork: true` also effectively requires
102
- `OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES` in Coatepec's own environment.
103
- Without it, a forked child that touches an Objective-C-initialized class
104
- aborts -- Coatepec retries via spawn, so it degrades silently to the slow
105
- path (no crash, no error surfaced) rather than failing loudly, and you
106
- simply never get the speedup. Set it on the MCP server process itself:
121
+ `OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES` in Coatepec's own environment is
122
+ recommended if you ever see `spawn_after_crash`. Without it, a forked child
123
+ that touches an Objective-C-initialized class aborts -- Coatepec retries via
124
+ spawn, so it degrades to the slow path for that call (no crash, no error
125
+ surfaced) rather than failing loudly, and you simply never get the speedup.
126
+ Set it on the MCP server process itself:
107
127
 
108
128
  ```json
109
129
  {
@@ -125,34 +145,78 @@ a no-op.
125
145
 
126
146
  | Tool | Input | Notes |
127
147
  |---|---|---|
128
- | `rails_spec_run` | `paths: string[1..100]`, `example?`, `seed?`, `fail_fast?`, `timeout_seconds?` (1..900, default 120) | Isolated per run; output capped at 256 KiB per stream |
129
- | `rails_runtime_status` | `{}` | Reports Ruby/Rails versions, worker PID, boot_id, lifecycle state |
130
- | `rails_runtime_restart` | `{}` | Unconditionally respawns the worker, discarding its warm boot |
131
- | `rails_routes` | `query?`, `limit?` (1..200, default 50), `offset?` | Case-insensitive filter across name/verb/path/controller/action |
132
- | `rails_model` | `name` (constant path, e.g. `Widget` or `Admin::Widget`) | ActiveRecord models only; columns, associations, validators, enums -- no row data |
148
+ | `rails_spec_run` | `paths: string[1..100]`, `example?`, `seed?`, `fail_fast?`, `timeout_seconds?` (1..900, default 120), `include_passing?`, `include_stdout?` (`failures` default, `always`, `never`) (all three default-able in `.coatepec.yml`) | RSpec (`spec/**/*_spec.rb`) or Minitest (`test/**/*_test.rb`), chosen from the paths; isolated per run; output capped at 256 KiB per stream; failure blocks in `stdout` that repeat an earlier block's error verbatim are replaced by a roll-up line naming the tests; returns `summary` counts (`example_count`, `failure_count`, `error_count`, `pending_count`, `assertion_count`, `duration`; the error and assertion counts come from Minitest and are `null` for RSpec) plus only the failed/pending `examples`, pass `include_passing: true` for the full roster (still capped at 500 examples); `stdout` is returned only for failing runs by default; `include_stdout: "always"` keeps it for green runs, `"never"` drops it always (the key stays, `null`); `stderr` is always returned; `child_pid`/`signaled`/`termsig`/`stopsig`/`coredump` appear only when the child did not exit normally (a timeout kill or a crash); every result carries `execution_mode` (`fork`, `spawn`, `spawn_fallback`, `spawn_after_crash`) |
149
+ | `rails_runtime_status` | `{}` | Reports Ruby/Rails versions, worker PID, boot_id, lifecycle state, the project root, `spec_strategy` (`fork`, `guarded_fork`, `spawn`), `fallbacks` (guarded-fork runs that fell back to spawn; `null` unless guarded) and `defaults` (the effective `rails_spec_run`/`rails_routes` defaults after `.coatepec.yml`) |
150
+ | `rails_runtime_restart` | `{}` | Unconditionally respawns the worker, discarding its warm boot; returns the fresh worker's status, including `spec_strategy` and `fallbacks` |
151
+ | `rails_routes` | `query?`, `limit?` (1..200, default 100), `offset?`, `engines?` (`exclude` default, `include`, `only`) (default-able in `.coatepec.yml`) | Returns `columns` (`name`, `verb`, `path`, `controller`, `action`, `engine`) and `rows` in that order; paths omit the `(.:format)` suffix Rails appends. Case-insensitive filter across name/verb/path/controller/action/engine. Application routes only by default; `engines: "include"` adds the routes of mounted engines (one level deep, paths prefixed with the mount point, `engine` naming the engine class; `null` for an application route), `"only"` returns just those. Every response carries `engines` (the filter applied) and `engines_excluded` (how many routes matching `query` the filter withheld), so nothing is hidden silently. The filter applies after `query` and before paging, so `matched`/`next_offset` describe the kept set. The engine's mount route counts as an application route. Routes Rails marks `internal` are omitted, like `bin/rails routes`. `next_offset` is the offset of the next page, or `null` on the last one |
152
+ | `rails_model` | `name` (constant path, e.g. `Widget` or `Admin::Widget`), `fields?` (any of `columns`, `associations`, `validators`, `enums`; omitted or `[]` means none) | Returns `name`, `table_name`, `primary_key`, `abstract_class` and `counts` (the size of each of the four lists, each capped at 200) by default and only the lists named in `fields` -- so a how-many question costs a few dozen bytes and `fields: ["columns"]` asks for the one list you need; ActiveRecord models only; columns, associations, validators, enums -- no row data; validators are de-duplicated by class, attributes and options (a concern and the model body declaring the same validation count once), so the count can be lower than `klass.validators.size`; an array-valued validator option longer than 20 entries (a country-code `inclusion` list, say) is cut to its first 20 with `<option>_count` and `<option>_truncated: true` beside it |
133
153
  | `rails_controller` | `name` (constant path, e.g. `WidgetsController` or `Admin::ReportsController`) | Actions, action callbacks, concerns, and the routes reaching each action -- no request dispatch |
134
- | `rails_spec_flaky_check` | `paths`, `example?`, `timeout_seconds?` (per round, 1..900), `runs?` (2..20, default 5) | Runs the selection `runs` times with a fresh random seed each round; reports examples whose status was inconsistent across runs |
154
+ | `rails_spec_flaky_check` | `paths`, `example?`, `timeout_seconds?` (per round, 1..900), `runs?` (2..20, default 5) | Runs the selection `runs` times with a fresh random seed each round; reports tests whose status was inconsistent across runs; RSpec or Minitest, chosen from the paths |
135
155
 
136
156
  ### Example queries
137
157
 
138
158
  `rails_routes`:
139
159
 
140
- - "What are all the routes in this app?" -- `rails_routes()`
160
+ - "What are all the routes in this app?" -- `rails_routes()`, the
161
+ application's own; add `engines: "include"` for the mounted engines' too.
141
162
  - "What's the URL for widgets?" -- `rails_routes(query: "widget")`. `query` is a
142
163
  case-insensitive substring match across name, verb, path, controller, *and*
143
164
  action -- not just the path -- so a resource name alone typically returns
144
165
  every route for that resource (index/create/new/...); narrow further with
145
- something like `query: "new_widget"` to hit one route by name.
166
+ something like `query: "new_widget"` to hit one route by name. The call
167
+ already leaves out the admin engine's scaffolding, which is often two thirds
168
+ of every match; the response's `engines_excluded` says how many engine routes
169
+ it held back. Pass `engines: "include"` to see them too, or
170
+ `engines: "only"` for just the engine's.
146
171
  - "Which routes accept POST?" -- `rails_routes(query: "POST")`, the same
147
172
  substring match applied to the verb column.
173
+ - "Which routes does the Avo engine add?" --
174
+ `rails_routes(query: "Avo::Engine", engines: "only")` -- the substring match
175
+ covers the `engine` column too, so an engine's class name returns exactly the
176
+ routes mounted from it. Application routes always sort before engine routes,
177
+ so an unfiltered listing (with `engines: "include"`) reads the way
178
+ `bin/rails routes` does.
179
+ - `engines_excluded` is a correctness signal, not only a saving. Engine
180
+ routes print their paths relative to the mount point, so
181
+ `bin/rails routes -g admin` cannot find an admin engine's routes at all: it
182
+ matches the mount line and answers "one admin route" with no hint that
183
+ anything is missing. `rails_routes(query: "admin")` returns that one row
184
+ plus `engines_excluded: 243`, so the reader knows the rest exists.
185
+ - Opting in is the expensive path, and it paginates. A broad query with
186
+ `engines: "include"` on an app with a mounted admin engine costs several
187
+ times the default response and can still stop at the 100-row page of a
188
+ larger match (`next_offset` says so), where the default returned the
189
+ application's routes complete. Reach for `engines: "only"` with the
190
+ engine's class name when the engine's routes are the question.
191
+ - An engine route's `name` is relative to its engine, not a top-level url
192
+ helper: the row `["audits", "GET", "/widget_admin/audits",
193
+ "widget_admin/audits", "index", "WidgetAdmin::Engine"]` is reached as
194
+ `widget_admin.audits_path` -- `<mount name>.<name>_path`, where the mount
195
+ name is the `name` of the mount route itself -- and `audits_path` alone
196
+ does not exist on the application.
197
+ - An engine mounted at two paths is listed under both mount points, once per
198
+ prefixed path, whenever engine routes are in scope, since each of those
199
+ paths is a real, reachable URL.
200
+ `bin/rails routes` keys engines by endpoint and so lists such an engine
201
+ only once.
202
+ - Paging: when `next_offset` is not `null`, call again with
203
+ `offset: next_offset` to get the next page.
148
204
 
149
205
  `rails_model`:
150
206
 
151
207
  - "What columns does Widget have, and which are nullable?" --
152
- `rails_model(name: "Widget")` -- see `columns[].null`, `columns[].sql_type`,
153
- `columns[].default`.
154
- - "What validations and associations does Widget enforce?" -- same call --
208
+ `rails_model(name: "Widget", fields: ["columns"])` -- see `columns[].null`,
209
+ `columns[].sql_type`, `columns[].default`.
210
+ - "What validations and associations does Widget enforce?" --
211
+ `rails_model(name: "Widget", fields: ["validators", "associations"])` --
155
212
  see `validators` and `associations`.
213
+ - "How many columns, associations, validators and enums does Event have?" --
214
+ `rails_model(name: "Event")` -- see `counts`; ask for the lists with
215
+ `fields` when you need them.
216
+ - A long allow-list is summarised, not enumerated: `validates :country_code,
217
+ inclusion: { in: ISO_CODES }` with 249 codes comes back as `"in"` holding
218
+ the first 20, `"in_count": 249` and `"in_truncated": true`. A list of 20 or
219
+ fewer has no `_count`/`_truncated` siblings.
156
220
  - "What happens if I ask about a non-model class, like a controller?" --
157
221
  `rails_model(name: "ApplicationController")` raises `not_active_record_model`
158
222
  rather than introspecting it (a nonexistent constant raises `model_not_found`
@@ -162,9 +226,10 @@ a no-op.
162
226
 
163
227
  - "What actions does WidgetsController define, and what routes reach them?" --
164
228
  `rails_controller(name: "WidgetsController")` -- see `actions[].routes`,
165
- each with `verb`, `path` (Rails' raw route spec, `(.:format)` suffix
166
- included -- byte-identical to the same route's `path` from `rails_routes`),
167
- and `route_name`.
229
+ each with `verb`, `path` (without the `(.:format)` suffix -- byte-identical
230
+ to the same route's `path` from `rails_routes`), `route_name`, and `engine`
231
+ (`null` for an application route, the engine's class name for a route that
232
+ reaches this controller through a mount).
168
233
  - "Does this controller have dead code, or a route that will 500?" -- same
169
234
  call -- see `unroutable_actions` (action methods no route reaches --
170
235
  probably dead code) and `routes_without_action` (action names the route
@@ -185,6 +250,12 @@ a no-op.
185
250
  `concerns`: app-defined modules only, whether included directly or
186
251
  inherited from a base class; framework modules (`ActionController::Base`
187
252
  and everything above it in the ancestor chain) are excluded.
253
+ - Unlike `concerns`, `callbacks[]` is *not* filtered to app code: it is the
254
+ controller's full callback chain, so callbacks Rails itself installs show
255
+ up too -- `verify_authenticity_token` / `verify_same_origin_request` from
256
+ the default forgery protection, and a `"(block)"` entry for macros like
257
+ `allow_browser` that register a Proc. Read the list as "everything that
258
+ runs around an action", not "everything this app wrote".
188
259
  - `rails_controller` admits `ActionController::API` controllers as well as
189
260
  `ActionController::Base` ones. A malformed constant name raises
190
261
  `invalid_controller_name`; a name that doesn't resolve raises
@@ -192,18 +263,22 @@ a no-op.
192
263
  `ActionController` descendant (a plain class, a model) raises
193
264
  `not_action_controller`.
194
265
 
195
- `rails_controller` has two deliberate limitations, matching a boundary
196
- `rails_routes` already has:
266
+ `rails_controller` has one deliberate limitation:
197
267
 
198
268
  - **Strong parameters are not reported.** `params.require(:widget).permit(:name,
199
269
  :size)` exists only as code inside a private method body, never as
200
270
  queryable class metadata -- the only way to recover a permit-list is to
201
271
  parse source, which this gem does not do (see `ROADMAP.md`'s "Considered
202
272
  and set aside" section for why source parsing is out of scope generally).
203
- - **Only the main app's route table is read.** A controller mounted inside an
204
- engine will have its actions reported under `unroutable_actions` even where
205
- the engine's own routes reach them -- `rails_routes` has the identical
206
- boundary today.
273
+
274
+ Routes drawn by an engine mounted in the application are cross-referenced
275
+ like any other, with the mount point on the path and the engine's class name
276
+ in `engine` -- so a controller that lives inside an engine reports its real
277
+ routes rather than listing every action as unroutable. Their `route_name` is
278
+ engine-local, reached as `<mount name>.<route_name>_path`, exactly as in
279
+ `rails_routes`. Expansion goes one level deep: an engine mounted inside
280
+ another engine stays an opaque mount route, the same boundary `bin/rails
281
+ routes` (and therefore `rails_routes`) draws.
207
282
 
208
283
  `rails_spec_flaky_check`:
209
284
 
@@ -223,6 +298,8 @@ a no-op.
223
298
  `rails_spec_run` already has), and each round samples a different subset,
224
299
  since execution order varies by design -- for suites this large, narrow
225
300
  `paths`/`example` rather than passing a very broad directory selection.
301
+ - `description` is only present when it differs from `id` (RSpec); Minitest
302
+ entries carry `id` only.
226
303
  - `statuses[]` only aligns positionally with `rounds[]` when no round
227
304
  crashed -- a round whose process itself failed contributes no entry to
228
305
  `statuses[]` (though it still appears in `rounds[]`), so treat positional
@@ -242,6 +319,30 @@ worker). For most apps this is invisible, but if you ever see behavior differ
242
319
  between Coatepec and your own `bundle exec rspec`, this is the first thing to
243
320
  suspect.
244
321
 
322
+ Minitest through `rails_spec_run`:
323
+
324
+ - "Run this Minitest file" -- `rails_spec_run(paths: ["test/models/widget_test.rb"])`;
325
+ the framework is decided by the path, and there is no separate
326
+ Minitest-named tool. `test/models/widget_test.rb:12` runs the one test
327
+ whose definition spans line 12, and a directory runs every `_test.rb`
328
+ under it.
329
+ - `example:` is a substring match on the test's method name
330
+ (`example: "reaches the"` matches `test_reaches_the_database`), passed to
331
+ Minitest as an escaped regex.
332
+ - Results use RSpec's vocabulary so the shape is identical: a Minitest skip
333
+ is `"pending"`, an error is `"failed"` and counts toward
334
+ `summary.failure_count` -- which is why that number can exceed the
335
+ `failures` Minitest prints in `stdout`; `summary.error_count` says how many
336
+ of them were errors, and `summary.assertion_count` is the assertion total
337
+ Minitest prints; `id` is `ClassName#test_method`.
338
+ - One call may not mix `spec/` and `test/` paths (`mixed_test_frameworks`).
339
+ - Rails' parallel testing is disabled in the child (`PARALLEL_WORKERS=1`), so
340
+ a large directory selection runs serially under the one timeout budget
341
+ rather than forking a worker tree.
342
+ - `file:LINE` works even when the app was generated with `--skip-test` (no
343
+ `rails/test_unit/railtie`) or runs Rails 7.1 with Minitest 6, both of which
344
+ break Rails' own line filtering; Coatepec installs its own.
345
+
245
346
  ### Restarts
246
347
 
247
348
  If any tool call fails with `sidecar_restart_required`, the target app's
@@ -262,11 +363,12 @@ it always respawns, even if the current worker looks healthy.
262
363
  ## Security boundary
263
364
 
264
365
  No eval, console, SQL/record access, shell, Rake, or file-write tool. Spec
265
- selectors must resolve inside an allowed spec root (`spec/`, `packs/*/spec/`,
266
- `engines/*/spec/`, `gems/*/spec/`); absolute paths, `..`, symlink escapes,
267
- non-`_spec.rb` files, and more than 100 selectors are rejected. RSpec still
268
- executes application-controlled code; only run Coatepec against a trusted
269
- checkout.
366
+ selectors must resolve inside an allowed spec or test root (`spec/`, `test/`,
367
+ and the `packs/*/`, `engines/*/`, `gems/*/` variants of each); absolute
368
+ paths, `..`, symlink escapes, files that are neither `_spec.rb` under a spec
369
+ root nor `_test.rb` under a test root, and more than 100 selectors are
370
+ rejected. RSpec and Minitest still execute application-controlled code; only
371
+ run Coatepec against a trusted checkout.
270
372
 
271
373
  ### Compared to Rails Active MCP
272
374
 
@@ -279,8 +381,8 @@ sophisticated bypasses of a denylist like that are always possible in
279
381
  principle.
280
382
 
281
383
  Coatepec takes the opposite approach: there's no eval, console, or SQL
282
- tool to begin with. `rails_spec_run` only ever executes RSpec files that
283
- already exist under the app's own allowed spec roots, and `rails_routes`/
384
+ tool to begin with. `rails_spec_run` only ever executes RSpec or Minitest
385
+ files that already exist under the app's own allowed spec/test roots, and `rails_routes`/
284
386
  `rails_model`/`rails_controller` only ever call structured, read-only Rails
285
387
  APIs (`Rails.application.routes.routes`, `ActiveRecord` reflection,
286
388
  `ActionController` callback/action-method metadata) -- never `eval`,
@@ -291,9 +393,10 @@ structure without ever handing it a REPL.
291
393
 
292
394
  ## Compatibility
293
395
 
294
- Ruby `>= 3.2`, Rails `>= 7.1, < 8.2`. CI tests three lanes: Rails 8.1 on
295
- Linux (primary), Rails 7.1 on Linux (compat), and Rails 8.1 on macOS (which
296
- is where the guarded-fork path above actually forks).
396
+ Ruby `>= 3.2`, Rails `>= 7.1, < 8.2`, Minitest 5.x and 6.x (the fixture
397
+ apps pin 6.0.6; the 5.x name-filter flag is unit-tested). CI tests three
398
+ lanes: Rails 8.1 on Linux (primary), Rails 7.1 on Linux (compat), and Rails
399
+ 8.1 on macOS (the lane that exercises the guarded-fork path above).
297
400
 
298
401
  ## Development
299
402
 
data/ROADMAP.md CHANGED
@@ -4,56 +4,19 @@ Ideas and known future work for Coatepec, roughly in the order they came up.
4
4
  Nothing here is committed to a release; this is a place to write things down
5
5
  before they're designed, not a promise.
6
6
 
7
- ## Minitest support
8
-
9
- Every current tool (`rails_spec_run`, `rails_spec_flaky_check`, and the
10
- paused `rails_spec_profile` below) is built around RSpec: `Coatepec::Spec::Runner`
11
- shells out to `bundle exec rspec`/forks an RSpec process, and
12
- `Coatepec::Spec::PathPolicy` validates selectors against RSpec's own
13
- `*_spec.rb` convention. None of that carries over to a Rails app using
14
- Minitest instead.
15
-
16
- The shape of the fix should mirror what already exists rather than
17
- invent something new: a parallel `Coatepec::Minitest::Runner` (or
18
- similarly named) implementing the same "validate selectors, build CLI
19
- args, run via the platform strategy, return a structured result" contract
20
- `Spec::Runner` already does, reusing `ForkStrategy`/`SpawnStrategy`/`GuardedForkStrategy`
21
- as-is where their logic is genuinely test-framework-agnostic (they mostly
22
- just fork/spawn a command and reap it — the RSpec-specific parts are
23
- `Runner#build_args` and the `--format json` output parsing in
24
- `Coatepec::Spec::Result`, both of which would need Minitest equivalents:
25
- Minitest's own JSON/machine-readable reporter, or `minitest-reporters`
26
- gem output, would need to be identified before designing that half).
27
-
28
- **Framework auto-detection via the Gemfile is straightforward and doesn't
29
- need new machinery** — `Coatepec::Worker::RailsRuntime#loaded_gem_names`
30
- already exists and is exactly the mechanism `GuardedForkStrategy` uses
31
- today to check for fork-unsafe gems, and `Coatepec::Spec::FactoryProfRunner`
32
- (see below) uses to check for `test-prof`. The same `loaded_gem_names.include?("rspec-rails")`
33
- vs. `.include?("minitest")` check (a Rails app's default `Gemfile` already
34
- declares one or the other, sometimes both) is enough to route
35
- `rails_spec_run` (or a to-be-decided `rails_test_run`, if the two
36
- frameworks' capabilities diverge enough to warrant separate tool names
37
- rather than one dispatching tool) to the right runner. `Coatepec::Spec::Runner#require_rspec!`
38
- already anticipates the gap in spirit: it raises `:unsupported_test_framework`
39
- today when `rspec-rails` isn't loadable, rather than assuming RSpec
40
- unconditionally.
41
-
42
- Open questions for whenever this gets designed properly:
43
- - One tool name that dispatches by detected framework, or separate
44
- `rails_spec_run`/`rails_minitest_run`-style tools? (Affects whether an
45
- agent needs to know which framework a given app uses before calling the
46
- right tool, vs. the tool figuring it out.)
47
- - What Minitest gives you for structured per-example output
48
- (pass/fail/pending, id, file/line) equivalent to RSpec's `--format json`
49
- — needed before `rails_spec_flaky_check`'s per-example flaky-detection
50
- logic (`Coatepec::Spec::FlakyChecker`, framework-agnostic in principle
51
- since it only depends on `Runner#run`'s result shape) could target
52
- Minitest too.
53
- - Whether Minitest's own `--seed` (it has one; Minitest randomizes test
54
- order by default too) is enough of an equivalent to RSpec's `--seed`
55
- for `rails_spec_flaky_check` to reuse the same "rerun N times with a
56
- fresh random seed" mechanism unchanged.
7
+ ## Minitest support -- shipped in 0.8.0
8
+
9
+ Shipped through the existing `rails_spec_run`/`rails_spec_flaky_check`
10
+ tools. The open questions resolved as: one tool, dispatching by
11
+ *selector shape* (`test/**/*_test.rb` vs `spec/**/*_spec.rb`) rather
12
+ than by Gemfile, since that is per-request and unambiguous for apps
13
+ with both; structured per-test output comes from an in-process
14
+ `Minitest::AbstractReporter` writing RSpec's JSON shape, so
15
+ `Spec::Result` and `FlakyChecker` were reused unchanged; and Minitest's
16
+ `--seed` is a direct equivalent (16-bit, random order by default), so
17
+ the flaky checker works as-is. See
18
+ `docs/superpowers/specs/2026-09-05-minitest-support-design.md` (local-only)
19
+ for the design and the line-filtering finding.
57
20
 
58
21
  ## Get stats on a spec/test run (TestProf / FactoryProf) — designed, implemented, paused
59
22
 
@@ -76,8 +39,9 @@ root-cause writeup and the fix options considered (forcing `SpawnStrategy`
76
39
  for profiled runs specifically, vs. detecting and raising a clear error on
77
40
  non-spawn strategies, vs. a Linux-only opt-out config knob). Paused
78
41
  specifically to wait for real signal on how coatepec is actually used
79
- (Linux vs. macOS, `macos_fork` adoption) before picking a fix, rather than
80
- guessing.
42
+ (Linux vs. macOS, `macos_fork` adoption) -- `macos_fork` is the default from
43
+ 0.9.0, so the fork path is now the common case on both platforms -- before
44
+ picking a fix, rather than guessing.
81
45
 
82
46
  ## Run Rails 8.1's built-in CI (`bin/ci`)
83
47
 
@@ -114,16 +78,16 @@ or folds into an existing one, and whether "security audits" specifically
114
78
  `bin/ci`) are worth exposing as a narrower, separate tool for apps that
115
79
  don't use Rails 8.1's `bin/ci` scaffold at all but do have those gems.
116
80
 
117
- ## Controller introspection
81
+ ## Controller introspection -- shipped in 0.7.0
118
82
 
119
- A `rails_controller`-style tool mirroring the existing `rails_model`
120
- (`Coatepec::Introspection::Model`) and `rails_routes`
121
- (`Coatepec::Introspection::Routes`) tools' shape: given a controller
122
- constant, return its actions, `before_action`/`around_action`/`after_action`
123
- filters (and which actions they apply to), strong-parameter method
124
- definitions, and included concerns -- via real Rails introspection APIs
125
- (`ActionController::Base` callback chains, not source parsing), the same
126
- trust boundary `rails_model` already holds to.
83
+ Shipped as `rails_controller` (`Coatepec::Introspection::Controller`) in
84
+ 0.7.0: actions, `before`/`after`/`around` callbacks with their `only`/`except`
85
+ restrictions and `if`/`unless` conditions, included concerns, and a
86
+ cross-reference of every action against `Rails.application.routes`
87
+ (`unroutable_actions` / `routes_without_action`). Strong parameters were
88
+ deliberately left out: a `permit` list exists only as code inside a method
89
+ body, and recovering it would mean source parsing (see "Considered and set
90
+ aside" below). See the README and CHANGELOG for the full contract.
127
91
 
128
92
  ## Background job introspection -- built, then parked (PR #14, closed unmerged)
129
93
 
@@ -171,9 +135,12 @@ Two constraints worth keeping if this is revisited:
171
135
  ## Cross-file consistency validation
172
136
 
173
137
  Distinct from anything coatepec does today: a tool that checks for drift
174
- *across* files rather than introspecting one thing at a time -- a route
175
- pointing at a controller action that doesn't exist, a `belongs_to`/`has_many`
176
- referencing a column or table that isn't in the schema, that kind of thing.
138
+ *across* files rather than introspecting one thing at a time -- a
139
+ `belongs_to`/`has_many` referencing a column or table that isn't in the
140
+ schema, that kind of thing. (The route-to-missing-action case is already
141
+ covered per controller by `rails_controller`'s `routes_without_action`,
142
+ shipped in 0.7.0; an app-wide sweep of the whole route table would be the
143
+ cross-file version of that same check.)
177
144
  Surfaced while researching prior art (a competing tool does this via
178
145
  source-code parsing); would need its own design for how to do it via
179
146
  structured Rails APIs instead, consistent with how every other coatepec