constable-rails 0.1.0 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3a0630c0ca6e000d33e21515a284616186d1939531ba5f5e9c5c61882d6dc9ec
-  data.tar.gz: 10fcc4a75763bec06db7049bd3815214e1320ecdb4f3ad2780ef0497b8f8af6b
+  metadata.gz: 05e7e65ec7f1f096713bc5cb7b891ce2297275876c01cf9151130dc13608befe
+  data.tar.gz: db0b094a1e70a0052f02bb343cd0d41c53b224e56df089638ad6f5be60025e36
 SHA512:
-  metadata.gz: 9a823bc22286ffaff8ab4c29f4f357ba40af37e423a9c6d1cab82382edf5d19c0ca0fae9796c0171f256bab6bdef93c9e2f14a65dd9a57e6e49e2cede204e959
-  data.tar.gz: 0c043327dc97ace117a1dae5919c9b066df22059deafa5a422066e96b3abafc6f67df7e33f911f68a8eb314b3cca6e7bab0150c7d91c3d6b26f92482baee8b0e
+  metadata.gz: 19e12d780b73ec73830eab6290b27d61f1fb8c9cd40894ff0c2832120883f06265e58b405ec107c1dab002d6bf484fb166a99cad273507d1b05e0da1234ae0df
+  data.tar.gz: 789ba56df5a2720e25ddbadec56d9916776f809c95b812748468addbdf05276292e76ee37686a40d8489c5678121d44d35e41a0b2b09ab6e72af82b016dc682a

data/CHANGELOG.md

@@ -5,6 +5,163 @@ All notable changes to this project are documented here. This project adheres to
 
 ## [Unreleased]
 
+## [1.0.0]
+
+The first release anybody should install.
+
+0.1.0 shipped the ideas; running it against a real Rails app for the first time found
+that several of them did not survive contact. Four bugs made the documented adoption
+path impossible to follow, three commands raised `NoMethodError` the moment they were
+run, and three separate ways of mistyping a command produced a **green build that ran no
+tests at all**. Those are all fixed, with tests, and the suite has grown from 738 runs to
+888.
+
+The API has not changed. Every 0.1.0 case file, config key and command still works.
+
+### The adoption path now actually works
+
+These four were found by installing 0.1.0 into a Rails app with a 188-example RSpec
+suite and following the README from the top.
+
+- **`rails generate constable:install` no longer breaks the Gemfile.** It appended a
+  `:cold_case` group declaring `rspec-rails` without checking whether the app already
+  had it. Any app adopting Constable *from RSpec* — which is the entire target audience
+  — was left with a Gemfile Bundler refused to parse: *"You cannot specify the same gem
+  twice with different version requirements."* The installer now adds only the engines
+  the repo has files for, and only ones the Gemfile does not already declare.
+
+- **Cold cases run `before(:suite)` and `after(:suite)` hooks.** `ColdCase::RSpec` drove
+  example groups directly and skipped RSpec's `with_suite_hooks`, so the hooks never
+  fired. That is where `webmock/rspec` calls `WebMock.enable!`, where VCR and
+  DatabaseCleaner install themselves, and where SimpleCov starts. It failed *open*: a
+  spec that stubbed HTTP opened a real socket instead of erroring. Each hook now runs
+  exactly once per run — after a file has loaded, since a legacy file's own
+  `require "rails_helper"` is what registers them.
+
+- **Parallel workers get their own database.** Constable forks its own workers and so
+  never picked up the per-worker databases Rails builds for `rails test`. Every worker
+  opened the same one. On SQLite a suite that passed 188/0 serially collapsed into 130
+  `database is locked` failures. On a client/server database it would have been quieter
+  and worse. A worker that cannot build its own database now raises rather than falling
+  back to the shared one, and an app that cannot shard runs serially with a warning.
+
+- **An outage no longer jails healthy tests.** Flake history reads "passed last run,
+  failed this run" as evidence about a test, so that one broken parallel run put 29
+  healthy tests on the docket marked *"passed, then failed with no code change"*. A run
+  where a quarter of the suite fails with the identical error is now recognised as one
+  broken run: the failures still stand and the build still goes red, but nothing moves
+  through the jail or parole state machine and nothing is written to flake history.
+
+- **`constable:install` writes the blotter into `.gitignore`.** It is this machine's
+  flake history and docket. Committing it hands CI somebody else's docket and conflicts
+  on every run.
+
+- **FactoryBot is wired into the tier base classes,** and `test/support/**` loads
+  *before* them. `create(:user)` is what a converted spec is full of, and the generated
+  `case_helper.rb` both omitted the include and told you to `include Authenticatable` in
+  a class defined thirty lines above the file that defines it.
+
+### Commands that had never been run
+
+There was no test file for the CLI at all. All three of these are in the published
+command reference and all three raised `NoMethodError` on their first line:
+
+- `constable jail parole PATH:LINE`
+- `constable jail release PATH:LINE`
+- `constable warrants release PATH:LINE` — twice over: after fixing the first bug it
+  went on to call a second method that does not exist either.
+
+Ambiguous targets are also refused rather than guessed at. `constable jail parole
+test/cases/reports_case.rb`, with three tests from that file on the docket, paroled one
+of them — not the first by line, whichever row the database happened to return. It now
+prints the candidates and exits.
+
+### Three ways to get a green build that ran nothing
+
+Each of these printed `0 passed, 0 failed` and exited **0**, so a typo in a CI script
+went green having tested nothing. The most expensive kind of bug a test runner can have,
+because it stays invisible for months.
+
+- `constable test test/cases/typo_case.rb` — no such file.
+- `constable test users_case.rb:999` — worse than nothing. `PATH:LINE` picks the
+  investigation declared nearest above the line so you can point anywhere inside a
+  block; unbounded, a line past the end of the file ran the **last** investigation in
+  it. Not the test you asked for, not an error, green either way.
+- `constable test --tier nonsense` — and `--tier UNIT`, which matched nothing because
+  tiers were case-sensitive.
+
+### Correctness
+
+- **Two tests no longer share one identity.** A test's key is a content hash of its
+  body, which is what lets history survive a rename — but two tests with byte-identical
+  bodies got the same key, in different classes, with different descriptions. The
+  blotter treated them as one test: jail either and both went, and their flake histories
+  merged. This is routine rather than exotic; the model generator writes an identical
+  first investigation into every file it touches.
+
+- **A witness can no longer replace the framework.** `witness` defines a real instance
+  method, so `witness(:class)` quietly replaced `Object#class` and every later failure
+  message reported the wrong thing. `witness(:attest)` was worse: it disabled assertions
+  outright, so the tests passed by doing nothing.
+
+- **`be(nil)` asserted the opposite of what it said.** `[nil].any?` is false — `Array#any?`
+  without a block tests the truthiness of the elements, not presence — so it fell through
+  to the truthiness branch.
+
+- **`match_array` was aliased to `contain_exactly`,** but RSpec's takes one array where
+  `contain_exactly` takes varargs, so `match_array([1, 2])` asserted the collection held
+  a single element which was itself `[1, 2]`.
+
+### Matchers
+
+`constable modernize` rewrote any matcher name straight into an `attest` call, so a spec
+using one Constable did not implement converted cleanly and then died at runtime.
+
+- Added: `contain_exactly`, `match_array`, `be` (identity, truthiness, and the
+  `be >= 0` operator form), `be_within(d).of(x)`, `start_with`, `end_with`,
+  `be_between`, `satisfy`.
+- `modernize` now **flags any matcher it does not recognize** instead of converting it.
+- `have_http_status` resolves names through `Rack::Utils` and knows
+  `:unprocessable_content` — the Rack 3.1 name for 422, and the one Rails 8.1 tells you
+  to use while Constable accepted only the deprecated spelling.
+- Helper specs are flagged. They converted reporting *"21 converted, 0 flagged"* and
+  then failed every example with `undefined local variable or method 'helper'`.
+
+### Output
+
+- **An `expanded` mode.** `output: concise | expanded` in `.constable/config.yml`, or
+  `--expanded` / `--concise` for one run. Concise is unchanged and still the default.
+  Expanded prints a line per test — glyph, description, duration — so you can see which
+  test is hanging while it hangs. Per-test durations are in milliseconds; the run-scale
+  format rendered every fast test as `0.0s`.
+- **Sections for the supervision states.** Jailed tests, warrants and tests on parole
+  were counted in the headline and then never mentioned again, so "2 jailed" was a
+  number with nothing behind it. `WARRANTS`, `JAILED` and `ON PAROLE` now print like
+  `FAILURES` does.
+- **Each one ends with what to do next.** "2 jailed" is a fact;
+  `constable jail parole PATH:LINE` is an action. Parole shows its progress toward
+  release.
+- Parole entries carry their `file:line` — the one thing their own hint asked you to
+  pass — and warning text wraps to the 60-column frame instead of spilling out of it.
+
+### Messages instead of stack traces
+
+- A YAML typo in `config.yml` raised a raw `Psych::SyntaxError`; a file that parsed but
+  was not a mapping raised *"no implicit conversion of Array into Hash"* from inside the
+  merge. Both now name the file and the problem.
+- A corrupt blotter raised *"file is not a database: PRAGMA journal_mode = WAL"*. It now
+  says what the file holds — flake history, the docket, warrants, never a test — and
+  that deleting it is safe.
+- `coverage_threshold` is clamped to a percentage, negative `warrant_retries` means off,
+  and `parole_period` clamps in one place. `Jail` already refused a period of zero, but
+  the reporter read the raw value and would print *"Day 1 of 0 — 0 clean runs to go"*
+  while the docket waited for ten.
+
+### Also
+
+- A new logo, with a dark variant, and a `<picture>` element so GitHub picks.
+
+
 ## [0.1.0]
 
 Initial release.
@@ -84,5 +241,6 @@ Initial release.
 - Diff-based coverage gate — only lines changed in the current diff are held to the
   threshold. `constable beat` for the full picture, `--html` for a browsable report.
 
-[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v0.1.0...HEAD
+[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v1.0.0...HEAD
+[1.0.0]: https://github.com/Ray-Hughes/constable/compare/v0.1.0...v1.0.0
 [0.1.0]: https://github.com/Ray-Hughes/constable/releases/tag/v0.1.0

data/README.md

@@ -1,8 +1,9 @@
 <div align="center">
 
-<img src="https://raw.githubusercontent.com/Ray-Hughes/constable/main/docs/assets/logo.png" alt="Constable" width="200">
-
-# Constable
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Ray-Hughes/constable/main/docs/assets/logo-dark.png">
+  <img src="https://raw.githubusercontent.com/Ray-Hughes/constable/main/docs/assets/logo.png" alt="Constable" width="340">
+</picture>
 
 **A strict Rails testing framework where fast and non-flaky are structural, not disciplinary.**
 
@@ -215,10 +216,16 @@ Constable::Matchers.define(:be_created) { |response| response.status == 201 }
 Constable::Matchers.define(:exist) { |model_class, attrs| model_class.exists?(attrs) }
 ```
 
-Built in: `eq`, `eql`, `include`, `match`, `raise_error`, `have_attributes`, `exist`,
-`be_created`, `redirect_to`, `have_http_status`, `change`, plus `be_a`, `be_nil`, `be_empty`,
-`be_truthy`, `be_falsey` and a `be_*` / `have_*` predicate fallback. Plain `assert_*` and
-`refute_*` primitives are always available alongside `attest`.
+Built in: `eq`, `eql`, `be`, `include`, `match`, `raise_error`, `have_attributes`,
+`exist`, `be_created`, `redirect_to`, `have_http_status`, `change`, `contain_exactly`,
+`match_array`, `start_with`, `end_with`, `be_between`, `be_within(d).of(x)`, `satisfy`,
+plus `be_a`, `be_nil`, `be_empty`, `be_truthy`, `be_falsey` and a `be_*` / `have_*`
+predicate fallback. `be` also takes the operator form — `attest(count).to be > 0`.
+Plain `assert_*` and `refute_*` primitives are always available alongside `attest`.
+
+The set is deliberately smaller than RSpec's, so `constable modernize` **flags any
+matcher it does not recognize** rather than converting it into a case that only fails
+once you run it.
 
 ### Shared behavior is just Ruby
 
@@ -369,6 +376,14 @@ Class name, description and file are stored alongside purely as a display label.
   new one appears and suggests `constable history relink OLD NEW`. Set `auto_relink: true` to
   confirm high-confidence matches automatically.
 
+Two tests with byte-identical bodies would otherwise share a key — and bodies repeat more
+than the phrase "content hash" suggests, since
+`attest(build(:thing, name: nil)).not_to be_valid` is the same handful of tokens in every
+model case. Constable re-keys colliding tests on their class and description once the
+suite is loaded, so no two tests ever share a docket row. Rename-survival is weaker for
+exactly those tests, which is the right trade: a history belonging to two tests at once is
+worse than one that resets.
+
 ### Command reference
 
 | Command | Runs |
@@ -387,22 +402,69 @@ Class name, description and file are stored alongside purely as a display label.
 | `constable import --from=rspec` | Adopt an existing suite as cold cases |
 | `constable modernize PATH` | Opt-in AST rewrite into the native DSL |
 
-Flags: `--full --unsafe --jail --warrants --coverage --seed N --workers N --verbose --tier T --no-color`.
+Flags: `--full --unsafe --jail --warrants --coverage --seed N --workers N --verbose --tier T\n--expanded --concise --output MODE --no-color`.
 
 Order is randomized every run for native cases, with the seed printed and replayable via
 `--seed`. Cold cases keep their own engine's order. Workers run in parallel by default,
 load-balanced by a cached per-test duration index.
 
+Each worker gets **its own database**, built from schema the way `rails test` does it.
+Sharing one would not be a speed/safety trade but a correctness bug: on SQLite the run
+dissolves into `database is locked`, and on a client/server database tests quietly see
+each other's rows. If your app has ActiveRecord but cannot shard, Constable runs serially
+and says why — slow is a trade-off, wrong is not.
+
 ### Output
 
 stdout is reserved for results. `Rails.logger`, SQL and request/response logging go to
 `log/test.log`; `--verbose` streams it back for active debugging.
 
+**While it runs**, the live stream has two modes. `concise` is the default: one glyph per
+test, grouped into a run per case, so a thousand-test suite stays inside one screen and a
+wall of green is the point.
+
+```
+  ProjectCase                       ✓✓✓✓✓✓✓✓✓✓✓✓
+  BillingCase                       ✓✓⚖⛓✓
+```
+
+`expanded` trades that for a line per test — glyph, name, duration — so you can see which
+test is hanging while it hangs, rather than after.
+
+```
+  ProjectCase
+    ✓ validations rejects a colour outside the palette  2ms
+    ✓ validations rejects a duplicate name for the same owner 12ms
+    ✗ #completion_ratio is the fraction of done tasks   8ms
+
+  BillingCase
+    ⚖ charges a card                                    310ms
+    ⛓ refunds a charge — assertion failed on the amount
+    ✓ issues a receipt                                  1.4s
+```
+
+Set it in `.constable/config.yml` (`output: concise` or `expanded`), or per run with
+`--expanded` / `--concise`. A jailed test never ran its body, so it is given no duration
+rather than a dishonest `0ms`. **The summary below is identical in both modes** — the mode
+only changes what you watch on the way there.
+
 ```
 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-  CONSTABLE            482 tests · 3 cases · 12.4s
+  CONSTABLE            6 tests · 3 cases · 12.4s
 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-  ✓ 478 passed   ✗ 2 failed   ⛓ 2 jailed (1 parole violation)   ◑ 1 on parole   ⚖ 1 warrant issued   ⚠ 3 warnings   ◐ 92% covered
+  ✓ 2 passed   ✗ 1 failed   ⛓ 2 jailed (1 parole violation)   ◑ 1 on parole   ⚖ 1 warrant issued   ⚠ 2 warnings   ◐ 92% covered (3 files unpatrolled)
+
+  PAROLE VIOLATED
+  ───────────────
+  ⛓ UsersController::CreatesUserCase
+    "creates a user with valid params"
+    spec/cases/users_controller/creates_user_case.rb:8
+    Failed on day 3 of a 10-run parole — back to jail. This is its 2nd time in jail.
+
+  → Somebody trusted this test again and it let them down,
+    so it is back on the docket. Fix it before the next
+    constable jail parole — a second violation is the signal
+    that the test, not the flake, is the problem.
 
   FAILURES
   ────────
@@ -417,11 +479,64 @@ stdout is reserved for results. `Rails.logger`, SQL and request/response logging
 
     Rerun just this test:
       constable test spec/cases/sessions_case.rb:12 --seed 8841
+
+  WARRANTS
+  ────────
+  ⚖ BillingCase
+    "charges a card"
+    spec/cases/sessions_case.rb:12
+    Failed, then passed 4 of 5 retries run in isolation.
+
+  → A warrant is "not reproducible", not "not a problem" —
+    it stops blocking the build and stays visible until
+    someone deals with it. Fixed the flake? constable
+    warrants release PATH:LINE
+
+  JAILED
+  ──────
+  ⛓ BillingCase
+    "refunds a charge"
+    spec/cases/sessions_case.rb:12
+    Assertion failed on the amount.
+
+  → Jailed means skipped and tracked, not passing. Think one
+    is fixed? constable jail parole PATH:LINE runs it for
+    real again — 10 clean runs and it releases itself.
+
+  ON PAROLE
+  ─────────
+  ◑ SessionsCase
+    "signs a user in"
+    spec/cases/sessions_case.rb:12
+    Day 4 of 10 — 6 clean runs to go.
+
+  → A paroled test runs for real and is watched: one failure
+    sends it straight back to jail. constable watchlist
+    shows everything under supervision.
+
+  WARNINGS
+  ────────
+  ⚠ spec/legacy/old_users_spec.rb
+    running as a cold case (Constable::ColdCase::RSpec) — 12
+    tests not yet under native rules
+
+  ⚠ spec/controllers/sessions_case.rb:44
+    unsafe { sleep(0.1) } — "testing an actual timeout path,
+    not a code smell"
+
+  SLOWEST
+  ────────
+  3.2s  UsersController::CreatesUserCase "creates a user with valid params"
+  1.1s  SessionsCase "times out after thirty seconds"
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 ```
 
-Sections print worst-to-least-urgent: parole violations, failures, warnings, slowest.
-Failures carry their own context and point at your `investigate` line, not at framework
-internals.
+Sections print worst-to-least-urgent: parole violations, failures, warrants, jailed, on
+parole, warnings, slowest. A section only appears when it has something to say.
+
+Each supervision section ends with one line saying what to do next, because "2 jailed" is
+a fact and `constable jail parole PATH:LINE` is an action. Failures don't get a hint —
+they already end with the exact command to rerun them.
 
 ### The blotter
 
@@ -462,6 +577,7 @@ coverage_threshold: 90           # diff-based — only lines changed in the curr
 coverage_html: false
 
 fail_on_warnings: false
+output: concise                  # live stream detail: concise | expanded
 parallel_workers: auto
 
 tiers:                           # fallback inference; base classes are primary

data/lib/constable/case.rb

@@ -60,6 +60,7 @@ module Constable
         raise ArgumentError, "witness(#{name.inspect}) requires a block" unless block
 
         name = name.to_sym
+        guard_witness_name!(name)
         own_witnesses[name] = block
         define_method(name) do
           constable_witnesses.fetch(name) { constable_witnesses[name] = instance_exec(&block) }
@@ -161,6 +162,38 @@ module Constable
         constable_lineage.flat_map(&:own_briefings)
       end
 
+      # Names a witness may not take.
+      #
+      # `witness` defines a real instance method, so `witness(:class) { ... }` quietly
+      # replaces Object#class on the case and every later `attest` failure reports the
+      # wrong thing. Worse, `witness(:attest)` disables assertions outright -- the tests
+      # then pass by doing nothing, which is the one failure mode a testing framework
+      # must never have.
+      #
+      # Deliberately a list rather than `method_defined?`: a blanket check would reject
+      # ordinary names a tier happens to define (`response` on an integration case), and
+      # shadowing those is a legitimate, if unusual, thing to want.
+      RESERVED_WITNESS_NAMES = %i[
+        class send __send__ __id__ object_id method methods freeze frozen? dup clone
+        hash inspect to_s instance_variable_get instance_variable_set instance_variables
+        attest unsafe witness briefing investigate docket tier setup teardown
+        assert refute flunk skip pass freeze_time travel_to travel_back
+      ].freeze
+
+      def guard_witness_name!(name)
+        if RESERVED_WITNESS_NAMES.include?(name)
+          raise ArgumentError,
+                "witness(:#{name}) would replace Constable::Case##{name}, which the " \
+                "framework needs. Pick another name."
+        end
+
+        return unless name.to_s.start_with?("constable_")
+
+        raise ArgumentError,
+              "witness(:#{name}) is in Constable's own namespace. Names beginning " \
+              "`constable_` belong to the framework."
+      end
+
       def witnesses
         constable_lineage.each_with_object({}) { |klass, out| out.merge!(klass.own_witnesses) }
       end

data/lib/constable/cli.rb

@@ -17,7 +17,10 @@ module Constable
     # wrong", and CI needs to. Commands return a status; this turns it into one.
     def self.dispatch!(argv)
       start(argv)
-    rescue Thor::Error => e
+    # Thor::Error is a malformed command; Constable::Error is a wrong path, an unknown
+    # tier, an unreadable config. Both are the user's mistake rather than a crash, and
+    # both deserve one sentence and a usage status instead of a backtrace.
+    rescue Thor::Error, Constable::Error => e
       warn e.message
       exit(EXIT_USAGE)
     rescue Interrupt
@@ -42,6 +45,9 @@ module Constable
     option :workers,  type: :numeric, desc: "Parallel workers (default: config, or auto)"
     option :verbose,  type: :boolean, default: false, desc: "Stream log/test.log to stdout"
     option :tier,     type: :string,  desc: "Run one tier only: unit, integration or system"
+    option :output,   type: :string,  desc: "Live stream detail: concise (default) or expanded"
+    option :expanded, type: :boolean, default: false, desc: "Shorthand for --output=expanded"
+    option :concise,  type: :boolean, default: false, desc: "Shorthand for --output=concise"
     def test(*paths)
       config = load_config
       LogRouter.route!(verbose: options[:verbose])
@@ -299,7 +305,9 @@ module Constable
       no_commands do
         def act(locator)
           jail = Jail.new(config: Constable.config, storage: Constable.storage)
-          identity = jail.identity_for(locator)
+          refuse_ambiguous(locator, jail.candidates(locator), "on the docket")
+
+          identity = jail.resolve(locator)
           unless identity
             warn "Nothing on the docket at #{locator}"
             exit(EXIT_USAGE)
@@ -307,6 +315,18 @@ module Constable
           yield jail, identity
         end
 
+        # A bare path naming several tests is a question. Answer it with the list rather
+        # than acting on whichever row the database happened to return first.
+        def refuse_ambiguous(locator, candidates, noun)
+          return if candidates.size <= 1
+
+          warn "#{locator} matches #{candidates.size} tests #{noun}. Name one:"
+          candidates.sort_by { |entry| entry.line.to_i }.each do |entry|
+            warn "  #{entry.location}  #{entry.label}"
+          end
+          exit(EXIT_USAGE)
+        end
+
         def repeat_note(entry)
           count = entry.times_jailed
           count > 1 ? " (this is its #{Reporter.ordinalize(count)} time in jail)" : ""
@@ -344,12 +364,14 @@ module Constable
       desc "release PATH:LINE", "Clear a warrant by hand"
       def release(locator)
         warrants = Warrants.new(config: Constable.config, storage: Constable.storage)
-        identity = warrants.identity_for(locator)
+        JailCommand.new.send(:refuse_ambiguous, locator, warrants.candidates(locator), "under warrant")
+
+        identity = warrants.resolve(locator)
         unless identity
           warn "No warrant at #{locator}"
           exit(EXIT_USAGE)
         end
-        warrants.clear(identity)
+        warrants.release(identity)
         say "Warrant cleared."
       end
     end
@@ -400,7 +422,18 @@ module Constable
       end
 
       def reporter(config)
-        Reporter.new(io: $stdout, config: config, color: color?)
+        Reporter.new(io: $stdout, config: config, color: color?, mode: output_mode)
+      end
+
+      # nil means "the config file decides". The explicit --output wins over the two
+      # shorthands, and --concise wins over --expanded if somebody passes both -- the
+      # quieter of two contradictory instructions is the safer one to obey.
+      def output_mode
+        return options[:output] if options[:output]
+        return :concise if options[:concise]
+        return :expanded if options[:expanded]
+
+        nil
       end
 
       def say_table(heading, entries)

data/lib/constable/cold_case/rspec.rb

@@ -120,7 +120,12 @@ module Constable
             with_engine do
               collector  = Collector.new
               load_error = capture_load(path)
-              run_world(collector) unless load_error
+              unless load_error
+                # After the file has loaded -- that load is what registers them -- and
+                # before its examples run. See #run_pending_before_suite_hooks.
+                run_pending_before_suite_hooks(::RSpec.configuration)
+                run_world(collector)
+              end
 
               ColdCase.warn_for_file(path, base_class_name, collector.examples.size, config: config)
               results = build_results(path, collector.examples, config: config, seed: seed,
@@ -134,14 +139,101 @@ module Constable
         # configuration -- which also means any RSpec.configure hooks a rails_helper
         # installed are gone, so this is a teardown call, not a between-files call.
         def reset_engine!
+          # Inside the global-state swap, not outside it. Building a SuiteHookContext
+          # makes rspec-core lazily construct a world and a configuration, so running
+          # these hooks bare would leave both behind in a host process that had none --
+          # exactly the leak #with_engine exists to prevent.
+          with_engine { run_after_suite_hooks! } if after_suite_hooks_pending?
+
           @session_world = nil
           @session_configuration = nil
           @session_prepared = false
+          @ran_before_suite_hooks = nil
           nil
         end
 
         private
 
+        # --- Suite hooks ------------------------------------------------------------
+        #
+        # RSpec's own Runner wraps its group loop in `configuration.with_suite_hooks`.
+        # We drive the groups directly (see #run_world), so without this a cold case
+        # never fires `before(:suite)` -- and that is exactly where webmock/rspec calls
+        # `WebMock.enable!`, where VCR and DatabaseCleaner install themselves, and where
+        # SimpleCov starts. Skipping them fails *open*: a spec that stubs HTTP opens a
+        # real socket instead of erroring, which is the worst direction for a testing
+        # tool to be wrong in.
+        #
+        # `with_suite_hooks` itself is the wrong shape here. It is a bracket around one
+        # block, but "suite" means the whole run rather than one file: wrapping each file
+        # would fire `after(:suite)` after file one and hand file two the wreckage. Nor
+        # can the hooks all be run up front, because a legacy file's own
+        # `require "rails_helper"` is what registers them -- before the first load there
+        # is nothing to run.
+        #
+        # So each hook runs exactly once, the first time we see it: after a file has
+        # loaded, before its examples. `after(:suite)` runs once, from #reset_engine!.
+        def run_pending_before_suite_hooks(configuration)
+          pending = suite_hooks(configuration, :@before_suite_hooks) - ran_before_suite_hooks
+          return if pending.empty?
+
+          ran_before_suite_hooks.concat(pending)
+          invoke_suite_hooks(configuration, "a `before(:suite)` hook", pending,
+                             scope: :before_suite_hook)
+        end
+
+        # Only worth running if we ever ran the matching before(:suite) half -- otherwise
+        # this is a teardown for setup that never happened.
+        def run_after_suite_hooks!
+          configuration = @session_configuration
+          return if configuration.nil? || ran_before_suite_hooks.empty?
+
+          hooks = suite_hooks(configuration, :@after_suite_hooks)
+          return if hooks.empty?
+
+          invoke_suite_hooks(configuration, "an `after(:suite)` hook", hooks,
+                             scope: :after_suite_hook)
+        end
+
+        def ran_before_suite_hooks
+          @ran_before_suite_hooks ||= []
+        end
+
+        # Nothing was set up, so there is nothing to tear down -- and no reason to build
+        # an RSpec world to discover that.
+        def after_suite_hooks_pending?
+          !@session_configuration.nil? &&
+            ran_before_suite_hooks.any? &&
+            suite_hooks(@session_configuration, :@after_suite_hooks).any?
+        end
+
+        # RSpec keeps these in plain ivars with no public reader. Read them defensively:
+        # a missing ivar means a version that stores them elsewhere, and running no suite
+        # hooks is the behavior we already had.
+        def suite_hooks(configuration, ivar)
+          return [] unless configuration.respond_to?(:instance_variable_defined?)
+          return [] unless configuration.instance_variable_defined?(ivar)
+
+          Array(configuration.instance_variable_get(ivar))
+        end
+
+        # `run_suite_hooks` is private on Configuration, and it is the part that builds a
+        # SuiteHookContext and keeps one failing before-hook from running the rest. Use it
+        # when it is there, and fall back to driving the hooks ourselves when it is not.
+        def invoke_suite_hooks(configuration, description, hooks, scope:)
+          previous = ::RSpec.current_scope if ::RSpec.respond_to?(:current_scope)
+          ::RSpec.current_scope = scope if ::RSpec.respond_to?(:current_scope=)
+
+          if configuration.respond_to?(:run_suite_hooks, true)
+            configuration.send(:run_suite_hooks, description, hooks)
+          else
+            context = ::RSpec::Core::SuiteHookContext.new(description, configuration.reporter)
+            hooks.each { |hook| hook.run(context) }
+          end
+        ensure
+          ::RSpec.current_scope = previous if previous && ::RSpec.respond_to?(:current_scope=)
+        end
+
         # Running RSpec in-process is a global-state problem: RSpec.world holds every
         # registered example group and RSpec.configuration holds every hook. We swap in a
         # session world/configuration for the duration of a file and put whatever was