constable-rails 1.3.0 → 1.3.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '092c12ca7debd9d093f51e3ce6d90332aef25dcb9c16a5cf79b67f716e15a6a5'
-  data.tar.gz: 0b82a9db3e6d66228103161d46635e394591223956aa67a4bc5f929075701954
+  metadata.gz: c9f421fcc02a9f1c30aae54819a543cf548bdd997f8e638899ac6e448d093907
+  data.tar.gz: 506be0d669595b7e3f965f81f6449b4c074f4ade3a98c15d9283d78eca1cd584
 SHA512:
-  metadata.gz: 9bbf3170f8fd728513e0384d36477d4281535276446c86d8e97b8d49b32d956101d1fe4fe30d19688b9a37f36d8159f8ee3061ebc1b2801a6ad55f718d721d27
-  data.tar.gz: 300f8c3ecbfb7075492638c355aceb83fc242808e8296ec21798bea71e23bee98ec6b28ffeaad84a11206fd0e6e65b9771c56a441f4e1d12cc730a969a4340ba
+  metadata.gz: bf1762345ec78b7e7133e724fc9e525d2b037013eb940d49c1fdb6de2940a56284fa67c476e93c229840410e344a92e342bd344f9d21ba7c18c3a6217c3e7a2c
+  data.tar.gz: 74b67cd3a65966adc81147e272d08a3b5f092e8602dc3b543c6b2aa878f1bbead4eaabf555d072cc66b97c2137d4bf9593658478235730f3229f15cac0ef1096

data/CHANGELOG.md

@@ -5,6 +5,98 @@ All notable changes to this project are documented here. This project adheres to
 
 ## [Unreleased]
 
+## [1.3.2]
+
+### stdout is results only — the app's stdout too, not just Constable's
+
+`log/test.log` has taken Rails' loggers since 1.0.0. It did not take anything a gem writes
+directly to `$stdout` or `$stderr`, and a warning fired once per file lands in the middle
+of the live stream:
+
+```
+Address    ✓✓✓✓✓✓✓✓✓✓✓To use retry middleware with Faraday v2.0+, install `faraday-retry` gem
+```
+
+Both streams are now pointed at the log for the duration of a run, and the reporter keeps
+the terminal it captured beforehand. `--verbose` tees them back, which is what that flag
+is for.
+
+Two ordering bugs came with it, both caught before release. The console has to be taken
+**before** the Rails check, because `route!` runs before the app boots and most of the
+noise is emitted *by* booting it. And colour detection was asking `$stdout.tty?` — which
+by then is a file, and a file is never a tty, so colour would have silently switched off
+for everybody.
+
+Not intercepted: a write straight to file descriptor 2. Capturing that would also swallow
+a real crash and break `binding.pry`, so `2>/dev/null` stays the user's decision.
+
+
+## [1.3.1]
+
+### Cold cases now read `.rspec`
+
+The bug that mattered. `.rspec` is where an RSpec suite says what to load before any spec
+file:
+
+```
+--require spec_helper
+--require rails_helper
+```
+
+That is what `rspec --init` generates, and it is why a real spec file usually has no
+`require` line of its own — there is nothing for it to repeat. RSpec's own runner reads
+those files. Constable's cold-case driver did not.
+
+So a cold case ran with no `rails_helper` at all: no FactoryBot, no shoulda-matchers, no
+`spec/support/**`. On a 1,277-file suite that is **entirely green under
+`bundle exec rspec`**, `constable test spec/models --full` reported:
+
+```
+✓ 774 passed   ✗ 3589 failed
+```
+
+2,637 of them `undefined method 'create'`, the rest `belong_to`,
+`validate_presence_of`, missing support constants. Every one of them Constable's fault,
+and every one of them looking like the user's.
+
+Cold cases now apply the `--require` directives from `.rspec`, `~/.rspec`, `.rspec-local`
+and `SPEC_OPTS`, parsed by RSpec itself so the precedence is its own rather than a guess.
+Only `--require` is taken: formatters, colour and output streams belong to Constable's
+reporter, ordering is Constable's job, and a `--tag` filter meant for a different run
+should not silently drop tests from this one.
+
+The same file is now 14 passed, 0 failed — matching `bundle exec rspec` exactly.
+
+This never showed up before because the app it was developed against wrote
+`require "rails_helper"` at the top of every spec, which is the one arrangement that
+hides it.
+
+### `constable import` says what it actually did
+
+It reported `did reopen 1277 file(s)`. "Reopen" is this code's internal word and means
+nothing to a reader, and "import" on its own suggests the files were copied somewhere —
+which is the opposite of what happened. A real user read it exactly that way and asked
+why their specs had not moved into `test/`.
+
+```
+Adopted 1277 rspec files as cold cases.
+
+  Your rspec files stay exactly where they are and are not changed.
+  One line was added to .constable/config.yml:
+
+    cold_cases:
+      - spec/**/*_spec.rb
+
+  Constable runs them from there, through real RSpec, and folds
+  the results into its own reporting, flake history and CI gate.
+
+  Next:  constable test --full        run everything, cold and native
+         constable test --unsafe      run only these
+         constable modernize PATH     see what one file would look like
+                                      as a native case (writes nothing)
+```
+
+
 ## [1.3.0]
 
 Adoption at scale. Everything here came from installing 1.2.0 into a 1,277-spec-file
@@ -462,7 +554,9 @@ Initial release.
 - Diff-based coverage gate — only lines changed in the current diff are held to the
   threshold. `constable beat` for the full picture, `--html` for a browsable report.
 
-[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v1.3.0...HEAD
+[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v1.3.2...HEAD
+[1.3.2]: https://github.com/Ray-Hughes/constable/compare/v1.3.1...v1.3.2
+[1.3.1]: https://github.com/Ray-Hughes/constable/compare/v1.3.0...v1.3.1
 [1.3.0]: https://github.com/Ray-Hughes/constable/compare/v1.2.0...v1.3.0
 [1.2.0]: https://github.com/Ray-Hughes/constable/compare/v1.1.0...v1.2.0
 [1.1.0]: https://github.com/Ray-Hughes/constable/compare/v1.0.0...v1.1.0

data/README.md

@@ -422,8 +422,16 @@ and says why — slow is a trade-off, wrong is not.
 
 ### Output
 
-stdout is reserved for results. `Rails.logger`, SQL and request/response logging go to
-`log/test.log`; `--verbose` streams it back for active debugging.
+stdout is reserved for results — not just Constable's own output, but the app's. Rails
+loggers, SQL, request/response logging, and anything a gem prints to `$stdout` or
+`$stderr` mid-run all go to `log/test.log`; `--verbose` streams it back for active
+debugging.
+
+That matters more than it sounds. A gem warning fired once per file lands in the middle of
+the live stream, and you get `Address ✓✓✓✓✓✓To use retry middleware with Faraday...`
+instead of a readable run. The one thing Constable deliberately does not intercept is a
+write straight to file descriptor 2 — capturing that would also swallow a real crash and
+break `binding.pry`, so `2>/dev/null` stays yours to decide on.
 
 **While it runs**, the live stream has two modes. `concise` is the default: one glyph per
 test, grouped into a run per case, so a thousand-test suite stays inside one screen and a

data/lib/constable/cli.rb

@@ -168,8 +168,8 @@ module Constable
         strategy: options[:strategy].to_sym
       )
 
+      # The summary says this in full now, in the reader's own terms.
       say result.summary
-      say "\nNothing was rewritten -- cold cases run through their own engine, unchanged." if result.any_changes?
       exit(EXIT_CLEAN)
     end
 
@@ -501,14 +501,19 @@ module Constable
 
       def color?
         return false if options[:"no-color"]
-        return false unless $stdout.tty?
+        # The console, not $stdout -- which by now is log/test.log, and a file is never a
+        # tty. Asking the wrong one silently turns colour off for everybody.
+        return false unless LogRouter.console.tty?
         return false unless ENV["NO_COLOR"].to_s.empty?
 
         true
       end
 
+      # LogRouter.console, not $stdout: by this point route! has pointed $stdout at
+      # log/test.log so a stray gem warning cannot land in the middle of the live stream.
+      # The reporter is the one thing that still writes to the terminal.
       def reporter(config)
-        Reporter.new(io: $stdout, config: config, color: color?, mode: output_mode)
+        Reporter.new(io: LogRouter.console, config: config, color: color?, mode: output_mode)
       end
 
       # nil means "the config file decides". The explicit --output wins over the two

data/lib/constable/cold_case/rspec.rb

@@ -278,9 +278,53 @@ module Constable
           # rspec-core's own default; restated because rspec-rails turns it off and a
           # verbatim legacy file may well open with a bare `describe`.
           configuration.expose_dsl_globally = true if configuration.respond_to?(:expose_dsl_globally=)
+          apply_rspec_options(configuration)
           @session_prepared = true
         end
 
+        # `.rspec` is where a suite says what to load before any spec file.
+        #
+        #   --require spec_helper
+        #   --require rails_helper
+        #
+        # That is what `rspec --init` generates, and it is why a real spec file usually
+        # has no `require` line of its own -- there is nothing for it to repeat. RSpec's
+        # own runner reads those files; driving example groups directly does not, so
+        # without this a cold case runs with no rails_helper at all: no FactoryBot, no
+        # shoulda-matchers, no spec/support. In one real suite that was 2,637 tests
+        # failing on `undefined method 'create'` -- a suite that is entirely green under
+        # `bundle exec rspec`.
+        #
+        # Only `--require` is taken. Formatters, colour and output streams belong to
+        # Constable's reporter, ordering is Constable's job, and a `--tag` filter from a
+        # file RSpec would apply to its own run should not silently drop tests from this
+        # one. Requires are the part that decides whether the suite can run at all.
+        def apply_rspec_options(configuration)
+          requires = rspec_option_requires
+          return if requires.empty?
+
+          # `requires=` rather than plain Kernel#require: it is RSpec's own accessor, and
+          # it puts `lib` and the default path (`spec`) on the load path first, which is
+          # what makes a bare `require "rails_helper"` resolve.
+          configuration.requires = requires
+        rescue StandardError => e
+          # A helper that will not load is the suite's problem to fix, and it will say so
+          # loudly on the first file. Constable's job here is not to disappear.
+          Constable.warn!("could not load what .rspec requires (#{e.class}: #{e.message}). " \
+                          "Cold cases will run without it.", kind: :cold_case)
+        end
+
+        # Parsed by RSpec itself, so `.rspec`, `~/.rspec`, `.rspec-local` and SPEC_OPTS are
+        # all read with its precedence rather than a guess at the format.
+        def rspec_option_requires
+          return [] unless defined?(::RSpec::Core::ConfigurationOptions)
+
+          options = ::RSpec::Core::ConfigurationOptions.new([]).options
+          Array(options[:requires])
+        rescue StandardError
+          []
+        end
+
         def clear_examples
           world = ::RSpec.instance_variable_get(:@world)
           world.reset if world.respond_to?(:reset)

data/lib/constable/importer/reopener.rb

@@ -114,12 +114,26 @@ module Constable
           }
         end
 
+        # Written for somebody who has just typed `constable import` and does not yet know
+        # what a cold case is. "reopen" is the word this code uses internally and it means
+        # nothing to a reader; "import" itself suggests files were copied somewhere, which
+        # is the opposite of what happened. So: say where the files are, say what changed,
+        # and say what to do next.
         def summary
-          verb = dry_run? ? "would" : "did"
-          lines = ["#{@from} import (#{@strategy}) -- #{verb} reopen #{imported_count} file(s)"]
+          verb = dry_run? ? "Would adopt" : "Adopted"
+          noun = imported_count == 1 ? "file" : "files"
+          lines = ["#{verb} #{imported_count} #{@from} #{noun} as cold cases."]
+
           unless @globs_added.empty?
-            lines << "  config path match: added #{@globs_added.size} glob(s) to #{@config_path}"
-            @globs_added.each { |glob| lines << "    - #{glob}   (#{covered_by(glob).size} files, 0 file changes)" }
+            lines << ""
+            lines << "  Your #{@from} files stay exactly where they are and are not changed."
+            lines << "  #{dry_run? ? "One line would be added to" : "One line was added to"} " \
+                     "#{@config_path}:"
+            lines << ""
+            @globs_added.each { |glob| lines << "    cold_cases:\n      - #{glob}" }
+            lines << ""
+            lines << "  Constable runs them from there, through real #{engine_label}, and folds"
+            lines << "  the results into its own reporting, flake history and CI gate."
             unless comments_preserved?
               lines << "  note: config.yml was rewritten from parsed YAML; comments were not preserved"
             end
@@ -133,9 +147,20 @@ module Constable
             @skipped.each { |s| lines << "    - #{s[:path]} (#{s[:reason]})" }
           end
           @errors.each { |e| lines << "  error: #{e[:path]} -- #{e[:message]}" }
+
+          unless dry_run? || imported_count.zero?
+            lines << ""
+            lines << "  Next:  constable test --full        run everything, cold and native"
+            lines << "         constable test --unsafe      run only these"
+            lines << "         constable modernize PATH     see what one file would look like"
+            lines << "                                      as a native case (writes nothing)"
+          end
+
           lines.join("\n")
         end
 
+        def engine_label = @from.to_s == "rspec" ? "RSpec" : "Minitest"
+
         def superclass_name = ENGINES.fetch(@from)[:superclass]
 
         def covered_by(glob)

data/lib/constable/log_router.rb

@@ -55,6 +55,14 @@ module Constable
       def sync         = true
       def tty?         = false
 
+      # Something that has been handed $stdout may ask it for a file descriptor. Answer
+      # with the first target that has one rather than raising NoMethodError from inside
+      # somebody else's gem.
+      def fileno
+        @targets.each { |t| return t.fileno if t.respond_to?(:fileno) }
+        nil
+      end
+
       def sync=(value)
         @targets.each { |target| target.sync = value if target.respond_to?(:sync=) }
       end
@@ -89,12 +97,18 @@ module Constable
       def route!(verbose: false, path: nil, io: $stdout, root: nil)
         target = path ? File.expand_path(path.to_s, root || Constable.root) : default_path(root)
 
+        # Taking the console does not depend on Rails, and the ordering matters: route! runs
+        # before the app boots, so anything that waited for Rails would miss every warning
+        # emitted *by* booting it -- which is most of them. A suite with no Rails at all
+        # still wants its stdout kept for results.
+        file = open_log(target)
+        capture_console!(file, verbose ? io : nil)
+
         unless rails_loaded?
-          reason = "Rails is not loaded — nothing to route, stdout is already results only"
+          reason = "Rails is not loaded — no loggers to route, but stdout is held for results"
           return @current = Routing.new(routed: false, path: target, verbose: verbose, reason: reason)
         end
 
-        file   = open_log(target)
         logger = build_logger(verbose ? Tee.new(file, io) : file)
 
         remember_previous!
@@ -103,9 +117,40 @@ module Constable
         @current = Routing.new(routed: true, path: target, verbose: verbose, reason: nil)
       end
 
+      # The terminal, as it was before route! took it. The reporter writes here; everything
+      # else writes to the log.
+      def console = @console_out || $stdout
+
+      # Rails loggers are not the only thing that writes to a terminal. A gem warning --
+      # Faraday's "install the faraday-retry gem", say -- goes straight to $stderr, once
+      # per file that triggers it, and lands in the middle of the live stream:
+      #
+      #   Address    ✓✓✓✓✓✓✓✓✓✓✓To use retry middleware with Faraday v2.0+...
+      #
+      # stdout is supposed to be results only. So the app's stdout and stderr are pointed
+      # at log/test.log for the duration of the run, and the reporter keeps the real
+      # terminal it captured beforehand. `--verbose` tees both back, which is the whole
+      # point of that flag.
+      def capture_console!(file, tee_to)
+        @console_out = $stdout
+        @console_err = $stderr
+
+        replacement = tee_to ? Tee.new(file, tee_to) : file
+        $stdout = replacement
+        $stderr = replacement
+      end
+
+      def restore_console!
+        $stdout = @console_out if @console_out
+        $stderr = @console_err if @console_err
+        @console_out = nil
+        @console_err = nil
+      end
+
       # Puts back whatever the app had before route!. Only useful in-process (our own
       # suite, or a REPL); a real run never wants its logs back on stdout.
       def restore!
+        restore_console!
         (@previous || {}).each do |constant_name, (setter, logger)|
           constant = resolve(constant_name)
           constant&.public_send(setter, logger)

data/lib/constable/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Constable
-  VERSION = "1.3.0"
+  VERSION = "1.3.2"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: constable-rails
 version: !ruby/object:Gem::Version
-  version: 1.3.0
+  version: 1.3.2
 platform: ruby
 authors:
 - Ray Hughes