constable-rails 1.3.1 → 1.3.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e3b589fed13c38522dab7443ab1bb967bedeaec7abb34beaecd5c4f2c9a4298a
-  data.tar.gz: b646c7160830f81bc2c6c5af9fe60728a8c7e1f40236e594fb64a51eed04080f
+  metadata.gz: fb4658a48f23de1dcb97417de5bab1deda4548a334e2a198835f93b7cac656c3
+  data.tar.gz: c844e9f7ac1c8d797d60349b7b15759bbcbbb0f1b56c914d43e2511e17f75489
 SHA512:
-  metadata.gz: ee0579e80cb81345601ce2c40090db63b87b1498698a77fbc8c34d0df478d36f3210295b3909bd8290835a1f245f2522952f491caca6776d7ac28981dcae7b74
-  data.tar.gz: aea8a0a67d5bdc8868706db9820af271ded921871da905c9e09fe83ac74a359c8c5e9ef8649d737367075f8faa9fabc32fff13156a8f2a9f6e9f69516464441a
+  metadata.gz: c3e78d5c2d2ab5314ee501795a92a1017fbfe0eb4cab6889c7ab6883d1ff898b02a595a693c3bb774c8e8e26caed2132c4f0c033723e7389735386f7b6c43668
+  data.tar.gz: 9f60ad3894558087d8747a5308e39558ce03d87c1518608dff21c9ca341fc776f11ceea45b7251435f052654ead1bbf348c489086b5857cea8e1b0a845781ff8

data/CHANGELOG.md

@@ -5,6 +5,97 @@ All notable changes to this project are documented here. This project adheres to
 
 ## [Unreleased]
 
+## [1.3.3]
+
+**1.3.2 was tagged twice.** The console fix in it was rebuilt after the gem had already
+been pushed, and RubyGems will not accept a second push of the same version — so the
+published 1.3.2 does not contain it. That fix is here, along with two more found on a
+1,277-file suite.
+
+### Shared examples survive across cold-case files
+
+A suite that keeps shared examples in a plain file beside its specs —
+`require_relative "appeal_shared_examples"` at the top of `appeal_spec.rb` — registers
+them the first time that file loads. `require` never fires again, so every later file
+that shares them died on load:
+
+```
+✗ spec/models/legacy_appeal_spec.rb  "failed to load"
+  ArgumentError: Could not find shared examples "toggle overtime"
+```
+
+Twelve files in one run, every one of which passes in isolation. RSpec never hits this
+because it loads every spec file and *then* runs them; Constable loads one at a time,
+which is what makes a cold case cheap, so the registry has to be carried across by hand.
+
+Same two files, before and after: 12 files failing to load → `458 passed, 0 failed`.
+
+### Rename suggestions are a section, not a wall
+
+They were printed unlabelled after SLOWEST, one long line per suggestion, each carrying
+two full test descriptions and two hashes. A real suite produced two hundred of them —
+several hundred lines with no heading, burying every section above.
+
+Now a `RENAMED?` section: eight at a time, three short lines each, with the rest counted.
+Descriptions are truncated to the frame, because an RSpec description built from a matcher
+carries the entire inspected object — every column of a record, ids and timestamps
+included.
+
+### The app's stdout, again (from 1.3.2, unpublished)
+
+Reassigning the `$stdout` object was not enough: a gem writing through the `STDERR`
+constant, or anything already holding the descriptor, goes straight past it, and in a
+terminal both streams land in the same place. The descriptors are now redirected with
+`IO#reopen`, with an `at_exit` guard so a crash still prints where it can be seen.
+
+CLI errors no longer use `Kernel#warn`. Rails apps routinely override `Warning.warn` to
+funnel Ruby warnings into `Rails.logger`, and a real one did: `no tests matched ...`
+arrived in `log/test.log` tagged `[RUBY WARNING]` while the terminal showed nothing at
+all, so the command looked like it had silently done nothing.
+
+
+## [1.3.2]
+
+### stdout is results only — the app's stdout too, not just Constable's
+
+`log/test.log` has taken Rails' loggers since 1.0.0. It did not take anything a gem writes
+directly to `$stdout` or `$stderr`, and a warning fired once per file lands in the middle
+of the live stream:
+
+```
+Address    ✓✓✓✓✓✓✓✓✓✓✓To use retry middleware with Faraday v2.0+, install `faraday-retry` gem
+```
+
+Both streams are now pointed at the log for the duration of a run, and the reporter keeps
+the terminal it captured beforehand. `--verbose` tees them back, which is what that flag
+is for.
+
+Two ordering bugs came with it, both caught before release. The console has to be taken
+**before** the Rails check, because `route!` runs before the app boots and most of the
+noise is emitted *by* booting it. And colour detection was asking `$stdout.tty?` — which
+by then is a file, and a file is never a tty, so colour would have silently switched off
+for everybody.
+
+Reassigning the `$stdout` *object* turned out not to be enough — a gem writing through
+the `STDERR` constant, or anything already holding the descriptor, goes straight past it,
+and in a terminal both streams land in the same place. So the **descriptors** are
+redirected with `IO#reopen`, which is the only thing that catches every writer, and the
+reporter is handed a dup of the real terminal taken beforehand. An `at_exit` guard puts
+them back, so an uncaught exception still prints its backtrace where you can see it.
+
+Two things that fell out of doing it properly:
+
+- **CLI errors no longer use `Kernel#warn`.** With the descriptors redirected an error
+  would have gone into the log — and `warn` would not have reached the terminal even
+  without that, because Rails apps routinely override `Warning.warn` to funnel Ruby
+  warnings into `Rails.logger`. A real one did: `no tests matched ...` arrived in
+  `log/test.log` tagged `[RUBY WARNING]` while the terminal showed nothing at all. Errors
+  now go to the console the reporter kept, on stderr.
+- **Colour detection was asking the wrong stream.** `$stdout.tty?`, when `$stdout` is by
+  then a log file and a file is never a tty — colour would have silently switched off for
+  everybody.
+
+
 ## [1.3.1]
 
 ### Cold cases now read `.rspec`
@@ -528,7 +619,9 @@ Initial release.
 - Diff-based coverage gate — only lines changed in the current diff are held to the
   threshold. `constable beat` for the full picture, `--html` for a browsable report.
 
-[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v1.3.1...HEAD
+[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v1.3.3...HEAD
+[1.3.3]: https://github.com/Ray-Hughes/constable/compare/v1.3.2...v1.3.3
+[1.3.2]: https://github.com/Ray-Hughes/constable/compare/v1.3.1...v1.3.2
 [1.3.1]: https://github.com/Ray-Hughes/constable/compare/v1.3.0...v1.3.1
 [1.3.0]: https://github.com/Ray-Hughes/constable/compare/v1.2.0...v1.3.0
 [1.2.0]: https://github.com/Ray-Hughes/constable/compare/v1.1.0...v1.2.0

data/README.md

@@ -422,8 +422,16 @@ and says why — slow is a trade-off, wrong is not.
 
 ### Output
 
-stdout is reserved for results. `Rails.logger`, SQL and request/response logging go to
-`log/test.log`; `--verbose` streams it back for active debugging.
+stdout is reserved for results — not just Constable's own output, but the app's. Rails
+loggers, SQL, request/response logging, and anything a gem prints to `$stdout` or
+`$stderr` mid-run all go to `log/test.log`; `--verbose` streams it back for active
+debugging.
+
+That matters more than it sounds. A gem warning fired once per file lands in the middle of
+the live stream, and you get `Address ✓✓✓✓✓✓To use retry middleware with Faraday...`
+instead of a readable run. The one thing Constable deliberately does not intercept is a
+write straight to file descriptor 2 — capturing that would also swallow a real crash and
+break `binding.pry`, so `2>/dev/null` stays yours to decide on.
 
 **While it runs**, the live stream has two modes. `concise` is the default: one glyph per
 test, grouped into a run per case, so a thousand-test suite stays inside one screen and a

data/lib/constable/cli.rb

@@ -21,13 +21,29 @@ module Constable
     # tier, an unreadable config. Both are the user's mistake rather than a crash, and
     # both deserve one sentence and a usage status instead of a backtrace.
     rescue Thor::Error, Constable::Error => e
-      warn e.message
+      complain(e.message)
       exit(EXIT_USAGE)
     rescue Interrupt
-      warn "\ninterrupted"
+      complain("\ninterrupted")
       exit(EXIT_FAILED)
     end
 
+    # Anything the user has to see, written where they will see it.
+    #
+    # Not Kernel#warn. A run points stdout and stderr at log/test.log, so an error has to
+    # go to the console the reporter kept -- and `warn` would not reach it even without
+    # that: Rails apps routinely override Warning.warn to funnel Ruby warnings into
+    # Rails.logger, and a real one did. The message arrived in log/test.log tagged
+    # "[RUBY WARNING]" while the terminal showed nothing at all.
+    def self.complain(message)
+      LogRouter.restore!
+      io = LogRouter.console_err
+      io.puts(message)
+      io.flush if io.respond_to?(:flush)
+    rescue StandardError
+      $stderr.puts(message) # rubocop:disable Style/StderrPuts
+    end
+
     class_option :"no-color", type: :boolean, default: false, desc: "Disable ANSI color"
 
     desc "test [PATH[:LINE]]", "Run the suite -- native and cold cases side by side"
@@ -184,7 +200,7 @@ module Constable
     option :"show-source", type: :boolean, default: false, desc: "Print the rewritten source"
     def modernize(*paths)
       if paths.empty?
-        warn "modernize needs at least one path"
+        CLI.complain("modernize needs at least one path")
         exit(EXIT_USAGE)
       end
 
@@ -309,7 +325,7 @@ module Constable
 
           identity = jail.resolve(locator)
           unless identity
-            warn "Nothing on the docket at #{locator}"
+            CLI.complain("Nothing on the docket at #{locator}")
             exit(EXIT_USAGE)
           end
           yield jail, identity
@@ -320,9 +336,9 @@ module Constable
         def refuse_ambiguous(locator, candidates, noun)
           return if candidates.size <= 1
 
-          warn "#{locator} matches #{candidates.size} tests #{noun}. Name one:"
+          CLI.complain("#{locator} matches #{candidates.size} tests #{noun}. Name one:")
           candidates.sort_by { |entry| entry.line.to_i }.each do |entry|
-            warn "  #{entry.location}  #{entry.label}"
+            CLI.complain("  #{entry.location}  #{entry.label}")
           end
           exit(EXIT_USAGE)
         end
@@ -368,7 +384,7 @@ module Constable
 
         identity = warrants.resolve(locator)
         unless identity
-          warn "No warrant at #{locator}"
+          CLI.complain("No warrant at #{locator}")
           exit(EXIT_USAGE)
         end
         warrants.release(identity)
@@ -413,7 +429,7 @@ module Constable
     def prepare
       config = load_config
       unless WorkerDatabases.shardable?
-        warn "This app has no ActiveRecord test databases to prepare."
+        CLI.complain("This app has no ActiveRecord test databases to prepare.")
         exit(EXIT_USAGE)
       end
 
@@ -429,7 +445,7 @@ module Constable
           "rebuilding them."
       0
     rescue Constable::Error => e
-      warn e.message
+      CLI.complain(e.message)
       exit(EXIT_FAILED)
     end
 
@@ -501,14 +517,19 @@ module Constable
 
       def color?
         return false if options[:"no-color"]
-        return false unless $stdout.tty?
+        # The console, not $stdout -- which by now is log/test.log, and a file is never a
+        # tty. Asking the wrong one silently turns colour off for everybody.
+        return false unless LogRouter.console.tty?
         return false unless ENV["NO_COLOR"].to_s.empty?
 
         true
       end
 
+      # LogRouter.console, not $stdout: by this point route! has pointed $stdout at
+      # log/test.log so a stray gem warning cannot land in the middle of the live stream.
+      # The reporter is the one thing that still writes to the terminal.
       def reporter(config)
-        Reporter.new(io: $stdout, config: config, color: color?, mode: output_mode)
+        Reporter.new(io: LogRouter.console, config: config, color: color?, mode: output_mode)
       end
 
       # nil means "the config file decides". The explicit --output wins over the two

data/lib/constable/cold_case/rspec.rb

@@ -149,6 +149,7 @@ module Constable
           @session_configuration = nil
           @session_prepared = false
           @ran_before_suite_hooks = nil
+          @shared_examples = nil
           nil
         end
 
@@ -252,9 +253,11 @@ module Constable
           ::RSpec.instance_variable_set(:@configuration, @session_configuration)
           prepare_session_configuration(::RSpec.configuration)
           clear_examples
+          restore_shared_examples!
 
           yield
         ensure
+          remember_shared_examples!
           @session_world         = ::RSpec.instance_variable_get(:@world)
           @session_configuration = ::RSpec.instance_variable_get(:@configuration)
           clear_examples
@@ -325,6 +328,38 @@ module Constable
           []
         end
 
+        # Shared examples are registered on the world, by `require`, once per process.
+        #
+        # A suite that keeps them in a plain file next to its specs -- `require_relative
+        # "appeal_shared_examples"` at the top of appeal_spec.rb -- registers them while
+        # the first file runs. `require` never fires again, so if the registry does not
+        # survive into the next file, every later file that shares them dies on load with
+        # `Could not find shared examples "toggle overtime"`.
+        #
+        # Under RSpec that never happens: it loads every spec file first, then runs them.
+        # Constable loads one file at a time, which is what makes a cold case cheap, so
+        # the registry has to be carried across by hand. Observed on a real suite: twelve
+        # files failing to load, all of which pass in isolation.
+        def remember_shared_examples!
+          world = ::RSpec.instance_variable_get(:@world)
+          return unless world.respond_to?(:shared_example_group_registry)
+
+          @shared_examples = world.shared_example_group_registry
+        rescue StandardError
+          nil
+        end
+
+        def restore_shared_examples!
+          return if @shared_examples.nil?
+
+          world = ::RSpec.world
+          return unless world.respond_to?(:shared_example_group_registry)
+
+          world.instance_variable_set(:@shared_example_group_registry, @shared_examples)
+        rescue StandardError
+          nil
+        end
+
         def clear_examples
           world = ::RSpec.instance_variable_get(:@world)
           world.reset if world.respond_to?(:reset)

data/lib/constable/log_router.rb

@@ -55,6 +55,14 @@ module Constable
       def sync         = true
       def tty?         = false
 
+      # Something that has been handed $stdout may ask it for a file descriptor. Answer
+      # with the first target that has one rather than raising NoMethodError from inside
+      # somebody else's gem.
+      def fileno
+        @targets.each { |t| return t.fileno if t.respond_to?(:fileno) }
+        nil
+      end
+
       def sync=(value)
         @targets.each { |target| target.sync = value if target.respond_to?(:sync=) }
       end
@@ -89,12 +97,18 @@ module Constable
       def route!(verbose: false, path: nil, io: $stdout, root: nil)
         target = path ? File.expand_path(path.to_s, root || Constable.root) : default_path(root)
 
+        # Taking the console does not depend on Rails, and the ordering matters: route! runs
+        # before the app boots, so anything that waited for Rails would miss every warning
+        # emitted *by* booting it -- which is most of them. A suite with no Rails at all
+        # still wants its stdout kept for results.
+        file = open_log(target)
+        capture_console!(file, verbose ? io : nil)
+
         unless rails_loaded?
-          reason = "Rails is not loaded — nothing to route, stdout is already results only"
+          reason = "Rails is not loaded — no loggers to route, but stdout is held for results"
           return @current = Routing.new(routed: false, path: target, verbose: verbose, reason: reason)
         end
 
-        file   = open_log(target)
         logger = build_logger(verbose ? Tee.new(file, io) : file)
 
         remember_previous!
@@ -103,9 +117,99 @@ module Constable
         @current = Routing.new(routed: true, path: target, verbose: verbose, reason: nil)
       end
 
+      # The terminal, as it was before route! took it. The reporter writes here; everything
+      # else writes to the log.
+      def console = @console_out || $stdout
+
+      # The same, for the stream errors belong on. Convention puts them on stderr, and a
+      # run has pointed the real one at the log.
+      def console_err = @console_err || $stderr
+
+      # Rails loggers are not the only thing that writes to a terminal. A gem warning --
+      # Faraday's "install the faraday-retry gem", say -- goes straight to $stderr, once
+      # per file that triggers it, and lands in the middle of the live stream:
+      #
+      #   Address    ✓✓✓✓✓✓✓✓✓✓✓To use retry middleware with Faraday v2.0+...
+      #
+      # stdout is supposed to be results only. So the app's stdout and stderr are pointed
+      # at log/test.log for the duration of the run, and the reporter keeps the real
+      # terminal it captured beforehand. `--verbose` tees both back, which is the whole
+      # point of that flag.
+      def capture_console!(file, tee_to)
+        # Reassigning the $stdout *object* is not enough. A gem that writes through the
+        # STDERR constant, a C extension, or anything that already holds file descriptor 2
+        # goes straight past it -- and in a terminal both streams land in the same place,
+        # so "it was only on stderr" is no comfort at all when it lands mid-glyph.
+        #
+        # So the descriptors themselves are pointed at the log, and the reporter is handed
+        # a dup of the real terminal taken beforehand. #reopen changes where fd 1 and 2
+        # write for the whole process, which is the only thing that catches every writer.
+        @console_out = $stdout.dup
+        @console_err = $stderr.dup
+        @reopened    = false
+
+        # --verbose means "show me everything", so nothing is redirected; the object swap
+        # is enough to tee. The same fallback covers anything that is not a real IO on
+        # both sides -- a StringIO standing in for the terminal in a test, most obviously,
+        # where IO#reopen has nothing to reopen onto.
+        if tee_to || !redirectable?(file)
+          @swapped_out = $stdout
+          @swapped_err = $stderr
+          replacement  = tee_to ? Tee.new(file, tee_to) : file
+          $stdout = replacement
+          $stderr = replacement
+          return
+        end
+
+        $stdout.reopen(file)
+        $stderr.reopen(file)
+        $stdout.sync = true
+        $stderr.sync = true
+        @reopened = true
+
+        # Whatever happens next -- a clean exit, a raise, an interrupt -- the descriptors
+        # go back. Without this an uncaught exception prints its backtrace into
+        # log/test.log and the terminal shows nothing at all, which is a far worse bug
+        # than the one being fixed.
+        install_exit_guard!
+      end
+
+      def install_exit_guard!
+        return if @exit_guard_installed
+
+        @exit_guard_installed = true
+        at_exit { restore_console! }
+      end
+
+      # Every party has to be a real IO: the log we are redirecting to, and the two streams
+      # we are redirecting away from and will later have to put back.
+      def redirectable?(file)
+        [file, $stdout, $stderr].all? { |io| io.is_a?(::IO) && io.respond_to?(:fileno) }
+      rescue StandardError
+        false
+      end
+
+      def restore_console!
+        if @reopened
+          $stdout.reopen(@console_out)
+          $stderr.reopen(@console_err)
+        elsif @swapped_out
+          $stdout = @swapped_out
+          $stderr = @swapped_err
+        end
+
+        [@console_out, @console_err].each { |io| io&.close unless io&.closed? } if @reopened
+        @console_out = nil
+        @console_err = nil
+        @swapped_out = nil
+        @swapped_err = nil
+        @reopened = false
+      end
+
       # Puts back whatever the app had before route!. Only useful in-process (our own
       # suite, or a REPL); a real run never wants its logs back on stdout.
       def restore!
+        restore_console!
         (@previous || {}).each do |constant_name, (setter, logger)|
           constant = resolve(constant_name)
           constant&.public_send(setter, logger)

data/lib/constable/reporter.rb

@@ -50,6 +50,10 @@ module Constable
     # thousand lines that say the same thing.
     COLD_CASE_LIST_LIMIT = 10
 
+    # How many rename suggestions are listed before the rest are counted instead. A real
+    # suite produced two hundred, several hundred lines of output, unlabelled.
+    RENAME_LIST_LIMIT = 8
+
     DEFAULT_SLOWEST = 5
 
     # Column the expanded stream right-aligns durations into. Descriptions are never
@@ -679,27 +683,58 @@ module Constable
 
     # Rename detection is the Runner's job; the reporter only says it out loud. Never
     # its own section -- it is a suggestion, not a finding.
+    # Renames get a section of their own, a cap, and one line each.
+    #
+    # They used to be dumped unlabelled after SLOWEST, one long line per suggestion,
+    # every line carrying both full test descriptions and two hashes. A real suite
+    # produced two hundred of them and several hundred lines of output -- read as a wall
+    # of text with no heading, and buried the sections above it. A suggestion nobody can
+    # read is not a suggestion.
     def rename_suggestions(suggestions)
-      suggestions = Array(suggestions).map { |s| suggestion_line(s) }.compact
+      suggestions = Array(suggestions).map { |s| suggestion(s) }.compact
       return if suggestions.empty?
 
-      writeln
-      suggestions.each { |line| writeln(INDENT + paint(line, :dim)) }
+      section("RENAMED?")
+      shown = suggestions.first(RENAME_LIST_LIMIT)
+
+      each_entry(shown) do |item|
+        writeln(INDENT + paint(item[:from].to_s, :dim))
+        writeln(ENTRY_INDENT + paint("→ #{item[:to]}", :dim))
+        writeln(ENTRY_INDENT + paint("constable history relink #{item[:old_hash]} #{item[:new_hash]}", :dim))
+      end
+
+      remaining = suggestions.size - shown.size
+      hint(if remaining.positive?
+             "...and #{remaining} more. A test's history is keyed on its body, so a test " \
+               "that was reworded and edited in one commit looks like a new one. Relink the " \
+               "ones you recognise; ignore the rest and they age out."
+           else
+             "A test's history is keyed on its body, so one that was reworded and edited " \
+               "in the same commit looks like a new test. Relink it to carry the history over."
+           end)
     end
 
-    def suggestion_line(suggestion)
-      return suggestion.to_s if suggestion.is_a?(String)
-      return nil unless suggestion.respond_to?(:to_h)
+    def suggestion(raw)
+      return { from: raw.to_s, to: nil, old_hash: nil, new_hash: nil } if raw.is_a?(String)
+      return nil unless raw.respond_to?(:to_h)
 
-      s = suggestion.to_h.transform_keys(&:to_sym)
+      s = raw.to_h.transform_keys(&:to_sym)
       from = s[:old_label] || s[:from] || label_for(s[:old_case], s[:old_description])
       to   = s[:new_label] || s[:to]   || label_for(s[:new_case], s[:new_description])
-      old_hash = s[:old_hash] || s[:old_identity]
-      new_hash = s[:new_hash] || s[:new_identity]
       return nil if from.nil? || to.nil?
 
-      "possible rename: #{from} → #{to}, " \
-        "run constable history relink #{old_hash} #{new_hash} to confirm"
+      { from: truncate_label(from), to: truncate_label(to),
+        old_hash: s[:old_hash] || s[:old_identity], new_hash: s[:new_hash] || s[:new_identity] }
+    end
+
+    # A description built by RSpec from a matcher carries the whole inspected object --
+    # ids, timestamps, every column of a record. Printed in full it is unreadable, and it
+    # is the same test either way.
+    def truncate_label(label)
+      text = label.to_s.tr("\n", " ").squeeze(" ")
+      return text if text.length <= RULE_WIDTH - INDENT.length
+
+      "#{text[0, RULE_WIDTH - INDENT.length - 1]}…"
     end
 
     def label_for(case_name, description)

data/lib/constable/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Constable
-  VERSION = "1.3.1"
+  VERSION = "1.3.3"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: constable-rails
 version: !ruby/object:Gem::Version
-  version: 1.3.1
+  version: 1.3.3
 platform: ruby
 authors:
 - Ray Hughes