constable-rails 1.1.0 → 1.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c4612ebf00dac193c1557bbd887beccfbd7e7864f157dd88a77386e476897a6e
-  data.tar.gz: f208f84658349658dccef504a21f6a62833d27b9b772d1d91ef5f26216b60ae5
+  metadata.gz: 65a31ecd5db11b141e18762227af70518749d77042d45fad1626cc1713e06289
+  data.tar.gz: 6fe6e92175d4d306e47626a4d66e6e1ed029817c80af644f6c70f88f4c15c9df
 SHA512:
-  metadata.gz: 296f8609681632e8427d481ecc94a15773b581c90dd7a88fa28c66eb584cd492c1706eec3faebc56aec385709ff0e72320db299c3901cec4659b333d8b94f97a
-  data.tar.gz: aeb4696fc4029685e7ffcab4c4b7d3e0287b6a153b86f6c331864353e56546d6b9a9eb0aef640a59dce748ed08e27f6f5e80251b7e521143f1490ed18356d67f
+  metadata.gz: 717a3d9604ef554b4216a0fc497626b7bbff207767904c7c73d66805d70b7e3e60c932d030c728123e414823a346040817ddeb715fe84260ff46c8038bb02181
+  data.tar.gz: 4f22e2278306e93af27767a71ae63931ea4b1752dc403ef5b33d604cb6ca8b4c136267f3c3d95f78e74bfdaa581eab2d8f343a65bc49bd8e1180a434bbf1ebeb

data/CHANGELOG.md

@@ -5,6 +5,56 @@ All notable changes to this project are documented here. This project adheres to
 
 ## [Unreleased]
 
+## [1.2.0]
+
+Two bugs found installing 1.1.0 into a large real Postgres app (~3,000-line schema,
+custom types, a factory directory). Both are the same shape as the ones 1.0.0 fixed:
+Constable was confidently wrong and said nothing useful about it.
+
+### A factory named `*_case.rb` was loaded as a test
+
+`spec/**/*_case.rb` is a generous net, and a real app has things in it that merely share
+the suffix. Caseflow has a FactoryBot factory at `spec/factories/distributed_case.rb`.
+Constable loaded it as a case file, FactoryBot raised `DuplicateDefinitionError` because
+the factory was already registered, and the run reported a failing test in a file that
+contains no tests:
+
+```
+✗ spec/factories/distributed_case.rb
+  "could not be loaded"
+  FactoryBot::DuplicateDefinitionError: Factory already registered: distributed_case
+```
+
+A filename is not evidence. A file outside the conventional `test/cases/` and
+`spec/cases/` directories now has to look like a case before it is loaded — a class
+declaration, or the DSL. Files under those directories are still taken at their word,
+since that is what they are for and an empty one there is a case somebody is part-way
+through writing.
+
+### An app that cannot be sharded now runs anyway
+
+Per-worker databases (1.0.0) are built by loading `schema.rb` into `<database>_<index>`.
+Not every app can do that: one with Postgres custom types, functions or triggers cannot
+rebuild itself from `schema.rb` at all, which is exactly why such apps keep a
+`structure.sql`. Rails' own `parallelize` fails the same way.
+
+Constable handled it about as badly as possible. Each worker raised, printing a full
+stack trace — four workers, four traces, several hundred lines — and the parent then
+reported a run that had never happened:
+
+```
+CONSTABLE            1 test · 1 case · 10.8s
+✓ 0 passed   ✗ 1 failed
+```
+
+A worker that cannot build its database now reports that home rather than raising. If no
+worker got started, nothing has run yet, so the parent simply runs the suite serially and
+says why in one sentence — including the real error and how to skip the attempt
+(`parallel_workers: 1`). The rule from 1.0.0 is unchanged: a worker never falls back to
+sharing the parent's database, because that is the corruption this whole mechanism
+exists to prevent.
+
+
 ## [1.1.0]
 
 Three things that were documented and did not work, plus the command for a docket
@@ -358,7 +408,8 @@ Initial release.
 - Diff-based coverage gate — only lines changed in the current diff are held to the
   threshold. `constable beat` for the full picture, `--html` for a browsable report.
 
-[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v1.1.0...HEAD
+[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v1.2.0...HEAD
+[1.2.0]: https://github.com/Ray-Hughes/constable/compare/v1.1.0...v1.2.0
 [1.1.0]: https://github.com/Ray-Hughes/constable/compare/v1.0.0...v1.1.0
 [1.0.0]: https://github.com/Ray-Hughes/constable/compare/v0.1.0...v1.0.0
 [0.1.0]: https://github.com/Ray-Hughes/constable/releases/tag/v0.1.0

data/lib/constable/runner.rb

@@ -337,9 +337,17 @@ module Constable
           reader.close
 
           # Before a single test runs: build this worker's own database and point the
-          # process at it. Raises rather than falling back to the shared one, because a
-          # silent fallback is the bug we are here to prevent.
-          WorkerDatabases.after_fork!(worker_index)
+          # process at it. Never falls back to the shared one -- that is the bug this
+          # exists to prevent -- but the failure is reported home rather than raised.
+          # A raise here dumps a full stack trace per worker and leaves the parent
+          # reporting a run that never happened.
+          begin
+            WorkerDatabases.after_fork!(worker_index)
+          rescue Constable::Error => e
+            write_message(writer, :worker_error, e.message)
+            writer.close
+            exit!(0)
+          end
 
           bucket.each do |item|
             run_item(item).each { |result| write_message(writer, :result, result.to_h) }
@@ -365,6 +373,17 @@ module Constable
       collected = drain(readers)
       pids.each { |pid| Process.waitpid(pid) rescue nil } # rubocop:disable Style/RescueModifier
 
+      # No worker could build itself a database, so no test ran. Not every app can be
+      # sharded: an app whose schema.rb cannot rebuild the database on its own -- Postgres
+      # custom types, functions and triggers are the usual reason, and are exactly why
+      # such apps use structure.sql -- will fail here every time. Rails' own `parallelize`
+      # fails the same way; the difference is that this is not the user's fault and they
+      # should not have to read four stack traces to find that out.
+      #
+      # Nothing has run yet, so falling back to a serial run costs a restart, not
+      # correctness.
+      return run_serially_after_worker_failure(items) if collected.empty? && worker_errors.any?
+
       # A warning raised inside a worker only ever reached that worker's memory, so the
       # results carry them home. Nothing that bends the rules is allowed to go missing
       # just because it happened in a subprocess.
@@ -372,6 +391,25 @@ module Constable
       collected
     end
 
+    def worker_errors = (@worker_errors ||= [])
+
+    def run_serially_after_worker_failure(items)
+      reason = worker_errors.first.to_s
+
+      Constable.warn!(
+        "no parallel worker could build its own test database, so the suite ran serially " \
+        "instead. This usually means the app's schema cannot rebuild the database by " \
+        "itself -- Postgres custom types, functions and triggers are the common reason, " \
+        "and `rails test` parallelization fails the same way. Set `parallel_workers: 1` " \
+        "to skip the attempt. The first worker said: #{reason}",
+        kind: :parallel
+      )
+
+      # The blotter handle was closed before forking, and the pool was cleared. Both come
+      # back on their next use, so there is nothing to reopen by hand.
+      run_serial(items)
+    end
+
     # Every message on the pipe is tagged, because results are not the only thing a worker
     # has to send home.
     def write_message(writer, kind, body)
@@ -416,6 +454,10 @@ module Constable
               @reporter.record(result)
             when :coverage
               @worker_coverage = Constable::Coverage.merge_raw(@worker_coverage, body)
+            when :worker_error
+              # A worker that could not start. Collected rather than raised, so the parent
+              # decides what to do once it knows whether any worker got going at all.
+              worker_errors << body
             end
           end
         end

data/lib/constable/selection.rb

@@ -17,6 +17,22 @@ module Constable
       def to_s    = line ? "#{path}:#{line}" : path.to_s
     end
 
+    # A filename is not evidence. `spec/**/*_case.rb` is a generous net, and in a real app
+    # it catches things that merely share the suffix -- a FactoryBot factory named
+    # spec/factories/distributed_case.rb, say. Loading one of those runs somebody's code
+    # twice and reports the resulting explosion as a failing test in a file that contains
+    # no tests.
+    #
+    # So the file has to look like a case before we load it: a class declaration, or the
+    # DSL. Deliberately a cheap read rather than a parse -- this runs over every candidate
+    # on every run, and anything a parse would catch that this misses would also have to
+    # be a file that defines a case without mentioning one.
+    NATIVE_MARKERS = /
+      <\s*(?:Constable::Case|\w*Case)\b   # class FooCase < UnitCase
+      | ^\s*investigate\s*[("]             # or the DSL, for a reopened class
+      | ^\s*tier\s+:
+    /x
+
     NATIVE_GLOBS = [
       "test/cases/**/*.rb",
       "spec/cases/**/*.rb",
@@ -183,6 +199,17 @@ module Constable
 
     def native_files
       @native_files ||= glob(NATIVE_GLOBS).reject { |f| @config.cold_case?(f) }
+                                          .select { |f| native_by_content?(f) }
+    end
+
+    def native_by_content?(path)
+      # Files under the conventional case directories are taken at their word: that is
+      # what the directory is for, and an empty one there is a case file being written.
+      return true if path.match?(%r{/(?:test|spec)/cases/})
+
+      File.read(path).match?(NATIVE_MARKERS)
+    rescue StandardError
+      false
     end
 
     def cold_files

data/lib/constable/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Constable
-  VERSION = "1.1.0"
+  VERSION = "1.2.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: constable-rails
 version: !ruby/object:Gem::Version
-  version: 1.1.0
+  version: 1.2.0
 platform: ruby
 authors:
 - Ray Hughes