constable-rails 1.2.0 → 1.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 65a31ecd5db11b141e18762227af70518749d77042d45fad1626cc1713e06289
-  data.tar.gz: 6fe6e92175d4d306e47626a4d66e6e1ed029817c80af644f6c70f88f4c15c9df
+  metadata.gz: '092c12ca7debd9d093f51e3ce6d90332aef25dcb9c16a5cf79b67f716e15a6a5'
+  data.tar.gz: 0b82a9db3e6d66228103161d46635e394591223956aa67a4bc5f929075701954
 SHA512:
-  metadata.gz: 717a3d9604ef554b4216a0fc497626b7bbff207767904c7c73d66805d70b7e3e60c932d030c728123e414823a346040817ddeb715fe84260ff46c8038bb02181
-  data.tar.gz: 4f22e2278306e93af27767a71ae63931ea4b1752dc403ef5b33d604cb6ca8b4c136267f3c3d95f78e74bfdaa581eab2d8f343a65bc49bd8e1180a434bbf1ebeb
+  metadata.gz: 9bbf3170f8fd728513e0384d36477d4281535276446c86d8e97b8d49b32d956101d1fe4fe30d19688b9a37f36d8159f8ee3061ebc1b2801a6ad55f718d721d27
+  data.tar.gz: 300f8c3ecbfb7075492638c355aceb83fc242808e8296ec21798bea71e23bee98ec6b28ffeaad84a11206fd0e6e65b9771c56a441f4e1d12cc730a969a4340ba

data/CHANGELOG.md

@@ -5,6 +5,60 @@ All notable changes to this project are documented here. This project adheres to
 
 ## [Unreleased]
 
+## [1.3.0]
+
+Adoption at scale. Everything here came from installing 1.2.0 into a 1,277-spec-file
+Postgres app.
+
+### `worker_databases: reuse`
+
+Per-worker databases have been rebuilt from schema on every run since 1.0.0, which is
+what Rails does for `rails test`. It is correct by construction — no drift is possible —
+and it is useless for an app whose schema **cannot** rebuild the database by itself. Any
+app with Postgres custom types is in that position: `CREATE TYPE` has no `schema.rb`
+representation, so a from-scratch load fails on a schema that references a type it never
+defines.
+
+```yaml
+worker_databases: schema   # schema (default) | reuse | off
+```
+
+`reuse` connects to `<database>_<index>` when it is already there and builds it from
+schema only when it is not — checking each database separately, which matters for a
+multi-database app where one may be prepared and another not. "Already there" means
+present *and* holding tables: an empty database is not a prepared one, and running a
+suite against no tables is the worst available outcome.
+
+It is also just faster for everyone. A 3,000-line schema is no longer reloaded once per
+worker per run.
+
+`off` skips sharding entirely — an explicit serial run, no attempt and no warning.
+
+New command, for the one-time setup `reuse` needs:
+
+```console
+$ constable prepare              # build <database>_0 .. _<N-1>, once
+$ constable prepare --workers 8
+```
+
+### Hundreds of cold cases no longer bury the summary
+
+Every cold-case file warns, once per run, so that "12 tests not yet under native rules"
+is never something the suite quietly forgets to mention. At 1,277 files that is four
+thousand lines of the same sentence, and the warnings that actually need a decision — an
+`unsafe` block, a jailed test — are lost inside it.
+
+Past ten files they collapse into one line that keeps the numbers, which are the part
+that is supposed to shrink:
+
+```
+⚠ 1277 files running as cold cases, 18432 tests — not yet under native rules.
+  `constable test --unsafe` runs just these.
+```
+
+Fewer than ten are still listed individually, and nothing else is ever collapsed.
+
+
 ## [1.2.0]
 
 Two bugs found installing 1.1.0 into a large real Postgres app (~3,000-line schema,
@@ -408,7 +462,8 @@ Initial release.
 - Diff-based coverage gate — only lines changed in the current diff are held to the
   threshold. `constable beat` for the full picture, `--html` for a browsable report.
 
-[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v1.2.0...HEAD
+[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v1.3.0...HEAD
+[1.3.0]: https://github.com/Ray-Hughes/constable/compare/v1.2.0...v1.3.0
 [1.2.0]: https://github.com/Ray-Hughes/constable/compare/v1.1.0...v1.2.0
 [1.1.0]: https://github.com/Ray-Hughes/constable/compare/v1.0.0...v1.1.0
 [1.0.0]: https://github.com/Ray-Hughes/constable/compare/v0.1.0...v1.0.0

data/README.md

@@ -399,6 +399,7 @@ worse than one that resets.
 | `constable status` | How the suite is doing over time |
 | `constable beat [--html]` | Coverage: overall %, per-file, the unpatrolled list |
 | `constable history relink OLD NEW` | Carry history across a real body change |
+| `constable prepare [--workers N]` | Build the per-worker test databases `worker_databases: reuse` needs |
 | `constable prune [--dry-run]` | Forget docket rows and warrants for tests that no longer exist |
 | `constable import --from=rspec` | Adopt an existing suite as cold cases |
 | `constable modernize PATH` | Opt-in AST rewrite into the native DSL |
@@ -409,7 +410,11 @@ Order is randomized every run for native cases, with the seed printed and replay
 `--seed`. Cold cases keep their own engine's order. Workers run in parallel by default,
 load-balanced by a cached per-test duration index.
 
-Each worker gets **its own database**, built from schema the way `rails test` does it.
+Each worker gets **its own database**, built from schema the way `rails test` does it —
+or kept between runs, with `worker_databases: reuse`, which is both faster and the only
+thing that works for an app whose schema cannot rebuild the database by itself (any app
+with Postgres custom types: `CREATE TYPE` has no `schema.rb` representation). Prepare
+those once with `constable prepare`.
 Sharing one would not be a speed/safety trade but a correctness bug: on SQLite the run
 dissolves into `database is locked`, and on a client/server database tests quietly see
 each other's rows. If your app has ActiveRecord but cannot shard, Constable runs serially

data/lib/constable/cli.rb

@@ -399,6 +399,40 @@ module Constable
       end
     end
 
+    desc "prepare", "Build the per-worker test databases parallel runs need"
+    long_desc <<~DESC
+      For `worker_databases: reuse`. Creates `<database>_0` .. `<database>_<N-1>` and
+      loads the schema into each, once, so later runs can connect straight to them.
+
+      Only useful for an app that has opted into reuse -- the default `schema` mode
+      rebuilds them on every run and needs no preparation. It is the answer for an app
+      whose schema cannot rebuild the database by itself: prepare these once, by whatever
+      means already works for you, and Constable will use them from then on.
+    DESC
+    option :workers, type: :numeric, desc: "How many to prepare (default: the configured worker count)"
+    def prepare
+      config = load_config
+      unless WorkerDatabases.shardable?
+        warn "This app has no ActiveRecord test databases to prepare."
+        exit(EXIT_USAGE)
+      end
+
+      count = (options[:workers] || config.parallel_workers).to_i.clamp(1, 64)
+      say "Preparing #{count} worker #{count == 1 ? "database" : "databases"}..."
+
+      count.times do |index|
+        built = WorkerDatabases.prepare!(index)
+        say "  worker #{index}: #{built.empty? ? "already prepared" : "built #{built.join(", ")}"}"
+      end
+
+      say "\nDone. Set `worker_databases: reuse` so runs connect to these instead of " \
+          "rebuilding them."
+      0
+    rescue Constable::Error => e
+      warn e.message
+      exit(EXIT_FAILED)
+    end
+
     desc "prune", "Forget docket rows and warrants for tests that no longer exist"
     long_desc <<~DESC
       A test's key is a content hash of its body, so editing a jailed test gives it a new

data/lib/constable/cold_case.rb

@@ -195,7 +195,11 @@ module Constable
           "running as a cold case (#{base_class_name}) — " \
           "#{count} #{count == 1 ? "test" : "tests"} not yet under native rules",
           location: location,
-          kind: :cold_case
+          kind: :cold_case,
+          # Carried so the reporter can total them when there are too many to list. A
+          # suite mid-adoption has hundreds of these, and printing every one buries the
+          # things that actually need a decision.
+          tests: count
         )
       end
 

data/lib/constable/config.rb

@@ -21,6 +21,7 @@ module Constable
       "fail_on_warnings" => false,
       "output" => "concise",
       "parallel_workers" => "auto",
+      "worker_databases" => "schema",
       "tiers" => {
         "unit" => "test/cases/models/**/*",
         "integration" => "test/cases/controllers/**/*",
@@ -103,6 +104,24 @@ module Constable
     #             but you can see which test is hanging without waiting for the summary.
     #
     # The summary itself is identical either way. This only affects the live stream.
+    # How a parallel worker gets a database of its own.
+    #
+    #   schema  rebuild `<database>_<index>` from schema on every run. What Rails does for
+    #           `rails test`, and correct by construction: no drift is possible.
+    #   reuse   connect to `<database>_<index>` when it already exists, and build it from
+    #           schema only when it does not. Faster -- a large schema is not reloaded on
+    #           every run -- and the only option that works for an app whose schema cannot
+    #           rebuild the database by itself, which is any app with Postgres custom
+    #           types. The cost is that keeping those databases current is now yours.
+    #   off     do not shard, so do not fork. An explicit serial run, with no attempt and
+    #           no warning.
+    WORKER_DATABASE_MODES = %i[schema reuse off].freeze
+
+    def worker_databases
+      mode = @raw["worker_databases"].to_s.strip.downcase.to_sym
+      WORKER_DATABASE_MODES.include?(mode) ? mode : :schema
+    end
+
     OUTPUT_MODES = %i[concise expanded].freeze
 
     def output_mode

data/lib/constable/reporter.rb

@@ -44,6 +44,12 @@ module Constable
     # without a limit a quiet case could hold the buffer forever.
     STREAM_FLUSH_THRESHOLD = 12
 
+    # How many cold-case files are listed one by one before they are summarised instead.
+    # A suite part-way through adoption has hundreds -- one real app had 1,277 -- and
+    # printing every one buries the warnings that actually need a decision under four
+    # thousand lines that say the same thing.
+    COLD_CASE_LIST_LIMIT = 10
+
     DEFAULT_SLOWEST = 5
 
     # Column the expanded stream right-aligns durations into. Descriptions are never
@@ -609,6 +615,7 @@ module Constable
     end
 
     def section_warnings(warnings)
+      warnings = collapse_cold_cases(warnings)
       return if warnings.empty?
 
       section("WARNINGS")
@@ -624,6 +631,25 @@ module Constable
       end
     end
 
+    # Cold cases are a fact about the suite, not a list of problems: every one says the
+    # same sentence about a different file. Past the limit they become one line that
+    # still carries the number -- which is the part that is supposed to shrink over time,
+    # and the reason the warning exists at all.
+    def collapse_cold_cases(warnings)
+      cold, rest = warnings.partition { |w| w[:kind] == :cold_case }
+      return warnings if cold.size <= COLD_CASE_LIST_LIMIT
+
+      tests = cold.sum { |w| w[:tests].to_i }
+      total = tests.positive? ? ", #{tests} #{pluralize(tests, "test")}" : ""
+
+      [{
+        kind: :cold_case,
+        location: nil,
+        message: "#{cold.size} files running as cold cases#{total} — not yet under " \
+                 "native rules. `constable test --unsafe` runs just these."
+      }] + rest
+    end
+
     # A warning carries the author's own words -- an unsafe block's reason, a cold case's
     # count -- and those are easily ninety columns. Wrapped to the frame, but respecting
     # any line breaks the message already chose.
@@ -707,7 +733,11 @@ module Constable
       w = warning.to_h.transform_keys(&:to_sym)
       return nil if w[:message].nil?
 
-      { message: w[:message].to_s, location: w[:location], kind: (w[:kind] || :unsafe).to_sym }
+      # `tests` rides along so cold cases can be totalled when there are too many to list.
+      # Normalizing is about the three fields the reporter needs, not about discarding
+      # everything a warning chose to carry.
+      { message: w[:message].to_s, location: w[:location], kind: (w[:kind] || :unsafe).to_sym,
+        tests: w[:tests] }.compact
     end
 
     def normalize_coverage(coverage)

data/lib/constable/runner.rb

@@ -283,6 +283,9 @@ module Constable
     #
     # When we cannot shard, we run serially and say why. Slow is a trade-off; wrong is not.
     def parallel_safe?
+      # An explicit opt-out. No attempt, and no warning about one -- the user has already
+      # told us they know.
+      return false if @config.worker_databases == :off
       return true unless WorkerDatabases.active_record?
       return true if WorkerDatabases.shardable?
 
@@ -342,7 +345,7 @@ module Constable
           # A raise here dumps a full stack trace per worker and leaves the parent
           # reporting a run that never happened.
           begin
-            WorkerDatabases.after_fork!(worker_index)
+            WorkerDatabases.after_fork!(worker_index, mode: @config.worker_databases)
           rescue Constable::Error => e
             write_message(writer, :worker_error, e.message)
             writer.close
@@ -400,8 +403,9 @@ module Constable
         "no parallel worker could build its own test database, so the suite ran serially " \
         "instead. This usually means the app's schema cannot rebuild the database by " \
         "itself -- Postgres custom types, functions and triggers are the common reason, " \
-        "and `rails test` parallelization fails the same way. Set `parallel_workers: 1` " \
-        "to skip the attempt. The first worker said: #{reason}",
+        "and `rails test` parallelization fails the same way. Set " \
+        "`worker_databases: reuse` to keep prepared databases between runs instead, or " \
+        "`worker_databases: off` to stop trying. The first worker said: #{reason}",
         kind: :parallel
       )
 

data/lib/constable/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Constable
-  VERSION = "1.2.0"
+  VERSION = "1.3.0"
 end

data/lib/constable/worker_databases.rb

@@ -46,24 +46,115 @@ module Constable
       false
     end
 
-    # Child side, immediately after the fork and before any test runs. Builds
-    # `<database>_<index>` from schema and points this process at it.
+    # Child side, immediately after the fork and before any test runs. Gives this process
+    # a database of its own, named `<database>_<index>`.
     #
-    # ENV["VERBOSE"] is silenced the way Rails silences it: schema loading is chatty, and
-    # stdout belongs to the reporter.
-    def after_fork!(index)
+    # `mode` is the `worker_databases` setting:
+    #
+    #   :schema  rebuild it from schema every run, which is what Rails does. Correct by
+    #            construction -- no drift is possible -- and the right default.
+    #   :reuse   connect to it when it is already there, build it from schema when it is
+    #            not. Skips reloading a large schema on every run, and is the only thing
+    #            that works when the schema cannot rebuild the database by itself.
+    def after_fork!(index, mode: :schema)
       return false unless shardable?
+      return reuse!(index) if mode.to_sym == :reuse
 
       ::ActiveRecord::TestDatabases.create_and_load_schema(index, env_name: env_name)
       true
     rescue StandardError => e
-      # A worker that cannot build its own database would otherwise silently fall back to
-      # sharing the parent's, which is the bug this module exists to prevent. Say so, and
-      # let the failure be a real one.
+      # A worker that cannot build its own database must never fall back to sharing the
+      # parent's -- that is the corruption this module exists to prevent. The runner turns
+      # this into a serial run rather than letting it kill the process.
       raise Constable::Error, "worker #{index} could not create its own test database: " \
                               "#{e.class}: #{e.message}"
     end
 
+    # `constable prepare`. Builds worker `index`'s databases from the parent process --
+    # the same work :reuse would do lazily inside a fork, done once, deliberately, where
+    # the output is visible and a failure is not four stack traces at once.
+    def prepare!(index)
+      raise Constable::Error, "this app has no ActiveRecord databases to prepare" unless shardable?
+
+      # Renaming is destructive, and here it happens in the parent rather than in a fork
+      # that is about to die. Without restoring the names afterwards this command would
+      # leave the process -- and, worse, anything else in it -- pointed at
+      # `<database>_<index>` instead of the real test database.
+      original = database_names
+      begin
+        reuse!(index)
+      ensure
+        restore_database_names(original)
+        ::ActiveRecord::Base.establish_connection
+      end
+    rescue Constable::Error
+      raise
+    rescue StandardError => e
+      raise Constable::Error, "could not prepare worker #{index}: #{e.class}: #{e.message}"
+    end
+
+    def database_names
+      ::ActiveRecord::Base.configurations
+                          .configs_for(env_name: env_name, include_hidden: true)
+                          .map(&:database)
+    end
+
+    def restore_database_names(names)
+      ::ActiveRecord::Base.configurations
+                          .configs_for(env_name: env_name, include_hidden: true)
+                          .zip(names).each { |config, name| config._database = name if name }
+    end
+
+    # The :reuse half. Points every database this environment declares at its `_<index>`
+    # sibling, and only builds the ones that are not there yet.
+    #
+    # "There" means present *and* populated: an empty database is not a prepared one, and
+    # connecting to it would hand the worker a suite with no tables. Deciding that per
+    # database rather than per worker matters for a multi-database app -- Caseflow has a
+    # primary and an ETL database -- where one may be prepared and the other not.
+    #
+    # Keeping these current is the user's job once they opt in, which is the trade the
+    # setting exists to let them make.
+    def reuse!(index)
+      built = []
+
+      each_worker_config(index) do |db_config|
+        next if populated?(db_config)
+
+        ::ActiveRecord::Tasks::DatabaseTasks.reconstruct_from_schema(db_config, nil)
+        built << db_config.database
+      end
+
+      built
+    ensure
+      # Rails does this after its own schema load: the pool has to be re-established
+      # against the renamed configuration before any test asks for a connection.
+      ::ActiveRecord::Base.establish_connection
+    end
+
+    # Every database config for this environment, renamed to its per-worker sibling.
+    # `_database=` is exactly how Rails' own TestDatabases does the renaming, and this
+    # runs in a forked child, so the mutation dies with the worker.
+    def each_worker_config(index)
+      configs = ::ActiveRecord::Base.configurations.configs_for(env_name: env_name,
+                                                                include_hidden: true)
+      configs.each do |db_config|
+        db_config._database = "#{db_config.database}_#{index}"
+        next unless db_config.database_tasks?
+
+        yield db_config
+      end
+    end
+
+    # Present and holding tables. A database that exists but is empty is not prepared, and
+    # silently running a suite against no tables is the worst of the available outcomes.
+    def populated?(db_config)
+      ::ActiveRecord::Base.establish_connection(db_config)
+      ::ActiveRecord::Base.connection.tables.any?
+    rescue StandardError
+      false
+    end
+
     def env_name
       if defined?(::ActiveRecord::ConnectionHandling::DEFAULT_ENV)
         ::ActiveRecord::ConnectionHandling::DEFAULT_ENV.call

data/lib/constable.rb

@@ -97,8 +97,8 @@ module Constable
 
     # Warnings are never silent and never fatal by default. They accumulate through a run
     # and always get their own section in the summary.
-    def warn!(message, location: nil, kind: :unsafe)
-      warnings << { message: message, location: location, kind: kind }
+    def warn!(message, location: nil, kind: :unsafe, **extra)
+      warnings << { message: message, location: location, kind: kind, **extra }
     end
 
     def warnings
@@ -149,7 +149,7 @@ module Constable
     SETTINGS = %i[
       cold_cases warrants warrant_retries auto_relink parole_period
       coverage coverage_threshold coverage_html fail_on_warnings parallel_workers
-      output tiers
+      worker_databases output tiers
     ].freeze
 
     # `storage` is the one setting that cannot live here, and the reason is ordering, not

data/lib/generators/constable/templates/case_helper.rb.tt

@@ -156,6 +156,7 @@ end
 #
 #      c.cold_cases         = []      # RSpec/Minitest globs to run as cold cases
 #      c.parallel_workers   = "auto"  # or an integer. Each worker gets its own database
+#      c.worker_databases   = :schema # or :reuse (keep them between runs) / :off
 #      c.output             = :concise    # or :expanded -- a line per test, with timings
 #      c.fail_on_warnings   = false   # CI: fail when the warning count is not trending down
 #      c.warrants           = false   # rerun a failure in isolation before believing it

data/lib/generators/constable/templates/config.yml.tt

@@ -73,6 +73,21 @@ fail_on_warnings: false  # CI: fail the build when the warning count doesn't tre
 
 parallel_workers: auto   # or an explicit integer
 
+# How a parallel worker gets a database of its own. Sharing one is not a speed/safety
+# trade but a correctness bug: on SQLite the run dissolves into "database is locked", and
+# on a client/server database tests quietly see each other's rows.
+#
+#   schema  rebuild <database>_<index> from schema on every run. What Rails does for
+#           `rails test`, and correct by construction -- no drift is possible.
+#   reuse   connect to <database>_<index> when it is already there, and build it from
+#           schema only when it is not. Faster, because a large schema is not reloaded
+#           every run -- and the only option that works at all when the schema cannot
+#           rebuild the database by itself, which is true of any app with Postgres custom
+#           types (`CREATE TYPE` has no schema.rb representation, so a from-scratch load
+#           fails). Keeping those databases current becomes your job.
+#   off     do not shard, so do not fork. An explicit serial run: no attempt, no warning.
+worker_databases: schema   # schema | reuse | off
+
 # How much the live stream says while the suite is running. The summary is identical
 # either way -- this only changes what you watch on the way there.
 #

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: constable-rails
 version: !ruby/object:Gem::Version
-  version: 1.2.0
+  version: 1.3.0
 platform: ruby
 authors:
 - Ray Hughes