rspecq-instructure 0.8.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 1df575298a210ef26b7f057dcfd500287519e2ec6f8d32b2296f6eea26bc05fc
+  data.tar.gz: fe66df1684e18af2501fb67cf26ce475a9db89e65717f83e0ff1f3bb9455041c
+SHA512:
+  metadata.gz: 52b16ee95c80f6a7da5246c6335910dde1a897d7ce37ac7d5f99b508aa306d3cdce350d47fc7d641d1c4947f894345eea3d49844df04aab5aa0ae7de97c51bc1
+  data.tar.gz: 31ec697c4f6b54c39a8dad7527d6151e177e45a81ad55e74e6a07ad459ae3c430bf02d35a8674817488b9e56008941875e371e73a293d69e8646ac446aefc7b2

data/CHANGELOG.md

@@ -0,0 +1,106 @@
+# Changelog
+
+Breaking changes are prefixed with a "[BREAKING]" label.
+
+## 0.8.1 (unreleased)
+
+- [BREAKING] The gem is published to rubygems.org as `rspecq-instructure`,
+  not to Artifactory as `rspecq`. Change the dependency to
+  `gem "rspecq-instructure"` and drop the Artifactory source; credentials are
+  no longer needed to `bundle install`.
+
+  Only the package name changed. Everything the gem provides keeps its old
+  name, so no code has to change: `require "rspecq"`, the `RSpecQ` namespace,
+  and the `rspecq` executable all still work as before.
+
+## 0.8.0.instructure2 (unreleased)
+
+- [BREAKING] Require Ruby >= 3.2 and raise the redis-rb floor to `>= 5.0`
+  (cap `< 7.0`). Verified compatible with redis-rb 6.0 / RESP3.
+- `REDIS_GEM` env var pins the redis-rb version in CI (mirrors `RSPEC_CORE`).
+
+## 0.7.1 (2021-04-08)
+
+- New env variable RSPECQ_REPORTER_RERUN_COMMAND_SKIP. When set, the reporter
+  does not include the flaky test's rerun command.
+
+## 0.7.0 (2021-04-01)
+
+- New cli parameter `reproduction`.
+  When passed, primary worker publishes the queue in the same order as passed
+  in the command.
+- Reporter now includes a reproduction command for flaky tests.
+
+## 0.6.0 (2021-03-23)
+
+- New cli parameter `seed`.
+  The seed is passed to the RSpec command.
+
+## 0.5.0 (2021-02-05)
+
+### Added
+
+- New cli parameter `queue_wait_timeout`.
+  It configured the time a queue can wait to be ready. The env equivalent
+  is `RSPECQ_QUEUE_WAIT_TIMEOUT`. [#51](https://github.com/skroutz/rspecq/pull/51)
+
+## 0.4.0 (2020-10-07)
+
+### Added
+
+- Builds can be configured to terminate after a specified number of failures,
+  using the `--fail-fast` option.
+
+
+## 0.3.0 (2020-10-05)
+
+### Added
+
+- Providing a Redis URL is now possible using the `--redis-url` option
+  [[#40](https://github.com/skroutz/rspecq/pull/40)]
+
+### Changed
+
+- [DEPRECATION] The `--redis` option is now deprecated. Use `--redis-host`
+  instead [[#40](https://github.com/skroutz/rspecq/pull/40)]
+
+## 0.2.2 (2020-09-10)
+
+### Fixed
+- Worker would fail if application code was writing to stderr
+ [[#35](https://github.com/skroutz/rspecq/pull/35)]
+
+## 0.2.1 (2020-09-09)
+
+### Changed
+
+- Sentry Integration: Changed the way events for flaky jobs are emitted to a
+  per-flaky-job fashion. This ultimately improves grouping and filtering of the
+  flaky events in Sentry [[#33](https://github.com/skroutz/rspecq/pull/33)]
+
+
+## 0.2.0 (2020-08-31)
+
+This is a feature release with no breaking changes.
+
+### Added
+
+- Flaky jobs are now printed by the reporter in the final build output and also
+  emitted to Sentry (if the integration is enabled) [[#26](https://github.com/skroutz/rspecq/pull/26)]
+
+## 0.1.0 (2020-08-27)
+
+### Added
+
+- Sentry integration for various RSpecQ-level events [[#16](https://github.com/skroutz/rspecq/pull/16)]
+- CLI: Flags can now be also set environment variables [[c519230](https://github.com/skroutz/rspecq/commit/c5192303e229f361e8ac86ae449b4ea84d42e022)]
+- CLI: Added shorthand specifiers versions for some flags [[df9faa8](https://github.com/skroutz/rspecq/commit/df9faa8ec6721af8357cfee4de6a2fe7b32070fc)]
+- CLI: Added `--help` and `--version` flags [[df9faa8](https://github.com/skroutz/rspecq/commit/df9faa8ec6721af8357cfee4de6a2fe7b32070fc)]
+- CLI: Max number of retries for failed examples is now configurable via the `--max-requeues` option [[#14](https://github.com/skroutz/rspecq/pull/14)]
+
+### Changed
+
+- [BREAKING] CLI: Renamed `--timings` to `--update-timings` [[c519230](https://github.com/skroutz/rspecq/commit/c5192303e229f361e8ac86ae449b4ea84d42e022)]
+- [BREAKING] CLI: Renamed `--build-id` to `--build` and `--worker-id` to `--worker` [[df9faa8](https://github.com/skroutz/rspecq/commit/df9faa8ec6721af8357cfee4de6a2fe7b32070fc)]
+- CLI: `--worker` is not required when `--reporter` is used [[4323a75](https://github.com/skroutz/rspecq/commit/4323a75ca357274069d02ba9fb51cdebb04e0be4)]
+- CLI: Improved help output [[df9faa8](https://github.com/skroutz/rspecq/commit/df9faa8ec6721af8357cfee4de6a2fe7b32070fc)]

data/LICENSE

@@ -0,0 +1,20 @@
+The MIT License
+
+Copyright (c) 2020 Skroutz S.A.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software is furnished to do so,
+subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,268 @@
+RSpec Queue
+=========================================================================
+[![Build Status](https://travis-ci.com/skroutz/rspecq.svg?branch=master)](https://travis-ci.com/github/skroutz/rspecq)
+[![Gem Version](https://badge.fury.io/rb/rspecq.svg)](https://badge.fury.io/rb/rspecq)
+
+RSpec Queue (RSpecQ) distributes and executes RSpec suites among parallel
+workers. It uses a centralized queue that workers connect to and pop off
+tests from. It ensures optimal scheduling of tests based on their run time,
+facilitating faster CI builds.
+
+RSpecQ is inspired by [test-queue](https://github.com/tmm1/test-queue)
+and [ci-queue](https://github.com/Shopify/ci-queue).
+
+## Features
+
+- Run an RSpec suite among many workers
+  (potentially located in different hosts) in a distributed fashion,
+  facilitating faster CI builds.
+- Consolidated, real-time reporting of a build's progress.
+- Optimal scheduling of test execution by using timings statistics from previous runs and
+  automatically scheduling slow spec files as individual examples. See
+  [*Spec file splitting*](#spec-file-splitting).
+- Automatic retry of test failures before being considered legit, in order to
+  rule out flakiness. Additionally, flaky tests are detected and provided to
+  the user. See [*Requeues*](#requeues).
+- Handles intermittent worker failures (e.g. network hiccups, faulty hardware etc.)
+  by detecting non-responsive workers and requeing their jobs. See [*Worker failures*](#worker-failures)
+- Sentry integration for monitoring build-level events. See [*Sentry integration*](#sentry-integration).
+  See [#2](https://github.com/skroutz/rspecq/issues/2).
+- Automatic termination of builds after a certain amount of failures. See [*Fail-fast*](#fail-fast).
+
+## Requirements
+
+- Ruby >= 3.2
+- redis-rb 5.x or 6.x (runs on RESP3, the redis-rb 6.0 default), backed by a Redis server >= 6.2 (the `COPY` command is used to promote build timings).
+
+## Usage
+
+A worker needs to be given a name and the build it will participate in.
+Assuming there's a Redis instance listening at `localhost`, starting a worker
+is as simple as:
+
+```shell
+$ rspecq --build=123 --worker=foo1 spec/
+```
+
+To start more workers for the same build, use distinct worker IDs but the same
+build ID:
+
+```shell
+$ rspecq --build=123 --worker=foo2
+```
+
+To view the progress of the build use `--report`:
+
+```shell
+$ rspecq --build=123 --report
+```
+
+For detailed info use `--help`:
+
+```
+NAME:
+    rspecq - Optimally distribute and run RSpec suites among parallel workers
+
+USAGE:
+    rspecq [<options>] [spec files or directories]
+
+OPTIONS:
+    -b, --build ID                   A unique identifier for the build. Should be common among workers participating in the same build.
+    -w, --worker ID                  An identifier for the worker. Workers participating in the same build should have distinct IDs.
+        --seed SEED                  The RSpec seed. Passing the seed can be helpful in many ways i.e reproduction and testing.
+    -r, --redis HOST                 --redis is deprecated. Use --redis-host or --redis-url instead. Redis host to connect to (default: 127.0.0.1).
+        --redis-host HOST            Redis host to connect to (default: 127.0.0.1).
+        --redis-url URL              Redis URL to connect to (e.g.: redis://127.0.0.1:6379/0).
+        --redis-connect-timeout N    Seconds to wait when establishing a Redis connection (default: 1.0).
+        --redis-read-timeout N       Seconds to wait for a Redis read (default: 5.0).
+        --redis-write-timeout N      Seconds to wait for a Redis write (default: 5.0).
+        --redis-reconnect-attempts LIST  Comma-separated backoff (seconds) before each Redis reconnect attempt, e.g. "0.05,0.1,0.25,0.5,1.0". Empty disables reconnects.
+        --update-timings             Update the global job timings key with the timings of this build. Note: This key is used as the basis for job scheduling.
+        --file-split-threshold N     Split spec files slower than N seconds and schedule them as individual examples.
+        --report                     Enable reporter mode: do not pull tests off the queue; instead print build progress and exit when it's finished.
+                                     Exits with a non-zero status code if there were any failures.
+        --report-timeout N           Fail if build is not finished after N seconds. Only applicable if --report is enabled (default: 3600).
+        --max-requeues N             Retry failed examples up to N times before considering them legit failures (default: 3).
+        --queue-wait-timeout N       Time to wait for a queue to be ready before considering it failed (default: 30).
+        --worker-liveness-sec N      Worker liveness timeout in seconds, node will be considered dead if it hasn't emitted a heartbeat for N seconds (default: 60).
+        --fail-fast N                Abort build with a non-zero status code after N failed examples.
+        --reproduction               Enable reproduction mode: Publish files and examples in the exact order given in the command. Incompatible with --timings.
+    -h, --help                       Show this message.
+    -v, --version                    Print the version and exit.
+```
+
+You can set most options using ENV variables:
+
+```shell
+$ RSPECQ_BUILD=123 RSPECQ_WORKER=foo1 rspecq spec/
+```
+
+### Supported ENV variables
+
+| Name | Desc |
+| --- | --- |
+| `RSPECQ_BUILD` | Build ID |
+| `RSPECQ_WORKER` | Worker ID |
+| `RSPECQ_SEED` | RSpec seed |
+| `RSPECQ_REDIS` | Redis HOST |
+| `RSPECQ_UPDATE_TIMINGS` | Timings |
+| `RSPECQ_FILE_SPLIT_THRESHOLD` | File split threshold |
+| `RSPECQ_REPORT` | Report |
+| `RSPECQ_REPORT_TIMEOUT` | Report Timeout |
+| `RSPECQ_MAX_REQUEUES` | Max requests |
+| `RSPECQ_QUEUE_WAIT_TIMEOUT` | Queue wait timeout |
+| `RSPECQ_REDIS_URL` | Redis URL |
+| `RSPECQ_REDIS_CONNECT_TIMEOUT` | Seconds to wait when establishing a Redis connection (default: 1.0) |
+| `RSPECQ_REDIS_READ_TIMEOUT` | Seconds to wait for a Redis read (default: 5.0) |
+| `RSPECQ_REDIS_WRITE_TIMEOUT` | Seconds to wait for a Redis write (default: 5.0) |
+| `RSPECQ_REDIS_RECONNECT_ATTEMPTS` | Comma-separated backoff (seconds) before each Redis reconnect attempt (default: `0.05,0.1,0.25,0.5,1.0`); empty disables reconnects |
+| `RSPECQ_WORKER_LIVENESS_SEC` | Worker liveness timeout in seconds |
+| `RSPECQ_FAIL_FAST` | Fail fast |
+| `RSPECQ_REPORTER_RERUN_COMMAND_SKIP` | Do not report flaky test's rerun command |
+
+### Sentry integration
+
+RSpecQ can optionally emit build events to a
+[Sentry](https://sentry.io) project by setting the
+`SENTRY_DSN` environment variable.
+
+This is convenient for monitoring important warnings/errors that may impact
+build times, such as the fact that no previous timings were found and
+therefore job scheduling was effectively random for a particular build.
+
+## How it works
+
+The core design is almost identical to ci-queue so please refer to its
+[README](https://github.com/Shopify/ci-queue/blob/master/README.md) instead.
+
+### Terminology
+
+- **Job**: the smallest unit of work, which is usually a spec file
+  (e.g. `./spec/models/foo_spec.rb`) but can also be an individual example
+  (e.g. `./spec/models/foo_spec.rb[1:2:1]`) if the file is too slow.
+- **Queue**: a collection of Redis-backed structures that hold all the necessary
+  information for an RSpecQ build to run. This includes timing statistics,
+  jobs to be executed, the failure reports and more.
+- **Build**: a particular test suite run. Each build has its own **Queue**.
+- **Worker**: an `rspecq` process that, given a build id, consumes jobs off the
+  build's queue and executes them using RSpec
+- **Reporter**: an `rspecq` process that, given a build id, waits for the build's
+  queue to be drained and prints the build summary report
+
+### Spec file splitting
+
+Particularly slow spec files may set a limit to how fast a build can be.
+For example, a single file may need 10 minutes to run while all other
+files finish after 8 minutes. This would cause all but one workers to be
+sitting idle for 2 minutes.
+
+To overcome this issue, RSpecQ can split files which their execution time is
+above a certain threshold (set with the `--file-split-threshold` option)
+and instead schedule them as individual examples.
+
+Note: In the future, we'd like for the slow threshold to be calculated and set
+dynamically (see #3).
+
+### Requeues
+
+As a mitigation technique against flaky tests, if an example fails it will be
+put back to the queue to be picked up by another worker. This will be repeated
+up to a certain number of times (set with the `--max-requeues` option), after
+which the example will be considered a legit failure and printed as such in the
+final report.
+
+Flaky tests are also detected and printed as such in the final report. They are
+also emitted to Sentry (see [Sentry integration](#sentry-integration)).
+
+### Fail-fast
+
+In order to prevent large suites running for a long time with a lot of
+failures, a threshold can be set to control the number of failed examples that
+will render the build unsuccessful. This is in par with RSpec's
+[--fail-fast](https://relishapp.com/rspec/rspec-core/docs/command-line/fail-fast-option).
+
+This feature is disabled by default, and can be controlled via the
+`--fail-fast` command line option.
+
+### Worker failures
+
+It's not uncommon for CI processes to encounter unrecoverable failures for
+various reasons: faulty hardware, network hiccups, segmentation faults in
+MRI etc.
+
+For resiliency against such issues, workers emit a heartbeat after each
+example they execute, to signal
+that they're healthy and performing jobs as expected. If a worker hasn't
+emitted a heartbeat for a given amount of time (set by the `worker_liveness_sec` option)
+it is considered dead and its reserved job will be put back to the queue, to
+be picked up by another healthy worker.
+
+
+## Rationale
+
+### Why didn't you use ci-queue?
+
+**Update**: ci-queue [deprecated support for RSpec](https://github.com/Shopify/ci-queue/pull/149).
+
+While evaluating ci-queue we experienced slow worker boot
+times (up to 3 minutes in some cases) combined with disk IO saturation and
+increased memory consumption. This is due to the fact that a worker in
+ci-queue has to load every spec file on boot. In applications with a large
+number of spec files this may result in a significant performance hit and
+in case of cloud environments, increased costs.
+
+We also observed slower build times compared to our previous solution which
+scheduled whole spec files (as opposed to individual examples), due to
+big differences in runtimes of individual examples, something common in big
+RSpec suites.
+
+We decided for RSpecQ to use whole spec files as its main unit of work (as
+opposed to ci-queue which uses individual examples). This means that an RSpecQ
+worker only loads the files needed and ends up with a subset of all the suite's
+files.  (Note: RSpecQ also schedules individual examples, but only when this is
+deemed necessary, see [Spec file splitting](#spec-file-splitting)).
+
+This kept boot and test run times considerably fast. As a side benefit, this
+allows suites to keep using `before(:all)` hooks (which ci-queue explicitly
+rejects).
+
+The downside of this design is that it's more complicated, since the scheduling
+of spec files happens based on timings calculated from previous runs. This
+means that RSpecQ maintains a key with the timing of each job and updates it
+on every run (if the `--update-timings` option was used). Also, RSpecQ has a
+"slow file threshold" which, currently has to be set manually (but this can be
+improved in the future).
+
+
+## Development
+
+Install the required dependencies:
+
+```
+$ bundle install
+```
+
+Then you can execute the tests after spinning up a Redis instance at
+`127.0.0.1:6379`:
+
+```
+$ bundle exec rake
+```
+
+To enable verbose output in the tests:
+
+```
+$ RSPECQ_DEBUG=1 bundle exec rake
+```
+
+## Redis
+
+RSpecQ by design doesn't expire its keys from Redis. It is left to the user
+to configure the Redis server to do so; see
+[Using Redis as an LRU cache](https://redis.io/topics/lru-cache) for more info.
+
+You can do this from a configuration file or with `redis-cli`.
+
+## License
+
+RSpecQ is licensed under MIT. See [LICENSE](LICENSE).

data/Rakefile

@@ -0,0 +1,10 @@
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new do |t|
+  t.libs << "test"
+  t.test_files = FileList["test/test_*.rb"]
+  t.verbose = true
+end
+
+task default: :test

data/bin/rspecq

@@ -0,0 +1,49 @@
+#!/usr/bin/env ruby
+require "rspecq"
+
+config = RSpecQ::Configuration.new(ARGV)
+
+# Log the Redis connection resilience settings so they can be
+# verified from the build console
+redis_conn = config.redis_opts
+puts "[rspecq] redis connection: " \
+     "connect_timeout=#{redis_conn[:connect_timeout]} " \
+     "read_timeout=#{redis_conn[:read_timeout]} " \
+     "write_timeout=#{redis_conn[:write_timeout]} " \
+     "reconnect_attempts=#{redis_conn[:reconnect_attempts].inspect}"
+
+Sentry.init if ENV["SENTRY_DSN"]
+
+if config.report?
+  reporter = RSpecQ::Reporter.new(
+    build_id: config.build,
+    timeout: config.report_timeout,
+    redis_opts: config.redis_opts,
+    queue_wait_timeout: config.queue_wait_timeout,
+    worker_liveness_sec: config.worker_liveness_sec,
+    update_timings: config.timings,
+    timings_key: config.timings_key
+  )
+
+  reporter.report
+else
+  worker = RSpecQ::Worker.new(
+    build_id: config.build,
+    worker_id: config.worker,
+    redis_opts: config.redis_opts,
+    worker_liveness_sec: config.worker_liveness_sec
+  )
+
+  worker.files_or_dirs_to_run = config.files_or_dirs_to_run if config.files_or_dirs_to_run
+  worker.rspec_args = config.rspec_args
+  worker.file_split_threshold = config.file_split_threshold
+  worker.chunk_target_duration = config.chunk_target_duration
+  worker.max_requeues = config.max_requeues
+  worker.queue_wait_timeout = config.queue_wait_timeout
+  worker.fail_fast = config.fail_fast
+  worker.junit_output = config.junit_output
+  worker.seed = Integer(config.seed, 10) if config.seed
+  worker.reproduction = config.reproduction
+  worker.tags = config.tags
+  worker.work
+end

data/lib/rspecq/configuration.rb

@@ -0,0 +1,103 @@
+module RSpecQ
+  # Use the RSpec parser to parse any command line args intended
+  # for rspec such as `-- --format JUnit -o foo.xml` so that we can
+  # pass these args to rspec while removing the
+  # files_or_dirs_to_run since we want to pull those from the
+  # queue. The RSpecQ::Parser will mutate args, removing any rspecq
+  # args so that RSpec::Core::Parser only sees the args intended
+  # for rspec.
+  Configuration = Struct.new(
+    :build,
+    :chunk_target_duration,
+    :exclude_pattern,
+    :fail_fast,
+    :files_or_dirs_to_run,
+    :file_split_threshold,
+    :include_pattern,
+    :junit_output,
+    :max_requeues,
+    :queue_wait_timeout,
+    :redis_host,
+    :redis_url,
+    :redis_opts,
+    :redis_connect_timeout,
+    :redis_read_timeout,
+    :redis_write_timeout,
+    :redis_reconnect_attempts,
+    :report,
+    :report_timeout,
+    :reproduction,
+    :rspec_args,
+    :seed,
+    :tags,
+    :timings,
+    :timings_key,
+    :worker,
+    :worker_liveness_sec,
+    keyword_init: true
+  ) do
+    def initialize(args)
+      super(**RSpecQ::Parser.parse!(args))
+
+      self.files_or_dirs_to_run = RSpec::Core::Parser.new(args).parse[:files_or_directories_to_run]
+      l = files_or_dirs_to_run.length
+      if l.zero?
+        self.files_or_dirs_to_run = nil
+        self.rspec_args = args
+      else
+        self.rspec_args = args[0...-l]
+      end
+
+      if include_pattern || exclude_pattern
+        self.files_or_dirs_to_run = filter_tests(files_or_dirs_to_run, self)
+      end
+
+      self.redis_opts = if redis_url
+                          { url: redis_url }
+                        else
+                          { host: redis_host }
+                        end
+
+      self.redis_opts = redis_opts.merge(
+        connect_timeout: redis_connect_timeout,
+        read_timeout: redis_read_timeout,
+        write_timeout: redis_write_timeout,
+        reconnect_attempts: parse_reconnect_attempts(redis_reconnect_attempts)
+      )
+    end
+
+    def report?
+      !!report
+    end
+
+    def filter_tests(tests, options = {})
+      suffix_pattern = /_spec\.rb$/
+      include_pattern = options[:include_pattern] || //
+      exclude_pattern = options[:exclude_pattern]
+      pattern = "**{,/*/**}/*"
+
+      (tests || []).flat_map do |file_or_folder|
+        if File.directory?(file_or_folder)
+          files = Dir[File.join(file_or_folder, pattern)].uniq.sort
+          files = files.grep(suffix_pattern).grep(include_pattern)
+          files -= files.grep(exclude_pattern) if exclude_pattern
+          files
+        else
+          file_or_folder
+        end
+      end.uniq
+    end
+
+    private
+
+    # Coerces the reconnect_attempts option into the form redis-client expects.
+    # The option is a comma-separated list of backoff durations in seconds
+    # (e.g. "0.05,0.1,0.25,0.5,1.0"); each entry is how long to sleep before
+    # that retry. An empty/blank value (including whitespace) disables
+    # reconnects (false).
+    def parse_reconnect_attempts(value)
+      attempts = value.to_s.split(",").map(&:strip).reject(&:empty?).map { |v| Float(v) }
+      attempts.empty? ? false : attempts
+    end
+  end
+end

data/lib/rspecq/formatters/README.md

@@ -0,0 +1,4 @@
+RSpec Formatters are used by RSpecQ as hooks for various execution events.
+
+For more info on formatters in general, see
+https://rubydoc.info/gems/rspec-core/RSpec/Core/Formatters.

data/lib/rspecq/formatters/example_count_recorder.rb

@@ -0,0 +1,15 @@
+module RSpecQ
+  module Formatters
+    # Increments the example counter after each job.
+    class ExampleCountRecorder
+      def initialize(queue)
+        @queue = queue
+      end
+
+      def dump_summary(summary)
+        n = summary.examples.count
+        @queue.increment_example_count(n) if n > 0
+      end
+    end
+  end
+end

data/lib/rspecq/formatters/failure_recorder.rb

@@ -0,0 +1,62 @@
+module RSpecQ
+  module Formatters
+    # Persists failed examples information (i.e. message and backtrace), so
+    # that they can be reported to the end user by the Reporter.
+    #
+    # Also persists non-example error information (e.g. a syntax error that
+    # in a spec file).
+    class FailureRecorder
+      def initialize(queue, job, max_requeues, worker_id)
+        @queue = queue
+        @job = job
+        @colorizer = RSpec::Core::Formatters::ConsoleCodes
+        @non_example_error_recorded = false
+        @max_requeues = max_requeues
+        @worker_id = worker_id
+      end
+
+      # Here we're notified about errors occuring outside of examples.
+      #
+      # NOTE: Upon such an error, RSpec emits multiple notifications but we only
+      # want the _first_, which is the one that contains the error backtrace.
+      # That's why have to keep track of whether we've already received the
+      # needed notification and act accordingly.
+      def message(n)
+        if RSpec.world.non_example_failure && !@non_example_error_recorded
+          @queue.record_non_example_error(@job, n.message)
+          @non_example_error_recorded = true
+        end
+      end
+
+      def example_failed(notification)
+        example = notification.example
+
+        presenter = RSpec::Core::Formatters::ExceptionPresenter.new(
+          example.exception, example
+        )
+
+        msg = presenter.fully_formatted(nil, @colorizer)
+
+        if @queue.requeue_job(example, @max_requeues, @worker_id)
+          # HACK: try to avoid picking the job we just requeued; we want it
+          # to be picked up by a different worker
+          @queue.record_flaky_failure(notification.example.id, msg)
+          sleep 0.5
+          return
+        end
+
+        msg << "\n"
+        msg << @colorizer.wrap(
+          "bin/rspec --seed #{RSpec.configuration.seed} #{example.location_rerun_argument}",
+          RSpec.configuration.failure_color
+        )
+
+        msg << @colorizer.wrap(
+          " # #{example.full_description}", RSpec.configuration.detail_color
+        )
+
+        @queue.record_example_failure(notification.example.id, msg)
+      end
+    end
+  end
+end

data/lib/rspecq/formatters/job_timing_recorder.rb

@@ -0,0 +1,23 @@
+module RSpecQ
+  module Formatters
+    # Persists each job's timing (in seconds). Those timings are used when
+    # determining the ordering in which jobs are scheduled (slower jobs will
+    # be enqueued first).
+    class JobTimingRecorder
+      def initialize(queue, job)
+        @queue = queue
+        @job = job
+      end
+
+      def dump_summary(summary)
+        if @job.include?("+")
+          summary.examples.each do |example|
+            @queue.record_build_timing(example.id, Float(example.execution_result.run_time))
+          end
+        else
+          @queue.record_build_timing(@job, Float(summary.duration))
+        end
+      end
+    end
+  end
+end