greenroom 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: d42a959640478bcc8b5b8fcc2e6c65773a8a6158fc4fdc7a7f9551b5023c2f9a
+  data.tar.gz: b0606a05f1aeb4d4ce8d573a197f79260b01a6fa54522d763377e74b04e47c8c
+SHA512:
+  metadata.gz: 76468d54ede95d81cd679caa2e221d690e52db078c8f64bd848d9a1a469d9a5ac8dec909719cf365fffc722a23badb798bf49f3db55c31fbf8225aa0db296ddc
+  data.tar.gz: 688524ef7f944fb15ca4bf29d24a6e1c56ab43b3d102dc868a065b1c6c1ad41a577c4fd34e3eb8b95a38699022838773fa9a31cc946cfafce04d2a1cda2a5bef

data/CHANGELOG.md

@@ -0,0 +1,16 @@
+## [Unreleased]
+
+## [0.1.0] - 2026-08-22
+
+- Add `Greenroom::ExperimentBehavior`, the `enabled?`, `publish`, and `raised`
+  methods a host mixes into its own Scientist experiment class.
+- Add `Greenroom::Recorder`, which counts rows, comparisons, mismatches, and
+  errors for a run, and carries its counts through an interruption.
+- Add the census core: `Greenroom::Census::Run`, `Cursor`, `Job`, and
+  `Middleware`.
+- Add `Greenroom.assert_registered!`, which fails at boot when no class has
+  registered with Scientist.
+- Add `greenroom check`, which reads a commit and confirms that the original
+  method body reached the `use` block unchanged.
+- Add `greenroom judge`, which reads census counts and returns a verdict and a
+  patch.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 meganemura
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,165 @@
+# Greenroom
+
+Turn a [Scientist](https://github.com/github/scientist) experiment into
+evidence, without running the candidate on a live request.
+
+An experiment stays inactive by default: `Scientist::Default#enabled?`
+returns false, so the candidate never runs and `publish` never sends
+anything. A nightly census job installs a recorder for the duration of its
+run. While that recorder is installed, the experiment turns on and the
+recorder counts every comparison. The run ends with one aggregate event,
+plus a limited number of mismatch samples. A sample carries a row
+identifier and a digest of the value, never the value itself.
+
+## What this gem provides
+
+- `Greenroom::ExperimentBehavior`, a module with the `enabled?`, `publish`,
+  and `raised` methods a Scientist experiment class needs.
+- `Greenroom::Recorder`, the thread-local counter a census job installs and
+  removes around its run.
+- `Greenroom::Census::Run`, the per-row work a census job performs, plus
+  `Greenroom::Census::Job`, a thin `Sidekiq::IterableJob` wrapper around it.
+- `Greenroom.assert_registered!`, a boot-time check for one common
+  misconfiguration (see below).
+- `Greenroom::Reporter`, the seam between a recorded event and wherever it
+  is sent. The gem ships `Reporter::Null` (the default, does nothing) and
+  `Reporter::Memory` (collects events, for tests). A New Relic adapter
+  lives in `greenroom/reporter/new_relic`, a separate file a host requires
+  on purpose.
+
+## Check a commit
+
+`greenroom check` accepts a commit when it contains one safe experiment
+change. The command compares the commit with its parent through Git and
+compares the two Ruby syntax trees. It accepts the change when the old method
+body moves into `use` unchanged and one private candidate method is added.
+
+```sh
+bundle exec greenroom check
+bundle exec greenroom check <commit>
+bundle exec greenroom check --json <commit>
+```
+
+An accepted change exits with status 0. A rejected change exits with status
+1, and a usage error exits with status 2. The command writes one result line.
+The JSON form writes one object for CI.
+
+This check has been measured with synthetic Git repositories. It has not been
+measured with a pull request from a production codebase.
+
+## Setup
+
+A host application writes three things.
+
+### 1. An experiment class
+
+`Scientist::Experiment` allows exactly one class per process to register
+as the default; a later `include` silently takes that slot from an
+earlier one. Because of this, `ExperimentBehavior` ships as a module, not
+a class: the host's own class claims the slot, and the gem never competes
+for it.
+
+```ruby
+class Pricing::Experiment
+  include Scientist::Experiment          # registers this class with Scientist
+  include Greenroom::ExperimentBehavior  # adds enabled?, publish, raised
+
+  attr_reader :name
+
+  def initialize(name)
+    @name = name
+  end
+end
+```
+
+Include `Scientist::Experiment` first. `ExperimentBehavior#raised` must
+come earlier in the ancestor chain than `Scientist::Experiment#raised` (the
+one that re-raises) to override it. Ruby puts the module included later closer
+to the class. The other include order would re-raise every internal failure
+into the caller.
+
+`Scientist.run("price-v2") { |e| ... }` and `Scientist::Experiment.new(name)`
+both look up the registered class, so the rest of a host's code calls
+Scientist exactly as the [Scientist README](https://github.com/github/scientist)
+describes; nothing else changes.
+
+### 2. A boot-time check
+
+A host that includes only `ExperimentBehavior`, and forgets
+`Scientist::Experiment`, gets no error: `enabled?` and `publish` both
+exist, but `Scientist::Experiment.new` still returns the inert
+`Scientist::Default`, so every experiment reports success while running no
+candidate, forever. Call this once at boot to turn that silent
+misconfiguration into a startup failure instead:
+
+```ruby
+# config/initializers/greenroom.rb
+Greenroom.assert_registered!
+```
+
+### 3. A Sidekiq server middleware
+
+A census job installs a recorder before it reads the first row and removes
+it when the run stops. If a row raises an exception that escapes
+`each_iteration`, Sidekiq skips the job's `on_stop` hook entirely, so that
+removal never happens. Sidekiq also reuses worker threads across jobs, so
+the next, unrelated job on that thread would otherwise find an experiment
+already turned on. Add this middleware to close that gap on every path out
+of a job, including the ones that hit an error:
+
+```ruby
+# config/initializers/sidekiq.rb
+Sidekiq.configure_server do |config|
+  config.server_middleware do |chain|
+    chain.add Greenroom::Census::Middleware
+  end
+end
+```
+
+## Writing a census target
+
+A census job reads rows through a small target object that answers three
+questions: which experiment it feeds, which rows to read, and how to call
+one row.
+
+```ruby
+class PriceV2Census
+  def experiment = "price-v2"
+  def scope = Order.all
+  def call(order) = Pricing.new(order).compute
+end
+```
+
+`Greenroom::Census::Run#visit(row)` freezes the row, counts it, tells the
+recorder which row is under examination, and calls the target through a
+`reader` -- `Greenroom::Census::Reader::Direct` by default, which just
+calls the block. A host reading from a database replica supplies its own
+reader with the same one-method interface; this gem does not ship a
+replica-reading implementation.
+
+`Greenroom::Census::Job` wires a target's `reader` and `each_iteration`
+into `Sidekiq::IterableJob`, installs and restores a `Greenroom::Recorder`
+keyed on the job's `jid`, and sends the `GreenroomCensusProgress` and
+`GreenroomCensus` events described below from `on_stop` and `on_complete`.
+Its default `row_enumerator` reads through Sidekiq's own
+`active_record_records_enumerator`, so a target whose `scope` is an
+ActiveRecord relation needs nothing beyond `target(*args)`; a host whose
+target's `scope` is something else (an array, a CSV) overrides
+`row_enumerator` instead.
+
+## Events
+
+| Name | Sent from | Carries |
+|---|---|---|
+| `GreenroomComparison` | `ExperimentBehavior#publish`, when no recorder is installed | one comparison's outcome |
+| `GreenroomInternalError` | `ExperimentBehavior#raised` | the operation and exception class that failed inside this gem's own instrumentation |
+| `GreenroomCensusProgress` | `Recorder#flush_segment` | a sign that the run is still alive, and any mismatch samples gathered so far |
+| `GreenroomMismatch` | `Recorder#flush_segment`, one per sample | a row identifier and digests, never the value itself |
+| `GreenroomCensus` | `Recorder#flush_total` | the run's full counts: `rows_scanned`, `comparisons`, `mismatches`, `ignored`, `candidate_errors`, `control_errors`, `row_errors` |
+
+A judge reads `comparisons` and the other counts from `GreenroomCensus`
+alone, never from `GreenroomCensusProgress`. A run sends one
+`GreenroomCensus` event for each recorded experiment name. It sends an event
+even when it triggered zero comparisons: the event's presence, with
+`comparisons` at zero, is how a reader tells "ran and found nothing to
+compare" apart from "did not run at all".

data/Rakefile

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "minitest/test_task"
+
+Minitest::TestTask.create
+
+require "standard/rake"
+
+task default: %i[test standard]

data/exe/greenroom

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require_relative "../lib/greenroom/cli"
+
+exit Greenroom::CLI.run(ARGV)

data/lib/greenroom/census/cursor.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+module Greenroom
+  module Census
+    # Bundles the recorder's counts into the cursor Sidekiq's Iteration
+    # persists to Redis on every row, so that a resume after an
+    # interruption restarts the counts at the same row it restarts
+    # iteration at.
+    #
+    # Sidekiq writes whichever cursor was attached to the row it stopped
+    # on, and resumes iteration from that row. If the counts were kept
+    # anywhere else (an instance variable, a separate Redis key updated on
+    # its own schedule), a resume could restart iteration at row N while
+    # the counts still reflected some other row -- double-counting or
+    # skipping whatever rows fell in the gap. Attaching the snapshot to the
+    # very cursor that names the resume position keeps position and counts
+    # describing the same instant, across a plain interruption and across
+    # `each_iteration` raising (Sidekiq flushes the last cursor either way).
+    module Cursor
+      module_function
+
+      # Wraps `enumerator`, which must itself yield `[object, position]`
+      # pairs. Built with `Enumerator.new` and read from `enumerator` lazily
+      # -- one pair at a time, only as the caller pulls -- rather than
+      # eagerly, because eagerly building every pair up front would read
+      # `recorder.snapshot` once, at build time, instead of once per row.
+      # Every row would then carry the same stale counts, and a resume from
+      # any of them would replay rows already processed.
+      def decorate(enumerator, recorder)
+        Enumerator.new do |yielder|
+          enumerator.each do |object, position|
+            yielder << [object, {"position" => position, "counts" => recorder.snapshot}]
+          end
+        end
+      end
+
+      # The inverse of the "counts" half of `decorate`: given the cursor
+      # Sidekiq handed back on resume (or nil, on a fresh start), returns
+      # `[position, counts]` so a caller can feed `position` back into its
+      # own enumerator and `counts` into `Recorder#restore`.
+      def split(cursor)
+        return [nil, nil] if cursor.nil?
+
+        [cursor["position"], cursor["counts"]]
+      end
+    end
+  end
+end

data/lib/greenroom/census/job.rb

@@ -0,0 +1,118 @@
+# frozen_string_literal: true
+
+# `sidekiq/iterable_job` alone leaves `Sidekiq::Job` (which supplies `jid`,
+# among other things) only partially loaded; the full mixin this job needs
+# requires `sidekiq` itself first.
+require "sidekiq"
+require "sidekiq/iterable_job"
+
+require "greenroom/recorder"
+require "greenroom/census/cursor"
+require "greenroom/census/middleware"
+require "greenroom/census/run"
+
+module Greenroom
+  module Census
+    # A Sidekiq iterable job that turns a host's target (see `Census::Run`)
+    # into a nightly census: a recorder is installed for exactly the
+    # duration of one run, so the experiment it drives is on only inside
+    # this job (see `ExperimentBehavior#enabled?`).
+    #
+    # A host subclasses this and defines `target(*args)`, returning an
+    # object with `experiment`, `scope`, and `call` (see the README). A
+    # target whose `scope` is an ActiveRecord relation needs nothing else;
+    # a target whose `scope` is something else overrides `row_enumerator`
+    # below instead.
+    class Job
+      include Sidekiq::IterableJob
+
+      # A host reading from a primary/replica split overrides this to
+      # return a reader that connects to the replica; the default reads
+      # through whatever connection is already active.
+      def reader
+        Reader::Direct.new
+      end
+
+      # The hook `Sidekiq::Job::Iterable#perform` calls to get this run's
+      # row source, on the first call and again on every resume. A
+      # subclass cannot supply its row source through `super`: the host's
+      # rows are only needed after `Cursor.split` has pulled greenroom's
+      # own position out of the combined cursor Sidekiq hands back, and
+      # `super` would still be carrying the cursor before that split.
+      # `row_enumerator` below is the seam a subclass overrides instead.
+      def build_enumerator(*args, cursor:)
+        assert_middleware_registered!
+        # A second, narrower guard than `Census::Middleware`'s own: this
+        # one only protects one census job from a recorder a *previous*
+        # census job left installed on this thread. It cannot help an
+        # unrelated job, which is why the middleware above is still
+        # required (see its own comment for that general case).
+        Recorder.uninstall
+        position, counts = Cursor.split(cursor)
+        @target = target(*args)
+        @recorder = Recorder.new(run_id: jid, experiment: @target.experiment)
+        @recorder.restore(counts) if counts
+        Recorder.install(@recorder)
+        @run = Run.new(target: @target, recorder: @recorder, reader: reader)
+        Cursor.decorate(row_enumerator(*args, cursor: position), @recorder)
+      end
+
+      # The default row source: `active_record_records_enumerator` comes
+      # from `Sidekiq::IterableJob` itself, so a target whose `scope` is an
+      # ActiveRecord relation needs nothing else. A target whose `scope` is
+      # not an ActiveRecord relation (an array, a CSV) overrides this
+      # method alone -- `build_enumerator` above stays correct regardless
+      # of where the rows actually come from.
+      #
+      # `cursor:` names the position alone here, in the vocabulary a host
+      # already knows from Sidekiq's own helper (its `cursor:` carries the
+      # same meaning). The combined `{"position" => ..., "counts" => ...}`
+      # cursor `build_enumerator` receives is greenroom's own bookkeeping;
+      # a subclass overriding this method never sees it.
+      def row_enumerator(*args, cursor:)
+        active_record_records_enumerator(@target.scope, cursor: cursor)
+      end
+
+      def each_iteration(row, *args)
+        @run.visit(row)
+      end
+
+      # Called on every interruption, not only on completion -- this is the
+      # "the run is still alive" signal (see `Recorder#flush_segment`), so
+      # it must fire whether the run finishes this perform or is interrupted
+      # and resumed later.
+      #
+      # Uninstalling here, rather than leaving it for the next
+      # `build_enumerator` to do, keeps the thread-local recorder from
+      # surviving into whatever unrelated work this thread picks up next in
+      # between Sidekiq perform calls.
+      def on_stop
+        @recorder.flush_segment(Greenroom.reporter)
+        Recorder.uninstall
+      end
+
+      # Reads `@recorder`, not `Recorder.current`: Sidekiq calls `on_stop`
+      # before `on_complete`, and `on_stop` has already uninstalled the
+      # thread-local by the time `on_complete` runs.
+      def on_complete
+        @recorder.flush_total(Greenroom.reporter)
+      end
+
+      private
+
+      # Raised before `build_enumerator` reads a single row (see the
+      # comment on `Census::Middleware` for why a missing registration
+      # matters): a host that forgot to add the middleware would otherwise
+      # only discover the gap the first time a row raises in production,
+      # when the leak it causes is already live on some other job.
+      def assert_middleware_registered!
+        return if Sidekiq.default_configuration.server_middleware.exists?(Middleware)
+
+        raise MiddlewareNotRegisteredError,
+          "Greenroom::Census::Middleware is not registered as a Sidekiq server " \
+          "middleware. Add it inside Sidekiq.configure_server: " \
+          "`config.server_middleware { |chain| chain.add Greenroom::Census::Middleware }`."
+      end
+    end
+  end
+end

data/lib/greenroom/census/middleware.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+require "greenroom/recorder"
+
+module Greenroom
+  module Census
+    # A Sidekiq server middleware that closes the one gap `Census::Job`
+    # cannot close on its own: when `each_iteration` raises, Sidekiq
+    # re-raises out of `perform` without calling `on_stop`, so the
+    # thread-local recorder `build_enumerator` installed is never
+    # uninstalled. Sidekiq reuses worker threads across jobs, so without
+    # this middleware, the next unrelated job Sidekiq happens to run on that
+    # thread would find an experiment already turned on -- the one safety
+    # property this gem promises (a candidate runs only inside a census)
+    # would be broken by an ordinary job failure.
+    #
+    # `Census::Job#build_enumerator` also uninstalls at its own start, as a
+    # second, narrower guard between one census job and the next -- but that
+    # only covers census jobs calling each other; it cannot help an
+    # unrelated job. A host must add this middleware to its Sidekiq server
+    # middleware chain for the guarantee to hold in general.
+    class Middleware
+      def call(_job_instance, _job_payload, _queue)
+        yield
+      ensure
+        Greenroom::Recorder.uninstall
+      end
+    end
+  end
+end

data/lib/greenroom/census/run.rb

@@ -0,0 +1,59 @@
+# frozen_string_literal: true
+
+module Greenroom
+  module Census
+    # Where a row is actually read from. `Direct` just calls the block; a
+    # replica-reading implementation (`ActiveRecord::Base.connected_to(role:
+    # :reading, prevent_writes: true)`) is a separate concern from the
+    # census core and is not implemented here -- this one-method interface
+    # (`#call { }`) is fixed now so a host can supply its own without
+    # touching `Run`.
+    module Reader
+      class Direct
+        def call
+          yield
+        end
+      end
+    end
+
+    # Does the work of one row: freeze it, count it, tell the recorder which
+    # row is being looked at, then call the host's target through the
+    # reader. Kept as a single, small method so a census job's
+    # `each_iteration` (see `Census::Job`) is just `@run.visit(row)`.
+    class Run
+      def initialize(target:, recorder:, reader:)
+        @target = target
+        @recorder = recorder
+        @reader = reader
+      end
+
+      # The control and candidate blocks run on the same row object, and
+      # Scientist shuffles their execution order; without freezing, a
+      # candidate's write would look like a mismatch caused by the control
+      # (or vice versa) rather than what it is, a bug in that one row's
+      # candidate.
+      #
+      # A raising row is caught here rather than left to escape: one bad row
+      # must not stop the rest of the census, and `record_row_error` keeps
+      # the count of rows that never reached a comparison visible instead of
+      # silently inflating `rows_scanned` beyond what was actually examined.
+      # `StandardError`, not `Exception`, so a signal like `SystemExit`
+      # still propagates; `Census::Middleware` is this gem's backstop for a
+      # row that escapes some other way.
+      def visit(row)
+        row.freeze
+        @recorder.count_row
+        @recorder.subject = subject_for(row)
+        @reader.call { @target.call(row) }
+      rescue => e
+        @recorder.record_row_error(e)
+      end
+
+      private
+
+      def subject_for(row)
+        row.id if row.respond_to?(:id)
+      end
+    end
+  end
+end