slopbouncer 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: ba4f63d3b0f813fcd42c469d4d4051ca2b0c93a3c12d8f34c4ebc8683831b166
+  data.tar.gz: 4504c98ddfbd09547290ab538c0b8a75d963a0b43d442788eec05c8349daf62e
+SHA512:
+  metadata.gz: a6a0e5a1f3bb6db696c32ecdf23807fcaadaa62b39d83d91341f9fbca6358cd70990c8d72ecd0aff1934a26d75da4aaa2c1925c13749603c7f7d4b42865c1856
+  data.tar.gz: '0838bd507b4124151b1d0aed78bafe574fc3c96eba3d306dbde56ff4a8cdb6896200663fcaf339b5a3ac3fc8153cd2502222ed996d01b22da13b274006603acc'

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Kipple Pty Ltd
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,119 @@
+# Slopbouncer
+
+A static check that flags tests which got weaker in a change.
+
+A coding agent's job is to make the checks pass. One way to do that is to weaken the check. Slopbouncer compares each test before and after a change and reports the ones that lost something, so a person can decide whether that was intended. It reads code and never runs it, so it takes seconds.
+
+Status: in development. It reads Ruby tests today (Minitest, as `it` blocks or `test_` methods).
+
+## What it flags
+
+| Rule | When |
+|---|---|
+| `assertion_removed` | An assertion is gone from a test that still exists |
+| `expected_value_changed` | The value an assertion expects was edited |
+| `test_deleted` | A test, or its whole file, was deleted |
+| `skip_added` | A test was marked to be skipped or pending |
+| `no_assertions` | A test the change touched asserts nothing |
+
+## What it leaves alone
+
+- New tests added with new code.
+- Tests that gained assertions.
+- A test that was reworded, moved to another file, or both, with its checks unchanged.
+- Files that are not tests.
+
+## Running it
+
+It needs Ruby 3.4 and git. It is not on RubyGems yet, so build the gem from a clone and install that:
+
+```shell
+git clone https://github.com/mainline-dev/slopbouncer.git
+cd slopbouncer
+gem build slopbouncer.gemspec
+gem install slopbouncer-0.1.0.gem
+```
+
+Then run it from the root of the repository you want checked:
+
+```shell
+slopbouncer main..HEAD
+```
+
+The argument is a range of commits. Give one commit instead, such as `HEAD` or `origin/main`, and it checks everything since then, committed or not. It prints one line for each finding:
+
+```
+test/basket_test.rb: Basket applies a voucher: skip_added: marked to be skipped
+test/checkout_test.rb: Checkout lets a guest pay: assertion_removed: assert_equal "paid", order.status
+test/refund_test.rb: Refund refunds part of an order: expected_value_changed: 100 → 90
+```
+
+| Option | What it does |
+|---|---|
+| `--json` | Prints the findings as JSON |
+| `--strict` | Exits with 1 when anything is flagged |
+| `--agent=NAME` | Runs as that coding agent's hook (see below) |
+
+Without `--strict` it exits with 0 whatever it finds: it flags for review and does not block. It exits with 2 when it is given nothing to compare.
+
+## Feeding it back to a coding agent
+
+Slopbouncer is most use when the agent that made the change hears about it. The agent knows why it changed a test, and it can put the test back.
+
+One command sets an agent up, from the root of your project:
+
+```shell
+slopbouncer install claude-code
+```
+
+| Agent | Command | What it adds |
+|---|---|---|
+| Claude Code | `slopbouncer install claude-code` | A `Stop` hook in `.claude/settings.json` |
+| Cursor | `slopbouncer install cursor` | A `stop` hook in `.cursor/hooks.json` |
+| Gemini CLI | `slopbouncer install gemini` | An `AfterAgent` hook in `.gemini/settings.json` |
+| Codex | `slopbouncer install codex` | A `Stop` hook in `.codex/hooks.json` |
+
+It adds one hook and keeps everything else in the file. Running it again changes nothing. For Claude Code, `--local` writes to `.claude/settings.local.json`, which keeps the hook to you and out of the settings your team shares.
+
+When the agent is about to finish and has weakened an existing test, it is sent back once and told which tests, with two ways to answer: put the test back and make the code pass it, or leave the change and tell the person which behaviour changed. The second time it is let finish, so its reasons are on the record for the person to read.
+
+The hook checks the work the agent has not committed. If your agent commits as it goes, edit the hook's command to name the commit its work started from, such as `origin/main`, in place of `HEAD`.
+
+Codex skips a new hook until you have trusted it: after installing, open `/hooks` in Codex and review it.
+
+Claude Code is the one this has been run under. The Cursor, Gemini CLI and Codex hooks are written to those agents' published hook contracts and tested against them, and have not yet been run inside the agents themselves. If one misbehaves, please open an issue.
+
+An agent set on a passing build can still get round a static check, so keep running the plain command where a person will see it.
+
+## Your own assertion helpers
+
+Slopbouncer counts calls such as `assert_equal`, `refute_nil`, `must_equal` and `expect` as assertions. If your tests assert through helpers of your own, name them in `.slopbouncer.yml` at the root of your repository:
+
+```yaml
+assertions:
+  - sees_total
+  - is_refused
+```
+
+A test that only calls helpers Slopbouncer has not been told about looks like a test with no assertions.
+
+## Its limits
+
+- It catches tests that are visibly weaker. A test that looks thorough and checks nothing needs mutation testing, which is slower and runs the code. Use both: Slopbouncer on every change, mutation testing behind it.
+- It cannot tell a stronger assertion from a weaker one. An assertion swapped for another kind is reported as removed.
+- It cannot see intent. A test changed on purpose, because the behaviour changed, is flagged the same way as one weakened to get a build to pass.
+
+On its first run, over the 706 commits in [Mainline](https://mainline.dev)'s history that touch a test, it flagged 210 of them. In a sample of those, every finding was a real change to a test and every one looked intended. So read a finding as "this test changed in a way worth a look".
+
+## Working on it
+
+```shell
+bundle install
+bundle exec rake
+```
+
+The tests are scenarios. Each one builds a throwaway git repository, makes a change in it, and runs the real command over that change. `PROJECT.md` holds the plan and the rules still to come.
+
+## Licence
+
+MIT. See `LICENSE`.

data/bin/slopbouncer

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require_relative '../lib/slopbouncer'
+
+exit Slopbouncer.run(ARGV)

data/lib/slopbouncer/agents.rb

@@ -0,0 +1,195 @@
+# frozen_string_literal: true
+
+require 'json'
+
+module Slopbouncer
+  # The coding agents Slopbouncer can run under. Each one knows where that
+  # agent keeps its hooks, how to add Slopbouncer to them, and how to tell the
+  # agent it may not finish yet. A new agent is one more module here.
+  module Agents
+    # Claude Code runs Stop hooks when the agent is about to finish. A hook
+    # that exits with 2 stops it, and what the hook wrote to standard error is
+    # handed to the agent as the reason.
+    module ClaudeCode
+      NAME = 'claude-code'
+      SHARED = '.claude/settings.json'
+      LOCAL = '.claude/settings.local.json'
+
+      def self.settings_path(local:)
+        local ? LOCAL : SHARED
+      end
+
+      def self.command(base)
+        "slopbouncer #{base} --agent=#{NAME}"
+      end
+
+      def self.installed?(settings)
+        stop_hooks(settings).any? { |hook| hook['command'].to_s.include?('slopbouncer') }
+      end
+
+      # The settings with Slopbouncer added to whatever runs at Stop already.
+      def self.add(settings, command)
+        hooks = settings.fetch('hooks', {})
+        entry = { 'hooks' => [{ 'type' => 'command', 'command' => command }] }
+        settings.merge('hooks' => hooks.merge('Stop' => hooks.fetch('Stop', []) + [entry]))
+      end
+
+      # Claude Code says on the hook's input whether it has stopped the agent
+      # once already.
+      def self.stopped_before?(input)
+        input['stop_hook_active'] == true
+      end
+
+      # What to write and exit with to send the agent back: [output, error, status].
+      def self.block(message)
+        ['', message, 2]
+      end
+
+      def self.stop_hooks(settings)
+        settings.dig('hooks', 'Stop').to_a.flat_map { |entry| entry['hooks'].to_a }
+      end
+      private_class_method :stop_hooks
+    end
+
+    # Cursor runs stop hooks when the agent loop ends. A hook answers on
+    # standard output with JSON, and a follow-up message there is given to the
+    # agent as its next turn. Project hooks run from the project root.
+    module Cursor
+      NAME = 'cursor'
+      SHARED = '.cursor/hooks.json'
+      VERSION = 1
+
+      # Cursor's own per-person file is in the home directory and its hooks
+      # do not run from the project, so there is nothing to offer here.
+      def self.settings_path(local:)
+        raise Error, "Cursor keeps a project's hooks in #{SHARED} only. Run it without --local." if local
+
+        SHARED
+      end
+
+      def self.command(base)
+        "slopbouncer #{base} --agent=#{NAME}"
+      end
+
+      def self.installed?(settings)
+        stop_hooks(settings).any? { |hook| hook['command'].to_s.include?('slopbouncer') }
+      end
+
+      def self.add(settings, command)
+        hooks = settings.fetch('hooks', {})
+        { 'version' => VERSION }.merge(settings).merge('hooks' => hooks.merge('stop' => stop_hooks(settings) + [{ 'command' => command }]))
+      end
+
+      # Cursor counts the follow-ups a stop hook has already caused, and says
+      # how the loop ended. Only an agent that finished by itself is sent back.
+      def self.stopped_before?(input)
+        input['loop_count'].to_i.positive? || (input.key?('status') && input['status'] != 'completed')
+      end
+
+      def self.block(message)
+        [JSON.generate('followup_message' => message), '', 0]
+      end
+
+      def self.stop_hooks(settings)
+        settings.dig('hooks', 'stop').to_a
+      end
+      private_class_method :stop_hooks
+    end
+
+    # Gemini CLI runs AfterAgent hooks once the agent has written its final
+    # response. A hook answers on standard output with JSON and nothing else:
+    # a decision to deny the response, with a reason that Gemini gives the
+    # agent as a new prompt.
+    module Gemini
+      NAME = 'gemini'
+      SHARED = '.gemini/settings.json'
+
+      # Gemini's per-person settings are in the home directory and apply to
+      # every project, so there is nothing to offer here.
+      def self.settings_path(local:)
+        raise Error, "Gemini CLI keeps a project's hooks in #{SHARED} only. Run it without --local." if local
+
+        SHARED
+      end
+
+      def self.command(base)
+        "slopbouncer #{base} --agent=#{NAME}"
+      end
+
+      def self.installed?(settings)
+        after_agent(settings).flat_map { |entry| entry['hooks'].to_a }.any? { |hook| hook['command'].to_s.include?('slopbouncer') }
+      end
+
+      def self.add(settings, command)
+        entry = { 'hooks' => [{ 'type' => 'command', 'name' => 'slopbouncer', 'command' => command }] }
+        settings.merge('hooks' => settings.fetch('hooks', {}).merge('AfterAgent' => after_agent(settings) + [entry]))
+      end
+
+      # Gemini says on the hook's input whether this is already a retry.
+      def self.stopped_before?(input)
+        input['stop_hook_active'] == true
+      end
+
+      def self.block(message)
+        [JSON.generate('decision' => 'deny', 'reason' => message), '', 0]
+      end
+
+      def self.after_agent(settings)
+        settings.dig('hooks', 'AfterAgent').to_a
+      end
+      private_class_method :after_agent
+    end
+
+    # Codex runs Stop hooks when a turn is about to end. A hook answers on
+    # standard output with JSON: a decision to block, whose reason Codex gives
+    # the agent as a new prompt. Codex skips a new or changed hook until a
+    # person has reviewed and trusted it.
+    module Codex
+      NAME = 'codex'
+      SHARED = '.codex/hooks.json'
+      AFTER = 'Codex skips a new hook until you trust it: open /hooks in Codex to review it.'
+
+      # Codex's per-person hooks are in the home directory and apply to every
+      # project, so there is nothing to offer here.
+      def self.settings_path(local:)
+        raise Error, "Codex keeps a project's hooks in #{SHARED} only. Run it without --local." if local
+
+        SHARED
+      end
+
+      def self.command(base)
+        "slopbouncer #{base} --agent=#{NAME}"
+      end
+
+      def self.installed?(settings)
+        stop(settings).flat_map { |entry| entry['hooks'].to_a }.any? { |hook| hook['command'].to_s.include?('slopbouncer') }
+      end
+
+      def self.add(settings, command)
+        entry = { 'hooks' => [{ 'type' => 'command', 'command' => command }] }
+        settings.merge('hooks' => settings.fetch('hooks', {}).merge('Stop' => stop(settings) + [entry]))
+      end
+
+      # Codex says on the hook's input whether Stop already continued this turn.
+      def self.stopped_before?(input)
+        input['stop_hook_active'] == true
+      end
+
+      def self.block(message)
+        [JSON.generate('decision' => 'block', 'reason' => message), '', 0]
+      end
+
+      def self.stop(settings)
+        settings.dig('hooks', 'Stop').to_a
+      end
+      private_class_method :stop
+    end
+
+    ALL = { ClaudeCode::NAME => ClaudeCode, Cursor::NAME => Cursor, Gemini::NAME => Gemini, Codex::NAME => Codex }.freeze
+    DEFAULT = ClaudeCode::NAME
+
+    def self.named(name)
+      ALL.fetch(name) { raise Error, "Slopbouncer does not know the agent #{name.inspect}. It knows: #{ALL.keys.join(', ')}" }
+    end
+  end
+end

data/lib/slopbouncer/compare.rb

@@ -0,0 +1,77 @@
+# frozen_string_literal: true
+
+module Slopbouncer
+  # Compares a test file's table of tests before and after a change and says
+  # which tests got weaker. Tests that are new are never flagged. It cannot
+  # tell a stronger assertion from a weaker one, so an assertion swapped for
+  # another kind is reported as removed, for a person to judge.
+  module Compare
+    Finding = Struct.new(:file, :test, :rule, :detail, :before, :after, keyword_init: true)
+    Result = Struct.new(:findings, :gone, :fresh, keyword_init: true)
+
+    # A reworded test is the same test: one that has gone is taken to be a
+    # new one when at least this share of their body lines are the same.
+    SAME_TEST = 0.5
+
+    # gone: tests no longer in the file. fresh: tests that are new to it.
+    # The caller decides whether a gone test turned up in another file.
+    def self.call(file, before, after)
+      now = after.to_h { |test| [test.name, test] }
+      gone = before.reject { |test| now.key?(test.name) }
+      fresh = after.reject { |test| before.any? { |was| was.name == test.name } }
+      reworded = reworded(gone, fresh)
+      pairs = before.filter_map { |was| [was, now[was.name]] if now.key?(was.name) } + reworded
+      Result.new(findings: pairs.flat_map { |was, test| weaker(file, was, test) },
+        gone: gone - reworded.map(&:first), fresh: fresh - reworded.map(&:last))
+    end
+
+    def self.deleted(file, test)
+      Finding.new(file: file, test: test.name, rule: 'test_deleted', detail: "#{test.assertions.size} assertions")
+    end
+
+    # Pairs each test that has gone with the new test most like it.
+    def self.reworded(gone, fresh)
+      left = fresh.dup
+      gone.filter_map do |was|
+        match = left.max_by { |test| likeness(was, test) }
+        next unless match && likeness(was, match) >= SAME_TEST
+
+        left.delete(match)
+        [was, match]
+      end
+    end
+
+    def self.likeness(was, now)
+      shared = (was.body & now.body).size
+      total = [was.body.size, now.body.size].max
+      total.zero? ? 0.0 : shared.to_f / total
+    end
+
+    def self.weaker(file, was, now)
+      finding = ->(rule, **rest) { Finding.new(file: file, test: now.name, rule: rule, **rest) }
+      lost = was.assertions.reject { |assertion| now.assertions.map(&:source).include?(assertion.source) }
+      gained = now.assertions.reject { |assertion| was.assertions.map(&:source).include?(assertion.source) }
+      found = lost.map do |assertion|
+        edit = gained.find { |other| edited_value(assertion, other) }
+        next finding.call('assertion_removed', detail: assertion.source) unless edit
+
+        gained.delete(edit)
+        before, after = edited_value(assertion, edit)
+        finding.call('expected_value_changed', detail: edit.source, before: before, after: after)
+      end
+      found << finding.call('skip_added', detail: 'marked to be skipped') if now.skipped && !was.skipped
+      found << finding.call('no_assertions', detail: 'asserts nothing') if now.assertions.empty? && was.body != now.body
+      found
+    end
+
+    # The one argument that differs between two calls of the same assertion,
+    # as [before, after], when everything else about them is the same.
+    def self.edited_value(was, now)
+      return unless was.method == now.method && was.arguments.size == now.arguments.size
+
+      differing = was.arguments.zip(now.arguments).reject { |before, after| before == after }
+      differing.first if differing.size == 1
+    end
+    private_class_method :reworded, :likeness, :weaker, :edited_value
+  end
+end

data/lib/slopbouncer/git.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+require 'open3'
+
+module Slopbouncer
+  # What git knows about a change: which files it touched, and what each
+  # held before and after.
+  module Git
+    Changed = Struct.new(:before_path, :after_path, keyword_init: true)
+
+    # The files changed between two commits, or between a commit and the
+    # working tree when no second commit is given. A file that was added has
+    # no before path, and one that was deleted has no after path.
+    def self.changed(base, head = nil)
+      run('diff', '--name-status', '-M', *[base, head].compact).lines.map do |line|
+        status, first, second = line.chomp.split("\t")
+        case status[0]
+        when 'A' then Changed.new(after_path: first)
+        when 'D' then Changed.new(before_path: first)
+        when 'R' then Changed.new(before_path: first, after_path: second)
+        else Changed.new(before_path: first, after_path: first)
+        end
+      end
+    end
+
+    # What a file holds at a commit, or in the working tree when none is given.
+    def self.show(revision, path)
+      revision ? run('show', "#{revision}:#{path}") : File.read(File.join(root, path))
+    end
+
+    def self.root
+      @root ||= run('rev-parse', '--show-toplevel').strip
+    end
+
+    def self.run(*arguments)
+      output, error, status = Open3.capture3('git', *arguments)
+      raise Error, "git #{arguments.first} failed: #{error.strip}" unless status.success?
+
+      output
+    end
+    private_class_method :run, :root
+  end
+end

data/lib/slopbouncer/install.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+require 'fileutils'
+require 'json'
+
+module Slopbouncer
+  # Sets a coding agent up to run Slopbouncer when it finishes, by adding one
+  # hook to that agent's settings in the current directory. Everything else in
+  # the settings is kept, and setting up twice changes nothing.
+  module Install
+    # Says what it did, in one line.
+    def self.call(agent_name, local: false, base: 'HEAD')
+      agent = Agents.named(agent_name)
+      path = agent.settings_path(local: local)
+      settings = read(path)
+      return "#{path} already runs Slopbouncer. Nothing changed." if agent.installed?(settings)
+
+      FileUtils.mkdir_p(File.dirname(path))
+      File.write(path, "#{JSON.pretty_generate(agent.add(settings, agent.command(base)))}\n")
+      ["Added Slopbouncer to #{path}. #{agent_name} will run it when it is about to finish.",
+       (agent::AFTER if agent.const_defined?(:AFTER))].compact.join("\n")
+    end
+
+    def self.read(path)
+      return {} unless File.exist?(path)
+
+      settings = JSON.parse(File.read(path))
+      raise Error, "#{path} does not hold settings Slopbouncer can add to. Nothing changed." unless settings.is_a?(Hash)
+
+      settings
+    rescue JSON::ParserError
+      raise Error, "#{path} could not be read as JSON. Nothing changed."
+    end
+    private_class_method :read
+  end
+end

data/lib/slopbouncer/ruby_tests.rb

@@ -0,0 +1,113 @@
+# frozen_string_literal: true
+
+require 'prism'
+
+module Slopbouncer
+  # Reads a Ruby test file into a table of its tests: each test's full name,
+  # the assertions it makes, whether it is marked to be skipped, and its body.
+  # It reads the code and never runs it.
+  module RubyTests
+    Test = Struct.new(:name, :assertions, :skipped, :body, keyword_init: true)
+    Assertion = Struct.new(:method, :arguments, :source, keyword_init: true)
+
+    GROUPS = %i[describe context feature].freeze
+    SKIPPED_GROUPS = %i[xdescribe xcontext xfeature].freeze
+    TESTS = %i[it test specify scenario example].freeze
+    SKIPPED_TESTS = %i[xit xtest xspecify xscenario xexample].freeze
+    SKIPS = %i[skip pending].freeze
+    ASSERTION = /\A(assert|refute|must|wont)(_|\z)|\A(expect|flunk)\z/
+
+    # helpers: the project's own method names that count as assertions.
+    def self.in(source, helpers: [])
+      Reader.new(helpers).read(Prism.parse(source).value)
+    end
+
+    # Walks the syntax tree, keeping the names of the groups it is inside.
+    class Reader
+      def initialize(helpers)
+        @helpers = helpers.map(&:to_sym)
+        @tests = []
+        @groups = []
+        @skipping = 0
+      end
+
+      def read(node)
+        visit(node)
+        @tests
+      end
+
+      private
+
+      def visit(node)
+        return unless node
+
+        case node
+        when Prism::ClassNode, Prism::ModuleNode then inside(node.constant_path.slice) { visit(node.body) }
+        when Prism::DefNode then node.name.start_with?('test_') ? test(node.name.to_s, node.body) : visit(node.body)
+        when Prism::CallNode then call(node)
+        else node.compact_child_nodes.each { |child| visit(child) }
+        end
+      end
+
+      def call(node)
+        label = label(node)
+        if !(node.block && label) then node.compact_child_nodes.each { |child| visit(child) }
+        elsif GROUPS.include?(node.name) then inside(label) { visit(node.block) }
+        elsif SKIPPED_GROUPS.include?(node.name) then inside(label, skipped: true) { visit(node.block) }
+        elsif TESTS.include?(node.name) then test(label, node.block)
+        elsif SKIPPED_TESTS.include?(node.name) then test(label, node.block, skipped: true)
+        else node.compact_child_nodes.each { |child| visit(child) }
+        end
+      end
+
+      # What a group or test is called: its first argument, as written.
+      def label(node)
+        first = node.arguments&.arguments&.first
+        return unless first
+
+        first.respond_to?(:unescaped) ? first.unescaped : first.slice
+      end
+
+      def inside(name, skipped: false)
+        @groups.push(name)
+        @skipping += 1 if skipped
+        yield
+      ensure
+        @skipping -= 1 if skipped
+        @groups.pop
+      end
+
+      def test(name, body, skipped: false)
+        calls = calls_in(body)
+        @tests << Test.new(
+          name:       [*@groups, name].join(' '),
+          assertions: calls.select { |call| assertion?(call) }.map { |call| assertion(call) },
+          skipped:    skipped || @skipping.positive? || calls.any? { |call| call.receiver.nil? && SKIPS.include?(call.name) },
+          body:       body_of(body)
+        )
+      end
+
+      def calls_in(node)
+        return [] unless node
+
+        own = node.is_a?(Prism::CallNode) ? [node] : []
+        own + node.compact_child_nodes.flat_map { |child| calls_in(child) }
+      end
+
+      def assertion?(call)
+        call.name.match?(ASSERTION) || @helpers.include?(call.name)
+      end
+
+      def assertion(node)
+        arguments = node.arguments&.arguments.to_a.map { |argument| argument.slice.split.join(' ') }
+        Assertion.new(method: node.name.to_s, arguments: arguments, source: node.slice.split.join(' '))
+      end
+
+      # The lines inside a test, without its opening and closing lines.
+      def body_of(node)
+        inner = node.is_a?(Prism::BlockNode) ? node.body : node
+        inner ? inner.slice.lines.map(&:strip).reject(&:empty?) : []
+      end
+    end
+  end
+end

data/lib/slopbouncer/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module Slopbouncer
+  VERSION = '0.1.0'
+end

data/lib/slopbouncer.rb

@@ -0,0 +1,140 @@
+# frozen_string_literal: true
+
+require 'json'
+require 'yaml'
+require_relative 'slopbouncer/version'
+require_relative 'slopbouncer/git'
+require_relative 'slopbouncer/ruby_tests'
+require_relative 'slopbouncer/compare'
+require_relative 'slopbouncer/agents'
+require_relative 'slopbouncer/install'
+
+# A static check that flags tests which got weaker in a change, so a person
+# can decide whether that was intended.
+module Slopbouncer
+  class Error < StandardError; end
+
+  # A project's own settings, in the directory the command is run from:
+  #   assertions: the names of its helper methods that count as assertions.
+  CONFIG = '.slopbouncer.yml'
+
+  TEST_FILE = %r{(\A|/)(test|spec)/.*\.rb\z|(\A|/)(test_[^/]*|[^/]*_(test|spec))\.rb\z}
+
+  USAGE = 'Give a range of commits, such as main..HEAD, or one commit to compare the working tree with, such as HEAD'
+
+  # The findings for the commits in a range such as main..HEAD, or for
+  # everything since one commit, committed or not, when given just that.
+  def self.check(target)
+    base, head = target.split('..', 2)
+    raise Error, USAGE if base.to_s.empty? || (target.include?('..') && head.to_s.empty?)
+
+    helpers = settings.fetch('assertions', [])
+    results = Git.changed(base, head).filter_map do |changed|
+      next unless [changed.before_path, changed.after_path].compact.all? { |path| path.match?(TEST_FILE) }
+
+      before = changed.before_path ? RubyTests.in(Git.show(base, changed.before_path), helpers: helpers) : []
+      after = changed.after_path ? RubyTests.in(Git.show(head, changed.after_path), helpers: helpers) : []
+      [changed.after_path || changed.before_path, Compare.call(changed.after_path || changed.before_path, before, after)]
+    end
+    results.flat_map { |_file, result| result.findings } + deleted(results)
+  end
+
+  # A test that has gone from one file is deleted unless the same test, body
+  # for body, is new in another file of the same change.
+  def self.deleted(results)
+    moved = results.flat_map { |_file, result| result.fresh.map(&:body) }
+    results.flat_map do |file, result|
+      result.gone.reject { |test| moved.include?(test.body) }.map { |test| Compare.deleted(file, test) }
+    end
+  end
+
+  def self.settings
+    File.exist?(CONFIG) ? YAML.safe_load_file(CONFIG) || {} : {}
+  rescue Psych::Exception => e
+    raise Error, "#{CONFIG} could not be read: #{e.message}"
+  end
+
+  def self.run(arguments, input: $stdin)
+    words = arguments.reject { |argument| argument.start_with?('--') }
+    return install(words.drop(1), arguments) if words.first == 'install'
+
+    target = words.first
+    agent = arguments.find { |argument| argument == '--agent' || argument.start_with?('--agent=') }
+    return stop_hook(target.to_s, input, agent.split('=', 2)[1] || Agents::DEFAULT) if agent
+
+    findings = check(target.to_s)
+    puts arguments.include?('--json') ? JSON.pretty_generate(findings.map { |finding| finding.to_h.compact }) : report(findings)
+    arguments.include?('--strict') && findings.any? ? 1 : 0
+  rescue Error => e
+    warn e.message
+    2
+  end
+
+  # Sets a coding agent up in the current directory: slopbouncer install claude-code
+  def self.install(words, arguments)
+    raise Error, "Say which agent to set up: #{Agents::ALL.keys.join(', ')}" if words.empty?
+
+    puts Install.call(words.first, local: arguments.include?('--local'))
+    0
+  rescue Error => e
+    warn e.message
+    2
+  end
+
+  # Run as a coding agent's stop hook. The agent is stopped from finishing
+  # once and told what it weakened, in the way its own harness expects. A
+  # mistake in how the hook is set up never stops the agent: it is said on
+  # standard error and the agent is let finish.
+  def self.stop_hook(target, input, agent_name)
+    agent = Agents.named(agent_name)
+    told = hook_input(input)
+    return 0 if agent.stopped_before?(told)
+
+    findings = Dir.chdir(working_directory(told)) { check(target) }
+    return 0 if findings.empty?
+
+    output, error, status = agent.block(to_the_agent(findings))
+    print output
+    warn error unless error.empty?
+    status
+  rescue Error => e
+    warn e.message
+    0
+  end
+
+  # Where the agent says it is working, when its harness says so on the
+  # hook's input. Some harnesses run hooks from somewhere else.
+  def self.working_directory(told)
+    said = told['cwd'] || Array(told['workspace_roots']).first
+    said.is_a?(String) && File.directory?(said) ? said : Dir.pwd
+  end
+
+  # What the agent's harness passes the hook on standard input, if anything.
+  def self.hook_input(input)
+    text = input.tty? ? '' : input.read.to_s
+    parsed = text.strip.empty? ? {} : JSON.parse(text)
+    parsed.is_a?(Hash) ? parsed : {}
+  rescue JSON::ParserError
+    {}
+  end
+
+  def self.to_the_agent(findings)
+    <<~MESSAGE
+      Slopbouncer: this change weakens #{findings.size} existing #{findings.size == 1 ? 'test' : 'tests'}.
+
+      #{report(findings)}
+
+      For each one, do one of two things before you finish:
+      1. Put the test back as it was, and make the code pass it.
+      2. Leave the change, and tell the person which behaviour changed and why the test had to change with it.
+    MESSAGE
+  end
+
+  def self.report(findings)
+    findings.map { |finding|
+      what = finding.before ? "#{finding.before} → #{finding.after}" : finding.detail
+      "#{finding.file}: #{finding.test}: #{finding.rule}: #{what}"
+    }.join("\n")
+  end
+  private_class_method :deleted, :settings, :install, :stop_hook, :working_directory, :hook_input, :to_the_agent
+end

metadata

@@ -0,0 +1,72 @@
+--- !ruby/object:Gem::Specification
+name: slopbouncer
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Kipple Pty Ltd
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: prism
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.5'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.5'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3'
+description: A static, diff-aware check for code written by coding agents. It compares
+  each test before and after a change and flags the ones that got weaker, and can
+  tell the agent that made the change before it finishes.
+executables:
+- slopbouncer
+extensions: []
+extra_rdoc_files: []
+files:
+- LICENSE
+- README.md
+- bin/slopbouncer
+- lib/slopbouncer.rb
+- lib/slopbouncer/agents.rb
+- lib/slopbouncer/compare.rb
+- lib/slopbouncer/git.rb
+- lib/slopbouncer/install.rb
+- lib/slopbouncer/ruby_tests.rb
+- lib/slopbouncer/version.rb
+homepage: https://github.com/mainline-dev/slopbouncer
+licenses:
+- MIT
+metadata:
+  rubygems_mfa_required: 'true'
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.4'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.6.9
+specification_version: 4
+summary: Flags tests that got weaker in a change
+test_files: []