karst 0.2.0 → 0.4.0

data/docs/request-reproduction.md

@@ -0,0 +1,242 @@
+# Request reproduction
+
+> Something calls this Rails endpoint. What request do I send to exercise the same behavior?
+
+That is the whole question this answers. A Jira story says an external system
+`POST`s to `/api/v1/inspections`; the endpoint exists, someone else built it,
+and reproducing it locally normally means reading routes, controllers,
+`before_action` filters, and old specs until you can guess a working request.
+
+Karst answers it the same way it answers every other question: by running the
+real application and reporting what actually happened.
+
+## What it is
+
+You give Karst a method, a path, and (for a body-carrying method) a body.
+Karst issues **exactly one** request through the real Rails stack, inside a
+rolled-back database transaction, and reports:
+
+- the controller and action that actually dispatched
+- whether the controller lifecycle *completed* (`controller_completed`) --
+  distinct from dispatching: a controller can dispatch and still raise
+- the route parameters the router actually bound
+- the halted callback, if a `before_action` stopped the request
+- any raised exception, and the most specific phase Karst can prove it
+  happened in -- `controller`, `render`, or `unknown`
+- which templates, partials, and layouts were observed rendering (or
+  raising), by structural virtual path only -- never their content
+- the response status and content type
+- how many database writes the request made
+- a **cURL command for exactly what Karst sent**
+
+Every one of these is scoped strictly to the one request you asked for. If it
+raised before producing a response, the response fields are `null` and named
+in `unobserved` -- never a status, content type, or redirect left over from an
+earlier request Karst ran on the same session (identity establishment, most
+often).
+
+Then you iterate. A first attempt that halts at `authenticate_api_key!` has
+told you the endpoint's real gate. Add the header, send again, get a `201`,
+and copy the cURL — which is now a command you have watched work, not one
+assembled from reading code.
+
+## This is observed behavior, not documentation
+
+Karst observes the requests it issues. It does not record your application's
+production traffic, and it never infers a request from routes, controller
+source, or strong-parameter declarations.
+
+The practical consequence: Karst can tell you **whether the request you
+described works, and what it did**, but it cannot invent a body it has never
+seen. What it removes is the two genuinely hard parts — knowing whether a
+request works, and knowing what stopped it when it doesn't.
+
+A field Karst could not observe is `null` and is named in `unobserved`, never
+filled in with a plausible-looking value. If the path did not route, Karst
+reports no controller rather than guessing one.
+
+### What Karst deliberately does not capture
+
+- **Your application's real traffic.** Karst installs no request recorder and
+  keeps no buffer of past requests. There is nothing to leak later, and
+  nothing running on an ordinary request.
+- **How an external client authenticates.** Karst can sign in as one of your
+  own existing users, and it reports the callback that halted a request — but
+  it never claims to know that an endpoint "uses an API key". The halted
+  callback's name is evidence; anything beyond it would be a guess.
+- **Response bodies.** Karst reports the response status and content type,
+  never the body.
+
+## Secrets
+
+A generated request that makes you fill in a credential is better than one
+that leaks one, so Karst always chooses the former.
+
+- **Parameters** pass through your application's own
+  `config.filter_parameters`, using Rails' own `ActiveSupport::ParameterFilter`
+  — the same rules, including nested hashes and Procs, that already keep those
+  values out of your logs. Karst then applies one additional, conservative
+  credential-name filter on top (`token`, `secret`, `api_key`, `password`,
+  `signature`, `session`, ...), because an application's `filter_parameters` is
+  tuned for its own logs and routinely misses credentials that only appear in a
+  third-party integration's payload.
+- **Headers** are handled by name, never by value. `Authorization`, `Cookie`,
+  `X-Api-Key`, `X-CSRF-Token` and friends become `<AUTH_TOKEN>`,
+  `<SESSION_COOKIE>`, `<API_KEY>`, `<CSRF_TOKEN>`. Their real values are never
+  read, compared, or echoed. Headers that describe a request rather than
+  authorize it (`Content-Type`, `Accept`, ...) are shown as sent; anything else
+  whose name looks credential-shaped becomes `<FILTERED>`.
+- **Referer and Origin are URLs, so name-based header handling isn't enough
+  for them.** A `Referer` routinely carries the query string of whatever page
+  linked here — `?token=`, `?code=`, `?api_key=` — and, for an OAuth-style
+  redirect, a fragment such as `#access_token=`. Karst keeps the scheme, host,
+  and path (useful context: "this came from the password-reset page") and
+  discards the query, fragment, and any userinfo — structurally, by parsing
+  the header as a URL, never by inspecting the value for what looks like a
+  credential. A value that does not parse as a URL at all is shown as
+  `<FILTERED>` rather than echoed.
+- **Credentials in the path** — a password-reset token, a signed id — are
+  substituted back out of the URL too, so `/reset/s3cret` becomes
+  `/reset/<TOKEN>` in both the recipe and the cURL command.
+- **A redirect Karst observes** has its query string and fragment stripped
+  the same way, for the same reason: an OAuth-style callback's `Location`
+  can carry `?code=` or `#access_token=` every bit as much as a `Referer`
+  can.
+- **A body Karst cannot parse** (anything but JSON or form encoding) is sent
+  as given but never echoed back; the command shows `<BODY>`.
+- **Nothing is persisted.** Karst writes no recipe, request, or credential to
+  disk, and `/karst` is served `no-store`.
+
+One deliberate exception, for usability: the `/karst` form re-fills the body
+and header boxes with what you just typed, so you can edit and resend. That is
+ordinary form behavior on your own machine — but it does mean a credential you
+paste there is visible on that page until you navigate away. The **recipe**
+Karst generates, which is the part you copy into a ticket, never contains one.
+
+## Reproducing a request
+
+### At `/karst`
+
+Open `/karst`, pick a URL, and expand **Reproduce request**. Fill in method,
+path, content type, body, and any headers, then press **Send once and build
+request**. The recipe appears below, ending in a copyable cURL command built
+against your own development origin.
+
+### From the shell
+
+```bash
+bin/rails karst:reproduce POST /api/v1/inspections \
+  --content-type application/json \
+  --body '{"serial_number":"ABC123","status":"passed"}' \
+  --header 'Authorization: Bearer real-key'
+```
+
+Add `--json` for the same schema-versioned evidence document the MCP tool
+returns. `--anonymous` sends without assuming any identity. `--as MODEL:ID`
+(e.g. `--as User:72`) sends as one specific existing record instead of the
+ordinary sampled one, resolved only through Karst's own configured principal
+source — see [Running as a specific principal](advanced-configuration.md#running-as-a-specific-principal)
+in the advanced configuration docs. `--as` is CLI-only and cannot be combined
+with `--anonymous`. Exit code `0` means Karst observed a response, `1` means
+the request raised, `2` means Karst could not issue it at all.
+
+### From a coding agent
+
+The MCP server exposes `reproduce_request(path:, method:, body:, content_type:,
+headers:, anonymous:, base_url:)`. It returns the same document as `--json`:
+request, identity, execution, response, `unobserved`, isolation, and the cURL
+command.
+
+MCP reproduction permits only `GET` and `HEAD` by default. `POST`, `PUT`,
+`PATCH`, and `DELETE` fail with a structured `method_disabled` error before
+Karst constructs or executes an application request. A host application can
+make those methods available to MCP explicitly:
+
+```ruby
+Karst.configure { |config| config.mcp_mutating_requests = true }
+```
+
+That opt-in applies only to the MCP surface. Human-driven reproduction through
+`bin/rails karst:reproduce` and the `/karst` reproduction form continues to
+support all six methods without it. Enabling the MCP option means an agent can
+issue a real mutating request as an application principal; it is not a claim
+that the request is safe or free of side effects.
+
+It is a second tool rather than a mode of `verify_access` because the two have
+different blast radii. `verify_access` answers "which existing user can reach
+this page" by issuing up to 25 bounded GET requests automatically, and is
+GET-only for a reason: running a mutating method as 25 users would mean 25
+real creates, 25 enqueued jobs, and 25 delivered mails. `reproduce_request`
+issues exactly one request that the caller fully specified.
+
+## Identity
+
+By default Karst sends the request as one existing user drawn from your
+application's own configured principal source — the same machinery `/karst`
+uses to answer "who can use this?", so you do not have to know how your app
+signs anyone in. Pass `--anonymous` (or `anonymous: true`) for a genuinely
+identity-free probe: Karst establishes no identity at all, and then verifies
+that the application actually resolved none, which is usually what you want
+for an endpoint an external system calls.
+
+Karst assumes an identity directly rather than authenticating as an external
+client would, so the recipe's `identity` document reports what it *observed*
+the application resolve while running the request — not just what it asked
+for — through the same `requested`/`observed`/`confirmation` evidence
+`verify_access` reports:
+
+- `confirmed` — the application ran as exactly the requested user.
+- `confirmed_anonymous` — the application ran with no principal, as asked.
+- `mismatch` — the application resolved a *different* principal than requested.
+- `absent` — Karst's identity setup did not take, and the application resolved
+  no principal at all.
+- `contaminated` — an anonymous probe was requested, but the application
+  resolved a principal anyway (stale session state, most often).
+- `unobservable` — Karst has no way to see what the application resolved
+  (configure `config.observe_identity`, or use Karst's Devise/Warden
+  integration).
+
+Only `confirmed` and `confirmed_anonymous` mean the recipe ran under the
+identity it claims to.
+
+## Limitations
+
+- **Same-connection rollback is not side-effect isolation.** Database rollback
+  is attempted for writes on the request's own connection. Background jobs,
+  mail, outbound HTTP, files, Redis/external stores, and other database
+  connections are not.
+  One `POST` that enqueues a job really enqueues it. MCP therefore refuses
+  mutating methods by default; opting in does not broaden transaction isolation.
+  Human reproduction remains explicit and issues exactly one request, never a
+  sweep.
+- **Development only.** Reproduction refuses to run outside
+  `Rails.env.development?`, refuses when `config.enabled` is false, and at
+  `/karst` requires a loopback peer, a same-origin POST, and Karst's own CSRF
+  token.
+- **Local paths only.** A target with a scheme, a host, or a leading `//` is
+  refused rather than normalized.
+- **`GET HEAD POST PUT PATCH DELETE` only.** Anything else is refused rather
+  than passed through.
+- **Header names must be header names.** A name outside `[A-Za-z0-9-]+`, or a
+  CGI variable such as `REQUEST_METHOD` or `PATH_INFO`, is refused. Those
+  would be written into the Rack environment rather than sent as headers, so
+  the request Karst issued would stop matching the request Karst reports.
+- **Multipart bodies are not parsed.** They are sent as given and shown as
+  `<BODY>`.
+- **Karst cannot invent a body.** If nothing has ever told you what the
+  external system sends, Karst will faithfully report that your guess returned
+  a `422` — which is still more than reading the controller gives you, but it
+  is not the payload.
+- **Redaction is name-based (and, for Referer/Origin/redirects, structural),
+  never value-based.** A custom header, or a query or body field, whose name
+  neither your application's `filter_parameters` nor Karst's own
+  credential-name net recognizes is shown exactly as sent — even if its value
+  happens to hold a secret (a bespoke `X-Callback-Url` header carrying a
+  token in its own query string, say). If your application accepts secrets
+  under names Karst has no way to guess, add them to
+  `config.filter_parameters`; Karst's net is a supplement to that, never a
+  replacement for it.
+
+This is not API documentation, an OpenAPI generator, a request collection, or
+an HTTP client. It makes one workflow easier: reproducing a request the
+application actually handles.

data/lib/generators/karst/install/templates/karst_initializer.rb

@@ -12,6 +12,12 @@ Karst.configure do |config|
   end
   config.clear_identity = ->(session) { session.delete "/karst_test_logout" }
 
+  # How Karst observes which principal the application itself resolved while
+  # running a probe. Without it Karst reports identity as unobservable rather
+  # than assuming the requested principal is the one that ran.
+  # TODO: return this app's runtime principal (nil when it has none).
+  config.observe_identity = ->(context) { context.controller&.current_account }
+
   # TODO: replace :account_id with this app's browser-session identity.
   config.assume_browser_identity = ->(request, principal) { request.session[:account_id] = principal.id }
   config.clear_browser_identity = ->(request) { request.session.delete(:account_id) }

data/lib/karst/access/approved_populations.rb

@@ -42,16 +42,19 @@ module Karst
         # populations extended by whatever its model has approved and
         # confirmed. Returns the argument untouched when nothing applies, so
         # the overwhelmingly common "no approvals" case costs one file stat.
-        def merge(sources)
+        # `record:`/`discovery:`, when given, are a caller's already-loaded
+        # PopulationApprovals::Record and PopulationDiscovery instance (see
+        # Karst::Identity::Snapshot, which resolves this once per operation
+        # and shares both with whatever else that operation needs to discover
+        # or confirm populations) -- reused instead of loading and parsing
+        # again. Omitted, each is loaded fresh exactly as before.
+        def merge(sources, record: nil, discovery: nil)
           return sources unless sources && local_environment?
 
-          record = PopulationApprovals.load
+          record ||= PopulationApprovals.load
           return sources if record.entries.empty?
 
-          discovery = PopulationDiscovery.new
-          sources.each_with_object({}) do |(name, source), merged|
-            merged[name] = extend_source(source, record.entries, discovery)
-          end
+          merge_entries(sources, record.entries, discovery || PopulationDiscovery.new)
         rescue StandardError
           # Approval is an optional convenience layered over configuration
           # Karst already had. If resolving it fails for any reason, the
@@ -88,6 +91,12 @@ module Karst
 
         private
 
+        def merge_entries(sources, entries, discovery)
+          sources.each_with_object({}) do |(name, source), merged|
+            merged[name] = extend_source(source, entries, discovery)
+          end
+        end
+
         def extend_source(source, entries, discovery)
           klass = source.record_klass
           return source unless klass

data/lib/karst/access/errors.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module Karst
+  module Access
+    class Error < StandardError; end
+    class UnsafeTarget < Error; end
+    class UnsupportedMethod < Error; end
+    class Unavailable < Error; end
+
+    # Karst could not observe a request it tried to run -- one of its own
+    # subscribers raised, or the request never reached the application. Never
+    # evidence about the application: an operation that raises this reports
+    # nothing for the request rather than incomplete evidence.
+    class ObservationError < Error; end
+  end
+end

data/lib/karst/access/identity_probe.rb

@@ -0,0 +1,151 @@
+# frozen_string_literal: true
+
+require_relative "../identity"
+require_relative "observed_endpoint"
+
+module Karst
+  module Access
+    # One probe request's complete identity lifecycle, kept out of
+    # Access::Probe so the rule it exists to enforce stays readable:
+    #
+    #   a requested identity is intent, an established one is setup, and only
+    #   an observed one is evidence.
+    #
+    # Deliberately never raises out of establishment or release. A probe whose
+    # identity setup failed is exactly the probe whose runtime observation
+    # matters most ("asked for User#123, application saw nobody, halted at
+    # authorize_admin"), so the request still runs and is still observed --
+    # the failure is recorded as evidence rather than swallowing the probe or
+    # being mistaken for an application exception.
+    class IdentityProbe
+      # Establishment outcomes:
+      #
+      #   :established        the requested identity's assume seam ran
+      #   :failed             that seam raised (identity was not established)
+      #   :cleared            an anonymous probe: no identity was established
+      #                       and any queued/configured identity was dropped
+      #   :clear_failed       an anonymous probe whose configured
+      #                       clear_identity hook raised
+      attr_reader :requested, :request_env
+
+      def initialize(principal)
+        @principal = principal
+        @anonymous = Identity.anonymous?(principal)
+        @requested = @anonymous ? nil : Identity.describe(principal)
+        @observations = {}
+        @thread = Thread.current
+      end
+
+      def anonymous?
+        @anonymous
+      end
+
+      def endpoint(application)
+        ObservedEndpoint.new(application, self)
+      end
+
+      def capture_env(env)
+        @request_env = env
+      end
+
+      # Called as the target request starts. Identity setup may already have
+      # issued requests on the same session (a sign-in endpoint, a clear
+      # hook), and only the target's own env may ever be observed -- so
+      # whatever setup left behind is forgotten here, and a target that never
+      # reaches the application leaves nothing to observe at all.
+      def begin_target
+        @request_env = nil
+      end
+
+      def establish(session)
+        @anonymous ? establish_anonymous(session) : establish_principal(session)
+      end
+
+      # Runs in the caller's ensure, so it must survive anything the request
+      # did. An anonymous probe established nothing, but still releases: a
+      # configured clear hook is the application's own "make this anonymous
+      # again" seam, and a Warden principal queued for a request that never
+      # reached Warden must not survive into the next probe.
+      def release(session)
+        Identity.clear(session, principal: @principal) unless @anonymous
+      rescue StandardError => e
+        @cleanup_error = describe_error(e)
+      ensure
+        Identity::WardenAdapter.discard_pending!
+      end
+
+      # Karst's own sign-in raised inside the target request itself (see
+      # Identity::EstablishmentError): identity was never established, and
+      # the request stopped before the application dispatched it. Whatever
+      # half-applied state its env holds is not the application's, so
+      # nothing is observed from it.
+      def establishment_failed(message)
+        @establishment = :failed
+        @establishment_error = message
+        @request_env = nil
+        @observations[:request_completion] ||=
+          Identity::Observation.new(principal: nil, source: nil,
+                                    error: "the request stopped during Karst's own sign-in, before the application ran")
+      end
+
+      # Called at each moment worth observing. Ignores events raised on
+      # another thread: ActiveSupport::Notifications subscriptions are
+      # process-wide, and a concurrent request in a real development server
+      # must never be mistaken for this probe.
+      def observe(phase)
+        return unless Thread.current == @thread
+
+        @observations[phase] ||= Identity::Observer.observe(@request_env, requested: observable_principal)
+      end
+
+      def observed_this_probe?(phase)
+        @observations.key?(phase)
+      end
+
+      def evidence
+        Identity::EvidenceBuilder.build(requested: @requested, halt: @observations[:halted_callback],
+                                        completion: @observations[:request_completion],
+                                        establishment: @establishment,
+                                        establishment_error: @establishment_error,
+                                        cleanup_error: @cleanup_error)
+      end
+
+      private
+
+      def observable_principal
+        @anonymous ? nil : @principal
+      end
+
+      def establish_principal(session)
+        Identity.assume(session, @principal)
+        @establishment = :established
+      rescue StandardError => e
+        @establishment = :failed
+        @establishment_error = describe_error(e)
+      end
+
+      # Every probe already runs in its own freshly built integration session,
+      # which carries no cookies and no Warden proxy, so anonymity is
+      # established by construction rather than by logging something out.
+      # What can still leak across probes is Karst's own queued identity, so
+      # that is dropped explicitly; a configured clear_identity hook is then
+      # run as well, because it is the only seam that can reach application
+      # state Karst does not know about.
+      def establish_anonymous(session)
+        Identity::WardenAdapter.discard_pending!
+        @establishment = :cleared
+        hook = Karst.config.clear_identity
+        return unless hook.respond_to?(:call)
+
+        hook.call(session)
+      rescue StandardError => e
+        @establishment = :clear_failed
+        @establishment_error = describe_error(e)
+      end
+
+      def describe_error(error)
+        "#{error.class}: #{error.message}"
+      end
+    end
+  end
+end

data/lib/karst/access/local_path.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+require "uri"
+require_relative "errors"
+
+module Karst
+  module Access
+    # The single definition of "a local application path Karst may execute".
+    # Access::Sweep and Reproduction::Exercise both route their caller-supplied
+    # target through here, so there is exactly one place to audit rather than
+    # one per caller -- a security check duplicated across callers is a
+    # security check that eventually diverges.
+    module LocalPath
+      Parsed = Struct.new(:path, :query, keyword_init: true)
+
+      class << self
+        # Splits a target into its path and raw query string, refusing
+        # anything that is not a local application path. A scheme, an
+        # authority, or a protocol-relative "//host" target is rejected
+        # outright rather than normalized into something local-looking.
+        def parse(value)
+          raw = value.to_s
+          path, query = raw.split("?", 2)
+          path = path.to_s
+          uri = URI.parse(path)
+          local = uri.relative? && path.start_with?("/") && !path.start_with?("//")
+          raise UnsafeTarget, "target must be a local application path" unless local
+
+          Parsed.new(path: path, query: query)
+        rescue URI::InvalidURIError
+          raise UnsafeTarget, "target must be a valid local application path"
+        end
+
+        # The query-stripped path, for callers (Access::Sweep) that
+        # deliberately never carry a query string.
+        def path(value)
+          parse(value).path
+        end
+      end
+    end
+  end
+end

data/lib/karst/access/observed_endpoint.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+module Karst
+  module Access
+    # Wraps the probe endpoint for exactly one probe request so Karst holds
+    # the request's own Rack env while it is still running.
+    #
+    # This is what makes halt-time identity observation possible: when
+    # "halted_callback.action_controller" fires, the request has not returned
+    # yet, so ActionDispatch::Integration::Session cannot hand Karst its env
+    # (it only records one afterwards, and not at all when the application
+    # raises). Rails has already put the dispatching controller instance on
+    # this same env object by then -- ActionController::Metal#set_request!
+    # sets env["action_controller.instance"] before any callback runs -- and
+    # every middleware in the probe stack mutates that one env in place, so
+    # holding it here is enough to observe the application's runtime identity
+    # both mid-request and after it, without instrumenting the application.
+    class ObservedEndpoint
+      def initialize(app, probe)
+        @app = app
+        @probe = probe
+      end
+
+      def call(env)
+        @probe.capture_env(env)
+        @app.call(env)
+      end
+
+      # ActionDispatch::Integration::Session asks the endpoint it was given
+      # for things other than #call (#routes, for url_options), so this must
+      # stay transparent rather than merely callable.
+      def respond_to_missing?(name, include_private = false)
+        @app.respond_to?(name, include_private) || super
+      end
+
+      # rubocop:disable Naming/BlockForwarding, Style/ArgumentsForwarding -- anonymous
+      # argument/block forwarding needs Ruby 3.x; this file runs on Ruby 2.7.
+      def method_missing(name, *arguments, &block)
+        return @app.public_send(name, *arguments, &block) if @app.respond_to?(name)
+
+        super
+      end
+      # rubocop:enable Naming/BlockForwarding, Style/ArgumentsForwarding
+    end
+  end
+end

data/lib/karst/access/outcome_groups.rb

@@ -0,0 +1,93 @@
+# frozen_string_literal: true
+
+require_relative "../value"
+
+module Karst
+  module Access
+    # The one definition of "these probes observed the same thing", shared by
+    # Access::Result#groups, the CLI/MCP evidence document, the human CLI
+    # summary, and the /karst panel -- so no surface can split or merge
+    # outcomes differently from another.
+    #
+    # A group is keyed by what Karst observed about the *request*: the
+    # response, where it dispatched, what it wrote, and whether the configured
+    # usable_access_outcome policy accepts it. Everything about the *probe*
+    # that made the request -- who it ran as, how that identity was observed,
+    # why that principal was sampled, how long it took -- is deliberately not
+    # part of the key: two users halted at the same callback are one outcome
+    # observed twice, not two outcomes. Those per-probe details stay attached
+    # to each member outcome, so nothing is lost by grouping.
+    #
+    # Nothing here normalizes observed values (no redirect path rewriting, no
+    # status bucketing): two outcomes group only when every semantic field is
+    # exactly equal, so grouping can merge equivalent observations but never
+    # reinterpret distinct ones as the same.
+    module OutcomeGroups
+      # Observed facts about the request. Two outcomes that differ in any of
+      # these are different outcomes.
+      SEMANTIC_FIELDS = %i[status redirect exception_class halted_callback controller action
+                           writes_observed write_count database_rollback_attempted].freeze
+
+      # Facts about the probe, not the request's outcome. Volatile (elapsed
+      # time), per-identity (principal, identity evidence), or sampling
+      # provenance -- never part of group identity.
+      PROBE_METADATA = %i[principal identity sampling_reasons elapsed_ms].freeze
+
+      # `usable` is the configured usable_access_outcome verdict, shared by
+      # every member (it is part of the key), so a group can always be
+      # described as verified usable or not without consulting any one member.
+      Group = Value.define(:key, :outcomes, :usable) do
+        def count
+          outcomes.size
+        end
+
+        # Every member agrees on every SEMANTIC_FIELDS value, so any member
+        # describes the group; the first one observed is used.
+        def representative
+          outcomes.first
+        end
+      end
+
+      class << self
+        # Groups `outcomes` and orders the groups deterministically: largest
+        # first, then by the semantic key itself, so the same observations
+        # produce the same order regardless of the order they were probed in.
+        # Members keep the order they were probed in. `usable` is evaluated
+        # exactly once per outcome.
+        def group(outcomes, usable: Karst.config.usable_access_outcome)
+          groups = partition(outcomes, usable).map do |group_key, (verdict, members)|
+            Group.new(key: group_key, outcomes: members.freeze, usable: verdict)
+          end
+          groups.sort_by { |item| [-item.count, sort_key(item.key)] }
+        end
+
+        # halted_callback is compared by its string form: Rails reports a
+        # named callback as a Symbol, and the serialized evidence reports it
+        # as a String, so the two must never become separate groups.
+        def key(outcome)
+          SEMANTIC_FIELDS.map do |field|
+            value = outcome.respond_to?(field) ? outcome.public_send(field) : nil
+            field == :halted_callback && !value.nil? ? value.to_s : value
+          end
+        end
+
+        private
+
+        # semantic key + verdict => [verdict, members in probe order]
+        def partition(outcomes, usable)
+          outcomes.each_with_object({}) do |outcome, grouped|
+            verdict = usable.call(outcome) ? true : false
+            (grouped[key(outcome) + [verdict]] ||= [verdict, []]).last << outcome
+          end
+        end
+
+        # A total order over heterogeneous values (Integer, String, booleans,
+        # nil) that never raises: nil sorts first, everything else by its
+        # string form.
+        def sort_key(values)
+          values.map { |value| value.nil? ? [0, ""] : [1, value.to_s] }
+        end
+      end
+    end
+  end
+end