squishling 0.4.0 → 0.5.0

data/docs/verifying.md

@@ -0,0 +1,84 @@
+# Verifying squished classes
+
+Most mistakes in a squished class fail when the class loads (a bad `escalation:`, `params:`, `harness:` or `squawk:`
+raises `ConfigurationError` as the declaration runs). The output schema and its typed result are built on the first
+call, and the keywords your `include_when:` and `squish_when` procs take are only used when a call is routed. So
+Squishling keeps a list of the classes that include it and can check them all without making a request, for example
+in CI.
+
+## `Squishling.verify` and `verify!`
+
+```ruby
+report = Squishling.verify            # every squished class
+report.ok?                            # => false
+puts report
+# Squishling found 2 problems:
+#   Triage#route: escalation step "claude-haiku-4-5" include_when: requires tier:, which is not an input of the method
+#   Invoice#call: invalid data member: total=
+
+Squishling.verify!                    # raises Squishling::VerificationError (a ConfigurationError) naming every problem
+Squishling.verify(Triage, Invoice)    # just these classes
+```
+
+`verify` returns a `Squishling::Verification::Report` (`ok?`, `size`, `problems`, `to_s`). Each problem has a
+`label` and a `message`. `verify!` returns the report when it is clean and otherwise raises
+`Squishling::VerificationError`, which has `report` and `problems`. Every problem is listed, not just the first.
+
+For each squished method (inherited ones too, with the subclass's settings) it checks:
+
+- **The output schema:** that it builds (and is strict), is valid JSON Schema, and produces a typed result, so a
+  property name that can't be a result member (`total=`) is reported. A method with no output schema or no purpose is reported, because its LLM path
+  would raise. The default `call` method is skipped when you don't implement or configure it.
+- **What the layers resolve to:** the escalation, generation params and `squawk` that the method, its class and
+  `Squishling.config` add up to.
+- **The harness:** that it is still registered (not removed or replaced since the class declared it), and that a
+  `type: :judgment` judge has an adapter that supports judgments.
+- **Keywords:** that each `include_when:` and the `squish_when` block can be called with the method's inputs as keywords
+  (`instance_exec(**inputs, &proc)`): no required keyword that isn't an input, and no input a lambda (or a proc that names
+  keywords) can't take. Add `**` to accept any. A method that takes `**` or has no implementation can receive any
+  keyword, so nothing is reported for it. `squish_fallback` and `squish_validate` are called with the inputs too, but
+  aren't checked.
+- **The adapter** (once): that `config.adapter`, or RubyLLM's, can be built.
+
+A problem in an inherited method is reported once for each subclass, since each is checked with its own settings.
+
+Nothing is sent to a provider, and your own code (the Ruby implementation, `squish_when`, the procs) is not run.
+
+## Only loaded classes are checked
+
+The list holds the classes that have been loaded. In Rails, load everything first:
+
+```ruby
+Rails.application.eager_load!
+Squishling.verify!
+```
+
+The list holds its classes weakly, so it doesn't keep classes alive across code reloads, and a named class whose
+constant now holds a different object (replaced by a reload but not yet collected) is left out. Anonymous classes are
+included. `Squishling.squished_classes` returns the list.
+
+## In CI
+
+**Rake.** In Rails the `squishling:verify` task is added for you and eager loads the app first. Elsewhere, load it in your
+`Rakefile`:
+
+```ruby
+load "squishling/tasks/squishling.rake"
+```
+
+`bundle exec rake squishling:verify` prints the report and exits non-zero when there are problems. Load your classes
+before the task runs (an `:environment` task, if you have one, is invoked first). The Railtie adds the task in Rails
+when Squishling is required after Rails (as Bundler does), so a Rails app needn't load the file itself.
+
+**RSpec.** `require "squishling/rspec"` adds a matcher whose failure message is the report:
+
+```ruby
+require "squishling/rspec"
+
+RSpec.describe "squished classes" do
+  before(:all) { Rails.application.eager_load! }
+
+  it { expect(Squishling).to be_squishling_valid }
+  it { expect(InvoiceParser).to be_squishling_valid }   # or just one class, or an Array of them
+end
+```

data/lib/squishling/adapter/error_body.rb

@@ -0,0 +1,47 @@
+# frozen_string_literal: true
+
+module Squishling
+  module Adapter
+    # Reads the machine-readable fields of a provider's error body: the status (OpenAI-style `type`, Gemini's
+    # `status`), the code, and the name of the field it objected to. Only short plain tokens come out; the message
+    # and anything else a provider might echo from the request never do.
+    module ErrorBody
+      # A status or code is an enumeration-like word (no hyphens, so an API key like sk-... never passes); a param is
+      # a field path (messages[0].content), which is only compared with the developer's own param names and never
+      # repeated.
+      WORD = /\A[A-Za-z0-9][\w.]{0,39}\z/
+      PATH = /\A[\w.\-\[\]]{1,64}\z/
+      private_constant :WORD, :PATH
+
+      module_function
+
+      # body: a JSON String, a Hash with String keys, or an Array wrapping the Hash (Gemini does that). http_status
+      # drops a code that only repeats it. Returns { status:, code:, param: }, each a String or nil.
+      def read(body, http_status: nil)
+        detail = error(body)
+        return { status: nil, code: nil, param: nil } unless detail
+
+        code = token(detail["code"], WORD)
+        { status: token(detail["status"], WORD) || token(detail["type"], WORD),
+          code: (code unless code == http_status.to_s),
+          param: token(detail["param"], PATH) }
+      end
+
+      # The body's `error` Hash, or nil when it has none (or isn't JSON). For adapters that need more of it than
+      # #read returns, e.g. to recognize a wording; they must not repeat it.
+      def error(body)
+        data = body.is_a?(String) ? JSON.parse(body.dup.force_encoding(Encoding::UTF_8)) : body
+        data = data.first if data.is_a?(Array)
+        detail = data["error"] if data.is_a?(Hash)
+        detail if detail.is_a?(Hash)
+      rescue JSON::ParserError
+        nil
+      end
+
+      def token(value, pattern)
+        value = value.to_s if value.is_a?(Integer)
+        value if value.is_a?(String) && value.match?(pattern)
+      end
+    end
+  end
+end

data/lib/squishling/adapter.rb

@@ -0,0 +1,275 @@
+# frozen_string_literal: true
+
+module Squishling
+  # How Squishling reaches a model. An adapter is the only code that talks to a provider: Squishling builds a
+  # Request (the system prompt, the conversation so far, the strict output schema, the resolved params), the adapter
+  # makes one completion and returns a Response. Squishling owns the conversation, the validation, and the retries.
+  # Two adapters ship (Adapters::RubyLLM, Adapters::OpenAICompatible); write your own by subclassing Adapter::Base
+  # and registering it (Squishling.register_adapter) or assigning an instance to config.adapter. See docs/adapters.md.
+  module Adapter
+    # One completion to make. messages holds the conversation so far as { role: :user | :assistant, content: String }
+    # Hashes, ending with the user message to answer; instructions is the system prompt. schema is the strict
+    # structured-output payload ({ "name", "description", "schema", "strict" => true }). params is the resolved
+    # generation params: the portable :temperature, :max_output_tokens, and :thinking, plus any provider-specific keys
+    # (see Params). label ("Class#method") prefixes the adapter's error messages. model and provider are the step's
+    # (nil when it names none).
+    Request = Data.define(:label, :model, :provider, :instructions, :messages, :schema, :params)
+
+    # Stands in for an empty reply when the conversation is replayed: some servers reject an empty assistant message.
+    EMPTY_REPLY = "(no output)"
+
+    # What a completion returns. content is the model's output: the raw String, or a Hash the adapter already parsed.
+    # model is the model id the provider reports. tokens is a Hash of counts (:input_tokens, :output_tokens, and
+    # whatever else the provider reports, such as :cache_read_tokens and :thinking_tokens), empty when unknown.
+    # finish_reason is why the provider stopped, as a Symbol: :stop, :max_tokens (the output cap cut the reply off),
+    # :content_filter, :tool_calls, or the provider's own spelling; nil when it reported none.
+    Response = Data.define(:content, :model, :tokens, :finish_reason) do
+      # model, tokens, and finish_reason may be left out; nil tokens count as none, and a String or blank
+      # finish_reason is made a Symbol or nil. Raises ConfigurationError for tokens that aren't a Hash.
+      def initialize(content:, model: nil, tokens: {}, finish_reason: nil)
+        tokens ||= {}
+        unless tokens.is_a?(Hash)
+          raise ConfigurationError, "Adapter::Response tokens must be a Hash, got #{tokens.class}"
+        end
+
+        finish_reason = finish_reason.to_s.strip.empty? ? nil : finish_reason.to_sym
+        super
+      end
+
+      # Whether the provider cut the reply off at the output token cap.
+      def truncated?
+        finish_reason == :max_tokens
+      end
+
+      # The token counts, or nil when the adapter reported none (what squawk's usage: carries).
+      def usage
+        tokens unless tokens.empty?
+      end
+
+      # The output as the text the model will see again in a retry: a Hash an adapter already parsed goes back as
+      # JSON, and nothing (or a value JSON can't represent, which is rejected as invalid output anyway) as
+      # EMPTY_REPLY.
+      def text
+        text = content.is_a?(String) ? utf8(content) : generated
+        text.strip.empty? ? EMPTY_REPLY : text
+      end
+
+      private
+
+      def generated
+        content.nil? ? "" : JSON.generate(content)
+      rescue JSON::JSONError
+        ""
+      end
+
+      # Text that goes back out in a request must be valid UTF-8, or serializing the history fails. A reply that
+      # wasn't is already reported as invalid output; only its undecodable bytes are replaced here.
+      def utf8(text)
+        (text.encoding == Encoding::BINARY ? text.dup.force_encoding(Encoding::UTF_8) : text).scrub("?")
+      end
+    end
+
+    # The adapter contract. Subclass it, implement #complete, and raise Squishling's errors:
+    #
+    #   class MyAdapter < Squishling::Adapter::Base
+    #     def complete(request)
+    #       reply = MyClient.chat(request.model, request.instructions, request.messages)
+    #       Squishling::Adapter::Response.new(content: reply.text, model: reply.model, tokens: reply.usage)
+    #     rescue MyClient::Unauthorized
+    #       raise configuration_error(request, "MyClient rejected the credentials")
+    #     rescue MyClient::Error => e
+    #       raise llm_error(request, "MyClient failed (#{e.class})")
+    #     end
+    #   end
+    #
+    # ConfigurationError is for setup mistakes (bad credentials, an invalid request, an unknown model): it is never
+    # retried or escalated. LLMError is for failures worth another attempt (rate limits, 5xx, timeouts, connection
+    # errors): the next attempt, or the last one's caller, sees it. Keep the provider's own message out of both: it can
+    # echo request content or part of a key. Any other exception propagates unwrapped. Adapters are called from worker
+    # threads when a harness takes samples, so they must not share unsynchronized state.
+    class Base
+      # One completion. Returns an Adapter::Response.
+      def complete(_request)
+        raise NotImplementedError, "#{self.class} must implement #complete(request)"
+      end
+
+      # Whether #judge is supported (judges with type: :judgment).
+      def judgments?
+        false
+      end
+
+      # A System One judgment: input (a Hash) and questions (a Hash of name => question), with the step's model and
+      # params. Returns an object answering [name] with an answer that responds to choice and probabilities. label
+      # ("Class#method") prefixes error messages. Called as judge(input, questions:, step:, label:).
+      def judge(_input, **_options)
+        raise ConfigurationError, "#{self.class} doesn't support judgments (type: :judgment judges)"
+      end
+
+      # Called on the caller's thread before a harness's sample requests fan out to worker threads. Returns whatever
+      # #with_context needs to re-establish the caller's request context (tracing, say) on a worker.
+      def capture_context
+        nil
+      end
+
+      # Runs the block on a worker thread inside the context #capture_context returned.
+      def with_context(_context)
+        yield
+      end
+
+      private
+
+      def configuration_error(request, message)
+        ConfigurationError.new("#{request.label}: #{message}")
+      end
+
+      def llm_error(request, message)
+        LLMError.new("#{request.label}: #{message}")
+      end
+
+      # A RequestRejectedError for a request the provider rejected (a 400). via names how it failed (the error class,
+      # the HTTP status); body is the provider's error body (a JSON String, or a Hash with String keys), of which only
+      # the status and code tokens are repeated, never its message. http_status, when given, drops a code that only
+      # repeats it. The params are pointed at only when the provider named one of them.
+      def rejected_request_error(request, via:, body:, http_status: nil)
+        found = ErrorBody.read(body, http_status:)
+        tokens = found.values_at(:status, :code).compact
+        RequestRejectedError.new(
+          "#{request.label}: the provider rejected the request (#{[via, *tokens].join(', ')}). " \
+          "#{rejection_hint(request.params, found[:param])}",
+          provider_status: found[:status], provider_code: found[:code]
+        )
+      end
+
+      def rejection_hint(params, field)
+        named = params.keys.find { |key| names?(field, key.to_s) }
+        return "The #{suspects(params)} may be invalid." unless named
+
+        "The provider named the #{named.inspect} param; check its value."
+      end
+
+      def suspects(params)
+        params.empty? ? "output schema or input" : "params, output schema, or input"
+      end
+
+      # Whether the provider's field is the param or inside it (thinking.budget_tokens, stop[0]).
+      def names?(field, name)
+        !field.nil? && (field == name || field.start_with?("#{name}.", "#{name}["))
+      end
+    end
+
+    # The adapters by name: the built-ins, registered as they load, and your own (Squishling.register_adapter).
+    module Registry
+      STORE = NamedRegistry.new("adapter")
+      private_constant :STORE
+
+      class << self
+        def register_built_in(name, klass)
+          STORE.register_built_in(name, klass)
+        end
+
+        # Registers an adapter class under a name. It is built with the options given next to its type: (or none).
+        def register(name, klass)
+          name = STORE.checked_name(name)
+          unless implements_complete?(klass)
+            raise ConfigurationError, "register_adapter needs a class that implements #complete, got #{klass.inspect}"
+          end
+
+          STORE.add(name, klass)
+        end
+
+        def fetch(name)
+          STORE.fetch(name)
+        end
+
+        def names
+          STORE.names
+        end
+
+        # Forgets the registered adapters; the built-ins stay.
+        def reset!
+          STORE.reset!
+        end
+
+        private
+
+        # A class that defines #complete itself, not one that only inherits Base's placeholder.
+        def implements_complete?(klass)
+          klass.is_a?(Class) && klass.method_defined?(:complete) && klass.instance_method(:complete).owner != Base
+        end
+      end
+    end
+
+    KEYWORD_KINDS = %i[key keyreq].freeze
+    private_constant :KEYWORD_KINDS
+
+    module_function
+
+    # The adapter a config.adapter value names: an instance (anything that responds to complete), a registered name,
+    # or a Hash with type: and that adapter's options. Built now, so a mistake (a missing gem, a bad option) fails at
+    # setup rather than on the first request.
+    def build(value, where)
+      case value
+      when Symbol, String then instantiate(value, {}, where)
+      when Hash
+        options = value.transform_keys(&:to_sym)
+        raise ConfigurationError, "#{where} adapter: needs a type: (#{type_list})" unless options.key?(:type)
+
+        instantiate(options.delete(:type), options, where)
+      else
+        return value if value.respond_to?(:complete)
+
+        raise ConfigurationError, "#{where} adapter must be an adapter (responding to complete), one of " \
+                                  "#{type_list}, or a Hash with type:, got #{value.inspect}"
+      end
+    end
+
+    # The adapter used when none is configured: RubyLLM's, if the ruby_llm gem is available.
+    def detect
+      Adapters::RubyLLM.new
+    rescue Adapters::RubyLLM::MissingGemError
+      raise ConfigurationError, "No adapter is configured and the ruby_llm gem isn't available. " \
+                                "Add `gem \"ruby_llm\"` to your Gemfile, or set config.adapter " \
+                                "(#{type_list}, or your own adapter)"
+    end
+
+    def type_list
+      Registry.names.map(&:inspect).join(", ")
+    end
+
+    def instantiate(type, options, where)
+      name = type.to_s.to_sym
+      klass = Registry.fetch(name)
+      unless klass
+        raise ConfigurationError, "#{where} adapter: unknown adapter #{type.inspect} (available: #{type_list})"
+      end
+
+      check_options!(klass, options, name, where)
+      options.empty? ? klass.new : klass.new(**options)
+    end
+
+    # The keywords the adapter's initialize takes, so a typo is a ConfigurationError rather than an ArgumentError.
+    def check_options!(klass, options, name, where)
+      known, required = keywords(klass)
+      return unless known
+
+      unknown = options.keys - known
+      missing = required - options.keys
+      return if unknown.empty? && missing.empty?
+
+      problem = unknown.empty? ? "needs #{missing.join(', ')}" : "doesn't take #{unknown.join(', ')}"
+      hint = known.empty? ? "" : " (options: #{known.join(', ')})"
+      raise ConfigurationError, "#{where} adapter #{name.inspect} #{problem}#{hint}"
+    end
+
+    # [all keywords, required keywords] of klass#initialize, or nil when it takes any (**options).
+    def keywords(klass)
+      parameters = klass.instance_method(:initialize).parameters
+      return if parameters.any? { |kind, _| kind == :keyrest }
+
+      [parameters.filter_map { |kind, key| key if KEYWORD_KINDS.include?(kind) },
+       parameters.filter_map { |kind, key| key if kind == :keyreq }]
+    end
+    private_class_method :check_options!, :keywords
+    private_class_method :instantiate
+  end
+end