ruby_llm-llm_judge 0.1.2 → 0.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 71f6bd771e7f6bec3a9dcdf7036ee1238110c63ee530b15d32c6dadcfb0d0eaa
-  data.tar.gz: e54b13efadda477c3d3713c66184dc4ce01061b740e06ab52e95df0b672ae7e9
+  metadata.gz: fc81f9575b6468ad1a7ea81b19d777ea169eca0619cf1995c59c0aab01128254
+  data.tar.gz: 6052e01d0b02b576c25fe5baa0545a8f89e9060e12d42ae65f3fa99e8e14f5f7
 SHA512:
-  metadata.gz: 4e29ff7f1b47222dfc71f6f096cd058615d2517b7e16b29f9c2a68f094f2af95be8a99232d5ba72cc1d6b9a932fa5f9bf290e701a9744ee1a241fc6a0c710834
-  data.tar.gz: 942f4a7da0c27d1741d3e0e2b80ae719335123d6ddf276b84b0e4d200ed315d5f839cfc1ea998e708b6b54f83d7b7e97c718f6d4a070e17d2bfe0be23e9b3541
+  metadata.gz: ba27da7ce65adbb3d39d6447decaa6b2454d2bbee7086577f81798cb213ccf573efd81c250d46a5bc29d7b58d811892a73918b7f89cddd1dec778b50620d8406
+  data.tar.gz: 411fd92bb9f7700b6075559ed5644cbdad5485a555fc6601249ebfc1c28d657dbc221255bf308f4358c1730cd3c506b08c553a7e2359ddb0e449a2c0e5aedaf7

data/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## 0.1.4 (2026-10-03)
+
+- `structured_output: true` sends a strict JSON Schema with one-call requests, requiring every question and option.
+- `strategy: :committed` asks for one option per question and returns it with probability 1.
+- The provider accepts RubyLLM main's `with:` attachments argument and raises `UnsupportedAttachmentError` when images are passed. Judgments on RubyLLM main failed with `ArgumentError` before.
+- The README explains when to enable reasoning: off suits classification, while rule-following questions need it. See [the decision-mode benchmarks](benchmarks/decision-modes.md).
+
+## 0.1.3 (2026-09-29)
+
+- Setting a `chat_provider_options` key to `nil` removes that field, including the Luna defaults and, before RubyLLM 2.1, the output limit field. A nested hash emptied this way is removed too.
+- One-call answers given as a list of single-question objects are accepted when each question appears once. Claude Haiku 4.5 returns this shape on its first attempt, which previously cost a corrective retry on nearly every judgment.
+- The one-call instructions now say that `answers` is an object keyed by question ID. Haiku no longer needs corrective retries (59 of 96 benchmark cases before), and Luna scored at least as well in a back-to-back comparison. See [the Haiku benchmark](benchmarks/haiku.md).
+- Luna defaults apply to every OpenAI Luna model ID (`gpt-6-luna`, `gpt-5.6-luna`, `gpt-luna-latest`) and to OpenRouter's `openai/` Luna IDs, not only `gpt-6-luna` on OpenAI. On OpenRouter they send temperature zero and `reasoning: { enabled: false }`. With reasoning on, Luna's ratings often returned no digit. Pro variants and prefixed gateway IDs, such as `us.openai.gpt-5.6-luna` through `:openai`, are excluded.
+
 ## 0.1.2 (2026-09-28)
 
 - When a rating call fails, no new calls start and the error is raised once calls in flight finish. Previously the remaining calls kept running, and were billed, after `judge` raised.

data/README.md

@@ -84,9 +84,11 @@ result = RubyLLM::LLMJudge.judge(
 )
 ```
 
-Both strategies return valid judgments with this model. Its one-call answers arrive in a Markdown code fence, which the gem removes before parsing.
+Both strategies return valid judgments with this model. Its one-call answers arrive in a Markdown code fence, which the gem removes before parsing. On the benchmark cases it scored 56–57/64 on AG News and 29–30/32 on SST-2 at about 1.1 s median, a little below Luna. [Haiku results](benchmarks/haiku.md).
 
-The default Luna rating request uses Chat Completions, temperature zero, reasoning disabled, `store: false`, and a four-token output limit. Tune requests with `scoring_protocol`, `temperature`, `max_output_tokens`, and `chat_provider_options`. `chat_provider_options` is merged over those defaults, so setting one key, such as `service_tier: 'flex'`, keeps reasoning disabled; set a key explicitly to change a default.
+The default Luna rating request uses Chat Completions, temperature zero, reasoning disabled, `store: false`, and a four-token output limit. Tune requests with `scoring_protocol`, `temperature`, `max_output_tokens`, and `chat_provider_options`. `chat_provider_options` is merged over those defaults, so setting one key, such as `service_tier: 'flex'`, keeps reasoning disabled; set a key explicitly to change a default. Set a key to `nil` to remove that field from the request, whether it is a default or, before RubyLLM 2.1, the output limit field. For example, an app that routes RubyLLM 1.x OpenAI chats through the Responses API can send `chat_provider_options: { max_completion_tokens: nil, max_output_tokens: 256 }`.
+
+Luna defaults apply to each provider's own Luna model IDs: `gpt-6-luna`, `gpt-5.6-luna`, or `gpt-luna-latest` on `:openai`, which sends temperature zero, `reasoning_effort: 'none'`, and `store: false`; and `openai/gpt-6-luna` or `openai/gpt-5.6-luna` on `:openrouter`, which sends temperature zero and `reasoning: { enabled: false }`. Pro variants, other providers such as `:bedrock`, and prefixed IDs such as `us.openai.gpt-5.6-luna` sent through `:openai` to an OpenAI-compatible gateway get no Luna defaults, because their request format may differ; set `temperature:` and disable reasoning in that endpoint's format yourself. With reasoning left on, the ratings strategy's four-token output limit is spent on reasoning: on eight AG News cases, `gpt-6-luna` through OpenRouter returned no digit for all eight, and `gpt-5.6-luna` for four. RubyLLM before 2.1 sends temperature 1.0 for any OpenAI model whose ID starts with `gpt-5`, so `gpt-5.6-luna` on `:openai` runs at 1.0 there unless you pass `chat_provider_options: { temperature: 0 }`.
 
 ## One-call typed decisions
 
@@ -140,6 +142,31 @@ result = RubyLLM::LLMJudge.judge(
 
 OpenRouter's [latency sorting](https://openrouter.ai/docs/guides/routing/provider-selection) chooses among its available providers using their observed response times. It sped up OpenRouter in our test, while direct OpenAI was faster still. The measured speed and answer-quality tradeoffs are below.
 
+### Strict response schemas
+
+Set `structured_output: true` to send a strict JSON Schema with the one-call request, through RubyLLM's `with_schema`. The schema requires every question and every option, and nothing else, so a provider that supports structured outputs cannot return the wrong shape. Without it, the gem validates the reply and makes its corrective retry. In our runs it had little effect on accuracy, cut GPT-6 Luna's corrective retries from 3.3 to 0.7 per 96 judgments, and added 40–350 ms. Models and providers without structured-output support reject the request, so leave it off for them.
+
+### Committed answers
+
+Set `strategy: :committed` to ask for one option per question instead of a distribution. The reply is shorter, and each answer comes back with probability 1 for the chosen option and 0 for the rest, so `confidence` is always 1.0 and the probabilities carry no calibration. Use it when you only need the decision; use `:single_request` or `:ratings` when you threshold on probabilities. It accepts `structured_output: true`, which limits each answer to that question's option IDs. On classification it was 100–300 ms faster than probabilities and as accurate or up to three cases in 64 less; on rule-following questions it was the most accurate mode with reasoning off.
+
+### Reasoning
+
+The Luna defaults disable reasoning, which suits classification: on the benchmarks below, Luna is as accurate with reasoning off and much faster. Questions that apply explicit rules to a state are different. On [StreamDecisionBench](https://github.com/JacobLinCool/StreamDecisionBench), whose questions carry decision policies, GPT-5.6 Luna through LLM Judge answered 47% of states correctly with reasoning off and 90% with reasoning effort low, at about twice the latency. The ratings strategy cannot apply a policy across a state and should not be used for these questions. [Classification and rule-following results by mode](benchmarks/decision-modes.md).
+
+To enable reasoning, override the default and leave room for it in the output limit:
+
+```ruby
+provider_options: {
+  strategy: :single_request,
+  max_output_tokens: 4096,
+  chat_provider_options: { reasoning_effort: 'low' }          # OpenAI
+  # chat_provider_options: { reasoning: { effort: 'low' } }   # OpenRouter
+}
+```
+
+Reasoning does not fit the `:ratings` strategy, whose four-token output limit is meant for a single digit.
+
 ## How scoring works
 
 The default `:ratings` strategy asks the model to rate every declared answer from 0 to 9. It makes one call per answer, with up to six calls running at once (`max_workers:` changes this), then applies softmax to the ratings. If a call fails after its retry, no new calls start, calls in flight finish, and the error is raised. It retries a malformed digit once. When the highest ratings tie on a Choice question, it makes a one-call JSON judgment for that question and uses its distribution. If that call also ties, it raises an error instead of choosing whichever option came first. `result.raw[:tie_breaks]` records each resolution, and token usage includes the extra call. Set `tie_breaker: :first` to use the original first-option rule, or `tie_break_max_output_tokens:` to change the tie-break response limit (default 8192).

data/lib/ruby_llm/llm_judge/engine.rb

@@ -6,24 +6,38 @@ module RubyLLM
   module LLMJudge
     class Engine
       MAX_WORKERS = 6
+      # Luna model IDs by provider, which get temperature zero and reasoning disabled in that provider's
+      # format. Only each provider's own IDs match: an OpenAI-compatible gateway such as Bedrock, reached
+      # through :openai with an ID like "us.openai.gpt-5.6-luna", may use another request format. Pro
+      # variants are excluded: they exist to reason more.
+      LUNA_IDS = {
+        openai: /\Agpt-(?:\d+(?:\.\d+)?-)?luna(?:-latest)?\z/,
+        openrouter: %r{\A~?openai/gpt-(?:\d+(?:\.\d+)?-)?luna(?:-latest)?(?::\w+)?\z}
+      }.freeze
       SYSTEM_INSTRUCTIONS = 'Return exactly one ASCII digit 0-9. 9 means very likely to be the correct answer; 0 means very unlikely. No explanation.'
       SINGLE_REQUEST_INSTRUCTIONS = 'Answer all questions using only a JSON object with an "answers" field. ' \
                                     'For each question, return an object mapping every supplied option ID to a ' \
                                     'probability between 0 and 1. Include every option exactly once and make ' \
                                     'each question\'s probabilities sum to 1. Evaluate each question independently ' \
-                                    'against the same state. Return no explanations or markdown.'
+                                    'against the same state. Return no explanations or markdown. The "answers" value ' \
+                                    'must be an object whose keys are the question IDs.'
+      COMMITTED_INSTRUCTIONS = 'Answer all questions using only a JSON object with an "answers" field whose keys are ' \
+                               'the question IDs. For each question, give the ID of the one option that best applies. ' \
+                               'Evaluate each question independently against the same state. Return no explanations ' \
+                               'or markdown.'
+      STRATEGIES = %i[ratings single_request committed].freeze
 
       def initialize(config:, model: DEFAULT_MODEL, provider_options: {}, scorer: nil, responder: nil)
         @config = config
         options = provider_options.transform_keys(&:to_sym)
         allowed = %i[scoring_provider scoring_protocol temperature max_output_tokens chat_provider_options
                      max_arms max_input_bytes strategy malformed_retries tie_breaker tie_break_max_output_tokens
-                     max_workers]
+                     max_workers structured_output]
         unknown = options.keys - allowed
         raise ArgumentError, "Unknown LLMJudge provider options: #{unknown.join(', ')}" unless unknown.empty?
 
         @strategy = options.fetch(:strategy, :ratings).to_sym
-        raise ArgumentError, 'strategy must be :ratings or :single_request' unless %i[ratings single_request].include?(@strategy)
+        raise ArgumentError, 'strategy must be :ratings, :single_request, or :committed' unless STRATEGIES.include?(@strategy)
         @tie_breaker = options.fetch(:tie_breaker, :single_request).to_sym
         raise ArgumentError, 'tie_breaker must be :single_request or :first' unless %i[single_request first].include?(@tie_breaker)
 
@@ -31,14 +45,15 @@ module RubyLLM
         @scoring_model = model
         raise ArgumentError, 'A model is required' unless @scoring_model.is_a?(String) && !@scoring_model.empty?
 
-        luna_defaults = @scoring_provider == :openai && @scoring_model == DEFAULT_MODEL
-        @scoring_protocol = options.fetch(:scoring_protocol, luna_defaults ? :chat_completions : nil)
+        luna_defaults = LUNA_IDS.fetch(@scoring_provider, nil)&.match?(@scoring_model) || false
+        @scoring_protocol = options.fetch(:scoring_protocol,
+                                          luna_defaults && @scoring_provider == :openai ? :chat_completions : nil)
         # Before 2.1, RubyLLM chats use each provider's one chat API (Chat Completions for OpenAI).
         if !NATIVE && ![nil, :chat_completions].include?(@scoring_protocol&.to_sym)
           raise ArgumentError, 'scoring_protocol requires RubyLLM 2.1'
         end
         @temperature = options.fetch(:temperature, luna_defaults ? 0 : nil)
-        @max_output_tokens = options.fetch(:max_output_tokens, @strategy == :single_request ? 8192 : 4)
+        @max_output_tokens = options.fetch(:max_output_tokens, @strategy == :ratings ? 4 : 8192)
         @tie_break_max_output_tokens = options.fetch(:tie_break_max_output_tokens, 8192)
         chat_provider_options = options.fetch(:chat_provider_options, {})
         raise ArgumentError, 'chat_provider_options must be a Hash' unless chat_provider_options.is_a?(Hash)
@@ -49,6 +64,8 @@ module RubyLLM
         @max_input_bytes = options[:max_input_bytes]
         @malformed_retries = options.fetch(:malformed_retries, 1)
         @max_workers = options.fetch(:max_workers, MAX_WORKERS)
+        @structured_output = options.fetch(:structured_output, false)
+        raise ArgumentError, 'structured_output must be true or false' unless [true, false].include?(@structured_output)
         raise ArgumentError, 'max_workers must be a positive integer' unless @max_workers.is_a?(Integer) && @max_workers.positive?
         raise ArgumentError, 'max_output_tokens must be positive' unless @max_output_tokens.is_a?(Integer) && @max_output_tokens.positive?
         unless @tie_break_max_output_tokens.is_a?(Integer) && @tie_break_max_output_tokens.positive?
@@ -62,11 +79,12 @@ module RubyLLM
         end
 
         @scorer = scorer || method(:score_with_model)
-        @responder = responder || method(:respond_with_model)
+        @responder = responder
       end
 
       def judge(input, questions:, model:)
-        return judge_single_request(input, questions, model) if @strategy == :single_request
+        return judge_one_call(input, questions, model, :probabilities) if @strategy == :single_request
+        return judge_one_call(input, questions, model, :committed) if @strategy == :committed
 
         jobs = build_jobs(input, questions)
         results = run(jobs) { |job| @scorer.call(job[:prompt]) }
@@ -78,7 +96,7 @@ module RubyLLM
           chosen = answer(question, softmax(ratings))
           if question.type == :choice && ratings.count(ratings.max) > 1 && @tie_breaker == :single_request
             # Raises if the one-call judgment also ties, after its corrective retry.
-            judgment = judge_single_request(input, { question.name => question }, model)
+            judgment = judge_one_call(input, { question.name => question }, model, :probabilities)
             chosen = judgment.answers.fetch(question.name)
             tie_breaks << { question: question.name, ratings:, judgment: }
           end
@@ -103,44 +121,98 @@ module RubyLLM
 
       private
 
-      def judge_single_request(input, questions, model)
+      # One request for every question. :probabilities asks for a distribution per question;
+      # :committed asks for one option per question and returns it with probability 1.
+      def judge_one_call(input, questions, model, mode)
         specs = validated_options(input, questions)
-        original_prompt = single_request_prompt(input, specs)
+        original_prompt = one_call_prompt(input, specs)
         prompt = original_prompt
         attempts = []
         loop do
-          result = @responder.call(prompt)
+          result = respond(prompt, mode, specs)
           attempts << result
           begin
-            answers, reported, normalization = parse_single_request(result.fetch(:content), specs)
+            answers, reported, normalization = mode == :committed ? parse_committed(result.fetch(:content), specs)
+                                                                  : parse_single_request(result.fetch(:content), specs)
+            raw = { method: mode == :committed ? 'committed_answers' : 'single_request_probabilities',
+                    scoring_provider: @scoring_provider, scoring_model: @scoring_model,
+                    reported_probabilities: reported, attempts: attempts.size }
+            raw[:reported_totals] = normalization if normalization
             return Types::Judgment.new(
-              answers:, model: model.id,
-              tokens: aggregate_tokens(attempts.map { |attempt| attempt.fetch(:tokens) }),
-              raw: { method: 'single_request_probabilities', scoring_provider: @scoring_provider,
-                     scoring_model: @scoring_model, reported_probabilities: reported,
-                     reported_totals: normalization, attempts: attempts.size }
+              answers:, model: model.id, raw:,
+              tokens: aggregate_tokens(attempts.map { |attempt| attempt.fetch(:tokens) })
             )
           rescue RubyLLM::Error => error
             raise if attempts.size > @malformed_retries
 
             prompt = "#{original_prompt}\n\nThe previous answer was invalid: #{error.message}. " \
                      "Previous answer: #{result.fetch(:content).to_s[0, 4000]}\n" \
-                     'Return a complete corrected JSON object with every question and option.'
+                     'Return a complete corrected JSON object with every question.'
           end
         end
       end
 
-      def parse_single_request(content, specs)
+      def respond(prompt, mode, specs)
+        return @responder.call(prompt) if @responder
+
+        respond_with_model(prompt, mode:, schema: @structured_output ? answer_schema(mode, specs) : nil)
+      end
+
+      # The answers object keyed by question ID, accepting a fenced reply or a list of one-question objects.
+      def reported_answers(content, specs)
         parsed = JSON.parse(strip_code_fence(content))
         raise Error, 'Scoring model returned a non-object response' unless parsed.is_a?(Hash)
 
         reported = parsed.fetch('answers')
+        # Some models mirror the prompt's question list: [{"id": {...}}, ...].
+        if reported.is_a?(Array) && reported.all? { |item| item.is_a?(Hash) }
+          ids = reported.flat_map(&:keys)
+          reported = reported.reduce({}, :merge) if ids.uniq.size == ids.size
+        end
         expected_questions = specs.map { |question, _| question.name.to_s }
         unless reported.is_a?(Hash) && reported.keys.sort == expected_questions.sort
           actual = reported.is_a?(Hash) ? reported.keys : reported.class.name
           raise Error, "Question IDs must be #{expected_questions.inspect}; got #{actual.inspect}"
         end
 
+        reported
+      end
+
+      def parse_committed(content, specs)
+        reported = reported_answers(content, specs)
+        answers = specs.to_h do |question, options|
+          keys = options.map { |name, _| name.to_s }
+          value = reported.fetch(question.name.to_s)
+          value = { 'yes' => 'true', 'no' => 'false' }.fetch(value.to_s.downcase, value.to_s) if question.type == :probability
+          index = keys.index(value.to_s) unless value.is_a?(Hash) || value.is_a?(Array)
+          raise Error, "The answer for #{question.name} must be one of #{keys.inspect}; got #{value.inspect}" unless index
+
+          [question.name, answer(question, keys.each_index.map { |position| position == index ? 1.0 : 0.0 })]
+        end
+        [answers, reported, nil]
+      rescue JSON::ParserError, KeyError, TypeError => error
+        raise Error, "Scoring model returned invalid JSON answers: #{error.message}"
+      end
+
+      # A strict JSON Schema that admits only complete answers to these questions.
+      def answer_schema(mode, specs)
+        properties = specs.to_h do |question, options|
+          keys = options.map { |name, _| name.to_s }
+          schema = if mode == :committed
+                     { type: 'string', enum: keys }
+                   else
+                     { type: 'object', properties: keys.to_h { |key| [key, { type: 'number' }] },
+                       required: keys, additionalProperties: false }
+                   end
+          [question.name.to_s, schema]
+        end
+        answers = { type: 'object', properties:, required: properties.keys, additionalProperties: false }
+        { name: 'judgment', strict: true,
+          schema: { type: 'object', properties: { answers: }, required: ['answers'], additionalProperties: false } }
+      end
+
+      def parse_single_request(content, specs)
+        reported = reported_answers(content, specs)
         normalization = {}
         answers = specs.to_h do |question, options|
           distribution = reported.fetch(question.name.to_s)
@@ -172,7 +244,9 @@ module RubyLLM
         raise Error, "Scoring model returned invalid JSON probabilities: #{error.message}"
       end
 
-      def single_request_prompt(input, specs)
+      # The state comes first: listing the questions first made Luna 6 retry more and answer less
+      # accurately on the benchmark cases.
+      def one_call_prompt(input, specs)
         request = {
           state: input,
           questions: specs.map do |question, options|
@@ -280,10 +354,13 @@ module RubyLLM
         end
       end
 
+      # Luna rates poorly with reasoning on: the output limit is spent before the answer.
       def default_chat_options(luna_defaults)
-        return {} unless @scoring_provider == :openai
-
-        luna_defaults ? { store: false, reasoning_effort: 'none' } : { store: false }
+        case @scoring_provider
+        when :openai then luna_defaults ? { store: false, reasoning_effort: 'none' } : { store: false }
+        when :openrouter then luna_defaults ? { reasoning: { enabled: false } } : {}
+        else {}
+        end
       end
 
       def score_with_model(prompt)
@@ -300,13 +377,16 @@ module RubyLLM
         end
       end
 
-      def respond_with_model(prompt)
+      def respond_with_model(prompt, mode:, schema: nil)
         limit = @strategy == :ratings ? @tie_break_max_output_tokens : @max_output_tokens
-        response = scoring_chat(instructions: SINGLE_REQUEST_INSTRUCTIONS, max_output_tokens: limit).ask(prompt)
-        { content: response.content.to_s, tokens: response.tokens }
+        instructions = mode == :committed ? COMMITTED_INSTRUCTIONS : SINGLE_REQUEST_INSTRUCTIONS
+        response = scoring_chat(instructions:, max_output_tokens: limit, schema:).ask(prompt)
+        # With a schema, RubyLLM parses the JSON reply into a Hash.
+        content = response.content
+        { content: content.is_a?(String) ? content : JSON.generate(content), tokens: response.tokens }
       end
 
-      def scoring_chat(instructions: SYSTEM_INSTRUCTIONS, max_output_tokens: @max_output_tokens)
+      def scoring_chat(instructions: SYSTEM_INSTRUCTIONS, max_output_tokens: @max_output_tokens, schema: nil)
         context = RubyLLM::Context.new(@config)
         chat = if NATIVE
                  context.chat(model: @scoring_model, provider: @scoring_provider,
@@ -316,15 +396,30 @@ module RubyLLM
                end
         chat.with_instructions(instructions)
         chat.with_temperature(@temperature) unless @temperature.nil?
+        chat.with_schema(schema) if schema
         if NATIVE
           chat.with_max_output_tokens(max_output_tokens)
-          chat.with_provider_options(@chat_provider_options)
+          chat.with_provider_options(deep_compact(@chat_provider_options))
         else
-          chat.with_params(**deep_merge(max_output_tokens_param(max_output_tokens), @chat_provider_options))
+          chat.with_params(**deep_compact(deep_merge(max_output_tokens_param(max_output_tokens), @chat_provider_options)))
         end
         chat
       end
 
+      # A nil in chat_provider_options removes that field: a default, or before 2.1
+      # the output limit field. A hash emptied by removal is removed too.
+      def deep_compact(hash)
+        hash.each_with_object({}) do |(key, value), compacted|
+          next if value.nil?
+
+          if value.is_a?(Hash) && !value.empty?
+            value = deep_compact(value)
+            next if value.empty?
+          end
+          compacted[key] = value
+        end
+      end
+
       # The request field each provider reads for the output limit, as RubyLLM 2.1 renders it.
       def max_output_tokens_param(limit)
         case @scoring_provider

data/lib/ruby_llm/llm_judge/provider.rb

@@ -7,7 +7,10 @@ module RubyLLM
         'http://127.0.0.1'
       end
 
-      def judge(input, questions:, model:, provider_options: {})
+      # RubyLLM passes images as +with:+; they are not supported yet.
+      def judge(input, questions:, model:, with: [], provider_options: {})
+        raise UnsupportedAttachmentError, Array(with).first.mime_type unless Array(with).empty?
+
         Engine.new(config:, model: model.id, provider_options:).judge(input, questions:, model:)
       end
 

data/lib/ruby_llm/llm_judge/version.rb

@@ -2,6 +2,6 @@
 
 module RubyLLM
   module LLMJudge
-    VERSION = '0.1.2'
+    VERSION = '0.1.4'
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ruby_llm-llm_judge
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 0.1.4
 platform: ruby
 authors:
 - JP Camara