aireview 2.0.0 → 2.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3f326ae1b11f59f7c447e8f128ab94845ef1543b0b377753f053fd4f69e03bad
-  data.tar.gz: c5908173ca8020a0563b6d4ee1fd4d8ce4ebe609fda7ded26d5b7eb40c84dda1
+  metadata.gz: 1e262ff91d3ae033f68f0a1a940af115dda8cf924e6715d7d494dfbe30e2e907
+  data.tar.gz: ea5bfca46774fe542338e67b198dc66e550b33404363646da055e8648c67f6b1
 SHA512:
-  metadata.gz: 2de9b7dd7e6caaa440287339d03eed843dfdb73010ae8cf6a01dea4e9348e3b072fc4fb61a55bf30eb4ed324e857ff883e35e31b711bc542cb8d1495645041bc
-  data.tar.gz: 5f0867f71415c019c03d99bd2215fc7f7ef7cadf98ab8b0db91e6eb98af54cfae3787b848c7611fc371cfa19cc94acd55e170047f094d8543f5c77f258e3313d
+  metadata.gz: 7fee8f2bbade71a38d72e66be714c748d38afc91643dde35329fbc64a357c88af6020bda5f1a89b9762922dc4ff4a413cdf777fab03b16dc9a09e073953ad400
+  data.tar.gz: 869d8f2e89afd98feab341028556c7a361ba46f9afb070cb728298a2b20aafc8df1aba5525f9ab9299c4b35169dcc3323cef72771764f6120ccfba0f34c34d69

data/CHANGELOG.md

@@ -1,5 +1,41 @@
 # Changelog
 
+## 2.1.0
+
+- An answer that is not valid JSON and that the provider cut off at the
+  output limit (`max_tokens`) or blocked (`content_filter`) sends the stage
+  to the next model without a repair request: the same model would cut the
+  repair off too. The log names the reason. A valid answer is taken
+  whatever the finish reason.
+- Every LLM request logs the token counts the provider reported
+  (`tokens: input=… output=… thinking=… cache_read=… cache_write=…`), one
+  line per attempt; cache counts only when not zero. They are not summed:
+  Gemini already counts thinking into output, and input leaves out cached
+  tokens, so the prompt size is input + cache_read + cache_write.
+- RubyLLM 2.0 (was 1.16). Output schemas are built with Schematist, which
+  RubyLLM now ships instead of `ruby_llm-schema`; `json` goes back to 2.x,
+  RubyLLM 2.0 requires `json < 3`.
+- A structured answer is read as with RubyLLM 1.16: JSON that breaks the
+  schema sends the stage to the next model at once, only a text that is not
+  JSON gets a repair request.
+- OpenAI stays on Chat Completions: RubyLLM 2.0 would switch it to the
+  Responses API, which a compatible server behind `LLM_API_BASE` usually
+  lacks.
+- The critique schema goes to Chat Completions providers (Ollama,
+  OpenRouter, OpenAI) with `strict: false`: its `refinement` is optional,
+  and OpenAI strict mode rejects optional properties. Generate stays strict;
+  Gemini gets no strict flag, as before.
+
+## 2.0.2
+
+- When the merge request changes while the review runs, the skip warning
+  names the fields that changed (`sha`, `diff_refs`, `target_branch`,
+  `title`, `description`) instead of printing only the new head.
+- In `review_mode: once` the skip message names both the mode and whether
+  the review is up to date (`the review is up to date`, `review inputs
+  changed`, `review freshness is unknown`); a stale review gets a hint:
+  retry the job in CI, `--force` outside of it.
+
 ## 2.0.0
 
 Breaking: the review key of a shared pool includes the pool order and the

data/README.md

@@ -248,7 +248,7 @@ A local Ollama needs no API key. The providers can be swapped by changing
 `LLM_GENERATE_PROVIDER`, `LLM_CRITIQUE_PROVIDER` and the corresponding models.
 To run both stages locally, set `ollama` in both provider variables. The
 address with `/v1` matches the
-[Ollama configuration in RubyLLM](https://rubyllm.com/configuration/#provider-configuration).
+[Ollama configuration in RubyLLM](https://rubyllm.com/configuration-providers/).
 `LLM_TIMEOUT` sets the timeout of every LLM request in seconds; for a slow
 local model it can be raised, but not without limit: a hung request holds
 the job for exactly that long while a fallback model sits idle. An
@@ -698,12 +698,35 @@ The template is written for shell-executor runners (the image runs through
 `docker run` with secrets passed by name) and carries `[skip review]`,
 `resource_group`, `allow_failure` and `REVIEW_MODE=once`. Models come from
 the image defaults, `.aireview.yml` is mounted into the container only when
-the project has one. The job runs in the `.post` stage — it exists in every
-pipeline, so a project does not declare `stages`; to move the review to
-another stage, add `aireview: {stage: review}` to the project file. Every
-variable the config reads (`Config.env_names`: models, providers, reserves,
-temperatures, limits, `OLLAMA_API_BASE` and so on) is passed into the
-container by name, so a project can override anything through its CI/CD
+the project has one. The job runs in the `.post` stage: it exists in every
+pipeline, so a project does not declare it.
+
+GitLab does not start a pipeline in which, after `rules` and `only` are
+applied, only `.pre` and `.post` jobs remain. That happens when a project has
+no other merge request jobs (a build that runs on tags only does not count).
+In that case add a `review` stage to the project's existing `stages`, without
+replacing them, and move the job there:
+
+```yaml
+stages:
+  - review   # added to the project's stages
+  - build
+
+aireview:
+  stage: review
+```
+
+To keep the review from waiting for other stages to finish, add `needs: []`
+to the `aireview` job.
+
+The template sets `REVIEW_MODE` as a job variable, and an environment variable
+beats `.aireview.yml`: `review_mode` in the project file has no effect once
+the template is included. Change the mode with a `REVIEW_MODE` CI/CD variable
+of the project or with `aireview: {variables: {REVIEW_MODE: update}}`.
+
+Every variable the config reads (`Config.env_names`: models, providers,
+reserves, temperatures, limits, `OLLAMA_API_BASE` and so on) is passed into
+the container by name, so a project can override anything through its CI/CD
 variables — for instance, swap an unavailable model with `LLM_CRITIQUE_MODEL`
 without waiting for an image release.
 

data/lib/aireview/cli.rb

@@ -212,18 +212,41 @@ module Aireview
       up_to_date = existing[:key] == key
       return false unless up_to_date || (mode == 'once' && !retried_ci_job?(gitlab_client))
 
-      reason = up_to_date ? 'existing review is up to date' : 'merge request already reviewed (review_mode=once)'
-      @out.puts("Review skipped: #{reason}")
+      @out.puts("Review skipped: #{skip_reason(existing, up_to_date: up_to_date, mode: mode)}")
       true
     end
 
+    # In once mode the review is not repeated on new pushes, even when the MR
+    # has changed. So besides the mode the message says whether the review is
+    # up to date: if it is not, the Retry button of the GitLab job updates it.
+    def skip_reason(existing, up_to_date:, mode:)
+      return 'existing review is up to date' unless mode == 'once'
+
+      state = if up_to_date then 'the review is up to date'
+              elsif existing[:key].nil? then "review freshness is unknown: #{update_hint}"
+              else "review inputs changed: #{update_hint}"
+              end
+      "merge request already reviewed (review_mode=once), #{state}"
+    end
+
+    def update_hint
+      ci_job_context ? 'retry the job to update' : 'use --force to review again'
+    end
+
     def retried_ci_job?(gitlab_client)
-      project_id, job_id = @env.values_at('CI_PROJECT_ID', 'CI_JOB_ID')
-      return false if Aireview::Utils.blank?(project_id) || Aireview::Utils.blank?(job_id)
+      project_id, job_id = ci_job_context
+      return false unless project_id
 
       gitlab_client.retried_job?(project_id, job_id)
     end
 
+    def ci_job_context
+      project_id, job_id = @env.values_at('CI_PROJECT_ID', 'CI_JOB_ID')
+      return if Aireview::Utils.blank?(project_id) || Aireview::Utils.blank?(job_id)
+
+      [project_id, job_id]
+    end
+
     def publish_review(review, context, publication)
       return if merge_request_moved?(context)
 
@@ -245,10 +268,13 @@ module Aireview
         context[:parser_result].project_id,
         context[:parser_result].iid
       )
-      return false if ReviewMarker.state(current) == ReviewMarker.state(context[:merge_request])
+      before = ReviewMarker.state(context[:merge_request])
+      after = ReviewMarker.state(current)
+      changed = after.keys.reject { |field| after[field] == before[field] }
+      return false if changed.empty?
 
-      @logger.warn("Merge request moved to #{current['sha']} (#{current['target_branch']}) " \
-                   'while review was running; skipping publication')
+      @logger.warn("Merge request changed while review was running (#{changed.join(', ')}); " \
+                   'skipping publication')
       true
     end
 

data/lib/aireview/llm_client.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+require 'json'
 require 'timeout'
 require_relative 'errors'
 require_relative 'utils'
@@ -22,8 +23,8 @@ module Aireview
       @contexts = {}
     end
 
-    # Returns the RubyLLM answer (content is text or a structure by the
-    # schema). A request error is re-raised as is — LlmFailure classifies it.
+    # Returns the RubyLLM answer; read it with LlmClient.content. A request
+    # error is re-raised as is — LlmFailure classifies it.
     def request(prompt, candidate:, key:, timeout:, key_index: 0)
       load_ruby_llm
       stage = prompt.stage.to_s
@@ -36,15 +37,44 @@ module Aireview
         .with_schema(prompt.schema)
       chat.with_instructions(prompt.system)
       response = Timeout.timeout(timeout) { chat.ask(prompt.user) }
-      @logger.info("LLM #{stage} request completed (model=#{model})")
+      @logger.info("LLM #{stage} request completed (model=#{model}#{token_counts(response)})")
       response
     rescue Timeout::Error
       @logger.warn("LLM #{stage} request timed out after #{timeout.round} seconds (model=#{model})")
       raise
     end
 
+    # The answer as RubyLLM 1.x gave it under a schema: the parsed JSON when
+    # the text is JSON, the text itself otherwise (the pipeline repairs it).
+    # RubyLLM 2 always returns the text, and a Hash that breaks the schema
+    # would go to a repair request instead of the next model. An empty
+    # answer stays an empty String (Message#parsed would turn it into nil);
+    # JSON null becomes nil, as in 1.x.
+    def self.content(response)
+      content = response.content
+      return content unless content.is_a?(String) && !content.empty?
+
+      response.parsed
+    rescue JSON::ParserError
+      content
+    end
+
     private
 
+    # Token counts as the provider reported them, one line per attempt. They
+    # are not summed: Gemini already counts thinking into output. Input is
+    # what was not read from or written to a cache; the prompt size is
+    # input + cache_read + cache_write, and a retry of the same prompt on
+    # Gemini is often served from its implicit cache.
+    def token_counts(response)
+      tokens = response.tokens
+      cached = {cache_read: tokens.cache_read, cache_write: tokens.cache_write}.reject { |_, count| count.to_i.zero? }
+      counts = {input: tokens.input, output: tokens.output, thinking: tokens.thinking}.compact.merge(cached)
+      return '' if counts.empty?
+
+      ", tokens: #{counts.map { |name, count| "#{name}=#{count}" }.join(' ')}"
+    end
+
     def load_ruby_llm
       require 'ruby_llm'
     rescue LoadError => e
@@ -103,8 +133,11 @@ module Aireview
       end
     end
 
+    # RubyLLM 2 sends OpenAI requests to the Responses API; a compatible
+    # server behind LLM_API_BASE usually has Chat Completions only.
     def configure_remote_provider(ruby_config, provider, api_key)
       ruby_config.public_send("#{provider}_api_key=", api_key)
+      ruby_config.openai_protocol = :chat_completions if provider == 'openai'
       return unless Aireview::Utils.present?(@config.llm_api_base)
 
       ruby_config.public_send("#{provider}_api_base=", @config.llm_api_base)

data/lib/aireview/model_checker.rb

@@ -111,7 +111,7 @@ module Aireview
         schema: stage == 'critique' ? CritiqueOutputSchema : GenerateOutputSchema
       )
       key = @config.provider_api_keys(candidate.provider).first
-      @client.request(request, candidate: candidate, key: key, timeout: @config.llm_timeout.to_f).content
+      LlmClient.content(@client.request(request, candidate: candidate, key: key, timeout: @config.llm_timeout.to_f))
     end
 
     # missing — the provider has no such model; unverified — the provider

data/lib/aireview/output_schemas.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
-require 'ruby_llm/schema'
+require 'schematist'
 
 module Aireview
   module OutputSchemaValues
@@ -18,7 +18,7 @@ module Aireview
     DECISIONS = %w[keep reject].freeze
   end
 
-  class GenerateOutputSchema < RubyLLM::Schema
+  class GenerateOutputSchema < Schematist::Schema
     string :summary
     array :candidates, max_items: 3 do
       object do
@@ -38,7 +38,7 @@ module Aireview
     end
   end
 
-  class CritiqueOutputSchema < RubyLLM::Schema
+  class CritiqueOutputSchema < Schematist::Schema
     array :verdicts do
       object do
         string :id

data/lib/aireview/review_pipeline.rb

@@ -26,6 +26,11 @@ module Aireview
     DRY_RUN_CANDIDATES_JSON = '[{"id":"C1","file":"path/from/diff.rb","line":1,' \
                               '"quoted_code":"...","problem":"...","why":"...","suggestion":"...",' \
                               '"category":"bug","severity":"major"}]'
+    # Finish reasons after which an unparsable answer gets no repair.
+    CUT_OFF_REASONS = {
+      max_tokens: 'cut off at the output limit (max_tokens)',
+      content_filter: 'blocked by the provider (content_filter)'
+    }.freeze
 
     def initialize(config:, reviewer: nil, context_builder: nil, logger: Logger.new($stderr))
       @config = config
@@ -214,6 +219,7 @@ module Aireview
     def parse_string_with_repair(raw:, kind:, expected:, repair_stage:, critique_candidate_ids: nil)
       parse_expected_result(raw, expected, critique_candidate_ids: critique_candidate_ids)
     rescue JSON::ParserError, SchemaError => e
+      raise_if_cut_off(stage: repair_stage, kind: kind, error: e)
       @logger.warn("Invalid #{kind} JSON, requesting one repair: #{e.message}")
       repaired = repair_json(
         raw: raw,
@@ -225,10 +231,19 @@ module Aireview
       begin
         parse_expected_result(repaired, expected, critique_candidate_ids: critique_candidate_ids)
       rescue JSON::ParserError, SchemaError => second_error
+        raise_if_cut_off(stage: repair_stage, kind: "#{kind} repair", error: second_error)
         raise ParseError, "LLM returned invalid #{kind} JSON after repair: #{second_error.message}"
       end
     end
 
+    # An answer the provider cut off or blocked is not a JSON mistake: the
+    # same model would cut the repair off too, so the stage goes to the next
+    # model at once. A valid answer is taken whatever the reason.
+    def raise_if_cut_off(stage:, kind:, error:)
+      reason = CUT_OFF_REASONS[@reviewer.finish_reason(stage)]
+      raise ParseError, "LLM #{kind} was #{reason}: #{error.message}" if reason
+    end
+
     def parse_expected_result(raw, expected, critique_candidate_ids: nil)
       @parser.parse(raw, expected: expected, critique_candidate_ids: critique_candidate_ids)
     end

data/lib/aireview/reviewer.rb

@@ -16,6 +16,7 @@ module Aireview
       @logger = logger
       @router = router || LlmRouter.new(config: config, logger: logger)
       @client = client || LlmClient.new(config: config, logger: logger)
+      @finish_reasons = {}
     end
 
     # pinned — a request only to the model that answered last in the stage
@@ -47,6 +48,12 @@ module Aireview
       @router.critique_weaker?
     end
 
+    # Why the last answer of the stage stopped (:stop, :max_tokens,
+    # :content_filter…), nil when the provider did not say or no answer came.
+    def finish_reason(stage)
+      @finish_reasons[stage.to_s]
+    end
+
     # An invalid result: the model is excluded for the stage, the next
     # request of the stage goes to another. Returns the excluded model or nil.
     def exclude_answered_model(stage:, reason:)
@@ -58,11 +65,13 @@ module Aireview
     # A pinned route giving up is not an API error for the pipeline but
     # "repair impossible": the same fate as an invalid result.
     def call_llm(prompt, pinned:)
+      @finish_reasons.delete(prompt.stage)
       response = @router.call(stage: prompt.stage, request_chars: prompt.chars, pinned: pinned) do |route, timeout|
         @client.request(prompt, candidate: route.candidate, key: route.key, key_index: route.key_index,
                                 timeout: timeout)
       end
-      response.content
+      @finish_reasons[prompt.stage] = response.finish_reason
+      LlmClient.content(response)
     rescue RouteExhaustedError => e
       raise RepairImpossibleError, e.message
     end

data/lib/aireview/version.rb

@@ -1,4 +1,4 @@
 # frozen_string_literal: true
 module Aireview
-  VERSION = '2.0.0'
+  VERSION = '2.1.0'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: aireview
 version: !ruby/object:Gem::Version
-  version: 2.0.0
+  version: 2.1.0
 platform: ruby
 authors:
 - Denis Levenko
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-09-19 00:00:00.000000000 Z
+date: 2026-09-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: dotenv
@@ -50,14 +50,14 @@ dependencies:
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 1.16.0
+        version: 2.0.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 1.16.0
+        version: 2.0.0
 description: 'Reviews self-hosted GitLab merge requests with a two-pass LLM pipeline:
   the first pass finds candidate findings, the second one critiques them and drops
   the weak ones. Supports Gemini and local Ollama, optional Jira context and posting