aireview 2.0.0 → 2.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3f326ae1b11f59f7c447e8f128ab94845ef1543b0b377753f053fd4f69e03bad
-  data.tar.gz: c5908173ca8020a0563b6d4ee1fd4d8ce4ebe609fda7ded26d5b7eb40c84dda1
+  metadata.gz: c37a072f5cb03c8a4e95944a3c6fc5386d1abe6c340c63ebee9715db5d44ec6a
+  data.tar.gz: b2e2ef104f8e4310dafc0dd5041a899c7ff06af5d2098cff25e0e38f010f9847
 SHA512:
-  metadata.gz: 2de9b7dd7e6caaa440287339d03eed843dfdb73010ae8cf6a01dea4e9348e3b072fc4fb61a55bf30eb4ed324e857ff883e35e31b711bc542cb8d1495645041bc
-  data.tar.gz: 5f0867f71415c019c03d99bd2215fc7f7ef7cadf98ab8b0db91e6eb98af54cfae3787b848c7611fc371cfa19cc94acd55e170047f094d8543f5c77f258e3313d
+  metadata.gz: '09524e49ed6896e32e6e73d3fcc5f5e27d19eaa255273017acf699e8e6992f44a8a950e34d96f446066e58b31a6ee625188891e913ac3a4c0db27ac2b39e95a8'
+  data.tar.gz: 4a193a705c06609c07ceb2833211d51d124e1299378368feb50fc08e0fac48a6ff09f553b47406b0bd367e83ebfe7b55fd66d018f4781551725aa27234e95113

data/CHANGELOG.md

@@ -1,5 +1,78 @@
 # Changelog
 
+## 2.2.0
+
+- Jev shadow mode (`llm.jev.shadow`, `LLM_JEV_SHADOW`, key `JEV_API_KEY`):
+  after the LLM Critique the same candidates go to Jev (TypeSafe), a fast
+  classifier, and its keep/reject/unverifiable decisions with the raw
+  probabilities are logged next to the Critique verdicts. The report and the
+  review key do not change; a Jev failure is a warning. The data is for
+  choosing between an LLM and Jev as the critic later.
+- Critique engine (`llm.critique.engine`, `LLM_CRITIQUE_ENGINE`,
+  `--critique-engine`): `model` (the default, the LLM Critique as before) or
+  `jev`. With `jev` Jev decides keep/reject without refining the wording;
+  the candidates it cannot judge, and all of them when Jev fails, go to the
+  LLM Critique with `llm.jev.fallback: model` (the default), or are
+  rejected / fail the run with `fail`. Duplicates are dropped once over the
+  merged verdicts. The report says how Jev took part. The Jev thresholds
+  default to a provisional 0.5 until they are chosen from the shadow logs.
+- The stages that need an LLM follow the engine: with Jev and
+  `fallback: fail` the critique model, its key, its share of the context
+  budget, the pool's critique policy and its `models check` probe are not
+  required. `models check` probes Jev as well (counted only for the engine).
+- The review key includes the Jev version, thresholds, fallback and
+  question templates with `engine: jev`; with the default engine it does not
+  change, except for runs with `--no-critique`: the unused critique policy
+  of the pool (`critique.start`, `rank`, `allow_weaker`) no longer enters
+  their key and is no longer validated, so such a review is redone once.
+- `--dry-run` prints the Jev request for the stub candidate.
+- The gem now packages `lib/**/*.yml` (the Jev questions).
+- A pool of models from several providers: OpenAI, Anthropic and OpenRouter
+  take their keys from `OPENAI_API_KEY(S)`, `ANTHROPIC_API_KEY(S)`,
+  `OPENROUTER_API_KEY(S)`, and a string like `anthropic/claude-opus-4.5` in
+  `LLM_MODELS` or a reserve names the provider (an OpenRouter model goes with
+  the prefix: `openrouter/qwen/qwen3-coder`).
+- `api_base` for a model of the pool, a stage and a reserve: several servers
+  of your own in one pool. Such a model is named with its address, gets
+  `LLM_API_KEY` or no auth — never a provider's key — and needs no key to
+  start. Models without an address keep their names and review keys.
+
+## 2.1.0
+
+- An answer that is not valid JSON and that the provider cut off at the
+  output limit (`max_tokens`) or blocked (`content_filter`) sends the stage
+  to the next model without a repair request: the same model would cut the
+  repair off too. The log names the reason. A valid answer is taken
+  whatever the finish reason.
+- Every LLM request logs the token counts the provider reported
+  (`tokens: input=… output=… thinking=… cache_read=… cache_write=…`), one
+  line per attempt; cache counts only when not zero. They are not summed:
+  Gemini already counts thinking into output, and input leaves out cached
+  tokens, so the prompt size is input + cache_read + cache_write.
+- RubyLLM 2.0 (was 1.16). Output schemas are built with Schematist, which
+  RubyLLM now ships instead of `ruby_llm-schema`; `json` goes back to 2.x,
+  RubyLLM 2.0 requires `json < 3`.
+- A structured answer is read as with RubyLLM 1.16: JSON that breaks the
+  schema sends the stage to the next model at once, only a text that is not
+  JSON gets a repair request.
+- OpenAI stays on Chat Completions: RubyLLM 2.0 would switch it to the
+  Responses API, which a compatible server behind `LLM_API_BASE` usually
+  lacks.
+- The critique schema goes to Chat Completions providers (Ollama,
+  OpenRouter, OpenAI) with `strict: false`: its `refinement` is optional,
+  and OpenAI strict mode rejects optional properties. Generate stays strict;
+  Gemini gets no strict flag, as before.
+
+## 2.0.2
+
+- When the merge request changes while the review runs, the skip warning
+  names the fields that changed (`sha`, `diff_refs`, `target_branch`,
+  `title`, `description`) instead of printing only the new head.
+- In `review_mode: once` the skip message names both the mode and whether
+  the review is up to date (`the review is up to date`, `review inputs
+  changed`, `review freshness is unknown`); a stale review gets a hint:
+  retry the job in CI, `--force` outside of it.
+
 ## 2.0.0
 
 Breaking: the review key of a shared pool includes the pool order and the

data/README.md

@@ -79,11 +79,18 @@ REVIEW_LANGUAGE=ru
 REVIEW_MODE=update
 ```
 
-At the moment only the `gemini` and `ollama` providers are supported. The
-provider and the model of each stage are set through `LLM_GENERATE_PROVIDER`,
-`LLM_GENERATE_MODEL`, `LLM_CRITIQUE_PROVIDER` and `LLM_CRITIQUE_MODEL`.
-`LLM_PROVIDER` stays the shared default when a stage has no provider of its
-own.
+The providers are `gemini`, `ollama`, `openai`, `anthropic` and
+`openrouter`, and any OpenAI-compatible server of your own (vLLM, llama.cpp,
+LM Studio) as `openai` with an `api_base`. The provider and the model of each
+stage are set through `LLM_GENERATE_PROVIDER`, `LLM_GENERATE_MODEL`,
+`LLM_CRITIQUE_PROVIDER` and `LLM_CRITIQUE_MODEL`. `LLM_PROVIDER` stays the
+shared default when a stage has no provider of its own. Gemini and Ollama
+are the ones the image defaults and the verified models are about; for the
+others run `aireview models check` before relying on a model.
+
+Each provider takes its keys from its own variables:
+`GEMINI_API_KEY(S)`, `OPENAI_API_KEY(S)`, `ANTHROPIC_API_KEY(S)`,
+`OPENROUTER_API_KEY(S)`; `LLM_API_KEY` is the shared fallback.
 
 `REVIEW_LANGUAGE` (or `review_language` in `.aireview.yml`) sets the language
 of the review: both the LLM answers and the headings of the rendered report.
@@ -175,6 +182,52 @@ the end of the run (see "Fallback models and keys"). The separate smoke test
 of every model with both schemas (`aireview models check`) runs on an image
 release and on a schedule, not on MRs.
 
+### A pool of models from several providers
+
+The critic is best the strongest model you have, and it does not have to be
+Gemini: the shared pool (see "Shared model pool") takes models of any
+provider, ordered by strength as you see it — the order is the only measure,
+names and release dates are not compared.
+
+```bash
+LLM_MODELS=anthropic/claude-opus-4.5,openai/gpt-5,gemini/gemini-3.8-flash,ollama/qwen2.5-coder:14b
+LLM_GENERATE_START=gemini-3.8-flash
+ANTHROPIC_API_KEY=xxx
+OPENAI_API_KEY=xxx
+GEMINI_API_KEY=xxx
+```
+
+In a string the provider goes in front of the model; an OpenRouter model has
+a slash of its own, so it is written with the prefix:
+`openrouter/qwen/qwen3-coder`. Servers of your own, one or several, are
+listed in `.aireview.yml` with their address:
+
+```yaml
+llm:
+  models:
+    - provider: anthropic
+      model: claude-opus-4.5
+    - provider: openai            # an OpenAI-compatible server of your own
+      model: qwen3-coder
+      api_base: http://gpu1:8000/v1
+    - provider: ollama
+      model: qwen2.5-coder:14b
+      api_base: http://gpu2:11434/v1
+  generate:
+    start: qwen2.5-coder:14b
+```
+
+`api_base` also works for a stage (`llm.generate.api_base`) and for a reserve
+in `fallbacks`. A model with its own `api_base` is named with the address
+(`openai/qwen3-coder@http://gpu1:8000/v1`), so two servers with the same model are
+two models for quarantine and the review key, and a start may name either.
+Such a server never gets a provider's key: it is called with `LLM_API_KEY`,
+or without auth when that is not set, and needs no key to start. Without an
+`api_base` a model goes to its provider's API, or to `LLM_API_BASE` for
+Gemini, OpenAI and OpenRouter as before.
+
+Instead of an LLM the critic can be Jev (see "Critique engine").
+
 ### Local Ollama
 
 Install Ollama following the
@@ -248,7 +301,7 @@ A local Ollama needs no API key. The providers can be swapped by changing
 `LLM_GENERATE_PROVIDER`, `LLM_CRITIQUE_PROVIDER` and the corresponding models.
 To run both stages locally, set `ollama` in both provider variables. The
 address with `/v1` matches the
-[Ollama configuration in RubyLLM](https://rubyllm.com/configuration/#provider-configuration).
+[Ollama configuration in RubyLLM](https://rubyllm.com/configuration-providers/).
 `LLM_TIMEOUT` sets the timeout of every LLM request in seconds; for a slow
 local model it can be raised, but not without limit: a hung request holds
 the job for exactly that long while a fallback model sits idle. An
@@ -522,6 +575,124 @@ Critique receives the result of the check in the candidate's `note` field
 and decides keep/reject with it in mind. With `--no-critique` the check works
 the same way, its marks just go straight to the report.
 
+### Critique engine: an LLM or Jev
+
+`llm.critique.engine` picks who checks the candidates of the first pass:
+
+| | `model` (default) | `jev` |
+|---|---|---|
+| What checks | the LLM Critique chain or pool (`LLM_CRITIQUE_MODEL`, `critique.rank`) | [Jev](https://docs.typesafe.ai), a classifier by TypeSafe |
+| Context | the whole prompt: MR, Jira, diff | the same, cut to Jev's limits (see below) |
+| Wording of findings | refined by the critic | the first pass's own |
+| Time and cost | tens of seconds, provider quota | under a second, per input token |
+| Depends on Gemini | when the critique model is Gemini | no, with `fallback: fail` |
+| Non-English reviews | fine | not declared by TypeSafe, check first |
+| Code leaves for | the LLM provider | TypeSafe as well |
+
+The default stays `model`. A stronger critic than the generator is set with
+the existing settings (`LLM_CRITIQUE_MODEL`, or the order of the pool); Jev is
+meant for when speed or independence from the LLM quota matters more than
+refined wording, and only after its thresholds are chosen from the shadow
+logs (see below).
+
+```yaml
+llm:
+  critique:
+    engine: jev           # LLM_CRITIQUE_ENGINE, --critique-engine jev
+  jev:
+    model: jev-1.13.0     # LLM_JEV_MODEL; a pinned version, an alias is an error here
+    fallback: model       # LLM_JEV_FALLBACK: model or fail
+    keep_above: 0.5       # LLM_JEV_KEEP_ABOVE; enough_context, version_claim, duplicate in YAML
+```
+
+With `engine: jev` Jev answers the questions of `prompts/jev_questions.yml`
+about every candidate and decides:
+
+- claims that a version does not exist — reject;
+- not enough in the request to judge it, or too large for a Jev request —
+  unverifiable: with `fallback: model` such candidates go to the LLM
+  Critique (only them), with `fallback: fail` they are rejected;
+- otherwise keep when `real_issue` reaches `keep_above`;
+- duplicates are dropped once, over what Jev and the LLM kept together: the
+  more severe finding stays, on a tie the one the first pass listed first.
+
+When Jev fails (network, 429/529 after one retry, an invalid answer), the
+LLM Critique takes over with `fallback: model`, and the run fails with
+`fallback: fail`. The report says how Jev took part: checked by Jev without
+refining the wording, partly by the LLM, or by the LLM because Jev was
+unavailable. `engine: jev` without `JEV_API_KEY` or with an alias is a
+configuration error at start; `--no-critique` switches off every engine.
+
+What an LLM is still needed for follows the engine: with `fallback: fail`
+there is no LLM Critique at all, so its model and key are not required
+(generate on Ollama with Jev needs no Gemini key), the context budget counts
+generate only, the pool's critique policy is not used and `models check`
+probes the generate stage only. `models check` also sends Jev a probe
+request: with `engine: jev` it counts, in shadow mode it is shown but never
+fails the check. `--dry-run` shows the Jev request for the stub candidate
+(`=== JEV STATE ===`, `=== JEV QUESTIONS ===`).
+
+The review key follows the engine too: with `engine: jev` it includes the Jev
+version, every threshold, the fallback and the question templates; the LLM
+Critique model counts only while Jev can fall back to it. With the default
+engine the key is what it was before Jev, whatever `llm.jev` says. Without
+an LLM Critique (Jev with `fallback: fail`, or `--no-critique`) the critique
+settings — model, limit, start, rank, allow_weaker — are neither validated
+nor part of the key, while the generate pool stays in it.
+
+### Jev shadow (experiment)
+
+Jev is a classifier, not a text model: it answers yes/no and choice
+questions about a given state with probabilities. Before it decides anything
+as the engine, it can run in shadow mode next to the default engine: after
+the LLM Critique the same candidates go to Jev, and its decisions are logged
+next to the Critique verdicts. The report and the review key do not change;
+with `engine: jev` the shadow is ignored.
+
+```yaml
+llm:
+  jev:
+    shadow: true          # LLM_JEV_SHADOW=true
+    model: jev-1.13.0     # LLM_JEV_MODEL; a pinned version, not jev-latest
+    timeout: 10           # seconds per request
+    keep_above: 0.5       # provisional thresholds, only for the log line
+    enough_context: 0.5
+    version_claim: 0.5
+    duplicate: 0.5
+```
+
+The key is `JEV_API_KEY`. For every candidate Jev is asked whether it is a
+real, well-supported problem (the rules of `prompts/critique.txt`, asked in
+`prompts/jev_questions.yml`), whether the state holds enough to judge it,
+whether it claims that a version does not exist, which other candidate it
+duplicates, and how severe it is. The log gets one line per candidate and a
+summary:
+
+```
+Jev shadow C1: keep (real issue; real_issue=0.91 enough_context=0.88 version_claim=0.02 duplicate_of=none/0.97 severity=major/0.74), critique: keep
+Jev shadow: agrees with critique on 2 of 2 decided candidate(s); keep 1, reject 1, unverifiable 1 (model=jev-1.13.0, requests=1, 0.4s)
+```
+
+The readable lines round the numbers; a third line, `Jev shadow data: {…}`,
+carries every decision with the probabilities exactly as Jev returned them,
+so thresholds can be chosen from the logs afterwards. The version that
+answered is logged too, and a pinned version answering as another one is a
+warning. A Jev failure (network, 429/529 after
+one retry, an invalid answer, no key) is a warning and never affects the
+review. Jev goes out through `LLM_HTTP_PROXY` like the LLM providers.
+
+What leaves for TypeSafe: the MR title and description, the Jira section,
+`review_instructions`, the diff shown to the models and the candidates, after
+the same secret scrubbing as the LLM prompts. Jev limits a request to 32k
+tokens for the state plus the longest question and 64k for the state plus all
+questions; the state is cut to what the questions leave (the diff first, then
+the MR and Jira sections, never the candidates and the hunks they point at),
+and when the candidates do not fit together, each goes in a request of its
+own. The size is estimated from characters; if Jev still rejects a request
+as too large (a 422 that explicitly says it exceeds a limit), its candidates
+are asked about one by one, and one that is too large alone stays
+unverifiable. Any other 422 is not retried.
+
 ## Usage
 
 ```bash
@@ -698,12 +869,35 @@ The template is written for shell-executor runners (the image runs through
 `docker run` with secrets passed by name) and carries `[skip review]`,
 `resource_group`, `allow_failure` and `REVIEW_MODE=once`. Models come from
 the image defaults, `.aireview.yml` is mounted into the container only when
-the project has one. The job runs in the `.post` stage — it exists in every
-pipeline, so a project does not declare `stages`; to move the review to
-another stage, add `aireview: {stage: review}` to the project file. Every
-variable the config reads (`Config.env_names`: models, providers, reserves,
-temperatures, limits, `OLLAMA_API_BASE` and so on) is passed into the
-container by name, so a project can override anything through its CI/CD
+the project has one. The job runs in the `.post` stage: it exists in every
+pipeline, so a project does not declare it.
+
+GitLab does not start a pipeline in which, after `rules` and `only` are
+applied, only `.pre` and `.post` jobs remain. That happens when a project has
+no other merge request jobs (a build that runs on tags only does not count).
+In that case add a `review` stage to the project's existing `stages`, without
+replacing them, and move the job there:
+
+```yaml
+stages:
+  - review   # added to the project's stages
+  - build
+
+aireview:
+  stage: review
+```
+
+To keep the review from waiting for other stages to finish, add `needs: []`
+to the `aireview` job.
+
+The template sets `REVIEW_MODE` as a job variable, and an environment variable
+beats `.aireview.yml`: `review_mode` in the project file has no effect once
+the template is included. Change the mode with a `REVIEW_MODE` CI/CD variable
+of the project or with `aireview: {variables: {REVIEW_MODE: update}}`.
+
+Every variable the config reads (`Config.env_names`: models, providers,
+reserves, temperatures, limits, `OLLAMA_API_BASE` and so on) is passed into
+the container by name, so a project can override anything through its CI/CD
 variables — for instance, swap an unavailable model with `LLM_CRITIQUE_MODEL`
 without waiting for an image release.
 

data/config/.aireview.yml.example

@@ -36,3 +36,9 @@ llm:
     model: gemini-3.8-flash
     fallbacks:
       - gemini-3.7-flash
+  # Jev as the critique engine (critique: {engine: jev}) or next to it, log only
+  # (see README, "Critique engine"); the key is JEV_API_KEY
+  # jev:
+  #   shadow: true
+  #   model: jev-1.13.0
+  #   fallback: model

data/lib/aireview/cli.rb

@@ -99,9 +99,11 @@ module Aireview
         critique_model: options[:critique_model],
         generate_temperature: options[:generate_temperature],
         critique_temperature: options[:critique_temperature],
-        no_fallbacks: options[:no_fallbacks] == true
+        critique_engine: options[:critique_engine],
+        no_fallbacks: options[:no_fallbacks] == true,
+        no_critique: options[:no_critique] == true
       )
-      config.require_llm_configuration!
+      config.require_llm_configuration!(critique: !options[:no_critique])
       config.warnings.each { |warning| @logger.warn(warning) }
       config
     end
@@ -212,18 +214,41 @@ module Aireview
       up_to_date = existing[:key] == key
       return false unless up_to_date || (mode == 'once' && !retried_ci_job?(gitlab_client))
 
-      reason = up_to_date ? 'existing review is up to date' : 'merge request already reviewed (review_mode=once)'
-      @out.puts("Review skipped: #{reason}")
+      @out.puts("Review skipped: #{skip_reason(existing, up_to_date: up_to_date, mode: mode)}")
       true
     end
 
+    # In once mode the review is not repeated on new pushes, even when the MR
+    # has changed. So besides the mode the message says whether the review is
+    # up to date: if it is not, the Retry button of the GitLab job updates it.
+    def skip_reason(existing, up_to_date:, mode:)
+      return 'existing review is up to date' unless mode == 'once'
+
+      state = if up_to_date then 'the review is up to date'
+              elsif existing[:key].nil? then "review freshness is unknown: #{update_hint}"
+              else "review inputs changed: #{update_hint}"
+              end
+      "merge request already reviewed (review_mode=once), #{state}"
+    end
+
+    def update_hint
+      ci_job_context ? 'retry the job to update' : 'use --force to review again'
+    end
+
     def retried_ci_job?(gitlab_client)
-      project_id, job_id = @env.values_at('CI_PROJECT_ID', 'CI_JOB_ID')
-      return false if Aireview::Utils.blank?(project_id) || Aireview::Utils.blank?(job_id)
+      project_id, job_id = ci_job_context
+      return false unless project_id
 
       gitlab_client.retried_job?(project_id, job_id)
     end
 
+    def ci_job_context
+      project_id, job_id = @env.values_at('CI_PROJECT_ID', 'CI_JOB_ID')
+      return if Aireview::Utils.blank?(project_id) || Aireview::Utils.blank?(job_id)
+
+      [project_id, job_id]
+    end
+
     def publish_review(review, context, publication)
       return if merge_request_moved?(context)
 
@@ -245,10 +270,13 @@ module Aireview
         context[:parser_result].project_id,
         context[:parser_result].iid
       )
-      return false if ReviewMarker.state(current) == ReviewMarker.state(context[:merge_request])
+      before = ReviewMarker.state(context[:merge_request])
+      after = ReviewMarker.state(current)
+      changed = after.keys.reject { |field| after[field] == before[field] }
+      return false if changed.empty?
 
-      @logger.warn("Merge request moved to #{current['sha']} (#{current['target_branch']}) " \
-                   'while review was running; skipping publication')
+      @logger.warn("Merge request changed while review was running (#{changed.join(', ')}); " \
+                   'skipping publication')
       true
     end
 
@@ -309,6 +337,11 @@ module Aireview
         options[:critique_temperature] = value
       end
 
+      parser.on('--critique-engine ENGINE', Aireview::Config::CRITIQUE_ENGINES,
+                'Who checks the candidates: model (an LLM) or jev') do |value|
+        options[:critique_engine] = value
+      end
+
       parser.on('--no-critique', 'Skip critique pass and render Generate candidates directly') do
         options[:no_critique] = true
       end
@@ -381,6 +414,8 @@ module Aireview
                            Override Generate pass temperature
           --critique-temperature VALUE
                            Override Critique pass temperature
+          --critique-engine ENGINE
+                           Who checks the candidates: model (an LLM, default) or jev
           --config PATH    Path to .aireview.yml
           --review-mode MODE
                            How to treat an existing review: update (default) or once

data/lib/aireview/config.rb

@@ -6,6 +6,7 @@ require_relative 'errors'
 require_relative 'utils'
 require_relative 'config_limits'
 require_relative 'config_fallbacks'
+require_relative 'config_jev'
 require_relative 'config_layers'
 require_relative 'config_loader'
 
@@ -16,6 +17,7 @@ module Aireview
   class Config
     include ConfigLimits
     include ConfigFallbacks
+    include ConfigJev
     include ConfigLayers
 
     DEFAULT_SECRET_FILES = [
@@ -76,20 +78,27 @@ module Aireview
 
     # CLI overrides change only the primary model of a stage, the reserves
     # from the config stay; no_fallbacks leaves one model and one key.
-    def with_overrides(
+    # no_critique — --no-critique: the run has no Critique, so its settings
+    # are neither routed nor validated (see llm_stages).
+    # Seven named flags of the CLI read better than a struct for its own sake.
+    def with_overrides( # rubocop:disable Metrics/ParameterLists
       generate_model: nil,
       critique_model: nil,
       generate_temperature: nil,
       critique_temperature: nil,
-      no_fallbacks: false
+      critique_engine: nil,
+      no_fallbacks: false,
+      no_critique: false
     )
       llm_config = {
         'generate' => stage_overrides(model: generate_model, temperature: generate_temperature),
         'critique' => stage_overrides(model: critique_model, temperature: critique_temperature)
+          .merge({'engine' => critique_engine}.compact)
       }.reject { |_, overrides| overrides.empty? }
       overrides = {}
       overrides['llm'] = llm_config unless llm_config.empty?
       overrides['fallbacks_disabled'] = true if no_fallbacks
+      overrides['critique_disabled'] = true if no_critique
       return self if overrides.empty?
 
       self.class.new(
@@ -137,8 +146,9 @@ module Aireview
       routing.primary('generate').model
     end
 
+    # nil when no LLM Critique can run (see llm_stages).
     def critique_model
-      routing.primary('critique').model
+      routing.stage?('critique') ? routing.primary('critique').model : nil
     end
 
     def generate_provider
@@ -147,18 +157,26 @@ module Aireview
 
     # Everything besides the prompt that affects the review result goes into
     # the note key (see ReviewMarker): provider, model and temperature of the
-    # stages, the shared pool with its critique policy. Reserves of per-stage
-    # chains do not change the result.
+    # stages, Jev as the critique engine, the shared pool with its critique
+    # policy. Reserves of per-stage chains do not change the result.
     def result_signature
       {
-        'generate' => [generate_provider, generate_model, generate_temperature],
-        'critique' => [critique_provider, critique_model, critique_temperature],
+        'generate' => stage_signature('generate', generate_temperature),
+        'critique' => (routing.stage?('critique') ? stage_signature('critique', critique_temperature) : nil),
+        'critique_engine' => jev_signature,
         'pool' => routing.signature
       }
     end
 
     def critique_provider
-      routing.primary('critique').provider
+      routing.stage?('critique') ? routing.primary('critique').provider : nil
+    end
+
+    # The primary of a stage on a server of your own goes into the review key
+    # with its address; without one the signature is what it was.
+    def stage_signature(stage, temperature)
+      primary = routing.primary(stage)
+      [primary.provider, primary.model, temperature, *primary.api_base]
     end
 
     def generate_temperature

data/lib/aireview/config_fallbacks.rb

@@ -27,6 +27,19 @@ module Aireview
       routing.chain(stage)
     end
 
+    # The keys a model is called with. A server of your own (a model with
+    # api_base) never gets the provider's key — OPENAI_API_KEY must not leave
+    # for a self-hosted server — but LLM_API_KEY, or a placeholder for a
+    # server without auth, since the client wants some key.
+    SELF_HOSTED_PLACEHOLDER_KEY = 'no-key'
+
+    def candidate_api_keys(candidate)
+      return provider_api_keys(candidate.provider) unless candidate.api_base
+      return [nil] if KEYLESS_PROVIDERS.include?(candidate.provider.to_s)
+
+      [Aireview::Utils.presence(llm_api_key) || SELF_HOSTED_PLACEHOLDER_KEY]
+    end
+
     # Keys in order of preference; a single nil for a keyless provider, so
     # that walking the chain does not depend on the provider.
     def provider_api_keys(provider)
@@ -47,9 +60,9 @@ module Aireview
     end
 
     # Only the number of keys per provider, for --dry-run; the values never
-    # leave.
+    # leave. Servers of your own are not counted: they take LLM_API_KEY.
     def api_key_counts(stages)
-      providers = stages.flat_map { |stage| stage_chain(stage).map(&:provider) }.uniq
+      providers = stages.flat_map { |stage| stage_chain(stage).reject(&:api_base).map(&:provider) }.uniq
       providers.reject { |provider| KEYLESS_PROVIDERS.include?(provider.to_s) }
         .to_h { |provider| [provider, provider_api_keys(provider).size] }
     end
@@ -67,39 +80,53 @@ module Aireview
                         'llm.overloaded_quarantine')
     end
 
-    def require_models!
+    # critique — the run has a critique (false with --no-critique); only the
+    # stages that need an LLM in such a run are required (see llm_stages).
+    def require_models!(critique: true)
+      stages = llm_stages(critique: critique)
       missing = []
       missing << 'llm.generate.model (or LLM_GENERATE_MODEL)' if Aireview::Utils.blank?(generate_model)
-      missing << 'llm.critique.model (or LLM_CRITIQUE_MODEL)' if Aireview::Utils.blank?(critique_model)
+      if stages.include?('critique') && Aireview::Utils.blank?(critique_model)
+        missing << 'llm.critique.model (or LLM_CRITIQUE_MODEL)'
+      end
       raise ConfigError, "LLM models are required: #{missing.join(', ')}" unless missing.empty?
     end
 
     # The plan is built whole (pool and critique policy validated) before the
-    # first request, not in Critique after a paid-for Generate.
-    def require_llm_configuration!
-      require_models!
+    # first request, not in Critique after a paid-for Generate. Keys are
+    # required for the stages that need an LLM: generate on Ollama with Jev
+    # as the critic and no fallback needs no Gemini key.
+    def require_llm_configuration!(critique: true)
+      require_models!(critique: critique)
+      require_jev!(critique: critique)
       routing
 
-      missing_keys = STAGES.flat_map do |stage|
-        providers = stage_chain(stage).map(&:provider).uniq.reject { |provider| provider_keys_present?(provider) }
-        providers.map { |provider| "#{stage}: API key is required for provider #{provider.inspect}" }
+      missing_keys = llm_stages(critique: critique).flat_map do |stage|
+        providers = stage_chain(stage).reject(&:api_base).map(&:provider).uniq
+        providers.reject { |provider| provider_keys_present?(provider) }
+          .map { |provider| "#{stage}: API key is required for provider #{provider.inspect}" }
       end
       raise ConfigError, missing_keys.join(', ') unless missing_keys.empty?
     end
 
     private
 
+    # Only the stages that go to an LLM are routed (llm_stages): without an
+    # LLM Critique (Jev with fallback: fail, --no-critique) its model, limit,
+    # start, rank and allow_weaker are not used, so they must neither fail
+    # the run nor change the review key, nor push the pool out of it.
     def build_routing
-      settings = STAGES.to_h { |stage| [stage, stage_settings(stage)] }
+      stages = llm_stages
+      settings = stages.to_h { |stage| [stage, stage_settings(stage)] }
       models = Array(dig('llm', 'models'))
       return StageChains.build(settings, only_primary: fallbacks_disabled?) if models.empty?
 
       own = settings.select { |_, stage_settings| Aireview::Utils.present?(stage_settings[:model]) }
       ModelPool.new(
-        items: models, provider: llm_provider,
-        limits: STAGES.to_h { |stage| [stage, max_prompt_chars(stage)] },
-        starts: STAGES.to_h { |stage| [stage, dig('llm', stage, 'start')] },
-        inherited_starts: STAGES.select { |stage| start_inherited?(stage) },
+        items: models, provider: llm_provider, stages: stages,
+        limits: stages.to_h { |stage| [stage, max_prompt_chars(stage)] },
+        starts: stages.to_h { |stage| [stage, dig('llm', stage, 'start')] },
+        inherited_starts: stages.select { |stage| start_inherited?(stage) },
         rank: dig('llm', 'critique', 'rank'), allow_weaker: dig('llm', 'critique', 'allow_weaker'),
         own_chains: StageChains.build(own, only_primary: fallbacks_disabled?), only_primary: fallbacks_disabled?
       )
@@ -109,6 +136,7 @@ module Aireview
       {
         provider: stage_setting(stage, 'provider') || llm_provider,
         model: dig('llm', stage, 'model'),
+        api_base: dig('llm', stage, 'api_base'),
         fallbacks: dig('llm', stage, 'fallbacks'),
         max_prompt_chars: max_prompt_chars(stage)
       }