aireview 0.3.0 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 394d2acbc7ed98d03e3a1f87e61fd9f07e1985b40ad47e4086fce47e5a18f5be
4
- data.tar.gz: 85598cb2c2e81f95ff59e40bb5d2cef47d4813caf49f0ef318cf5931651adbf7
3
+ metadata.gz: 3f326ae1b11f59f7c447e8f128ab94845ef1543b0b377753f053fd4f69e03bad
4
+ data.tar.gz: c5908173ca8020a0563b6d4ee1fd4d8ce4ebe609fda7ded26d5b7eb40c84dda1
5
5
  SHA512:
6
- metadata.gz: c45f78bc422def8e2392499ba1dc0e7f87e67183bad8b9e9a29b3b242512caf0a006511922ad32f6a237d6e971578861a86ebb5107790e86e4c3e34598d9c6e6
7
- data.tar.gz: fb33990e5743821e8bef7900241ff6b5b7d7f575d3a22ac3eb645990c7050e792d46927c71d401ab659e200366e23acacb9d4f62eef6c74e70e78e7a76bc9111
6
+ metadata.gz: 2de9b7dd7e6caaa440287339d03eed843dfdb73010ae8cf6a01dea4e9348e3b072fc4fb61a55bf30eb4ed324e857ff883e35e31b711bc542cb8d1495645041bc
7
+ data.tar.gz: 5f0867f71415c019c03d99bd2215fc7f7ef7cadf98ab8b0db91e6eb98af54cfae3787b848c7611fc371cfa19cc94acd55e170047f094d8543f5c77f258e3313d
data/CHANGELOG.md CHANGED
@@ -1,5 +1,51 @@
1
1
  # Changelog
2
2
 
3
+ ## 2.0.0
4
+
5
+ Breaking: the review key of a shared pool includes the pool order and the
6
+ critique policy; a single chain without reserves gets three requests instead
7
+ of a seventeen-minute schedule; `LLM_PROVIDER` no longer re-targets models
8
+ that carry an explicit provider.
9
+
10
+ - Defaults shipped in the image: `config/defaults.yml` is a configuration
11
+ layer between the built-in values and the project's `.aireview.yml`,
12
+ enabled by `AIREVIEW_DEFAULTS` (set in the Dockerfile). A project needs no
13
+ model names; `--dry-run` prints the layer every setting came from and
14
+ warns when inherited reserves silently change provider.
15
+ - Shared model pool: `llm.models` (`LLM_MODELS`) in order of priority,
16
+ `generate.start`, `critique.rank: not_below_generate|any` and
17
+ `critique.allow_weaker`. Generate walks the pool round from its start;
18
+ Critique takes the first live model not below the one that answered in
19
+ Generate, never weaker unless allowed. A stage with a `model` of its own
20
+ keeps an independent chain.
21
+ - Router: an overloaded model or a timeout gets one short retry, then a
22
+ quarantine (`llm.overloaded_quarantine`, `LLM_OVERLOADED_QUARANTINE`,
23
+ 120 s) and the next model; the walk goes round the pool once every model
24
+ has been tried, waiting for the nearest quarantine to end. Three requests
25
+ per model per stage, counted across keys and including the JSON repair;
26
+ the last model has no special schedule. A model the provider does not
27
+ have (retired, a typo, not pulled into Ollama) is excluded for the run
28
+ instead of failing it; a bare 404 stays fatal.
29
+ - An invalid result that the same model cannot repair excludes that model
30
+ for the stage and restarts the stage on the next model with the original
31
+ request; Critique keeps the candidates already obtained.
32
+ - `aireview models check`: two probe requests per model with the production
33
+ schemas and validation, statuses `ok / missing / invalid / unverified /
34
+ failed / skipped`, `--strict` turns a skipped (unreachable) Ollama into a
35
+ failure. Meant for release pipelines and schedules.
36
+ - `templates/review.gitlab-ci.yml`: a job template projects include with
37
+ `ref: stable`; every config variable is passed into the container by name.
38
+ - The log names the model that answered each stage and its place in the
39
+ chain; the report notes a critique that ran on a weaker model.
40
+ - Fixed: with `json` 3.0 every provider answer failed to parse
41
+ (`Faraday::ParsingError`, "wrong number of arguments") — `faraday` 2.14.4
42
+ in the lockfile. The suite did not catch it because it stubs the network;
43
+ `aireview models check` did.
44
+ - Internals: routing plans (`StageChains`, `ModelPool`) and the router's
45
+ `ModelState` are objects, `ConfigLoader` owns loading and env parsing,
46
+ `ResultParser` parses LLM results, `LlmClient` makes one request while
47
+ the router retries. Specs run against the real RubyLLM classes.
48
+
3
49
  ## 0.3.0
4
50
 
5
51
  - Fallback models per stage (`llm.<stage>.fallbacks`,
data/README.md CHANGED
@@ -49,8 +49,10 @@ installed, replace it with plain `aireview`.
49
49
 
50
50
  ## Configuration
51
51
 
52
- Secrets live in environment variables or in a local `.env` file. In `.env` the
53
- Generate and Critique models are set explicitly:
52
+ Secrets live in environment variables or in a local `.env` file. The models of
53
+ both stages come from the defaults shipped in the Docker image (see "Defaults
54
+ shipped in the image"); outside the image they are set in `.env` or
55
+ `.aireview.yml`:
54
56
 
55
57
  ```bash
56
58
  GITLAB_URL=https://gitlab.company.com
@@ -91,6 +93,88 @@ If only the LLM traffic has to go through a proxy, set `LLM_HTTP_PROXY` or
91
93
  `llm.http_proxy`. That configures RubyLLM only; requests to GitLab and Jira
92
94
  keep going directly.
93
95
 
96
+ ### Defaults shipped in the image
97
+
98
+ Models, timeouts and the report language are baked into the image as
99
+ `config/defaults.yml`; inside the image `AIREVIEW_DEFAULTS` points at it. A
100
+ project in an organization has nothing to configure: no `.aireview.yml`, no
101
+ variables with model names. Changing the model for every project is one
102
+ edit of that file and a new image release. Outside the image the variable is
103
+ unset and the layer does not exist, so local runs are unaffected; point
104
+ `AIREVIEW_DEFAULTS` at any file to get the same layer elsewhere. The
105
+ defaults define a shared pool of five Gemini models (see "Shared model
106
+ pool"): Generate starts from a mid-range model and goes round the pool,
107
+ Critique takes the strongest live model not below the one Generate answered
108
+ with; every model of the pool is confirmed by `aireview models check` on
109
+ release.
110
+
111
+ The configuration layers, weakest first:
112
+
113
+ 1. built-in values (`Config::DEFAULTS`);
114
+ 2. image defaults — the file from `AIREVIEW_DEFAULTS`; no variable, no layer;
115
+ 3. the project's `.aireview.yml`;
116
+ 4. environment variables (`LLM_GENERATE_MODEL` and the rest);
117
+ 5. CLI flags (`--generate-model`, `--critique-model`, the temperatures).
118
+
119
+ Layers merge by key, so a project overrides only what it needs — with two
120
+ caveats about the pool:
121
+
122
+ - a `llm.generate.model` of its own (in YAML or `LLM_GENERATE_MODEL`) **takes
123
+ the stage out of the pool**: its chain is that model plus its own
124
+ `fallbacks`, if any; the image models are not picked up for that stage and
125
+ the critique rank rule does not apply. To change only the starting model
126
+ while staying in the pool, set `llm.generate.start` / `LLM_GENERATE_START`
127
+ — or replace the whole pool through `llm.models` / `LLM_MODELS`;
128
+ - every pool model in the image carries an explicit provider (`gemini`), so
129
+ `LLM_PROVIDER=ollama` on its own does not move the Gemini pool to Ollama:
130
+ it only changes the default provider of models without one. A project on
131
+ Ollama sets its own models — its own pool (`LLM_MODELS=ollama/qwen2.5-coder:7b,…`)
132
+ or per-stage `model`s.
133
+
134
+ The shared settings `llm.temperature`, `llm.max_prompt_chars` and the default
135
+ provider `llm.provider` (env: `LLM_TEMPERATURE`, `LLM_MAX_PROMPT_CHARS`,
136
+ `LLM_PROVIDER`) apply to the stages layer by layer: a stage value from the
137
+ image defaults does not beat a shared value from the project or the
138
+ environment, while within one layer the stage value still wins. For stages
139
+ on their own chains the `fallbacks` array is replaced as a whole when given
140
+ explicitly; `fallbacks: []` removes the reserves; a reserve without a
141
+ `provider` of its own inherits the stage provider — if a project overrode the
142
+ stage provider (`LLM_GENERATE_PROVIDER=ollama`) while inherited reserves
143
+ without an explicit provider remain in the chain, they silently become
144
+ "models" of the new provider. `--dry-run` warns about that and prints, for
145
+ every model, the layer it came from:
146
+
147
+ ```
148
+ === LLM SETTINGS ===
149
+ Config: image defaults /app/config/defaults.yml, .aireview.yml /app/.aireview.yml
150
+ Generate: gemini-3.7-flash temperature=0.1 (model from image defaults, provider from image defaults)
151
+ fallbacks: gemini/gemini-3.6-flash -> ... (fallbacks from image defaults)
152
+ Critique: qwen2.5-coder:7b temperature=0 (model from .aireview.yml, provider from .aireview.yml)
153
+ Time budget: 1800s, overloaded quarantine: 120s
154
+ ```
155
+
156
+ In pool mode the model source is the layer that set the stage `start` or the
157
+ `llm.models` list itself; `--critique-model` with a pool model shows up as
158
+ `model from cli`.
159
+
160
+ An `AIREVIEW_DEFAULTS` that points at a missing file is a configuration
161
+ error: such an image was built wrong, and failing at once beats reviewing
162
+ with the wrong models. Keys and tokens never live in `config/defaults.yml`.
163
+
164
+ Changing the default models or the pool order changes the review key (see "A
165
+ single comment per merge request"). What happens to open MRs depends on
166
+ `review_mode`: in `update` the review re-runs on the next push to the MR, in
167
+ `once` (as in the CI template) only on a job Retry. In `update` this is a
168
+ one-off burst of requests to the provider — better not to ship such a
169
+ release at peak hours.
170
+
171
+ No model is probed before a review: there are no test requests to five
172
+ models per MR, availability is learned from the working requests — the first
173
+ Generate request is the probe, and what the router learns it remembers until
174
+ the end of the run (see "Fallback models and keys"). The separate smoke test
175
+ of every model with both schemas (`aireview models check`) runs on an image
176
+ release and on a schedule, not on MRs.
177
+
94
178
  ### Local Ollama
95
179
 
96
180
  Install Ollama following the
@@ -169,15 +253,16 @@ address with `/v1` matches the
169
253
  local model it can be raised, but not without limit: a hung request holds
170
254
  the job for exactly that long while a fallback model sits idle. An
171
255
  "overloaded" (503) answer from the provider or a timeout does not fail the
172
- run right away: without a fallback model such a request gets up to five
173
- attempts, the original one and four retries with pauses of about 2, 5, 5 and
174
- 5 minutes; with a fallback model, one short retry and a switch (see
175
- "Fallback models and keys"). Only the failed stage is repeated, not the
176
- whole run.
177
-
178
- Project rules live in `.aireview.yml`. In YAML the `generate.model` and
179
- `critique.model` settings are required for each stage and are not inherited
180
- from the base `llm` settings. `llm.provider` is used as the default when
256
+ run right away: the model gets one short retry (~30 s), then goes into
257
+ quarantine for two minutes while the request goes to the next model;
258
+ without a fallback model the router waits for the quarantine to end and
259
+ tries once more. Three requests per model per stage in total (see "Fallback
260
+ models and keys"). Only the failed stage is repeated, not the whole run.
261
+
262
+ Project rules live in `.aireview.yml`. The `generate.model` and
263
+ `critique.model` settings are not inherited from the base `llm` settings: a
264
+ review does not start when a stage has no model in the image defaults, the
265
+ YAML or the environment. `llm.provider` is used as the default when
181
266
  `generate.provider` or `critique.provider` is not set:
182
267
 
183
268
  ```yaml
@@ -300,31 +385,119 @@ fallback model and a provider can have a fallback key:
300
385
  forbid circumventing its limits regardless of the project's billing. Keys
301
386
  live only in the environment, never in `.aireview.yml`.
302
387
 
388
+ #### Shared model pool
389
+
390
+ Instead of two independent chains a single pool can be configured:
391
+ `llm.models` in order of priority (the first is the preferred one for
392
+ Critique). Generate goes through the pool from `generate.start` downwards
393
+ and round again; Critique takes the first live model **not below the one
394
+ that actually answered in Generate** (`critique.rank: not_below_generate`,
395
+ the default), with self-critique by the same model as the last permitted
396
+ option. "Above" and "below" are positions in the list, they are not derived
397
+ from model names:
398
+
399
+ ```yaml
400
+ llm:
401
+ provider: gemini
402
+ models: # order = priority for Critique
403
+ - gemini-3.8-flash
404
+ - gemini-3.7-flash
405
+ - gemini-3.6-flash
406
+ - provider: ollama
407
+ model: qwen2.5-coder:7b
408
+ max_prompt_chars: 20000
409
+ generate:
410
+ start: gemini-3.7-flash # Generate starts here, then downwards and round
411
+ critique:
412
+ rank: not_below_generate # or any — the chains are independent
413
+ allow_weaker: false # true — go below when nothing above is alive
414
+ ```
415
+
416
+ Environment equivalents: `LLM_MODELS=gemini/gemini-3.8-flash,gemini/gemini-3.7-flash,…`,
417
+ `LLM_GENERATE_START`, `LLM_CRITIQUE_START`, `LLM_CRITIQUE_RANK`,
418
+ `LLM_CRITIQUE_ALLOW_WEAKER`. The rules:
419
+
420
+ - no permitted live model for Critique and `allow_weaker: false` — the run
421
+ fails, candidates are not published without a critique; with
422
+ `allow_weaker: true` Critique goes below Generate and the report gets a
423
+ "Critique ran on a model weaker than Generate" line;
424
+ - a stage with a `model` of its own (in YAML, the environment or
425
+ `--generate-model` with a model outside the pool) does not use the pool:
426
+ its chain is independent and the rank rule does not apply, as with
427
+ `rank: any`; `--generate-model` / `--critique-model` with a model **from**
428
+ the pool bring the stage back into the pool and make that model the start
429
+ (the stage's own `model` and `fallbacks` are reset), with a model outside
430
+ the pool — a single chain without reserves;
431
+ - a project that replaces the whole pool (`llm.models` in YAML or
432
+ `LLM_MODELS`) need not repeat `start`: a start inherited from the image
433
+ that the new pool does not contain is replaced by the first model of the
434
+ new pool with a warning in the log. A start set in the same layer as the
435
+ pool or above it must be in the pool — a typo there stays a configuration
436
+ error;
437
+ - `critique.start` goes first only when the rank permits it; a start below
438
+ the model that answered in Generate is skipped with a warning when
439
+ `allow_weaker` is off — it does not bypass the ban on a weaker critique;
440
+ - the review key (see "A single comment per merge request") includes the
441
+ whole pool together with the critique policy: its order decides which
442
+ model checks the findings, not just what to fall back to. Reordering the
443
+ pool, changing `start`, `rank` or `allow_weaker` — a new key, open MRs are
444
+ re-reviewed on the next push; moving from per-stage chains to the pool —
445
+ the same, once;
446
+ - `--dry-run` prints the chains of both stages and a `Critique rule: …` line.
447
+
303
448
  What happens on which error:
304
449
 
305
450
  | Error | Reaction |
306
451
  |---|---|
307
- | Daily quota (`quotaId` like `…PerDay…` in Google's answer) | No retries: the next key on the same model. The "key + model" pair is remembered until the end of the run so that Critique and the JSON repair do not hit it again. Out of keys: the next model with the first key. |
308
- | Per-minute limit (429 with a "retry in N s" hint) | Retry after the hinted delay; one retry while there is somewhere to switch to, up to three on the last route. Then the next key, then the next model. |
309
- | Overloaded (503) or timeout | One short retry (~30 s) and the next model with the same key. The last model in the chain gets the full 2/5/5/5 minute schedule. |
310
- | Other API errors (schema, context length, auth) | The run fails right away: a fallback model would answer the same. |
311
-
312
- All of this is bounded by a time budget for the LLM part of the run:
313
- `llm.time_budget` / `LLM_TIME_BUDGET`, 1800 seconds by default. A pause that
314
- does not fit into the remainder is skipped and the request timeout is capped
315
- by the remainder; when the time is up, the run fails with an error listing
316
- everything that was tried. Because of this keep `LLM_TIMEOUT` around 120–300
317
- seconds: one hung request must not eat the whole budget.
452
+ | Daily quota (`quotaId` like `…PerDay…` in Google's answer) | No retries: the next key on the same model. The "key + model" pair is remembered until the end of the run so that Critique and the JSON repair do not hit it again. Out of keys: the model is excluded until the end of the run, the request goes to the next one. |
453
+ | Per-minute limit (429 with a "retry in N s" hint) | One retry after the hinted delay, then the next key. Out of keys: the model is quarantined for the hinted time, the request goes to the next one. |
454
+ | Overloaded (503) or timeout | One short retry (~30 s), then a quarantine of `llm.overloaded_quarantine` / `LLM_OVERLOADED_QUARANTINE` (120 s by default) and the next model with the same key. |
455
+ | The provider has no such model (the answer text names the model: retired, a typo, not pulled into Ollama) | No retries: the model is excluded until the end of the run, the request goes to the next one. A bare 404 without such text is fatal: a wrong `LLM_API_BASE` answers the same, and the next model will not help. |
456
+ | An invalid result (not JSON, not the schema, foreign ids in the verdicts) — and still invalid after one repair by the same model | The model is excluded for this stage, the stage starts over on the next model with the original request. For Critique the candidates already obtained are kept, Generate is not repeated. |
457
+ | Other API errors (context length, auth) | The run fails right away: a fallback model would answer the same. |
458
+
459
+ The chain is walked round: once every model has been tried, the router
460
+ returns to those whose quarantine has expired; when all are quarantined, it
461
+ waits for the nearest release. The last model in the list is nothing
462
+ special, nobody gets a long retry schedule. Two limits keep the walk
463
+ finite:
464
+
465
+ - **three requests per model per stage** (`MAX_ATTEMPTS_PER_MODEL`). Every
466
+ request sent counts regardless of its outcome, including the short retry
467
+ and the JSON repair; the keys of one model share one counter, a quarantine
468
+ does not reset it. Out of requests — the model leaves the stage; a JSON
469
+ repair without requests left is not sent, and the result counts as invalid
470
+ (see the table). When every model of the stage is out or excluded, the
471
+ stage ends at once, quarantines are not waited for;
472
+ - **the time budget** of the LLM part of the run: `llm.time_budget` /
473
+ `LLM_TIME_BUDGET`, 1800 seconds by default. A pause or a quarantine wait
474
+ that does not fit into the remainder is skipped, the request timeout is
475
+ capped by the remainder; when the time is up, the run fails with an error
476
+ listing everything that was tried. Because of this keep `LLM_TIMEOUT`
477
+ around 120–300 seconds: one hung request must not eat the whole budget.
478
+
479
+ **A single chain** — one model without reserves, typically Ollama on its own
480
+ box — behaves differently from earlier versions: instead of seventeen minutes
481
+ of waiting (retries after 2, 5, 5 and 5 minutes) the model gets three
482
+ requests — the original, a short retry after ~30 s and one more after the
483
+ two-minute quarantine; after the third failure the stage fails. A timeout is
484
+ a failure too, and a quarantine will not help a slow model: it only
485
+ lengthens the pause between attempts. A slow local model needs an
486
+ `LLM_TIMEOUT` with room for the largest request and an `LLM_TIME_BUDGET`
487
+ that fits three such requests with their pauses; `LLM_OVERLOADED_QUARANTINE`
488
+ is about a model that is temporarily overloaded and should come back.
318
489
 
319
490
  The review key (see "A single comment per merge request") is computed from
320
491
  the configured primary model, not from the one that answered: a review made
321
492
  by a fallback model is not rewritten on the next push without changes in the
322
- MR, and adding a fallback model to the config does not re-run the review on
323
- every open MR. When a stage went to a fallback model, the report ends with a
324
- `Fallback model used: critique — …` line. A key switch stays in the log only,
325
- and key values never reach the log. `--dry-run` prints the model chains and
326
- the number of keys per provider, `--no-fallbacks` leaves one model and one
327
- key per stage.
493
+ MR, and adding a fallback model to a per-stage chain does not re-run the
494
+ review on every open MR. When a stage went to a fallback model, the report
495
+ ends with a `Fallback model used: critique — …` line; the log names, for
496
+ every stage, the model that answered and its place in the chain
497
+ (`model=gemini/gemini-3.6-flash (2/5)`). A key switch stays in the log only,
498
+ and key values never reach the log. `--dry-run` prints the model chains, the
499
+ number of keys per provider, the time budget and the quarantine length;
500
+ `--no-fallbacks` leaves one model and one key per stage.
328
501
 
329
502
  ### Checking that findings point at the diff
330
503
 
@@ -373,6 +546,42 @@ bundle _2.3.26_ exec bin/aireview review https://gitlab.company.com/team/project
373
546
  - `--review-mode MODE` sets the behaviour when a review has already been published: `update` or `once`.
374
547
  - `--force` reviews again even when a review for this state of the MR is already published.
375
548
 
549
+ ### Checking the models
550
+
551
+ ```bash
552
+ bundle _2.3.26_ exec bin/aireview models check
553
+ bundle _2.3.26_ exec bin/aireview models check --config .aireview.yml --strict --verbose
554
+ ```
555
+
556
+ Every model of both stage chains gets two small requests — one with the
557
+ production Generate schema and one with the Critique schema — on a tiny
558
+ synthetic MR, with the same system prompts as a review. The answer goes
559
+ through the same validation as in a run (JSON shape, candidate and verdict
560
+ ids). The provider's catalog is not consulted: "the model is listed" does
561
+ not mean "our request with the schema passes on it". There are no reserves,
562
+ retries or quarantine here — this is a check, not a review; only a
563
+ per-minute limit gets one retry after the provider's hint.
564
+
565
+ ```
566
+ Checking 5 model(s) with the generate and critique schemas
567
+ gemini/gemini-3.7-flash generate ok (4.6s)
568
+ gemini/gemini-3.7-flash critique ok (6.9s)
569
+ gemini/gemini-3.6-flash generate unverified: This model is currently experiencing high demand. …
570
+ gemini/gemini-9.9-nope generate missing: models/gemini-9.9-nope is not found for API version v1beta, …
571
+ ollama/qwen2.5-coder:7b generate skipped: Connection refused
572
+ Result: 6 ok, 1 unverified, 2 missing, 2 skipped -> FAILED
573
+ ```
574
+
575
+ Statuses: `ok` — an answer matching the schema; `missing` — the provider
576
+ has no such model (retired, a typo, not pulled into Ollama); `invalid` — it
577
+ answered, but not by the schema; `unverified` — the provider could not
578
+ answer right now (overload, quota, timeout), the model is not at fault but
579
+ not confirmed either; `failed` — other API errors; `skipped` — Ollama is
580
+ unreachable where the check runs (a CI runner has none). The exit code is 0
581
+ only when every model is `ok` or `skipped`; with `--strict` — only `ok`: on
582
+ a box where Ollama must be running, an unreachable Ollama is a failed check,
583
+ not a skip.
584
+
376
585
  ### A single comment per merge request
377
586
 
378
587
  With `--post` the review is not published as a new note every time; it updates
@@ -473,6 +682,46 @@ local HTTP proxy (`wireproxy`, for instance) before `aireview` starts and point
473
682
  GitLab and Jira stay on direct connections while RubyLLM goes through the
474
683
  tunnel.
475
684
 
685
+ ### Including the job template
686
+
687
+ Instead of copying the job into every project, it lives in this repository
688
+ as `templates/review.gitlab-ci.yml`. A project's `.gitlab-ci.yml` keeps only:
689
+
690
+ ```yaml
691
+ include:
692
+ - project: your-group/aireview
693
+ ref: stable
694
+ file: /templates/review.gitlab-ci.yml
695
+ ```
696
+
697
+ The template is written for shell-executor runners (the image runs through
698
+ `docker run` with secrets passed by name) and carries `[skip review]`,
699
+ `resource_group`, `allow_failure` and `REVIEW_MODE=once`. Models come from
700
+ the image defaults, `.aireview.yml` is mounted into the container only when
701
+ the project has one. The job runs in the `.post` stage — it exists in every
702
+ pipeline, so a project does not declare `stages`; to move the review to
703
+ another stage, add `aireview: {stage: review}` to the project file. Every
704
+ variable the config reads (`Config.env_names`: models, providers, reserves,
705
+ temperatures, limits, `OLLAMA_API_BASE` and so on) is passed into the
706
+ container by name, so a project can override anything through its CI/CD
707
+ variables — for instance, swap an unavailable model with `LLM_CRITIQUE_MODEL`
708
+ without waiting for an image release.
709
+
710
+ Secrets stay CI/CD variables **of the project**: group variables are readable
711
+ by any merge request of any project in the group.
712
+
713
+ The `stable` branch is managed: the copy of the template there pins
714
+ `AIREVIEW_IMAGE_TAG` to a verified release. Updating or rolling back every
715
+ project that includes the template is one edit of `stable`. A project can
716
+ pin another version by overriding `AIREVIEW_IMAGE_TAG` in its `variables`.
717
+
718
+ A release of the image is: tag → `aireview models check` on the image
719
+ defaults (an API error or an answer off the schema for any model stops the
720
+ pipeline, the image is not built, the previous version stays in the
721
+ registry) → build and push → set the new tag in `stable`. The same check on
722
+ a schedule (once a day) is the early signal that the provider retired a
723
+ model — otherwise the first to learn about it is a live MR.
724
+
476
725
  ## Docker
477
726
 
478
727
  ```bash
@@ -0,0 +1,47 @@
1
+ # Defaults baked into the aireview image. This layer sits between the
2
+ # built-in values and the project's .aireview.yml: a project overrides
3
+ # anything here, the environment and CLI flags override the project. No
4
+ # keys live here.
5
+ #
6
+ # One shared pool of models (see README, "Shared model pool"): the order is
7
+ # the priority for Critique, the first model is the preferred one. Generate
8
+ # starts at generate.start and goes down the list and round again; Critique
9
+ # takes the first live model not below the one that answered in Generate
10
+ # and never goes lower.
11
+ #
12
+ # Every model carries an explicit provider: a model without one inherits
13
+ # llm.provider, and a project that switched it to ollama would otherwise get
14
+ # "Ollama models" with Gemini names.
15
+ #
16
+ # Model names come from the Gemini catalog (GET /v1beta/models), text models
17
+ # without "preview" in the name only; each one is confirmed by
18
+ # `aireview models check` with both schemas (2026-09-18).
19
+
20
+ review_language: en
21
+
22
+ llm:
23
+ provider: gemini
24
+ temperature: 0
25
+ # Three attempts per model × timeout is the ceiling for one hung model:
26
+ # 300 s meant 15 minutes, 180 s means 9.
27
+ timeout: 180
28
+ time_budget: 1800
29
+ overloaded_quarantine: 120
30
+ models:
31
+ - provider: gemini
32
+ model: gemini-3.8-flash
33
+ - provider: gemini
34
+ model: gemini-3.7-flash
35
+ - provider: gemini
36
+ model: gemini-3.6-flash
37
+ - provider: gemini
38
+ model: gemini-3.5-flash
39
+ - provider: gemini
40
+ model: gemini-3.5-flash-lite
41
+ generate:
42
+ start: gemini-3.7-flash
43
+ temperature: 0.1
44
+ critique:
45
+ rank: not_below_generate
46
+ allow_weaker: false
47
+ temperature: 0
@@ -3,11 +3,12 @@ require 'logger'
3
3
  require 'set'
4
4
 
5
5
  module Aireview
6
- # Механическая проверка привязки кандидата к коду по тому диффу, который
7
- # реально ушёл в модель: файл, строка, цитата. Проверяется привязка, а не
8
- # сам баг: ненайденная цитата повод критику присмотреться, не
9
- # доказательство выдумки. Файл, которого нет среди изменений MR, другое
10
- # дело: такого кандидата проверять нечем, он отбрасывается.
6
+ # A mechanical check that a candidate points at the code, against the diff
7
+ # the model actually saw: file, line, quote. The anchoring is checked, not
8
+ # the bug itself: a quote that is not found is a reason for Critique to
9
+ # look closer, not proof of a fabrication. A file that is not among the MR
10
+ # changes is a different matter: there is nothing to check such a
11
+ # candidate against, it is dropped.
11
12
  class CandidateChecker
12
13
  HUNK_HEADER = /^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@/
13
14
  FILE_HEADER = %r{^diff --git a/(.+?) b/(.+)$}
@@ -15,12 +16,12 @@ module Aireview
15
16
  NOTE_LINE_RESET = 'line was outside the shown hunks and has been reset to null'
16
17
  NOTE_NOT_VERIFIED = 'file shown partially or without a diff, location not verified'
17
18
 
18
- # Один файл из контекста: диапазоны строк нового файла по показанным
19
- # хункам и нормализованный текст каждого хунка для поиска цитаты
20
- # отдельно новая сторона (контекст + добавленные строки) и старая
21
- # (контекст + удалённые), чтобы многострочная цитата из одной версии
22
- # файла находилась целиком. Хунки не склеиваются: между ними пропущенный
23
- # код, и цитата через границу хунка не цитата.
19
+ # One file of the context: the new-file line ranges of the shown hunks
20
+ # and the normalized text of every hunk for the quote search the new
21
+ # side (context + added lines) and the old side (context + removed
22
+ # lines) separately, so that a multi-line quote from one version of the
23
+ # file is found whole. Hunks are not glued together: there is skipped
24
+ # code between them, and a quote across a hunk boundary is not a quote.
24
25
  Section = Struct.new(:ranges, :hunks, :partial, keyword_init: true) do
25
26
  def include?(quote)
26
27
  hunks.any? { |hunk| hunk[:new_text].include?(quote) || hunk[:old_text].include?(quote) }
@@ -34,9 +35,9 @@ module Aireview
34
35
  @sections = parse_sections(diff_text.to_s)
35
36
  end
36
37
 
37
- # Возвращает кандидатов с пометками: note для критика, quote_missing для
38
- # отчёта, line сброшена в null, если не подтвердилась. Кандидаты с файлом
39
- # не из MR отброшены.
38
+ # Returns the candidates with marks: a note for Critique, quote_missing
39
+ # for the report, line reset to null when it was not confirmed.
40
+ # Candidates with a file outside the MR are dropped.
40
41
  def check(candidates)
41
42
  candidates.filter_map do |candidate|
42
43
  id = value(candidate, 'id')
@@ -85,10 +86,10 @@ module Aireview
85
86
  NOTE_QUOTE_NOT_FOUND
86
87
  end
87
88
 
88
- # Дифф уже собран под бюджет: у показанного частично файла хунков может
89
- # не хватать, у файла без диффа хунков нет вовсе. Заголовки --- / +++
90
- # бывают только между заголовком файла и первым @@; внутри хунка строка
91
- # «+++ x» это добавленный код «++ x».
89
+ # The diff is already packed to the budget: a partially shown file may
90
+ # lack hunks, a file without a diff has none at all. The --- / +++
91
+ # headers occur only between the file header and the first @@; inside a
92
+ # hunk a "+++ x" line is the added code "++ x".
92
93
  def parse_sections(diff_text)
93
94
  sections = {}
94
95
  section = nil
@@ -108,8 +109,8 @@ module Aireview
108
109
  normalize_hunks(sections)
109
110
  end
110
111
 
111
- # Хунк без новых строк (удаление, @@ -1 +0,0 @@) диапазона не даёт:
112
- # строки 0 в новом файле нет.
112
+ # A hunk without new lines (a deletion, @@ -1 +0,0 @@) gives no range:
113
+ # there is no line 0 in the new file.
113
114
  def start_hunk(section, header)
114
115
  start = header[1].to_i
115
116
  length = header[2] ? header[2].to_i : 1
@@ -142,8 +143,9 @@ module Aireview
142
143
  partial.each { |path| sections[path]&.partial = true }
143
144
  end
144
145
 
145
- # Сначала точное совпадение: каталоги a/ и b/ бывают настоящими. Префикс
146
- # из заголовка диффа снимается только если точного пути в MR нет.
146
+ # An exact match first: a/ and b/ directories can be real. The prefix
147
+ # from the diff header is stripped only when the exact path is not in
148
+ # the MR.
147
149
  def resolve_path(path)
148
150
  path = path.to_s.strip
149
151
  return path if @mr_paths.include?(path)