aireview 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0ceb8f061ed3a8172b46df1af17fa9961282858657aaf347215b42d5824acf7b
-  data.tar.gz: e345026c2323be4af4fa9ff2e00fb552685d9e9aec040a691a347cc063ce9d88
+  metadata.gz: 100adbc2d1510ad1f6c0e9b152decbfa46e71f8db30dfde36a8ba977528ae215
+  data.tar.gz: 071d392ef71d935a6b6a055e7e86fbbf06e8f4517251d460d8bbff8087cb8ecf
 SHA512:
-  metadata.gz: 3fbbfb3540d90c6d5cf35bbe5e403ba0d4abc17fab630b464d9a03bd9748563cd3ed11192673ed5bca48e1d01c393b25bd5ad25324c4ccf7c3e4548695d336e0
-  data.tar.gz: 33f0bbba90317654a12835bd8328f6ef58a46b89b7d312aad6da46b76033495d708dc8da6a25772c7b8815811c0cc882878f0a58ac7cf418d0328af29821fe60
+  metadata.gz: 48d8222728f5bd0cdb27790c9971c316060c9efd89479e6be58ee6f41066e237e4e94cf2f864297b03a3012e1f225b92c4ece4b84ffeca3ad3ef66831a176e09
+  data.tar.gz: e4cdac4c17edc362e318b548ac5d25198eaf9c916edd670424c7d21b0cdf4baab2f9f61527bb05547e737e30f192d243ee7dfcb80062a483b08bf410466bfcef

data/CHANGELOG.md

@@ -1,5 +1,12 @@
 # Changelog
 
+## 0.1.1
+
+- The prompts no longer ask the model to check whether dependency and image
+  versions exist; candidates about nonexistent versions are rejected.
+- An overloaded LLM (503) is retried up to three times with pauses of about
+  2, 5 and 5 minutes instead of two retries of 1.5-2.5 minutes.
+
 ## 0.1.0
 
 Initial release.

data/README.md

@@ -162,7 +162,10 @@ To run both stages locally, set `ollama` in both provider variables. The
 address with `/v1` matches the
 [Ollama configuration in RubyLLM](https://rubyllm.com/configuration/#provider-configuration).
 `LLM_TIMEOUT` sets the timeout of every LLM request in seconds; for a slow
-local model it can be raised.
+local model it can be raised. It does not apply to an "overloaded" (503)
+answer from the provider: such a request gets up to four attempts, the
+original one and three retries with pauses of about 2, 5 and 5 minutes, and
+only the failed stage is repeated, not the whole run.
 
 Project rules live in `.aireview.yml`. In YAML the `generate.model` and
 `critique.model` settings are required for each stage and are not inherited
@@ -316,6 +319,7 @@ aireview:
     LLM_HTTP_PROXY: "http://127.0.0.1:8888"
   script:
     - bundle _2.3.26_ exec bin/aireview review "$MR_URL" --verbose
+  timeout: 45m
   retry:
     max: 1
     when:
@@ -323,6 +327,11 @@ aireview:
       - stuck_or_timeout_failure
 ```
 
+`timeout: 45m` is a chosen ceiling, not a guarantee that every retry fits in:
+when the provider is overloaded, one stage can wait up to ~14 minutes of
+pauses plus up to four requests of `LLM_TIMEOUT` each, and there are two
+stages.
+
 Set secrets such as `GITLAB_TOKEN`, `GEMINI_API_KEY` and the optional Jira
 credentials in the GitLab CI/CD variables. If the job should publish the result
 back to the merge request, add `--post` to the review command.
@@ -351,8 +360,27 @@ bundle _2.3.26_ exec rspec spec/secret_scrubber_spec.rb
 
 ## Notes
 
+- The reviewer does not check whether the specified versions of dependencies and images exist: the model's knowledge of releases is outdated, and that is what CI is for. Syntax errors and contradictions with the MR/Jira requirements are checked as usual.
 - The CLI looks for `.aireview.yml` and `.env` walking up from the current working directory, so the project config can be kept in the repository root even when the tool is run from `aireview/`.
 
+## Releasing
+
+Releases are published to RubyGems.org by the `Release` workflow through
+[trusted publishing](https://guides.rubygems.org/trusted-publishing/), so no
+API key is stored anywhere. To cut a release:
+
+1. Bump `Aireview::VERSION` in `lib/aireview/version.rb` and move the
+   `Unreleased` section of `CHANGELOG.md` under the new version.
+2. Commit, then tag the commit with the same version and push the tag:
+
+   ```bash
+   git tag v0.2.0
+   git push github main v0.2.0
+   ```
+
+The workflow refuses to run when the tag does not match `Aireview::VERSION`,
+runs the test suite, builds the gem and pushes it.
+
 ## Changelog
 
 See [CHANGELOG.md](CHANGELOG.md).

data/lib/aireview/prompts/critique.txt

@@ -23,6 +23,12 @@ Implementation details (config flags, deploy settings, new fields, helper
 methods), accompanying refactoring and any changes whose relation to the task
 is plausible are NOT task_mismatch. Reject such candidates.
 
+Do not verify that the specified versions of packages, libraries, tools and
+image tags exist. Do not report that a version does not exist or has not been
+released yet. Check syntax errors and explicit contradictions with the MR/Jira
+requirements as usual.
+Reject candidates claiming that a version does not exist.
+
 Reject a candidate if it:
 - is not confirmed by the diff or Jira;
 - is based on an assumption about code outside the diff;

data/lib/aireview/prompts/generate.txt

@@ -3,6 +3,11 @@ You are doing the first pass of a merge request review.
 Look only at the diff, the MR description and the Jira context, if present.
 Do not draw conclusions about code outside the diff.
 
+Do not verify that the specified versions of packages, libraries, tools and
+image tags exist. Do not report that a version does not exist or has not been
+released yet. Check syntax errors and explicit contradictions with the MR/Jira
+requirements as usual.
+
 Look only for the most important and well-supported findings:
 - bugs, regressions, data loss;
 - security and performance risks;

data/lib/aireview/reviewer.rb

@@ -6,11 +6,12 @@ require 'timeout'
 
 module Aireview
   class Reviewer
-    MAX_TRANSIENT_RETRIES = 2
+    MAX_TRANSIENT_RETRIES = 3
     TRANSIENT_RETRY_BASE_DELAY = 2.0
     TRANSIENT_RETRY_JITTER_RANGE = 2.0..5.0
     PROVIDER_RETRY_DELAY_MULTIPLIER_RANGE = 2.0..2.4
-    OVERLOADED_RETRY_DELAY_RANGE = 90.0..150.0
+    OVERLOADED_RETRY_DELAYS = [120.0, 300.0, 300.0].freeze
+    OVERLOADED_RETRY_JITTER_RANGE = 0.85..1.15
     RETRY_WAIT_LOG_FORMAT = 'LLM %<stage>s request will sleep %<delay>.1fs before retry%<source>s ' \
                             '(attempt %<next_attempt>d/%<max_attempts>d, model=%<model>s)'
 
@@ -184,10 +185,12 @@ module Aireview
       end
 
       if overloaded_llm_error?(error)
+        base_delay = OVERLOADED_RETRY_DELAYS.fetch(attempt - 1, OVERLOADED_RETRY_DELAYS.last)
+        multiplier = rand(OVERLOADED_RETRY_JITTER_RANGE)
         return {
-          delay: rand(OVERLOADED_RETRY_DELAY_RANGE),
-          provider_delay: nil,
-          multiplier: nil,
+          delay: base_delay * multiplier,
+          base_delay: base_delay,
+          multiplier: multiplier,
           strategy: :overloaded
         }
       end
@@ -202,7 +205,13 @@ module Aireview
     end
 
     def retry_delay_source(retry_delay)
-      return ' (overloaded backoff)' if retry_delay[:strategy] == :overloaded
+      if retry_delay[:strategy] == :overloaded
+        return format(
+          ' (overloaded backoff %<base_delay>.0fs, multiplier %<multiplier>.2fx)',
+          base_delay: retry_delay[:base_delay],
+          multiplier: retry_delay[:multiplier]
+        )
+      end
       return '' unless retry_delay[:provider_delay]
 
       format(

data/lib/aireview/version.rb

@@ -1,4 +1,4 @@
 # frozen_string_literal: true
 module Aireview
-  VERSION = '0.1.0'
+  VERSION = '0.1.1'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: aireview
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
 platform: ruby
 authors:
 - Denis Levenko