@krotovm/gitlab-ai-review 1.0.39 → 1.0.40

package/README.md

@@ -58,6 +58,7 @@ Set these in your project/group CI settings:
 - `AI_PROMPT_PROFILE` (optional, default: `default`; one of `default` \| `weak`). Use `weak` for small or heavily-quantized models (≤ ~7B, local Ollama, etc.). The `weak` profile uses shorter system prompts, positive rules instead of negations, and inline few-shot examples for triage / per-file review / consolidation / verification, which dramatically improves output-format adherence on weak models. When the variable is not set, the profile is auto-detected: if the triage pass returns unparseable JSON with `default` prompts, triage is retried once with `weak` and, on success, the whole run continues with it. Set the variable explicitly to pin a profile and disable auto-detection.
 - `PROJECT_ACCESS_TOKEN` (optional for public projects, but required for most private projects; token with `api` scope)
 - `GITLAB_TOKEN` (optional alias for `PROJECT_ACCESS_TOKEN`)
+- `AI_REVIEW_CONCURRENCY` (optional) — same as `--max-review-concurrency`, handy as a CI/CD variable; the flag wins when both are set.
 - `AI_REVIEW_ARTIFACT_HTML_FILE` (optional, default: `ai-review-report.html`; used with `--include-artifacts`)
 
 `OPENAI_BASE_URL` is passed through to the `openai` SDK client, so you can use any OpenAI-compatible gateway/provider endpoint.
@@ -77,11 +78,29 @@ GitLab provides these automatically in Merge Request pipelines:
 - `--max-total-prompt-chars=220000` - Final hard cap for prompt size (single-pass fallback only).
 - `--triage-diff-chars=2000` - Max chars per file diff sent to the triage pass (Pass 1). Increase for large diffs where the first hunks are mostly git headers.
 - `--max-findings=5` - Max findings in the final review (CI multi-pass only).
-- `--max-review-concurrency=5` - Parallel per-file review API calls (CI multi-pass only).
+- `--max-review-concurrency=2` - Parallel per-file review API calls (CI multi-pass only). Set to 1 for single-GPU backends that queue-starve long requests.
 - `--debug` - Print full error details (stack and API error fields).
 - `--include-artifacts` - Generate a local HTML debug artifact with per-pass outputs/tokens.
 - `--help` - Show help output.
 
+## Benchmark
+
+Head-to-head against [Alibaba OpenCodeReview](https://github.com/alibaba/open-code-review) v1.8.10 (via its official GitLab CI recipe) on a demo MR with 4 seeded bugs: an auth bypass (`API_KEYS` unset → empty key accepted), a rate limiter whose window never resets, an unbounded per-IP counters Map, and a null-deref crash in a stats endpoint. Same MR, same commit range, same backends.
+
+| Tool | Model | Seeded bugs found | Noise (dups / off-target) | Tokens | LLM time |
+| --- | --- | --- | --- | --- | --- |
+| **gitlab-ai-review** | gpt-5.4 | 3 / 4 | 0 | 15.7k | ~17s |
+| OpenCodeReview | gpt-5.4 | 3 / 4 | 2 | 41.5k | ~11s |
+| **gitlab-ai-review** | self-hosted (single GPU) | **4 / 4** | **0** | 37.2k | 7m 24s |
+| OpenCodeReview | self-hosted (single GPU) | 3 / 4 | 8 (incl. 5 duplicates) | 151.8k | 11m 45s |
+
+Notes:
+
+- On gpt-5.4 both tools found the same 3 bugs and both missed the memory leak. OpenCodeReview added two design opinions, one of which (severity "high") suggested removing the very auth feature the MR introduces.
+- On the self-hosted model the multi-pass pipeline (triage → per-file review → consolidate → verify) paid off: all 4 seeded bugs, zero noise. OpenCodeReview posted 11 comments: 3 unique real findings, 5 duplicates, and 3 off-target — its only rate-limiter comment misdiagnosed the bug, and several comments were attributed to the wrong file or line.
+- Weak-model reviews are output-bound on a single GPU: OpenCodeReview generated 40.2k output tokens vs our 18.5k, and consumed 6× our prompt tokens (111.6k vs 18.7k) — that difference is the wall-clock gap.
+- Methodology caveats: single run per cell on one MR; LLM variance applies.
+
 ## Architecture
 
 The reviewer uses a three-pass pipeline optimized for large merge requests:

package/dist/cli.js

@@ -33,6 +33,7 @@ function printHelp() {
         "  OPENAI_API_KEY (required)  OpenAI API key.",
         "  OPENAI_BASE_URL (optional)  Custom OpenAI-compatible API base URL.",
         "  AI_MODEL      (optional)  OpenAI chat model, e.g. gpt-4o. Default: gpt-4o-mini.",
+        "  AI_REVIEW_CONCURRENCY (optional)  Same as --max-review-concurrency; the flag wins.",
         "  AI_PROMPT_PROFILE (optional)  Prompt style: \"default\" | \"weak\". Default: default.",
         "                              Use \"weak\" for small/quantized models — shorter prompts,",
         "                              positive rules, and few-shot examples.",
@@ -82,7 +83,10 @@ async function main() {
     const promptLimits = parsePromptLimits(process.argv);
     const triageDiffChars = parseTriageDiffChars(process.argv);
     const maxFindings = parseNumberFlag(process.argv, "max-findings", DEFAULT_MAX_FINDINGS, 1);
-    const reviewConcurrency = parseNumberFlag(process.argv, "max-review-concurrency", DEFAULT_REVIEW_CONCURRENCY, 1);
+    const envConcurrency = Number(envOrUndefined("AI_REVIEW_CONCURRENCY"));
+    const reviewConcurrency = parseNumberFlag(process.argv, "max-review-concurrency", Number.isFinite(envConcurrency) && envConcurrency >= 1
+        ? Math.floor(envConcurrency)
+        : DEFAULT_REVIEW_CONCURRENCY, 1);
     const aiModel = envOrDefault("AI_MODEL", "gpt-4o-mini");
     const promptProfile = readPromptProfileFromEnv();
     const allowProfileAutoDemote = envOrUndefined("AI_PROMPT_PROFILE") == null;

package/dist/prompt/index.js

@@ -53,8 +53,8 @@ export const buildPrompt = ({ changes, limits, allowTools = false, profile = DEF
 // Multi-pass pipeline prompts
 // ---------------------------------------------------------------------------
 export const DEFAULT_MAX_FINDINGS = 5;
-// ponytail: 2 keeps self-hosted gateways under their proxy timeout; raise via
-// --max-review-concurrency on backends that batch well.
+// ponytail: 2 suits most gateways; single-GPU backends that queue-starve past
+// the per-call deadline should set AI_REVIEW_CONCURRENCY=1 (or the flag).
 export const DEFAULT_REVIEW_CONCURRENCY = 2;
 export function buildTriagePrompt(changes, profile = DEFAULT_PROMPT_PROFILE, triageDiffChars = DEFAULT_TRIAGE_DIFF_CHARS) {
     return [

package/package.json

@@ -1,7 +1,7 @@
 {
   "type": "module",
   "name": "@krotovm/gitlab-ai-review",
-  "version": "1.0.39",
+  "version": "1.0.40",
   "description": "CLI tool to generate AI code reviews for GitLab merge requests.",
   "main": "dist/cli.js",
   "bin": {