lcode-cli 0.33.1__tar.gz → 0.35.0__tar.gz

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/CHANGELOG.md

@@ -6,6 +6,53 @@ All notable changes to lcode are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.35.0] - 2026-10-14
+
+### Added
+
+- A stronger model when stuck: with `escalate_model` set (a cloud model with your API key, such as
+  `anthropic/…`, `openai/…` or `xai/…`, or a bigger local one, `ollama/qwen3.8-27b`), lcode
+  offers to ask it for advice when the loop breaker would stop a request or the checks fail twice
+  in a row, and `/escalate` asks at any time. Nothing is sent without asking, in any permission
+  mode: you see why, what goes (your requests, the task sheet, the failure, the last steps and the
+  files in question, never the whole session), where, and the cost (`escalate_price`), and can
+  read it all first. Keys come only from environment variables and go only to their provider.
+  `lcode -p --escalate` allows it for one run, and `lcode bench --retry --escalate MODEL`
+  measures it. Measured on a 12 GB laptop with qwen3.6-35b and the bigger local qwen3.8-27b as
+  the advisor, on `runtime-bug` with a second try: 1 of 3 passed with its advice, as without it
+  (6 of 9 hidden-test problems fixed against 4), at about 9 minutes per question. A local advisor
+  helps a little; a cloud model is the one to try. See
+  [A stronger model when stuck](https://nasser1941.github.io/lcode/usage/#a-stronger-model-when-stuck).
+
+### Fixed
+
+- Long command output can't fill the disk any more. lcode keeps the full text of long outputs so the
+  model can read parts of them, but a command that printed gigabytes was saved whole, and the
+  saved outputs had no total limit, only an age (7 days). Now one is kept to its first 20 MB (the
+  model still gets the end and the error lines), and all of them together to 200 MB, oldest first.
+
+- A screenshot of a web app no longer starts its server a second time when one already answers at
+  the URL (one the model started itself, say), which used to fail on the busy port and put the
+  error in the result. A page that answers also counts when the server command itself exits.
+
+## [0.34.0] - 2026-10-14
+
+### Added
+
+- Screenshots without a screen: a `screenshot` tool runs a desktop program or opens a web page the
+  way you would see it, saves the shot in `.lcode/screenshots/`, and returns what it shows with the
+  sizes behind it. For a Python Qt program, that's each window against the screen, the size its
+  layout needs, and the widgets that stick out or overlap. For a web page or a local server
+  (headless Chrome), it's the page's size, what scrolls sideways, and the console errors and
+  failed requests. Other desktop programs run on a virtual display (Xvfb). See
+  [Screenshots](https://nasser1941.github.io/lcode/usage/#screenshots).
+
+### Fixed
+
+- In a project with a virtual environment (`.venv`, `venv` or `env`), the model is told to run its
+  Python (`.venv/bin/python`) rather than plain `python`, which lacks the project's packages. Found
+  when the model checked a PySide6 app with the system Python, failed, and stopped checking.
+
 ## [0.33.1] - 2026-10-14
 
 ### Changed
@@ -722,7 +769,9 @@ First public release.
 - `AGENTS.md` project instructions and `/init` to generate them.
 - One-line installer for Linux and macOS.
 
-[Unreleased]: https://github.com/nasser1941/lcode/compare/v0.33.1...HEAD
+[Unreleased]: https://github.com/nasser1941/lcode/compare/v0.35.0...HEAD
+[0.35.0]: https://github.com/nasser1941/lcode/compare/v0.34.0...v0.35.0
+[0.34.0]: https://github.com/nasser1941/lcode/compare/v0.33.1...v0.34.0
 [0.33.1]: https://github.com/nasser1941/lcode/compare/v0.33.0...v0.33.1
 [0.33.0]: https://github.com/nasser1941/lcode/compare/v0.32.0...v0.33.0
 [0.32.0]: https://github.com/nasser1941/lcode/compare/v0.31.0...v0.32.0

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: lcode-cli
-Version: 0.33.1
+Version: 0.35.0
 Summary: A local-first terminal coding agent powered by open-weight models on your own GPU or Mac, via Ollama.
 Project-URL: Homepage, https://nasser1941.github.io/lcode/
 Project-URL: Documentation, https://nasser1941.github.io/lcode/

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/docs/bench.md

@@ -45,6 +45,22 @@ cause. Neither is in the default set, so results stay comparable with earlier ve
 `--code-search` indexes each task's folder for [semantic search](search.md#semantic-code-search);
 both are off in `lcode bench` by default, for the same reason.
 
+### A second try, and a stronger model
+
+`--retry` gives a failed task one more try, as a user would ask: "That didn't fix it: it still
+doesn't work right. Look again." `--escalate MODEL` sets a [stronger model](usage.md#a-stronger-model-when-stuck)
+that's asked when the model is stuck and, with `--retry`, before the second try, as `/escalate`
+would: no questions asked, since the flag is your OK. Comparing the two shows what the stronger
+model's advice adds to the same second chance:
+
+```bash
+lcode bench --tasks runtime-bug --retry --rounds 3
+lcode bench --tasks runtime-bug --retry --rounds 3 --escalate ollama/qwen3.8-27b
+```
+
+The JSON report has, for each task, `first_passed` (with `--retry`, whether the first try passed)
+and `escalations`.
+
 ### Long sessions
 
 Each task normally starts a new conversation. `--session` runs them all in **one** conversation

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/docs/configuration.md

@@ -45,6 +45,9 @@ lcode config path                      # print the file location
 | `notify_after` | `30` | Seconds a request runs before it notifies |
 | `review` | `off` | Before the answer, [review a change](usage.md#a-review-before-the-answer) against the request and send problems back to the model: `auto` or `off` |
 | `checks` | `auto` | Run the project's [checks](usage.md#checks-before-done) after a request changed files: `auto` (found from the project), `off`, or a command |
+| `escalate_model` | — | A [stronger model](usage.md#a-stronger-model-when-stuck) to ask, with your OK, when the model is stuck: `anthropic/…`, `openai/…`, `xai/…` (keys from `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `XAI_API_KEY`), `ollama/<model>`, or `custom/<model>` with `escalate_url` |
+| `escalate_url` | — | Its server, when it isn't the provider's usual one: another Ollama, or any OpenAI-compatible API |
+| `escalate_price` | — | Its price in dollars per million tokens, input/output (e.g. `3/15`), for the cost shown before each use |
 | `repeat_limit` | `3` | Identical tool calls with identical results before lcode [steps in](how-it-works.md#when-the-model-repeats-itself); `0` turns it off |
 | `secret_check` | `true` | Before `git commit` and `git push`, look for passwords, keys and tokens they'd add ([details](usage.md#secrets-in-commits)) |
 | `checkpoints` | `true` | Save a checkpoint before the model changes files, so [`/undo`](usage.md#undo-and-checkpoints) can restore them |

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/docs/quickstart.md

@@ -12,7 +12,7 @@ lcode
 
 ```text
 ╭──────────────────────────────────────────────╮
-│ lcode v0.33.1 — local coding agent           │
+│ lcode v0.35.0 — local coding agent           │
 │                                              │
 │ model    lcode-qwen3.6-35b                   │
 │ context  256K tokens                         │

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/docs/usage.md

@@ -83,6 +83,7 @@ reasoning is on.
 | `/cd DIR` | Change the working directory |
 | `/todos` | Show the model's task list |
 | `/status` | The task sheet: your latest requests, the plan and the model's notes, files changed, the last check |
+| `/escalate` | Ask the [stronger model](#a-stronger-model-when-stuck) for advice on the work, with your OK; `/escalate <what's wrong>` says what to look at |
 | `/jobs [stop ID]` | The [background commands](#background-commands) the model started; stop one |
 | `/commit [notes]` | [Commit](git.md#commit) the changes with a message in the repository's style, once you approve it |
 | `/review [base]` | [Review](git.md#review) the uncommitted changes, or the branch against a base |
@@ -106,6 +107,7 @@ reasoning is on.
 | `web_search` | Search the web for current information (needs a [search provider](#web-search)) |
 | `web_fetch` | Read a web page or text file by URL as clean text |
 | `view_image` | Look at an image: a model that can see describes it ([Images](#images)) |
+| `screenshot` | Run a desktop program or open a web page without a screen, and see it with its sizes ([Screenshots](#screenshots)) |
 | `agent` | Hand a task to a [subagent](agents.md) with its own context; only its report comes back |
 | `memory` | Save, update or delete [notes](memory.md) that later sessions load |
 | `repo_map` | The [repository map](search.md): important files, classes and functions, ranked (larger repositories; small ones get it in the instructions) |
@@ -207,6 +209,51 @@ The review is asked on top of the conversation rather than in a fresh one. The m
 the processed conversation cached, and a separate review would push it out, so the next request
 would have to read the whole conversation again. On a small GPU that's tens of seconds.
 
+## A stronger model when stuck
+
+Some problems are beyond a local model. When it's stuck, lcode can ask a stronger one for
+advice: a cloud model with your API key, or a bigger local model. The local model gets the advice
+and carries on with its own tools. It's off until you choose the model:
+
+```bash
+lcode config set escalate_model anthropic/<model>   # with ANTHROPIC_API_KEY in your environment
+lcode config set escalate_model openai/<model>      # OPENAI_API_KEY
+lcode config set escalate_model xai/<model>         # XAI_API_KEY
+lcode config set escalate_model ollama/qwen3.8-27b  # a bigger local model: free, and nothing leaves
+lcode config set escalate_price 3/15                # dollars per million tokens in/out, for the cost
+```
+
+**When.** lcode offers it when the [loop breaker](how-it-works.md#when-the-model-repeats-itself)
+would stop the request, and when the project's [checks](#checks-before-done) fail twice in a row
+after the model's fixes, at most once per request. You can also ask at any time: `/escalate`, or
+`/escalate the window still grows` to say what's wrong.
+
+**Nothing is sent without asking**, in any permission mode:
+
+```text
+╭─ Ask claude-… for help? ──────────────────────────────────────────────────╮
+│ Why: the project's checks (`pytest -q`) failed 2 times in a row          │
+│ What goes: your requests, the task sheet, the failure, the last steps     │
+│ and these files: wall.py, simulate.py (3.1K tokens)                       │
+│ Where: claude-… (Anthropic), at api.anthropic.com                         │
+│ Cost: about $0.009 to $0.071 (3.1K tokens in at $3/M, up to 4.1K out …)  │
+╰───────────────────────────────────────────────────────────────────────────╯
+  Send it? [y]es / [n]o / [s]how what's sent:
+```
+
+- **What goes:** your last requests, the [task sheet](how-it-works.md#context-management), why
+  it's stuck (the failing check's output, the repeated call), the last few steps, and the files the
+  request changed or read: never the whole session or the rest of the repository. Press `s` to read
+  all of it before you decide.
+- **Keys** come from environment variables, are read when needed, and go only to their own
+  provider. lcode never stores them.
+- **Cost.** The token count is always shown; the price only when you set `escalate_price`, since
+  prices change. A local model is free.
+- **Another server:** `escalate_url` points at another Ollama (`ollama/<model>`) or any OpenAI
+  API (`custom/<model>`, key in `LCODE_ESCALATE_API_KEY`).
+- `lcode -p` can't ask, so it doesn't escalate unless you add `--escalate`, which is your OK for
+  that run. `--output json` lists what was asked in `escalations`.
+
 ## Background commands
 
 Some commands keep running: a dev server, a test watcher, a build in watch mode. The model starts
@@ -342,6 +389,43 @@ Loading a second model takes a moment: on a 12 GB GPU, describing an image with
 `qwen3.6:35b-a3b-coding` takes about a minute, and the next request reloads the main model. Turn
 image support off with `lcode config set vision_model off`.
 
+## Screenshots
+
+Layout bugs are invisible in the code: a window bigger than the screen, panels cut off, a page that
+scrolls sideways. With the `screenshot` tool the model runs the program, or opens the page, without
+a screen and sees what you would, measured:
+
+```text
+● screenshot(python camera_wall.py)
+  ⎿ .lcode/screenshots/20261014-120211-camera-wall.png (Qt offscreen (PyQt6), 1366x768 screen)
+```
+
+```text
+- Window "Camera Wall" (CameraWall): 1362x764 at (0, 0)
+  Its layout needs at least 1950x984, more than the screen has
+  Sticks out of the window: Panel #panel-4: 640x480 at (9, 385)
+  Overlap (190x480): Panel #panel-1: 640x480 at (9, 9) and Panel #panel-2: 640x480 at (459, 9)
+```
+
+| What | How | Besides the picture |
+|---|---|---|
+| A Python Qt program (PyQt5/6, PySide2/6) | Qt's offscreen platform, on a screen of the size asked for (Qt 6) | Each window's size and position, the size its layout needs, widgets that stick out or overlap |
+| Any other desktop program (Linux) | A virtual display, Xvfb | The size and position of each window |
+| A web page, an `.html` file or a local server | Headless Chrome, Chromium or Edge (Firefox takes the picture only) | The page's size against the window and what sticks out sideways, console errors, failed requests |
+
+- For a web app, the model gives the command that starts the server and the page's URL: lcode
+  starts the server, waits for the page to answer, takes the shot and stops the server.
+- A model that can see describes the picture with the model's question in mind (see
+  [Which model looks](#images)). Without one, the measurements still come back.
+- Shots are saved in `.lcode/screenshots/` in the project, which keeps itself out of git, so you
+  can look too.
+- Running a program asks for permission like any command. A web page that isn't local follows the
+  `web` setting. With the [sandbox](sandbox.md) on, programs can't be shown, since they don't run on
+  your machine.
+- Needs: Xvfb for other desktop programs (`sudo apt install xvfb x11-apps`), and Chrome or
+  Chromium for web pages. On macOS only Qt programs and web pages work, since there's no virtual
+  display.
+
 ## Permissions
 
 | Mode | File edits | Shell commands |

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/pyproject.toml

@@ -82,6 +82,7 @@ ignore = [
 "src/lcode/subagents.py" = ["E501"]
 "src/lcode/planning.py" = ["E501"]
 "src/lcode/review.py" = ["E501"] # the review prompt is prose
+"src/lcode/escalate.py" = ["E501"] # the advisor prompt is prose
 "src/lcode/extensions.py" = ["E501"]
 "src/lcode/context.py" = ["E501"]
 "src/lcode/lsp.py" = ["E501"]

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/src/lcode/__init__.py

@@ -3,7 +3,7 @@
 From a program: `from lcode import Session` (see lcode.api).
 """
 
-__version__ = "0.33.1"
+__version__ = "0.35.0"
 
 API = ("Session", "Result", "Options", "SetupError")
 

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/src/lcode/agent.py

@@ -29,6 +29,7 @@ from lcode import (
     catalog,
     checks,
     codesearch,
+    escalate,
     extensions,
     limits,
     notify,
@@ -48,7 +49,7 @@ from lcode.config import format_tokens
 from lcode.jobs import Jobs
 from lcode.mcp import McpManager
 from lcode.ollama import Ollama, OllamaError
-from lcode.permissions import Permissions
+from lcode.permissions import LEADING_CD, Permissions
 from lcode.render import MarkdownStreamer
 from lcode.repeats import Repeats
 from lcode.sandbox import Sandbox, SandboxError, project_root
@@ -122,7 +123,7 @@ SYSTEM_PROMPT = """You are lcode, an autonomous software-engineering agent runni
 # Verifying your work
 - A syntax check or an import only shows that code loads. Run what you changed the way it will be used (the script, the tests, the app) and fix what fails.
 - To fix a bug, first reproduce each problem the user reports with a test, a command or a short script, then run the same checks to show they're gone.
-- For what you can't see, like a GUI, a web page or a layout, check it without a screen where you can: `QT_QPA_PLATFORM=offscreen` for Qt, a headless browser, or a script that prints sizes and positions or renders to an image{look}. Measure instead of assuming.
+- For what you can't see, like a GUI, a web page or a layout, use the screenshot tool: it runs the program (or opens the page) without a screen and returns what it shows, with the sizes of windows, widgets or the page measured{look}. Measure instead of assuming.
 - Never claim something works, mark a todo done, or call it tested unless you ran it. If you couldn't check something, say so and tell the user exactly what to look for. If something fails, say so and show the relevant error.{checks}
 
 # Communication
@@ -131,7 +132,7 @@ SYSTEM_PROMPT = """You are lcode, an autonomous software-engineering agent runni
 
 # Environment
 - Working directory: {cwd}
-- OS: {os}; shell: bash; Python command: `{python}`
+- OS: {os}; shell: bash; Python command: {python}
 - Date: {date}
 - Git: {git}
 
@@ -218,6 +219,30 @@ def call_arguments(call: dict) -> dict:
     return args if isinstance(args, dict) else {}
 
 
+VENV_DIRS = (".venv", "venv", "env")
+
+
+def project_python(cwd: Path) -> Path | None:
+    """The project's virtual environment's Python (.venv, venv or env, here or up to the repository's root)."""
+    for folder in [cwd, *cwd.parents][:4]:
+        for name in VENV_DIRS:
+            path = folder / name / "bin" / "python"
+            if path.exists() and (folder / name / "pyvenv.cfg").exists():
+                return path
+        if (folder / ".git").exists():
+            break  # not beyond the repository
+    return None
+
+
+def python_line(cwd: Path) -> str:
+    """How the system prompt names the Python command."""
+    venv = project_python(cwd)
+    if venv is None:
+        return f"`{'python' if shutil.which('python') else 'python3'}`"
+    shown = venv.relative_to(cwd) if cwd in venv.parents else venv
+    return f"`{shown}` (the project's virtual environment: plain python lacks its packages)"
+
+
 class Progress:
     """The status line under a running request.
 
@@ -283,6 +308,9 @@ class Settings:
     repeat_limit: int = 3  # identical calls with identical results before the model is told (lcode.repeats)
     checks: str = "off"  # the project's checks after a change (lcode.checks): auto, off or a command
     review: str = "off"  # check a change against the request before the answer (lcode.review): auto or off
+    escalate_model: str | None = None  # a stronger model to ask when stuck (lcode.escalate): provider/model
+    escalate_url: str | None = None  # its server, when not the provider's usual one
+    escalate_price: str | None = None  # dollars per million tokens in/out, e.g. "3/15", for the cost estimate
 
 
 class Agent:
@@ -345,6 +373,8 @@ class Agent:
         self._check: tuple[Path, checks.Check | None] | None = None
         self.check_results: list[dict] = []  # the checks of the last request: command, exit code, who ran them
         self.review_results: list[str] | None = None  # what the last request's review found (None: no review)
+        self.escalations: list[dict] = []  # the stronger model asked in the last request (lcode.escalate)
+        self.escalate_approved: bool | None = None  # decided in advance (lcode -p --escalate, the benchmark)
         self.sent_chars = 0  # characters of the conversation the model had already read
         self.counted_chars: int | None = None  # characters that ctx_used covers (None: all of them)
         self.turn_status = ""  # how the last request ended: success, max_steps, loop
@@ -372,7 +402,7 @@ class Agent:
             model=self.settings.model,
             cwd=self.cwd,
             os=f"{platform.system()} {platform.release()} ({platform.machine()})",
-            python="python" if shutil.which("python") else "python3",
+            python=python_line(self.cwd),
             date=dt.date.today().isoformat(),
             git=git_info(self.cwd),
             tree=tree(self.cwd, depth=1, limit=120),
@@ -383,7 +413,7 @@ class Agent:
             skills=extensions.skills_prompt(self.extensions().skills, self.settings.context),
             code=self.code_prompt(),
             map=self.map_prompt(),
-            look=" you look at with view_image" if self.vision_model() else "",
+            look="; look at other images with view_image" if self.vision_model() else "",
             checks=self.checks_prompt(),
         )
 
@@ -928,9 +958,8 @@ class Agent:
             )
             return None
         if round_ >= checks.MAX_ROUNDS:
-            self.say(
-                f"Checks still fail ({check.command}, exit code {code}): the model will say what's left.", warning=True
-            )
+            then = "" if self.can_escalate() else ": the model will say what's left"
+            self.say(f"Checks still fail ({check.command}, exit code {code}){then}.", warning=True)
         else:
             self.say(
                 f"Checks fail ({check.command}, exit code {code}): the model gets the output to fix it.", warning=True
@@ -1000,6 +1029,9 @@ class Agent:
             return f"agent({args.get('type', '')}: {args.get('description') or str(args.get('task', ''))[:50]})"
         if name == "bash":
             return f"$ {str(args.get('command', ''))[:60]}"
+        if name == "screenshot":
+            what = args.get("url") or LEADING_CD.sub("", str(args.get("command") or ""))
+            return f"screenshot({what[:60]})"
         if name in ("web_search", "web_fetch", "view_image"):
             return f"{name}({args.get('query') or args.get('url') or args.get('path') or ''})"
         if self.mcp and self.mcp.owns(name):
@@ -1076,6 +1108,8 @@ class Agent:
         self.repeats.reset()
         self.check_results = []
         self.review_results = None
+        self.escalations = []
+        escalated = False  # the stronger model is asked at most once per request
         self.tools.before = {}
         reviewed = False
         changed = verified = False  # files changed in this request; the checks passed since
@@ -1139,6 +1173,15 @@ class Agent:
                     check_rounds += 1
                     feedback = self.run_check(check, check_rounds)
                     if feedback:
+                        if check_rounds >= checks.MAX_ROUNDS and not escalated and self.can_escalate():
+                            escalated = True
+                            why = f"the project's checks (`{check.command}`) failed {check_rounds} times in a row"
+                            advice = escalate.run(self, why, failure=feedback)
+                            if advice:
+                                self.messages.append({"role": "user", "content": advice})
+                                check_rounds -= 1  # the checks run once more after the advice
+                                changed = False
+                                continue
                         self.messages.append({"role": "user", "content": feedback})
                         changed = False  # run them again only if the model changes something more
                         continue
@@ -1232,6 +1275,14 @@ class Agent:
                             skipped["tool_call_id"] = rest["id"]
                         self.messages.append(skipped)
                     break
+            if looping and not escalated and self.can_escalate():
+                escalated = True
+                advice = escalate.run(self, f"it kept making the same call with the same result: {looping}")
+                if advice:
+                    self.messages.append({"role": "user", "content": advice})
+                    self.repeats.reset()
+                    looping = ""
+                    continue
             if looping:
                 self.turn_status = "loop"
                 self.console.print(
@@ -1249,6 +1300,11 @@ class Agent:
         self.turn_status = "success"
         self.print_stats()
 
+    def can_escalate(self) -> bool:
+        """Whether a stronger model may be asked here: one is set, and this isn't a subagent or plan mode."""
+        model = (self.settings.escalate_model or "").strip().lower()
+        return bool(model) and model != "off" and self.allowed_tools is None and not self.planning()
+
     def steer(self) -> None:
         """Between two steps: pause if the user pressed Esc, then hand the model the notes typed meanwhile."""
         steering = self.steering

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/src/lcode/api.py

@@ -78,6 +78,7 @@ class Result:
     usage: dict = field(default_factory=dict)  # requests, prompt_tokens, output_tokens
     checks: list[dict] = field(default_factory=list)  # {"command", "exit_code", "by": "lcode" | "model"}
     review: list[str] | None = None  # problems the review sent back ([] when it found none; None: no review)
+    escalations: list[dict] = field(default_factory=list)  # a stronger model asked: model, why, tokens, seconds
     seconds: float = 0.0
     error: str = ""
 
@@ -200,6 +201,9 @@ def open_agent(cwd: Path, options: Options, console: Console, hw: Hardware | Non
         repeat_limit=cfg["repeat_limit"],
         checks=cfg["checks"],
         review=cfg["review"],
+        escalate_model=cfg["escalate_model"],
+        escalate_url=cfg["escalate_url"],
+        escalate_price=cfg["escalate_price"],
         compact_at=cfg["compact_at"],
     )
     agent = Agent(ollama, settings, cwd, console=console)
@@ -309,6 +313,7 @@ def run_request(agent: Agent, prompt: str) -> Result:
         usage=usage,
         checks=list(agent.check_results),
         review=agent.review_results,
+        escalations=list(agent.escalations),
         seconds=round(time.monotonic() - started, 2),
         error=error,
     )

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/src/lcode/backends.py

@@ -265,11 +265,16 @@ class OpenAICompatible:
             return self.chat({k: v for k, v in payload.items() if k not in ("format", "tools")}, timeout)
         if r.status_code != 200:
             raise self._failed(r.status_code, r.text)
-        choice = (r.json().get("choices") or [{}])[0].get("message") or {}
+        data = r.json()
+        choice = (data.get("choices") or [{}])[0].get("message") or {}
         message: dict = {"role": "assistant", "content": strip_thinking(choice.get("content") or "")}
         if choice.get("tool_calls"):
             message["tool_calls"] = [self._ollama_call(c) for c in choice["tool_calls"]]
-        return {"message": message}
+        reply: dict = {"message": message}
+        usage = data.get("usage") or {}
+        if usage.get("prompt_tokens") is not None:  # as Ollama counts them
+            reply["prompt_eval_count"], reply["eval_count"] = usage["prompt_tokens"], usage.get("completion_tokens")
+        return reply
 
     @staticmethod
     def _ollama_call(call: dict) -> dict:

{lcode_cli-0.33.1 → lcode_cli-0.35.0}/src/lcode/bench.py

@@ -29,7 +29,7 @@ from rich.markup import escape
 from rich.table import Table
 from rich.text import Text
 
-from lcode import __version__
+from lcode import __version__, escalate
 from lcode.agent import Agent, Settings
 from lcode.config import format_tokens
 from lcode.hardware import Hardware
@@ -790,6 +790,8 @@ class TaskResult:
     tool_errors: int = 0  # failed tool calls, including malformed ones
     error: str | None = None  # why the run stopped early: a timeout or an Ollama error
     errors_by_tool: dict[str, int] = field(default_factory=dict)  # which tools failed, "malformed" for unreadable calls
+    first_passed: bool | None = None  # with --retry: whether the first try passed (None: no retry was needed)
+    escalations: int = 0  # times the stronger model was asked (--escalate)
 
 
 @dataclass
@@ -887,8 +889,13 @@ def run_task(
     keep: bool,
     usage: dict,
     agent: Agent | None = None,
+    retry: bool = False,
 ) -> TaskResult:
-    """Run one task in a new folder: in a new conversation, or continuing `agent`'s (--session)."""
+    """Run one task in a new folder: in a new conversation, or continuing `agent`'s (--session).
+
+    With `retry`, a failed task gets one more try, as a user would ask: "That didn't fix it. Look again." With a
+    stronger model set (--escalate), its advice goes along, as with /escalate.
+    """
     folder = Path(tempfile.mkdtemp(prefix=f"lcode-bench-{task.id}-"))
     try:
         for rel, content in task.files.items():
@@ -906,25 +913,28 @@ def run_task(
             if model:
                 codesearch.Index(folder, model).update(ollama)  # on the CPU: lcode's model stays loaded
             agent._code_index = None
+        agent.escalate_approved = True if settings.escalate_model else None  # the --escalate flag is the OK
         before = dict(agent.usage)
         first = len(agent.messages)
-        error = None
         started = time.monotonic()
-        try:
-            with time_limit(timeout):
-                agent.run_turn(task.prompt)
-        except TaskTimeout:
-            error = f"timed out after {timeout:.0f}s"
-        except OllamaError as e:
-            error = str(e).splitlines()[0][:200]
+        escalations = 0
+        error = attempt(agent, task.prompt, timeout)
+        escalations += len(agent.escalations)
+        check = checked(task, folder, agent)
+        first_passed = None
+        if retry and not (check.passed and error is None):
+            first_passed = False
+            text = RETRY
+            if error is None and agent.can_escalate():
+                advice = escalate.run(agent, "the user says the fix didn't work")
+                escalations += 1 if advice else 0
+                text = f"{RETRY}\n\n{advice}" if advice else RETRY
+            error = attempt(agent, text, timeout)
+            escalations += len(agent.escalations)
+            check = checked(task, folder, agent)
         seconds = time.monotonic() - started
         for key, value in agent.usage.items():
             usage[key] = usage.get(key, 0) + value - before.get(key, 0)
-        answer = next((m.get("content", "") for m in reversed(agent.messages) if m.get("role") == "assistant"), "")
-        try:
-            check = task.check(folder, answer)
-        except Exception as e:  # a check must never crash the benchmark
-            check = Check(False, f"check failed: {type(e).__name__}: {e}")
     finally:  # also when Ctrl+C stops the task
         from lcode.codesearch import index_dir
 
@@ -942,9 +952,34 @@ def run_task(
         tool_errors=count_tool_errors(agent.messages[first:] if len(agent.messages) > first else agent.messages),
         errors_by_tool=errors_by_tool(agent.messages[first:] if len(agent.messages) > first else agent.messages),
         error=error,
+        first_passed=first_passed,
+        escalations=escalations,
     )
 
 
+RETRY = "That didn't fix it: it still doesn't work right. Look again."
+
+
+def attempt(agent: Agent, text: str, timeout: float) -> str | None:
+    """One request of a task; why it stopped early, if it did."""
+    try:
+        with time_limit(timeout):
+            agent.run_turn(text)
+    except TaskTimeout:
+        return f"timed out after {timeout:.0f}s"
+    except OllamaError as e:
+        return str(e).splitlines()[0][:200]
+    return None
+
+
+def checked(task: Task, folder: Path, agent: Agent) -> Check:
+    answer = next((m.get("content", "") for m in reversed(agent.messages) if m.get("role") == "assistant"), "")
+    try:
+        return task.check(folder, answer)
+    except Exception as e:  # a check must never crash the benchmark
+        return Check(False, f"check failed: {type(e).__name__}: {e}")
+
+
 def prompt_speed(ollama: Ollama, settings: Settings, options: dict) -> float | None:
     """Prompt tokens per second for prompts the model hasn't seen: the faster of two readings.
 
@@ -998,6 +1033,7 @@ def run_model(
     keep: bool = False,
     verbose: bool = False,
     session: bool = False,
+    retry: bool = False,
 ) -> ModelRun:
     run = ModelRun(name, settings.model, settings.context, settings.num_batch, settings.think, session=session)
     shared: Agent | None = None  # with --session, one conversation for all tasks
@@ -1038,10 +1074,10 @@ def run_model(
         try:
             if verbose:
                 console.rule(f"{task.id}: {task.title}")
-                result = run_task(ollama, settings, task, timeout, log, keep, run.usage, shared)
+                result = run_task(ollama, settings, task, timeout, log, keep, run.usage, shared, retry)
             else:
                 with console.status(Elapsed(f"  {task.id:<13} {task.title}")):
-                    result = run_task(ollama, settings, task, timeout, log, keep, run.usage, shared)
+                    result = run_task(ollama, settings, task, timeout, log, keep, run.usage, shared, retry)
         except KeyboardInterrupt:
             console.print("  [yellow]Stopped.[/]")
             run.interrupted = True