virtuai-cli 1.0.3__tar.gz → 1.0.4__tar.gz

{virtuai_cli-1.0.3 → virtuai_cli-1.0.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: virtuai-cli
-Version: 1.0.3
+Version: 1.0.4
 Summary: Run VirtuAI deep agents on your local machine
 Author-email: uCloudStore <lmoreno@ucloudstore.com>
 License: Proprietary
@@ -138,7 +138,7 @@ Each model call goes to the runner of the person the turn is for; if they have n
 
 At connect and then every minute: the models it has, the Ollama version, what is loaded and how much memory it really took. With `llmfit` installed it also estimates which pulled models fit this machine, so the server does not send a model to a runner that cannot hold it until it has actually run there. A model that spills to the CPU gets half the context on its next call on that machine (down to 4k).
 
-The context sent to Ollama is 8k by default (set on the server); `OLLAMA_NUM_CTX` overrides it on a machine with less memory. For qwen3 the runner turns Ollama's thinking mode off: left on, it reasoned for minutes before every tool call.
+The context sent to Ollama is 16k by default (set on the server). A model that does not fit at that context — `qwen3:14b` on a 16 GB Mac spills 7% to the CPU — is capped at half on that machine after its first call, and the server is told in the same reply: from then on it sends prompts that fit, compacting the conversation when they would not, instead of letting Ollama cut the start of the prompt. `OLLAMA_NUM_CTX` sets a lower context on a machine with little memory. For qwen3 the runner turns Ollama's thinking mode off: left on, it reasoned for minutes before every tool call.
 
 ### oMLX
 
@@ -163,7 +163,7 @@ mean "allow more commands at once" silently launched more browsers.
 | `VIRTUAI_SANDBOX_OWNER` | unset | The one person this runner serves, reported to the server so a browser action reaches *their* browser. Leave it unset on a laptop or a container shared by a team: the server then routes to it for anyone, which is the behaviour that existed before per-person sandboxes. Set it when a workspace has a sandbox per person, or actions land on whichever runner connected last. |
 | `VIRTUAI_SERVE_MODELS` | `own` | Whose local-model turns this runner answers: `own`, `workspace` or `none`. Also set by `--serve-models`. |
 | `OLLAMA_BASE_URL` | `http://localhost:11434` | Where to reach the local Ollama. |
-| `OLLAMA_NUM_CTX` | unset | The context to ask Ollama for, instead of the server's (8k). Lower it on a machine with little memory. |
+| `OLLAMA_NUM_CTX` | unset | The context to ask Ollama for, instead of the server's (16k). Lower it on a machine with little memory. |
 | `VIRTUAI_OMLX_URL` | from `~/.omlx/settings.json`, else `http://127.0.0.1:8000` | Where to reach oMLX. |
 | `OMLX_API_KEY` | from `~/.omlx/settings.json` | oMLX's API key. Stays on this machine. |
 

{virtuai_cli-1.0.3 → virtuai_cli-1.0.4}/README.md

@@ -109,7 +109,7 @@ Each model call goes to the runner of the person the turn is for; if they have n
 
 At connect and then every minute: the models it has, the Ollama version, what is loaded and how much memory it really took. With `llmfit` installed it also estimates which pulled models fit this machine, so the server does not send a model to a runner that cannot hold it until it has actually run there. A model that spills to the CPU gets half the context on its next call on that machine (down to 4k).
 
-The context sent to Ollama is 8k by default (set on the server); `OLLAMA_NUM_CTX` overrides it on a machine with less memory. For qwen3 the runner turns Ollama's thinking mode off: left on, it reasoned for minutes before every tool call.
+The context sent to Ollama is 16k by default (set on the server). A model that does not fit at that context — `qwen3:14b` on a 16 GB Mac spills 7% to the CPU — is capped at half on that machine after its first call, and the server is told in the same reply: from then on it sends prompts that fit, compacting the conversation when they would not, instead of letting Ollama cut the start of the prompt. `OLLAMA_NUM_CTX` sets a lower context on a machine with little memory. For qwen3 the runner turns Ollama's thinking mode off: left on, it reasoned for minutes before every tool call.
 
 ### oMLX
 
@@ -134,7 +134,7 @@ mean "allow more commands at once" silently launched more browsers.
 | `VIRTUAI_SANDBOX_OWNER` | unset | The one person this runner serves, reported to the server so a browser action reaches *their* browser. Leave it unset on a laptop or a container shared by a team: the server then routes to it for anyone, which is the behaviour that existed before per-person sandboxes. Set it when a workspace has a sandbox per person, or actions land on whichever runner connected last. |
 | `VIRTUAI_SERVE_MODELS` | `own` | Whose local-model turns this runner answers: `own`, `workspace` or `none`. Also set by `--serve-models`. |
 | `OLLAMA_BASE_URL` | `http://localhost:11434` | Where to reach the local Ollama. |
-| `OLLAMA_NUM_CTX` | unset | The context to ask Ollama for, instead of the server's (8k). Lower it on a machine with little memory. |
+| `OLLAMA_NUM_CTX` | unset | The context to ask Ollama for, instead of the server's (16k). Lower it on a machine with little memory. |
 | `VIRTUAI_OMLX_URL` | from `~/.omlx/settings.json`, else `http://127.0.0.1:8000` | Where to reach oMLX. |
 | `OMLX_API_KEY` | from `~/.omlx/settings.json` | oMLX's API key. Stays on this machine. |
 

{virtuai_cli-1.0.3 → virtuai_cli-1.0.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "virtuai-cli"
-version = "1.0.3"
+version = "1.0.4"
 description = "Run VirtuAI deep agents on your local machine"
 readme = "README.md"
 requires-python = ">=3.11"

{virtuai_cli-1.0.3 → virtuai_cli-1.0.4}/src/virtuai_cli/__init__.py

@@ -1,2 +1,2 @@
 """VirtuAI local CLI."""
-__version__ = "1.0.3"
+__version__ = "1.0.4"

{virtuai_cli-1.0.3 → virtuai_cli-1.0.4}/src/virtuai_cli/ollama_state.py

@@ -120,7 +120,7 @@ async def after_call(model: str) -> None:
     info = (state.get("loaded") or {}).get(model)
     if not info or info.get("on_gpu_pct", 100) >= 100:
         return
-    current = info.get("num_ctx") or _ctx_cap.get(model) or 8_192
+    current = info.get("num_ctx") or _ctx_cap.get(model) or SERVER_NUM_CTX
     lower = max(MIN_NUM_CTX, current // 2)
     if lower < current:
         _ctx_cap[model] = lower
@@ -215,11 +215,16 @@ async def current(num_ctx: int) -> dict:
     return state
 
 
+#: The context the server asks for unless told otherwise (its
+#: VIRTUAI_LOCAL_NUM_CTX); what the fit report estimates at.
+SERVER_NUM_CTX = 16_384
+
+
 def _num_ctx() -> int:
     try:
-        return int(os.environ.get("OLLAMA_NUM_CTX") or 8_192)
+        return int(os.environ.get("OLLAMA_NUM_CTX") or SERVER_NUM_CTX)
     except ValueError:
-        return 8_192
+        return SERVER_NUM_CTX
 
 
 async def snapshot() -> dict:

{virtuai_cli-1.0.3 → virtuai_cli-1.0.4}/src/virtuai_cli/pull.py

@@ -303,9 +303,18 @@ class _Session:
                     continue
             if sock.done.is_set():
                 break
+            # The beat's clock starts when it is sent, and a slow one gives up
+            # at two intervals: it used to wait out the client's 40 s read
+            # timeout, and the beats behind it waited too — on a laptop busy
+            # with a local model and a `find /`, long enough for the server
+            # to declare the job lost while it ran.
+            last_beat = time.monotonic()
             try:
-                reply = await self.http.post(f"{base}/heartbeat", json={"events": sock.take_events()})
-                last_beat = time.monotonic()
+                # wait_for bounds the whole request; httpx's timeout is per
+                # read, and a reply trickling in would get past it.
+                reply = await asyncio.wait_for(
+                    self.http.post(f"{base}/heartbeat", json={"events": sock.take_events()}),
+                    timeout=max(2.0, 2 * self.heartbeat))
                 if reply.status_code == 200 and reply.json().get("cancel"):
                     self.on_status("warn", f"Job {job_id} cancelled by the server")
                     if task is not None:
@@ -314,7 +323,7 @@ class _Session:
                         "v": 1, "id": job.get("id"), "type": "error", "error": "cancelled by the server"}
                     sock.done.set()
             except Exception as exc:
-                logger.debug("Heartbeat for %s failed: %s", job_id, exc)
+                logger.warning("Heartbeat for %s failed: %s", job_id, exc)
 
         final = sock.final or runner._busy_reply(job, "the runner finished this request without a reply") or {
             "v": 1, "id": job.get("id"), "type": "error", "error": "no reply"}

{virtuai_cli-1.0.3 → virtuai_cli-1.0.4}/src/virtuai_cli/runner.py

@@ -222,18 +222,20 @@ async def _handle_model_generate(frame: dict, ws, on_status: StatusCallback) ->
 
         tool_calls = list(getattr(gathered, "tool_calls", []) or []) if gathered else []
         usage = dict(getattr(gathered, "usage_metadata", None) or {}) if gathered else {}
-        await ws.send(json.dumps({
-            "v": 1, "id": fid, "type": "model_result",
-            "tool_calls": tool_calls, "usage": usage,
-        }))
-        # After the reply, so the user never waits on it: what the model
-        # really took decides the context of its next call here.
+        result = {"v": 1, "id": fid, "type": "model_result", "tool_calls": tool_calls, "usage": usage}
         if not omlx:
+            # What the model really took decides the context of its next call
+            # here, and the server hears it in this reply — not a minute later
+            # in a status report, by which time it may have sent a prompt this
+            # context would cut. A local /api/ps: milliseconds.
             from virtuai_cli import ollama_state
             try:
                 await asyncio.wait_for(ollama_state.after_call(model), timeout=5.0)
             except Exception:
                 pass
+            if ollama_state.context_for(model, None):
+                result["context_cap"] = ollama_state.context_for(model, None)
+        await ws.send(json.dumps(result))
     except Exception as exc:
         logger.warning("model_generate failed: %s", exc)
         await _error(str(exc))

{virtuai_cli-1.0.3 → virtuai_cli-1.0.4}/src/virtuai_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: virtuai-cli
-Version: 1.0.3
+Version: 1.0.4
 Summary: Run VirtuAI deep agents on your local machine
 Author-email: uCloudStore <lmoreno@ucloudstore.com>
 License: Proprietary
@@ -138,7 +138,7 @@ Each model call goes to the runner of the person the turn is for; if they have n
 
 At connect and then every minute: the models it has, the Ollama version, what is loaded and how much memory it really took. With `llmfit` installed it also estimates which pulled models fit this machine, so the server does not send a model to a runner that cannot hold it until it has actually run there. A model that spills to the CPU gets half the context on its next call on that machine (down to 4k).
 
-The context sent to Ollama is 8k by default (set on the server); `OLLAMA_NUM_CTX` overrides it on a machine with less memory. For qwen3 the runner turns Ollama's thinking mode off: left on, it reasoned for minutes before every tool call.
+The context sent to Ollama is 16k by default (set on the server). A model that does not fit at that context — `qwen3:14b` on a 16 GB Mac spills 7% to the CPU — is capped at half on that machine after its first call, and the server is told in the same reply: from then on it sends prompts that fit, compacting the conversation when they would not, instead of letting Ollama cut the start of the prompt. `OLLAMA_NUM_CTX` sets a lower context on a machine with little memory. For qwen3 the runner turns Ollama's thinking mode off: left on, it reasoned for minutes before every tool call.
 
 ### oMLX
 
@@ -163,7 +163,7 @@ mean "allow more commands at once" silently launched more browsers.
 | `VIRTUAI_SANDBOX_OWNER` | unset | The one person this runner serves, reported to the server so a browser action reaches *their* browser. Leave it unset on a laptop or a container shared by a team: the server then routes to it for anyone, which is the behaviour that existed before per-person sandboxes. Set it when a workspace has a sandbox per person, or actions land on whichever runner connected last. |
 | `VIRTUAI_SERVE_MODELS` | `own` | Whose local-model turns this runner answers: `own`, `workspace` or `none`. Also set by `--serve-models`. |
 | `OLLAMA_BASE_URL` | `http://localhost:11434` | Where to reach the local Ollama. |
-| `OLLAMA_NUM_CTX` | unset | The context to ask Ollama for, instead of the server's (8k). Lower it on a machine with little memory. |
+| `OLLAMA_NUM_CTX` | unset | The context to ask Ollama for, instead of the server's (16k). Lower it on a machine with little memory. |
 | `VIRTUAI_OMLX_URL` | from `~/.omlx/settings.json`, else `http://127.0.0.1:8000` | Where to reach oMLX. |
 | `OMLX_API_KEY` | from `~/.omlx/settings.json` | oMLX's API key. Stays on this machine. |
 

{virtuai_cli-1.0.3 → virtuai_cli-1.0.4}/tests/test_ollama_options.py

@@ -134,3 +134,18 @@ def test_the_runner_reports_whose_turns_it_serves(monkeypatch, env, expected):
     payload = asyncio.run(R._ready_payload(Path("/tmp")))
     assert payload["models"] == {"serve": expected}
 
+
+
+def test_a_model_that_spilled_reports_its_cap_in_the_same_reply(monkeypatch):
+    """The server hears it on this call, not a minute later in a status report."""
+    from virtuai_cli import ollama_state
+
+    async def spilled(model):
+        ollama_state._ctx_cap[model] = 8192
+
+    monkeypatch.setattr(ollama_state, "after_call", spilled)
+    monkeypatch.setattr(ollama_state, "_ctx_cap", {})
+    built, sent = _run({**FRAME, "num_ctx": 16384}, monkeypatch)
+    assert built["num_ctx"] == 16384 and sent[-1]["context_cap"] == 8192
+    built, sent = _run({**FRAME, "num_ctx": 16384}, monkeypatch)
+    assert built["num_ctx"] == 8192, "and asks Ollama for less from then on"