whileai 0.58__tar.gz → 0.60__tar.gz

{whileai-0.58/whileai.egg-info → whileai-0.60}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: whileai
-Version: 0.58
+Version: 0.60
 Summary: While Python SDK: trace ingestion and verification client, plus agent simulations grounded in your tools and system prompt.
 Author: While
 License: Apache-2.0
@@ -51,7 +51,7 @@ Two ways in, one engine. Give it the agent's tools and system prompt and it samp
 
 ![How a row gets made: the draw, the coverage grid, the search arms, the rollout, the split](docs/how-a-row-gets-made.svg)
 
-A situation is drawn across the world axes (from the agent's tools) and the human axes (from a separate writer). It fills a cell in the coverage grid, nudges the five search arms, and the agent plays it against a world that breaks on schedule. The row that comes out splits into `Task`, `Rollout`, `Judgment`, and `Marker`, and every training target is a projection of some of those four. The engine on one page, with references: [docs/engine.md](docs/engine.md), also at [while.ai/docs/engine](https://while.ai/docs/engine).
+A situation is drawn across the world axes (from the agent's tools) and the human axes (from a separate writer). It fills a cell in the coverage grid, nudges the five search arms, and the agent plays it against a world that breaks on schedule. The row that comes out splits into `Task`, `Rollout`, `Judgment`, and `Marker`, and every training target is a projection of some of those four. The engine on one page, with references: [docs/engine.md](docs/engine.md), also at [zeroproofai.com/docs/engine](https://zeroproofai.com/docs/engine).
 
 ## Overview
 
@@ -241,6 +241,33 @@ data = wai.simulate(
 )
 ```
 
+A model spec names the backend and the model. Four are built in:
+
+- `ollama:<model>`: a local Ollama server, no key.
+- `vllm:<model>@<url>`: any vLLM or OpenAI-compatible endpoint you serve.
+- `openai:<model>`: `OPENAI_API_KEY`, and `OPENAI_BASE_URL` for a
+  compatible endpoint that is not OpenAI's.
+- `anthropic:<model>`: the Claude Messages API on `ANTHROPIC_API_KEY`
+  (`WHILEAI_ANTHROPIC_API_KEY` overrides it).
+
+A spec works everywhere one is accepted: `agent=`, `simulator=` for the
+situation writer, `user_model=` for the simulated person, and `spec=` for the
+judge.
+
+```bash
+export ANTHROPIC_API_KEY=...
+```
+
+```python
+data = wai.simulate(
+    agent="anthropic:claude-haiku-4-5",
+    tools=my_tools,
+    system_prompt=my_system_prompt,
+    simulator="anthropic:claude-sonnet-5",  # the writer, on the same key
+    output="rollout.jsonl",
+)
+```
+
 ## Five calls
 
 Agent to gated dataset. Everything else in this README is one layer down.

{whileai-0.58 → whileai-0.60}/README.md

@@ -15,7 +15,7 @@ Two ways in, one engine. Give it the agent's tools and system prompt and it samp
 
 ![How a row gets made: the draw, the coverage grid, the search arms, the rollout, the split](docs/how-a-row-gets-made.svg)
 
-A situation is drawn across the world axes (from the agent's tools) and the human axes (from a separate writer). It fills a cell in the coverage grid, nudges the five search arms, and the agent plays it against a world that breaks on schedule. The row that comes out splits into `Task`, `Rollout`, `Judgment`, and `Marker`, and every training target is a projection of some of those four. The engine on one page, with references: [docs/engine.md](docs/engine.md), also at [while.ai/docs/engine](https://while.ai/docs/engine).
+A situation is drawn across the world axes (from the agent's tools) and the human axes (from a separate writer). It fills a cell in the coverage grid, nudges the five search arms, and the agent plays it against a world that breaks on schedule. The row that comes out splits into `Task`, `Rollout`, `Judgment`, and `Marker`, and every training target is a projection of some of those four. The engine on one page, with references: [docs/engine.md](docs/engine.md), also at [zeroproofai.com/docs/engine](https://zeroproofai.com/docs/engine).
 
 ## Overview
 
@@ -205,6 +205,33 @@ data = wai.simulate(
 )
 ```
 
+A model spec names the backend and the model. Four are built in:
+
+- `ollama:<model>`: a local Ollama server, no key.
+- `vllm:<model>@<url>`: any vLLM or OpenAI-compatible endpoint you serve.
+- `openai:<model>`: `OPENAI_API_KEY`, and `OPENAI_BASE_URL` for a
+  compatible endpoint that is not OpenAI's.
+- `anthropic:<model>`: the Claude Messages API on `ANTHROPIC_API_KEY`
+  (`WHILEAI_ANTHROPIC_API_KEY` overrides it).
+
+A spec works everywhere one is accepted: `agent=`, `simulator=` for the
+situation writer, `user_model=` for the simulated person, and `spec=` for the
+judge.
+
+```bash
+export ANTHROPIC_API_KEY=...
+```
+
+```python
+data = wai.simulate(
+    agent="anthropic:claude-haiku-4-5",
+    tools=my_tools,
+    system_prompt=my_system_prompt,
+    simulator="anthropic:claude-sonnet-5",  # the writer, on the same key
+    output="rollout.jsonl",
+)
+```
+
 ## Five calls
 
 Agent to gated dataset. Everything else in this README is one layer down.

{whileai-0.58 → whileai-0.60}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "whileai"
-version = "0.58"
+version = "0.60"
 description = "While Python SDK: trace ingestion and verification client, plus agent simulations grounded in your tools and system prompt."
 readme = "README.md"
 license = {text = "Apache-2.0"}

{whileai-0.58 → whileai-0.60}/whileai/auth.py

@@ -34,12 +34,38 @@ from pathlib import Path
 from whileai._env import env_name, getenv
 
 DEFAULT_API_URL = "https://api.zeroproofai.com"
+SIGN_IN_URL = "https://www.zeroproofai.com/sign-in"
+#: the trial allowance the gate hands out, used when the reply does not say
+DEFAULT_TRIAL_INPUT_TOKENS = 25_000
+#: Input tokens one hosted situation spends, measured on a 4-tool spec: a
+#: 12-situation run of one cost 28,490, so round it to 2,000 a situation.
+#: The trial allowance is small enough that the count is the first thing
+#: anyone needs to know about it.
+INPUT_TOKENS_PER_SITUATION = 2_000
 
 
 class LoginError(RuntimeError):
     pass
 
 
+def trial_situations(daily_input_tokens: float | None = None) -> int:
+    """About how many hosted situations a trial day buys, in round numbers."""
+    tokens = float(daily_input_tokens or DEFAULT_TRIAL_INPUT_TOKENS)
+    return max(1, int(tokens / INPUT_TOKENS_PER_SITUATION))
+
+
+def trial_note(daily_input_tokens: float | None = None) -> str:
+    """The two facts a trial key needs before its first hosted run: how
+    far the daily allowance goes, and the offline writer that has no
+    allowance at all."""
+    return (
+        f"That is about {trial_situations(daily_input_tokens)} hosted situations a day "
+        f"(a 4-tool spec spends around {INPUT_TOKENS_PER_SITUATION:,} input tokens per "
+        "situation); simulate(..., simulator=False) writes situations offline with no quota "
+        f"and no network; signing in once at {SIGN_IN_URL} lifts the limit."
+    )
+
+
 def _api_url() -> str:
     return getenv("API_URL", DEFAULT_API_URL).rstrip("/")
 
@@ -291,6 +317,7 @@ def signup(email: str, *, name: str | None = None, out: Callable[[str], None] |
                 f"output tokens a day, {int(trial.get('storage_bytes', 104857600)) // 1048576} MB, "
                 f"{trial.get('datasets', 10)} datasets, expires {str(trial.get('expires_at', ''))[:10]}."
             )
+            say(trial_note(trial.get("daily_input_tokens")))
             say(
                 trial.get("lift")
                 or "Sign in once at https://www.zeroproofai.com/sign-in with an email code to lift trial limits."
@@ -321,7 +348,12 @@ def logout() -> bool:
 
 
 def status() -> dict:
-    """What the SDK would use right now, with the key masked."""
+    """What the SDK would use right now, with the key masked.
+
+    On a trial key ``trial_note`` says how many hosted situations the
+    daily allowance covers and names the offline writer that has none.
+    ``tier`` comes from ``GET /me``, so reading it costs one call.
+    """
     env = getenv("API_KEY")
     env_var = env_name("API_KEY")
     saved = _read(credentials_path()) or {}
@@ -354,4 +386,5 @@ def status() -> dict:
                 trial = me.get("trial") or {}
                 out["trial_expires_at"] = trial.get("expires_at")
                 out["lift"] = trial.get("lift")
+                out["trial_note"] = trial_note(trial.get("daily_input_tokens"))
     return out

{whileai-0.58 → whileai-0.60}/whileai/cli.py

@@ -79,6 +79,8 @@ def main(argv: list[str] | None = None) -> int:
     if args.command == "status":
         shown = auth.status()
         print(json.dumps(shown, indent=2))
+        if shown.get("trial_note"):
+            print(shown["trial_note"])
         if not shown.get("configured"):
             print(
                 "no API key configured: run `whileai login` or set WHILEAI_API_KEY",

{whileai-0.58 → whileai-0.60}/whileai/simulations/__init__.py

@@ -147,7 +147,14 @@ from .score.optimize import (
 )
 from .score.pairwise import judge_pairs, pairwise_judge
 from .score.passat import PassAt, pass_at
-from .score.preflight import FAILURE_CLASSES, classify_failure, dataset_report, preflight
+from .score.preflight import (
+    FAILURE_CLASSES,
+    classify_failure,
+    coverage_gap,
+    dataset_report,
+    format_coverage_gap,
+    preflight,
+)
 from .score.privileged import format_leak_report, leak_report
 from .score.publish_gate import PublishGateError, calibrate, publish_gate
 from .score.quality import rank_rows, score_row
@@ -252,6 +259,7 @@ __all__ = [
     "conduct_grade",
     "connect",
     "conversation",
+    "coverage_gap",
     "curriculum",
     "cut",
     "cuts",
@@ -277,6 +285,7 @@ __all__ = [
     "filter_rl_rows",
     "flaw_rows",
     "format_audit",
+    "format_coverage_gap",
     "format_curriculum",
     "format_delta_report",
     "format_hack_monitor",

{whileai-0.58 → whileai-0.60}/whileai/simulations/data.py

@@ -670,6 +670,10 @@ class SimulationData:
         ``ScoredData.rows`` is a list attribute and this used to be a
         method only, so ``for r in data.rows`` raised ``TypeError``
         without hinting at the missing parentheses.
+
+        The other direction still bites: ``ScoredData.rows``, what
+        ``evaluate`` and ``grade`` hand back, is a plain list, so
+        ``scored.rows()`` is a ``TypeError``. Method here, list there.
         """
         return RowList(export_row(t) for t in self.trajectories)
 

{whileai-0.58 → whileai-0.60}/whileai/simulations/generate/agents.py

@@ -15,8 +15,14 @@ from typing import Any
 from urllib.parse import urlparse
 
 from whileai._env import getenv
+from whileai.auth import SIGN_IN_URL
 
 from ..world.sandbox import MockEnvironment
+from .anthropic_backend import ANTHROPIC_BASE_URL, is_anthropic_url
+from .anthropic_backend import DEFAULT_MODEL as ANTHROPIC_DEFAULT_MODEL
+from .anthropic_backend import complete as anthropic_complete
+from .anthropic_backend import missing_key as missing_anthropic_key
+from .anthropic_backend import resolve_key as anthropic_key
 from .diversity import running_turn_mean, sample_turn_budget
 from .usage_meter import report_usage
 
@@ -60,7 +66,7 @@ current_rollout = _CurrentRollout()
 
 
 def parse_backend_spec(spec: str) -> tuple[str, str]:
-    """Return (base_url, model) for ollama:/vllm:/openai: specs."""
+    """Return (base_url, model) for ollama:/vllm:/openai:/anthropic: specs."""
     kind, _, rest = str(spec).partition(":")
     if kind == "ollama":
         return "http://localhost:11434/v1", rest or "llama3.1:8b"
@@ -74,9 +80,14 @@ def parse_backend_spec(spec: str) -> tuple[str, str]:
             os.environ.get("OPENAI_BASE_URL") or "https://api.openai.com/v1",
             rest or "gpt-4o-mini",
         )
+    if kind == "anthropic":
+        # The Messages API, on ANTHROPIC_API_KEY. The URL is fixed, so the
+        # spec is just the model name and every caller (writer, user model,
+        # agent, judge) records that name the way the other backends do.
+        return ANTHROPIC_BASE_URL, rest or ANTHROPIC_DEFAULT_MODEL
     raise ValueError(
         f"unsupported backend spec {spec!r}; use ollama:<model>, "
-        "vllm:<model>@<url>, or openai:<model>"
+        "vllm:<model>@<url>, openai:<model>, or anthropic:<model>"
     )
 
 
@@ -227,6 +238,8 @@ def resolve_completion_key(base_url: str | None = None, api_key: str | None = No
     """
     if api_key:
         return str(api_key).strip()
+    if is_anthropic_url(base_url):
+        return anthropic_key()
     vllm = str(os.environ.get("VLLM_API_KEY") or "").strip()
     if not base_url:
         # no URL means the default agent, whichever route that resolves to
@@ -264,6 +277,8 @@ def missing_hosted_key(base_url: str | None = None, api_key: str | None = None)
     endpoints run without one. Without this a bring-your-own run with no
     key spent its whole time budget on 401s and returned nothing.
     """
+    if is_anthropic_url(base_url):
+        return missing_anthropic_key(api_key)
     key = resolve_completion_key(base_url, api_key)
     if key:
         return None
@@ -285,12 +300,20 @@ MISSING_HOSTED_KEY = (
     "VLLM_API_KEY for the shared pool."
 )
 QUOTA_MARK = "quota exceeded"
+#: What to do about a spent daily allowance. A trial day is about 12
+#: hosted situations (``auth.trial_note``), which one real run spends, so
+#: the error names the writer that has no allowance and the sign-in that
+#: lifts the limit instead of leaving the run dead with a number.
+QUOTA_FIX = (
+    "simulate(..., simulator=False) writes the situations offline with no quota and no "
+    f"network; signing in once at {SIGN_IN_URL} lifts a trial key's daily limit."
+)
 
 
 def _quota_error(status: int, body: str) -> str | None:
     """The proxy's 429 for a spent daily allowance, or None. Not transient:
     every later call today answers the same, so the run stops instead of
-    retrying into the clock."""
+    retrying into the clock. Carries ``QUOTA_FIX``, the two ways on."""
     if int(status) != 429:
         return None
     text = str(body or "")
@@ -300,7 +323,8 @@ def _quota_error(status: int, body: str) -> str | None:
         msg = json.loads(text).get("error", {}).get("message") or text
     except (ValueError, AttributeError):
         msg = text
-    return f"Hosted model daily {msg[msg.lower().find('quota') :]}"
+    head = f"Hosted model daily {msg[msg.lower().find('quota') :]}".rstrip(". ")
+    return f"{head}. {QUOTA_FIX}"
 
 
 def _client_metered(base_url: str | None) -> bool:
@@ -541,7 +565,30 @@ def complete(
     reply then carries ``_logprobs`` (sum, n, and with ``"tokens"`` the
     per-token list). ``_finish_reason`` is always set from the first choice.
     A server that rejects ``logprobs`` gets the request again without it.
+
+    An ``anthropic:`` spec goes to the Messages API instead, translated to
+    and from this same shape by ``anthropic_backend``. That API returns no
+    log-probabilities, so ``logprobs`` yields no ``_logprobs`` there.
     """
+    if is_anthropic_url(base_url):
+        # The Messages API, translated at the boundary. It runs before the
+        # context squeeze below because that budget is sized to hosted Qwen's
+        # 4k window, not to a 200k one; report_usage stays out of it because a
+        # bring-your-own model is the customer's own bill.
+        reply = anthropic_complete(
+            base_url,
+            model,
+            messages,
+            tools=tools,
+            api_key=api_key,
+            temperature=temperature,
+            max_tokens=max_tokens,
+            timeout=timeout,
+            n=n,
+            extra=extra,
+        )
+        _trim_length_cut({"finish_reason": reply.get("_finish_reason"), "message": reply})
+        return reply
     key = resolve_completion_key(base_url, api_key)
     auth_err = missing_hosted_key(base_url, key)
     if auth_err: