whileai 0.59__tar.gz → 0.61__tar.gz

{whileai-0.59/whileai.egg-info → whileai-0.61}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: whileai
-Version: 0.59
+Version: 0.61
 Summary: While Python SDK: trace ingestion and verification client, plus agent simulations grounded in your tools and system prompt.
 Author: While
 License: Apache-2.0
@@ -41,7 +41,9 @@ The While Python SDK. One package, two importable modules:
 - `whileai`: the platform client. OTLP trace ingest and trace-dataset listing against the token gate.
 - `whileai.simulations`: post-training data for an agent. Give it the agent's traces, or its tools and system prompt; it simulates the situations, the people, and the world, plays the agent through multi-turn tool-calling conversations, and returns rows for your grader.
 
-**Renamed.** This SDK was `zeroproof` (ZeroProof is now While). `pip install zeroproof` still works: it installs `whileai`, and `import zeroproof` (or the older `zeroproof_simulations`) resolves to the same modules with a deprecation warning. `ZEROPROOF_*` environment variables and a saved `~/.zeroproof/credentials.json` are still read. Change the import when you can; new releases land under `whileai`. zp, ZeroProof and While all name this one product: the package is `whileai`, the import is `whileai.simulations`, keys start with `zp_`. A machine with the old package still picks up `~/.zeroproof/credentials.json`; set `WHILEAI_HOME` to a fresh directory to isolate a new account from it.
+Have an agent and want a pass rate with an interval? Start at [docs/evals.md](docs/evals.md) (offline, seconds, `coverage_gap` names what your tests miss).
+
+**Renamed.** This SDK was `zeroproof` (ZeroProof is now While). `pip install zeroproof` still works: it installs `whileai`, and `import zeroproof` (or the older `zeroproof_simulations`) resolves to the same modules with a deprecation warning. `ZEROPROOF_*` environment variables and a saved `~/.zeroproof/credentials.json` are still read. Change the import when you can; new releases land under `whileai`. zp, ZeroProof and While all name this one product: the package is `whileai`, the import is `whileai.simulations`, keys start with `zp_`. `zp`, `wai` and `whileai` run the same CLI, so `zp login` and `whileai login` do the same thing (the help text says `whileai`). A machine with the old package still picks up `~/.zeroproof/credentials.json`; set `WHILEAI_HOME` to a fresh directory to isolate a new account from it.
 
 Releases of `whileai` before 0.3 were an unrelated encrypted agent-to-agent messaging client. That code was removed in 0.04; pin `whileai<0.3` if you still depend on it.
 
@@ -51,7 +53,7 @@ Two ways in, one engine. Give it the agent's tools and system prompt and it samp
 
 ![How a row gets made: the draw, the coverage grid, the search arms, the rollout, the split](docs/how-a-row-gets-made.svg)
 
-A situation is drawn across the world axes (from the agent's tools) and the human axes (from a separate writer). It fills a cell in the coverage grid, nudges the five search arms, and the agent plays it against a world that breaks on schedule. The row that comes out splits into `Task`, `Rollout`, `Judgment`, and `Marker`, and every training target is a projection of some of those four. The engine on one page, with references: [docs/engine.md](docs/engine.md), also at [while.ai/docs/engine](https://while.ai/docs/engine).
+A situation is drawn across the world axes (from the agent's tools) and the human axes (from a separate writer). It fills a cell in the coverage grid, nudges the five search arms, and the agent plays it against a world that breaks on schedule. The row that comes out splits into `Task`, `Rollout`, `Judgment`, and `Marker`, and every training target is a projection of some of those four. The engine on one page, with references: [docs/engine.md](docs/engine.md), also at [zeroproofai.com/docs/engine](https://zeroproofai.com/docs/engine).
 
 ## Overview
 
@@ -195,7 +197,9 @@ that reads the trajectory, run the asks `k` times each, and read pass@1
 with its interval. Offline first, then the hosted writer. The how-to is
 [docs/evals.md](docs/evals.md); the runnable version is
 [`recipes/02-measure/eval-your-agent`](recipes/02-measure/eval-your-agent),
-which ends at a CI gate, not a push.
+which ends at a CI gate, not a push. `whileai init-evals` writes those
+four files for you, wired to the tools, system prompt and callable it
+finds in the project, and prints what it picked.
 
 ```python
 data = wai.simulate(
@@ -241,6 +245,33 @@ data = wai.simulate(
 )
 ```
 
+A model spec names the backend and the model. Four are built in:
+
+- `ollama:<model>`: a local Ollama server, no key.
+- `vllm:<model>@<url>`: any vLLM or OpenAI-compatible endpoint you serve.
+- `openai:<model>`: `OPENAI_API_KEY`, and `OPENAI_BASE_URL` for a
+  compatible endpoint that is not OpenAI's.
+- `anthropic:<model>`: the Claude Messages API on `ANTHROPIC_API_KEY`
+  (`WHILEAI_ANTHROPIC_API_KEY` overrides it).
+
+A spec works everywhere one is accepted: `agent=`, `simulator=` for the
+situation writer, `user_model=` for the simulated person, and `spec=` for the
+judge.
+
+```bash
+export ANTHROPIC_API_KEY=...
+```
+
+```python
+data = wai.simulate(
+    agent="anthropic:claude-haiku-4-5",
+    tools=my_tools,
+    system_prompt=my_system_prompt,
+    simulator="anthropic:claude-sonnet-5",  # the writer, on the same key
+    output="rollout.jsonl",
+)
+```
+
 ## Five calls
 
 Agent to gated dataset. Everything else in this README is one layer down.

{whileai-0.59 → whileai-0.61}/README.md

@@ -5,7 +5,9 @@ The While Python SDK. One package, two importable modules:
 - `whileai`: the platform client. OTLP trace ingest and trace-dataset listing against the token gate.
 - `whileai.simulations`: post-training data for an agent. Give it the agent's traces, or its tools and system prompt; it simulates the situations, the people, and the world, plays the agent through multi-turn tool-calling conversations, and returns rows for your grader.
 
-**Renamed.** This SDK was `zeroproof` (ZeroProof is now While). `pip install zeroproof` still works: it installs `whileai`, and `import zeroproof` (or the older `zeroproof_simulations`) resolves to the same modules with a deprecation warning. `ZEROPROOF_*` environment variables and a saved `~/.zeroproof/credentials.json` are still read. Change the import when you can; new releases land under `whileai`. zp, ZeroProof and While all name this one product: the package is `whileai`, the import is `whileai.simulations`, keys start with `zp_`. A machine with the old package still picks up `~/.zeroproof/credentials.json`; set `WHILEAI_HOME` to a fresh directory to isolate a new account from it.
+Have an agent and want a pass rate with an interval? Start at [docs/evals.md](docs/evals.md) (offline, seconds, `coverage_gap` names what your tests miss).
+
+**Renamed.** This SDK was `zeroproof` (ZeroProof is now While). `pip install zeroproof` still works: it installs `whileai`, and `import zeroproof` (or the older `zeroproof_simulations`) resolves to the same modules with a deprecation warning. `ZEROPROOF_*` environment variables and a saved `~/.zeroproof/credentials.json` are still read. Change the import when you can; new releases land under `whileai`. zp, ZeroProof and While all name this one product: the package is `whileai`, the import is `whileai.simulations`, keys start with `zp_`. `zp`, `wai` and `whileai` run the same CLI, so `zp login` and `whileai login` do the same thing (the help text says `whileai`). A machine with the old package still picks up `~/.zeroproof/credentials.json`; set `WHILEAI_HOME` to a fresh directory to isolate a new account from it.
 
 Releases of `whileai` before 0.3 were an unrelated encrypted agent-to-agent messaging client. That code was removed in 0.04; pin `whileai<0.3` if you still depend on it.
 
@@ -15,7 +17,7 @@ Two ways in, one engine. Give it the agent's tools and system prompt and it samp
 
 ![How a row gets made: the draw, the coverage grid, the search arms, the rollout, the split](docs/how-a-row-gets-made.svg)
 
-A situation is drawn across the world axes (from the agent's tools) and the human axes (from a separate writer). It fills a cell in the coverage grid, nudges the five search arms, and the agent plays it against a world that breaks on schedule. The row that comes out splits into `Task`, `Rollout`, `Judgment`, and `Marker`, and every training target is a projection of some of those four. The engine on one page, with references: [docs/engine.md](docs/engine.md), also at [while.ai/docs/engine](https://while.ai/docs/engine).
+A situation is drawn across the world axes (from the agent's tools) and the human axes (from a separate writer). It fills a cell in the coverage grid, nudges the five search arms, and the agent plays it against a world that breaks on schedule. The row that comes out splits into `Task`, `Rollout`, `Judgment`, and `Marker`, and every training target is a projection of some of those four. The engine on one page, with references: [docs/engine.md](docs/engine.md), also at [zeroproofai.com/docs/engine](https://zeroproofai.com/docs/engine).
 
 ## Overview
 
@@ -159,7 +161,9 @@ that reads the trajectory, run the asks `k` times each, and read pass@1
 with its interval. Offline first, then the hosted writer. The how-to is
 [docs/evals.md](docs/evals.md); the runnable version is
 [`recipes/02-measure/eval-your-agent`](recipes/02-measure/eval-your-agent),
-which ends at a CI gate, not a push.
+which ends at a CI gate, not a push. `whileai init-evals` writes those
+four files for you, wired to the tools, system prompt and callable it
+finds in the project, and prints what it picked.
 
 ```python
 data = wai.simulate(
@@ -205,6 +209,33 @@ data = wai.simulate(
 )
 ```
 
+A model spec names the backend and the model. Four are built in:
+
+- `ollama:<model>`: a local Ollama server, no key.
+- `vllm:<model>@<url>`: any vLLM or OpenAI-compatible endpoint you serve.
+- `openai:<model>`: `OPENAI_API_KEY`, and `OPENAI_BASE_URL` for a
+  compatible endpoint that is not OpenAI's.
+- `anthropic:<model>`: the Claude Messages API on `ANTHROPIC_API_KEY`
+  (`WHILEAI_ANTHROPIC_API_KEY` overrides it).
+
+A spec works everywhere one is accepted: `agent=`, `simulator=` for the
+situation writer, `user_model=` for the simulated person, and `spec=` for the
+judge.
+
+```bash
+export ANTHROPIC_API_KEY=...
+```
+
+```python
+data = wai.simulate(
+    agent="anthropic:claude-haiku-4-5",
+    tools=my_tools,
+    system_prompt=my_system_prompt,
+    simulator="anthropic:claude-sonnet-5",  # the writer, on the same key
+    output="rollout.jsonl",
+)
+```
+
 ## Five calls
 
 Agent to gated dataset. Everything else in this README is one layer down.

{whileai-0.59 → whileai-0.61}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "whileai"
-version = "0.59"
+version = "0.61"
 description = "While Python SDK: trace ingestion and verification client, plus agent simulations grounded in your tools and system prompt."
 readme = "README.md"
 license = {text = "Apache-2.0"}
@@ -38,6 +38,10 @@ Datasets = "https://huggingface.co/datasets/zero-proof-ai/agent-simulations"
 
 [project.scripts]
 whileai = "whileai.cli:main"
+# same CLI under the two names people type: the product is "zp"
+# (keys start with zp_) and "wai" is the short import alias.
+zp = "whileai.cli:main"
+wai = "whileai.cli:main"
 whileai-simulations = "whileai.simulations.score.quality:main"
 
 [project.optional-dependencies]

{whileai-0.59 → whileai-0.61}/whileai/auth.py

@@ -34,12 +34,59 @@ from pathlib import Path
 from whileai._env import env_name, getenv
 
 DEFAULT_API_URL = "https://api.zeroproofai.com"
+SIGN_IN_URL = "https://www.zeroproofai.com/sign-in"
+#: the trial allowance the gate hands out, used when the reply does not say
+DEFAULT_TRIAL_INPUT_TOKENS = 25_000
+#: Input tokens one hosted situation spends, measured on a 4-tool spec: a
+#: 12-situation run of one cost 28,490, so round it to 2,000 a situation.
+#: The trial allowance is small enough that the count is the first thing
+#: anyone needs to know about it.
+INPUT_TOKENS_PER_SITUATION = 2_000
 
 
 class LoginError(RuntimeError):
     pass
 
 
+def trial_situations(daily_input_tokens: float | None = None) -> int:
+    """About how many hosted situations a trial day buys, in round numbers."""
+    tokens = float(daily_input_tokens or DEFAULT_TRIAL_INPUT_TOKENS)
+    return max(1, int(tokens / INPUT_TOKENS_PER_SITUATION))
+
+
+def trial_note(daily_input_tokens: float | None = None) -> str:
+    """The two facts a trial key needs before its first hosted run: how
+    far the daily allowance goes, and the offline writer that has no
+    allowance at all."""
+    return (
+        f"That is about {trial_situations(daily_input_tokens)} hosted situations a day "
+        f"(a 4-tool spec spends around {INPUT_TOKENS_PER_SITUATION:,} input tokens per "
+        "situation); simulate(..., simulator=False) writes situations offline with no quota "
+        f"and no network; signing in once at {SIGN_IN_URL} lifts the limit."
+    )
+
+
+def trial_prerun_note() -> str | None:
+    """The one line a trial key needs before a hosted run spends it, or ``None``.
+
+    Read from the tier the credentials file recorded at sign-up or at the
+    last ``whileai status`` / ``whileai login``, so a run can say this
+    without a network call. A key from the environment has no recorded
+    tier, so this says nothing rather than guess at one.
+    """
+    if getenv("API_KEY"):
+        return None
+    saved = _read(credentials_path()) or {}
+    if not saved.get("api_key") or str(saved.get("tier") or "") != "trial":
+        return None
+    tokens = int(float(saved.get("daily_input_tokens") or DEFAULT_TRIAL_INPUT_TOKENS))
+    return (
+        f"trial key: the hosted writer covers about {trial_situations(tokens)} situations a day "
+        f"({tokens // 1000}k input tokens); simulator=False writes them offline with no quota; "
+        f"sign in once at {SIGN_IN_URL} to lift it"
+    )
+
+
 def _api_url() -> str:
     return getenv("API_URL", DEFAULT_API_URL).rstrip("/")
 
@@ -86,6 +133,37 @@ def _read(path: Path) -> dict | None:
     return data if isinstance(data, dict) else None
 
 
+def _tier_fields(payload: dict) -> dict:
+    """The tier facts worth keeping next to the key, from ``/signup`` or ``/me``."""
+    tier = str((payload or {}).get("tier") or "").strip()
+    if not tier:
+        return {}
+    fields: dict = {"tier": tier}
+    trial = (payload or {}).get("trial") or {}
+    if isinstance(trial, dict):
+        if trial.get("daily_input_tokens") is not None:
+            fields["daily_input_tokens"] = trial["daily_input_tokens"]
+        if trial.get("expires_at"):
+            fields["expires_at"] = str(trial["expires_at"])
+    return fields
+
+
+def remember_account(payload: dict) -> None:
+    """Record a ``/signup`` or ``/me`` reply's tier in the credentials file.
+
+    Only for the key that file holds: a key from the environment may belong
+    to another account, and writing its tier here would mislabel this one.
+    """
+    fields = _tier_fields(payload)
+    saved = _read(credentials_path())
+    if not fields or not saved or not saved.get("api_key"):
+        return
+    if all(saved.get(key) == value for key, value in fields.items()):
+        return
+    saved.update(fields)
+    _write_private(credentials_path(), saved)
+
+
 def stored_api_key() -> str | None:
     """The key saved by ``whileai login``, or ``None``."""
     data = _read(credentials_path())
@@ -238,6 +316,10 @@ def login(
             )
             with contextlib.suppress(OSError):
                 _pending_path().unlink()
+            # the tier the key carries, saved next to it, so a run can
+            # name a trial limit before it spends one
+            with contextlib.suppress(LoginError, OSError, ValueError):
+                remember_account(account(data["api_key"]))
             say(f"Logged in. Key saved to {credentials_path()}")
             return data["api_key"]
         error = data.get("error", "")
@@ -280,6 +362,9 @@ def signup(email: str, *, name: str | None = None, out: Callable[[str], None] |
                 "user_id": data.get("user_id"),
                 "email": data.get("email", email),
                 "created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
+                # so simulate() can warn about the trial before a hosted
+                # run spends it, without a call to /me
+                **_tier_fields(data),
             },
         )
         say(f"Account created for {data.get('email', email)}. Key saved to {credentials_path()}")
@@ -291,6 +376,7 @@ def signup(email: str, *, name: str | None = None, out: Callable[[str], None] |
                 f"output tokens a day, {int(trial.get('storage_bytes', 104857600)) // 1048576} MB, "
                 f"{trial.get('datasets', 10)} datasets, expires {str(trial.get('expires_at', ''))[:10]}."
             )
+            say(trial_note(trial.get("daily_input_tokens")))
             say(
                 trial.get("lift")
                 or "Sign in once at https://www.zeroproofai.com/sign-in with an email code to lift trial limits."
@@ -321,7 +407,12 @@ def logout() -> bool:
 
 
 def status() -> dict:
-    """What the SDK would use right now, with the key masked."""
+    """What the SDK would use right now, with the key masked.
+
+    On a trial key ``trial_note`` says how many hosted situations the
+    daily allowance covers and names the offline writer that has none.
+    ``tier`` comes from ``GET /me``, so reading it costs one call.
+    """
     env = getenv("API_KEY")
     env_var = env_name("API_KEY")
     saved = _read(credentials_path()) or {}
@@ -350,8 +441,11 @@ def status() -> dict:
             out["tier"] = f"unknown ({err})"
         else:
             out["tier"] = me.get("tier")
+            if not env:
+                remember_account(me)
             if me.get("tier") == "trial":
                 trial = me.get("trial") or {}
                 out["trial_expires_at"] = trial.get("expires_at")
                 out["lift"] = trial.get("lift")
+                out["trial_note"] = trial_note(trial.get("daily_input_tokens"))
     return out

{whileai-0.59 → whileai-0.61}/whileai/cli.py

@@ -1,4 +1,4 @@
-"""``whileai`` command line: login, signup, logout, status."""
+"""``whileai`` command line: login, signup, logout, status, init-evals."""
 
 from __future__ import annotations
 
@@ -7,6 +7,7 @@ import json
 import sys
 
 from . import auth
+from .init_evals import add_arguments as init_evals_args
 
 
 def main(argv: list[str] | None = None) -> int:
@@ -33,6 +34,12 @@ def main(argv: list[str] | None = None) -> int:
     sub.add_parser("logout", help="delete the saved key")
     sub.add_parser("status", help="show which key the SDK will use")
 
+    p_init = sub.add_parser(
+        "init-evals",
+        help="write an eval harness (agent, judge, run, test) wired to this project",
+    )
+    init_evals_args(p_init)
+
     p_purge = sub.add_parser(
         "purge", help="delete an agent's traces, datasets and record, or empty datasets"
     )
@@ -72,6 +79,17 @@ def main(argv: list[str] | None = None) -> int:
             return 1
         return 0
 
+    if args.command == "init-evals":
+        from .init_evals import init_evals
+
+        return init_evals(
+            agent=args.agent,
+            tools=args.tools,
+            system_prompt=args.system_prompt,
+            out=args.dir,
+            force=args.force,
+        )
+
     if args.command == "logout":
         print("Logged out." if auth.logout() else "No saved key.")
         return 0
@@ -79,6 +97,8 @@ def main(argv: list[str] | None = None) -> int:
     if args.command == "status":
         shown = auth.status()
         print(json.dumps(shown, indent=2))
+        if shown.get("trial_note"):
+            print(shown["trial_note"])
         if not shown.get("configured"):
             print(
                 "no API key configured: run `whileai login` or set WHILEAI_API_KEY",