jevkit-runtime 0.1.0__tar.gz → 0.2.0__tar.gz

{jevkit_runtime-0.1.0 → jevkit_runtime-0.2.0}/.github/workflows/downstream.yml

@@ -14,8 +14,6 @@ permissions:
   contents: read
 jobs:
   consumers:
-    # Enabled after all five migration PRs land; manual runs remain available.
-    if: vars.JEVKIT_CONSUMERS_READY == 'true' || github.event_name == 'workflow_dispatch'
     runs-on: ubuntu-latest
     strategy:
       fail-fast: false

jevkit_runtime-0.2.0/PKG-INFO

@@ -0,0 +1,94 @@
+Metadata-Version: 2.5
+Name: jevkit-runtime
+Version: 0.2.0
+Summary: Shared transport, configuration, caching, and accounting for JevKit tools
+Project-URL: Homepage, https://github.com/keltokhy/jevkit-core
+Project-URL: Issues, https://github.com/keltokhy/jevkit-core/issues
+Author: Khaled Eltokhy
+License-Expression: MIT
+License-File: LICENSE
+Requires-Python: >=3.10
+Requires-Dist: httpx>=0.27
+Provides-Extra: http2
+Requires-Dist: httpx[http2]>=0.27; extra == 'http2'
+Description-Content-Type: text/markdown
+
+# JevKit core
+
+Distribution **`jevkit-runtime`**, import **`jevkit_runtime`**. The PyPI name `jevkit-core` belongs
+to a different project.
+
+One request pipeline, one answer store, one provider catalog for jgrep, jsort, jlink, jselect,
+and jcol. Each tool remains its own package and repository; the core imports none of them, and a
+tool's adapter is a few lines naming which providers it offers.
+
+## What a tool gets
+
+```python
+from jevkit_runtime import AnswerStore, Client, catalog, resolve
+
+PROVIDERS = catalog("typesafe", "openrouter", "gateway")
+backend = resolve(PROVIDERS, name=None, model=None)          # or JEV_API / JEV_MODEL, else the first configured
+async with Client(backend, store=AnswerStore()) as client:
+    answers = await client.ask(state, {"q": {"type": "noul", "instructions": "..."}})
+```
+
+`Client.ask` does the whole thing: computes each question's identity, serves what the store already
+knows, joins an identical request already in flight, sends only the misses, validates the entire
+response before storing any of it, and meters the call before validation so a billed but malformed
+answer still counts. It returns `Answers`, a dict by question id whose `origins` say who answered
+each one and whether it came from the API, the store, or a shared call. Per-call policy is keyword
+arguments: `allow_paid=False` for cache-only runs, `on_cost` for the caller who should be charged,
+`hedge_after` to resend a slow call, and `keys` for callers whose reuse unit is not the request.
+HTTP/2 is used whenever the `http2` extra is installed.
+
+| Module | Owns |
+|---|---|
+| `settings.py` | Every environment and filesystem convention, read in one place: `XDG_*`, `JEV_API`, `JEV_URL`, `JEV_MODEL`, `JEV_PRICE_PER_MTOK`, provider keys and URL files |
+| `providers.py` | The catalog (`Provider`), a tool's selection of it or its own entries, and `resolve()` to one `Backend`: endpoint, model, key |
+| `protocol.py` | Request bodies, typed answer validation (`noul`, `choice`, `score`), usage parsing, answer identity, provenance |
+| `transport.py` | One HTTP call with a total deadline, retries with backoff and `Retry-After`, structured status errors |
+| `store.py` | SQLite answers with their provenance in one row, one versioned schema |
+| `client.py` | The pipeline above, request sharing, hedging |
+| `meter.py` | Calls, cache hits, retries, hedges, tokens, cost, and which models actually answered |
+| `errors.py` | `JevError`, `JevFatal`, `JevBudgetExceeded`, `RequestExhausted`, `ProviderError`, `ProviderFatal` |
+
+## Conventions every tool shares
+
+- **Answer identity** is `answer_key(backend, state, question)`: provider, endpoint, model, state and
+  question. An answer from one provider or model is never served for another.
+- **The store** lives at `$XDG_CACHE_HOME/jev/answers.sqlite` (default `~/.cache/jev`), is created
+  private to the user, and resets itself when it finds an older schema. Version 0.2 cannot read
+  caches written by 0.1 tools; the first run after upgrading re-asks.
+- **Credentials** come from the provider's variable, then `$XDG_CONFIG_HOME/jev/<provider>.key`.
+  Gateways take their URL from `JEV_GATEWAY_URL` or `<provider>.url`. `JEV_URL` overrides any endpoint.
+- **Metering** refuses malformed usage rather than under-counting; a response without a reported
+  cost is priced from its tokens at the run's list price.
+- **Errors** keep their wording across tools: a fatal status reads `PROVIDER said 401: detail`, a
+  bad request reads `HTTP 400: detail`, and exhaustion reads `gave up after 15s (last failure)`.
+  Both status errors carry `provider`, `status` and `detail` for tools that word or redact them.
+
+## Development
+
+Keep the six checkouts as siblings. Each consumer depends on `jevkit-runtime>=0.2.0,<0.3.0` and
+overrides it for development with `jevkit-runtime = { path = "../jevkit-core", editable = true }`
+under `[tool.uv.sources]`.
+
+```bash
+python3 scripts/dev.py setup          # uv sync every checkout, fetch jselect's tokenizer data
+python3 scripts/dev.py check          # core and consumer suites, credentials stripped, sockets blocked
+python3 scripts/dev.py wheel-check    # build and exercise real wheel installs in temporary environments
+python3 scripts/dev.py run jgrep -- --help
+```
+
+`check` also proves each consumer imports this exact source tree; `wheel-check` proves the installed
+wheel, not the checkout. `--tool NAME` limits either to one consumer and `--suffix` selects
+alternatively named checkouts. The GitHub workflows run the core suite on Python 3.10 and 3.13 and
+the downstream matrix against each consumer's main branch.
+
+## Releasing
+
+Tag the verified core `vX.Y.Z` and dispatch the publish workflow with that tag; the workflow checks
+the tag matches the package version and publishes through PyPI Trusted Publishing. Then release each
+consumer through its own process, bumping its supported core range, lockfile, and CI core reference
+together.

jevkit_runtime-0.2.0/README.md

@@ -0,0 +1,79 @@
+# JevKit core
+
+Distribution **`jevkit-runtime`**, import **`jevkit_runtime`**. The PyPI name `jevkit-core` belongs
+to a different project.
+
+One request pipeline, one answer store, one provider catalog for jgrep, jsort, jlink, jselect,
+and jcol. Each tool remains its own package and repository; the core imports none of them, and a
+tool's adapter is a few lines naming which providers it offers.
+
+## What a tool gets
+
+```python
+from jevkit_runtime import AnswerStore, Client, catalog, resolve
+
+PROVIDERS = catalog("typesafe", "openrouter", "gateway")
+backend = resolve(PROVIDERS, name=None, model=None)          # or JEV_API / JEV_MODEL, else the first configured
+async with Client(backend, store=AnswerStore()) as client:
+    answers = await client.ask(state, {"q": {"type": "noul", "instructions": "..."}})
+```
+
+`Client.ask` does the whole thing: computes each question's identity, serves what the store already
+knows, joins an identical request already in flight, sends only the misses, validates the entire
+response before storing any of it, and meters the call before validation so a billed but malformed
+answer still counts. It returns `Answers`, a dict by question id whose `origins` say who answered
+each one and whether it came from the API, the store, or a shared call. Per-call policy is keyword
+arguments: `allow_paid=False` for cache-only runs, `on_cost` for the caller who should be charged,
+`hedge_after` to resend a slow call, and `keys` for callers whose reuse unit is not the request.
+HTTP/2 is used whenever the `http2` extra is installed.
+
+| Module | Owns |
+|---|---|
+| `settings.py` | Every environment and filesystem convention, read in one place: `XDG_*`, `JEV_API`, `JEV_URL`, `JEV_MODEL`, `JEV_PRICE_PER_MTOK`, provider keys and URL files |
+| `providers.py` | The catalog (`Provider`), a tool's selection of it or its own entries, and `resolve()` to one `Backend`: endpoint, model, key |
+| `protocol.py` | Request bodies, typed answer validation (`noul`, `choice`, `score`), usage parsing, answer identity, provenance |
+| `transport.py` | One HTTP call with a total deadline, retries with backoff and `Retry-After`, structured status errors |
+| `store.py` | SQLite answers with their provenance in one row, one versioned schema |
+| `client.py` | The pipeline above, request sharing, hedging |
+| `meter.py` | Calls, cache hits, retries, hedges, tokens, cost, and which models actually answered |
+| `errors.py` | `JevError`, `JevFatal`, `JevBudgetExceeded`, `RequestExhausted`, `ProviderError`, `ProviderFatal` |
+
+## Conventions every tool shares
+
+- **Answer identity** is `answer_key(backend, state, question)`: provider, endpoint, model, state and
+  question. An answer from one provider or model is never served for another.
+- **The store** lives at `$XDG_CACHE_HOME/jev/answers.sqlite` (default `~/.cache/jev`), is created
+  private to the user, and resets itself when it finds an older schema. Version 0.2 cannot read
+  caches written by 0.1 tools; the first run after upgrading re-asks.
+- **Credentials** come from the provider's variable, then `$XDG_CONFIG_HOME/jev/<provider>.key`.
+  Gateways take their URL from `JEV_GATEWAY_URL` or `<provider>.url`. `JEV_URL` overrides any endpoint.
+- **Metering** refuses malformed usage rather than under-counting; a response without a reported
+  cost is priced from its tokens at the run's list price.
+- **Errors** keep their wording across tools: a fatal status reads `PROVIDER said 401: detail`, a
+  bad request reads `HTTP 400: detail`, and exhaustion reads `gave up after 15s (last failure)`.
+  Both status errors carry `provider`, `status` and `detail` for tools that word or redact them.
+
+## Development
+
+Keep the six checkouts as siblings. Each consumer depends on `jevkit-runtime>=0.2.0,<0.3.0` and
+overrides it for development with `jevkit-runtime = { path = "../jevkit-core", editable = true }`
+under `[tool.uv.sources]`.
+
+```bash
+python3 scripts/dev.py setup          # uv sync every checkout, fetch jselect's tokenizer data
+python3 scripts/dev.py check          # core and consumer suites, credentials stripped, sockets blocked
+python3 scripts/dev.py wheel-check    # build and exercise real wheel installs in temporary environments
+python3 scripts/dev.py run jgrep -- --help
+```
+
+`check` also proves each consumer imports this exact source tree; `wheel-check` proves the installed
+wheel, not the checkout. `--tool NAME` limits either to one consumer and `--suffix` selects
+alternatively named checkouts. The GitHub workflows run the core suite on Python 3.10 and 3.13 and
+the downstream matrix against each consumer's main branch.
+
+## Releasing
+
+Tag the verified core `vX.Y.Z` and dispatch the publish workflow with that tag; the workflow checks
+the tag matches the package version and publishes through PyPI Trusted Publishing. Then release each
+consumer through its own process, bumping its supported core range, lockfile, and CI core reference
+together.

{jevkit_runtime-0.1.0 → jevkit_runtime-0.2.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "jevkit-runtime"
-version = "0.1.0"
+version = "0.2.0"
 description = "Shared transport, configuration, caching, and accounting for JevKit tools"
 readme = "README.md"
 license = "MIT"
@@ -23,7 +23,7 @@ requires = ["hatchling"]
 build-backend = "hatchling.build"
 
 [tool.hatch.build.targets.wheel]
-packages = ["src/jevkit_core"]
+packages = ["src/jevkit_runtime"]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]

{jevkit_runtime-0.1.0 → jevkit_runtime-0.2.0}/scripts/dev.py

@@ -12,6 +12,10 @@ from pathlib import Path
 
 CORE = Path(__file__).resolve().parents[1]
 TOOLS = ("jgrep", "jsort", "jlink", "jselect", "jcol")
+PACKAGED_ASSETS = {
+    "jcol": ["static/index.html"],
+    "jlink": ["assets/review.html", "assets/review.js", "assets/review.css"],
+}
 
 
 def command(argv, *, cwd=CORE, env=None):
@@ -39,10 +43,34 @@ def python(repo):
     return repo / ".venv" / ("Scripts/python.exe" if os.name == "nt" else "bin/python")
 
 
+def runtime_environment(executable, env=None):
+    """Use the selected environment for child CLI processes as well as Python."""
+    result = dict(os.environ if env is None else env)
+    result["PATH"] = os.pathsep.join((str(executable.parent), result.get("PATH", os.defpath)))
+    result["VIRTUAL_ENV"] = str(executable.parent.parent)
+    return result
+
+
+def expect_core(executable, location, *, cwd, env):
+    """The consumer's interpreter must import the core from `location`, not a stray install."""
+    command(
+        [
+            executable,
+            "-c",
+            "import sys, pathlib, jevkit_runtime; "
+            "here = pathlib.Path(jevkit_runtime.__file__).resolve().parent; "
+            "assert here == pathlib.Path(sys.argv[1]).resolve(), here",
+            location,
+        ],
+        cwd=cwd,
+        env=env,
+    )
+
+
 def main():
     parser = argparse.ArgumentParser(description=__doc__)
     parser.add_argument("--repos-root", type=Path, default=CORE.parent)
-    parser.add_argument("--suffix", default="", help="e.g. --suffix=-jevkit for isolated migration worktrees")
+    parser.add_argument("--suffix", default="", help="checkout name suffix, e.g. --suffix=-wip")
     parser.add_argument("--tool", choices=TOOLS, help="operate on only one consumer")
     parser.add_argument(
         "--tokenizer-cache", type=Path, default=Path(tempfile.gettempdir()) / "data-gym-cache"
@@ -63,7 +91,11 @@ def main():
     if args.action == "run":
         repo = root / (args.name + args.suffix)
         arguments = args.arguments[1:] if args.arguments[:1] == ["--"] else args.arguments
-        command([python(repo), "-m", args.name, *arguments], cwd=Path.cwd())
+        command(
+            [python(repo), "-m", args.name, *arguments],
+            cwd=Path.cwd(),
+            env=runtime_environment(python(repo)),
+        )
         return
     for repo in repos.values():
         if not (repo / "pyproject.toml").is_file():
@@ -89,25 +121,11 @@ def main():
         temp = Path(temporary)
         env = environment(temp, args.tokenizer_cache)
         if args.action == "check":
-            baselines = json.loads((CORE / "consumer-baselines.json").read_text())
-            command([python(CORE), "-m", "pytest", "-q"], env=env)
-            for name, repo in repos.items():
-                command(
-                    [
-                        python(repo),
-                        CORE / "scripts/probe_consumer.py",
-                        name,
-                        "--expect-core",
-                        CORE / "src/jevkit_core",
-                        "--baseline-repo",
-                        repo,
-                        "--baseline-ref",
-                        baselines[name],
-                    ],
-                    cwd=temp,
-                    env=env,
-                )
-                command([python(repo), "-m", "pytest", "-q"], cwd=repo, env=env)
+            command([python(CORE), "-m", "pytest", "-q"], env=runtime_environment(python(CORE), env))
+            for repo in repos.values():
+                consumer_env = runtime_environment(python(repo), env)
+                expect_core(python(repo), CORE / "src/jevkit_runtime", cwd=temp, env=consumer_env)
+                command([python(repo), "-m", "pytest", "-q"], cwd=repo, env=consumer_env)
             return
         wheels = temp / "wheels"
         command(["uv", "build", "--no-sources", "--out-dir", wheels])
@@ -119,47 +137,27 @@ def main():
             venv = temp / name
             command(["uv", "venv", "--python", python(repo), venv])
             executable = venv / ("Scripts/python.exe" if os.name == "nt" else "bin/python")
+            wheel_env = runtime_environment(executable, env)
             command(["uv", "pip", "install", "--python", executable, core_wheel, package])
             site = subprocess.check_output(
                 [str(executable), "-c", "import sysconfig; print(sysconfig.get_path('purelib'))"], text=True
             ).strip()
-            command(
-                [
-                    executable,
-                    CORE / "scripts/probe_consumer.py",
-                    name,
-                    "--expect-core",
-                    Path(site) / "jevkit_core",
-                ],
-                cwd=temp,
-                env=env,
-            )
-            command([executable, "-m", name, "--version"], cwd=temp, env=env)
-            if name == "jcol":
-                cli = venv / ("Scripts/jcol.exe" if os.name == "nt" else "bin/jcol")
-                command([executable, repo / "tests/test_process.py", cli], cwd=temp, env=env)
+            expect_core(executable, Path(site) / "jevkit_runtime", cwd=temp, env=wheel_env)
+            command([executable, "-m", name, "--version"], cwd=temp, env=wheel_env)
+            for asset in PACKAGED_ASSETS.get(name, []):
                 command(
                     [
                         executable,
                         "-c",
                         "from importlib.resources import files; "
-                        "assert files('jcol').joinpath('static/index.html').is_file()",
+                        f"assert files({name!r}).joinpath({asset!r}).is_file()",
                     ],
                     cwd=temp,
-                    env=env,
-                )
-            if name == "jlink":
-                command(
-                    [
-                        executable,
-                        "-c",
-                        "from importlib.resources import files; "
-                        "assert all(files('jlink').joinpath('assets', f).is_file() "
-                        "for f in ('review.html', 'review.js', 'review.css'))",
-                    ],
-                    cwd=temp,
-                    env=env,
+                    env=wheel_env,
                 )
+            if name == "jcol":
+                cli = venv / ("Scripts/jcol.exe" if os.name == "nt" else "bin/jcol")
+                command([executable, repo / "tests/test_process.py", cli], cwd=temp, env=wheel_env)
         print(json.dumps({"wheel_checks": names, "status": "passed"}))
 
 

jevkit_runtime-0.2.0/src/jevkit_runtime/__init__.py

@@ -0,0 +1,63 @@
+"""Shared runtime for the JevKit tools: one pipeline, one store, one provider catalog."""
+
+from .client import Answers, Client
+from .errors import (
+    JevBudgetExceeded,
+    JevError,
+    JevFatal,
+    ProviderError,
+    ProviderFatal,
+    ProviderStatus,
+    RequestExhausted,
+)
+from .meter import Meter
+from .protocol import (
+    QUESTION_TYPES,
+    Usage,
+    answer_key,
+    digest,
+    error_detail,
+    parse_answers,
+    parse_usage,
+    request_body,
+    validate_answer,
+)
+from .providers import PROVIDERS, Backend, Provider, catalog, resolve
+from .settings import DEFAULT_PRICE_PER_MTOK, Settings
+from .store import AnswerStore, Entry
+from .transport import FATAL, RETRYABLE, post
+
+__version__ = "0.2.0"
+__all__ = [
+    "AnswerStore",
+    "Answers",
+    "Backend",
+    "Client",
+    "DEFAULT_PRICE_PER_MTOK",
+    "Entry",
+    "FATAL",
+    "JevBudgetExceeded",
+    "JevError",
+    "JevFatal",
+    "Meter",
+    "PROVIDERS",
+    "ProviderError",
+    "ProviderFatal",
+    "ProviderStatus",
+    "Provider",
+    "QUESTION_TYPES",
+    "RETRYABLE",
+    "RequestExhausted",
+    "Settings",
+    "Usage",
+    "answer_key",
+    "catalog",
+    "digest",
+    "error_detail",
+    "parse_answers",
+    "parse_usage",
+    "post",
+    "request_body",
+    "resolve",
+    "validate_answer",
+]

jevkit_runtime-0.2.0/src/jevkit_runtime/client.py

@@ -0,0 +1,213 @@
+"""The request pipeline: identity, store, sharing, transport, validation, storage, metering."""
+
+from __future__ import annotations
+
+import asyncio
+import importlib.util
+from collections.abc import Callable, Coroutine, Iterable, Mapping
+from typing import Any
+
+import httpx
+
+from . import transport
+from .errors import JevBudgetExceeded
+from .meter import Meter
+from .protocol import (
+    answer_key,
+    answer_origin,
+    parse_answers,
+    parse_usage,
+    request_body,
+    resolved_model,
+    validate_answer,
+)
+from .providers import Backend
+from .store import AnswerStore
+
+Flight = tuple[dict[str, dict], dict]  # answers by key, and the origin they share
+
+
+class Answers(dict):
+    """Answers by question id, plus where each came from.
+
+    `origins[qid]` carries `provider`, `requested_model`, `resolved_model`, `answered_at`, and a
+    `source` of `cache`, `api`, or `shared`. An answer stored without provenance has only `source`.
+    """
+
+    def __init__(self, answers: Mapping[str, dict], origins: Mapping[str, dict]):
+        super().__init__(answers)
+        self.origins: dict[str, dict] = dict(origins)
+
+
+def http2_available() -> bool:
+    """HTTP/2 whenever the optional `h2` package is installed; the `http2` extra pulls it in."""
+    return importlib.util.find_spec("h2") is not None
+
+
+class Client:
+    """Ask System One questions about a state. Every tool gets the same pipeline; policy is per call."""
+
+    def __init__(
+        self,
+        backend: Backend,
+        *,
+        timeout: float = 15.0,
+        attempts: int = 4,
+        concurrency: int = 32,
+        store: AnswerStore | None = None,
+        transport: httpx.AsyncBaseTransport | None = None,
+    ):
+        self.backend = backend
+        self.timeout, self.attempts, self.store = timeout, attempts, store
+        self.concurrency, self.transport = concurrency, transport
+        self.http2 = transport is None and http2_available()
+        self.meter = Meter(provider=backend.name, requested_model=backend.model)
+        self._flights: dict[str, asyncio.Task[Flight]] = {}
+        self._http: httpx.AsyncClient | None = None
+
+    @property
+    def model(self) -> str:
+        return self.backend.model
+
+    @property
+    def url(self) -> str:
+        return self.backend.url
+
+    @property
+    def http(self) -> httpx.AsyncClient:
+        """Opened on first use, so cache-only clients never hold a connection pool."""
+        if self._http is None:
+            headers = {"X-Title": "jev tools"}
+            if self.backend.key:
+                headers["Authorization"] = f"Bearer {self.backend.key}"
+            limits = (
+                httpx.Limits(max_connections=16, max_keepalive_connections=16, keepalive_expiry=120)
+                if self.http2
+                else httpx.Limits(
+                    max_connections=self.concurrency + 4, max_keepalive_connections=self.concurrency + 4
+                )
+            )
+            self._http = httpx.AsyncClient(
+                headers=headers, limits=limits, transport=self.transport, http2=self.http2
+            )
+        return self._http
+
+    async def close(self) -> None:
+        if self._http is not None:
+            await self._http.aclose()
+            self._http = None
+
+    async def __aenter__(self) -> Client:
+        return self
+
+    async def __aexit__(self, *exc) -> None:
+        await self.close()
+
+    def key(self, state, question: dict) -> str:
+        return answer_key(self.backend, state, question)
+
+    async def ask(
+        self,
+        state,
+        questions: dict[str, dict],
+        *,
+        keys: Mapping[str, str] | None = None,
+        allow_paid: bool = True,
+        on_cost: Callable[[float], None] | None = None,
+        hedge_after: float | None = None,
+    ) -> Answers:
+        """Answer every question, sending only those the store cannot answer.
+
+        `keys` overrides the identity of each question for callers whose reuse unit is not the
+        request. `allow_paid=False` still serves store hits and joins an in-flight request.
+        `on_cost` is charged only by the caller whose request actually went out. `hedge_after`
+        sends a slow call a second time and keeps the first answer.
+        """
+        keys = dict(keys) if keys is not None else {qid: self.key(state, q) for qid, q in questions.items()}
+        answers: dict[str, dict] = {}
+        origins: dict[str, dict] = {}
+        if self.store is not None:
+            for qid, question in questions.items():
+                if (entry := self.store.entry(keys[qid])) is not None:
+                    validate_answer(qid, question, entry.answer)
+                    answers[qid] = entry.answer
+                    origins[qid] = dict(entry.metadata or {}) | {"source": "cache"}
+        misses = {qid: q for qid, q in questions.items() if qid not in answers}
+        if not misses:
+            self.meter.cached += 1
+        else:
+            miss_keys = {qid: keys[qid] for qid in misses}
+
+            def start() -> Coroutine[Any, Any, Flight]:
+                if not allow_paid:
+                    raise JevBudgetExceeded("a new paid request is not allowed by the budget")
+                return self._request(state, misses, miss_keys, on_cost)
+
+            task, owner = self._share(miss_keys.values(), start)
+            if hedge_after is None:
+                by_key, origin = await task
+            else:
+                by_key, origin = await self._hedged(task, hedge_after, state, misses, miss_keys, on_cost)
+            for qid, key in miss_keys.items():
+                answers[qid] = by_key[key]
+                origins[qid] = dict(origin) | {"source": "api" if owner else "shared"}
+        for origin in origins.values():
+            self.meter.note_answer(origin)
+        return Answers({qid: answers[qid] for qid in questions}, {qid: origins[qid] for qid in questions})
+
+    def _share(self, keys: Iterable[str], start: Callable[[], Coroutine[Any, Any, Flight]]):
+        """Join an identical in-flight request, or start one. Only the starter pays."""
+        flight = "|".join(sorted(keys))
+        if (task := self._flights.get(flight)) is not None:
+            self.meter.cached += 1
+            return task, False
+        task = asyncio.ensure_future(start())
+        self._flights[flight] = task
+
+        def discard(done):
+            if self._flights.get(flight) is done:
+                del self._flights[flight]
+
+        task.add_done_callback(discard)
+        return task, True
+
+    async def _hedged(self, first, after, state, questions, keys, on_cost) -> Flight:
+        done, _ = await asyncio.wait({first}, timeout=after)
+        if done:
+            return first.result()
+        second = asyncio.ensure_future(self._request(state, questions, keys, on_cost))
+        self.meter.hedges += 1
+        pending = {first, second}
+        error: BaseException | None = None
+        while pending:
+            done, pending = await asyncio.wait(pending, return_when=asyncio.FIRST_COMPLETED)
+            for task in done:
+                if task.exception() is None:
+                    if second in pending:
+                        second.cancel()  # never cancel `first`: other callers may be sharing it
+                    return task.result()
+                error = task.exception()
+        assert error is not None
+        raise error
+
+    async def _request(self, state, questions: dict[str, dict], keys: dict[str, str], on_cost) -> Flight:
+        def retry():
+            self.meter.retries += 1
+
+        data = await transport.post(
+            self.http,
+            self.backend.url,
+            request_body(self.backend.model, state, questions),
+            provider=self.backend.name,
+            timeout=self.timeout,
+            attempts=self.attempts,
+            on_retry=retry,
+        )
+        usage = parse_usage(data.get("usage"), price_per_mtok=self.backend.price_per_mtok)
+        self.meter.record_call(usage, on_cost)
+        answers = parse_answers(data, questions, provider=self.backend.name)
+        origin = answer_origin(self.backend, resolved_model(data))
+        if self.store is not None:
+            for qid, answer in answers.items():
+                self.store.put(keys[qid], answer, origin)
+        return {keys[qid]: answer for qid, answer in answers.items()}, origin

jevkit_runtime-0.2.0/src/jevkit_runtime/errors.py

@@ -0,0 +1,43 @@
+"""Failure types shared by every JevKit tool."""
+
+
+class JevError(Exception):
+    """One request failed; the rest of the run may continue."""
+
+
+class JevFatal(Exception):
+    """Stop the run until configuration, credentials, or metering is corrected."""
+
+
+class JevBudgetExceeded(Exception):
+    """No cached or in-flight answer exists and a new paid request is forbidden."""
+
+
+class ProviderStatus(Exception):
+    """An HTTP status the provider answered with, kept structured so tools can word or redact it."""
+
+    def __init__(self, provider: str, status: int, detail: str):
+        self.provider, self.status, self.detail = provider, status, detail
+        super().__init__(self.message())
+
+    def message(self) -> str:
+        return f"HTTP {self.status}: {self.detail}"
+
+
+class ProviderError(ProviderStatus, JevError):
+    """A status that is neither retryable nor a reason to stop the run, such as a bad request."""
+
+
+class ProviderFatal(ProviderStatus, JevFatal):
+    """Authentication, payment, or permission failed; no further request can succeed."""
+
+    def message(self) -> str:
+        return f"{self.provider} said {self.status}: {self.detail}"
+
+
+class RequestExhausted(JevError):
+    """Every attempt within the deadline failed; `last` names the final failure."""
+
+    def __init__(self, timeout: float, last: str, *, timed_out: bool = False):
+        self.timeout, self.last, self.timed_out = timeout, last, timed_out
+        super().__init__(f"gave up after {timeout:g}s ({last})")