giul 0.1.0__tar.gz

giul-0.1.0/.gitignore

@@ -0,0 +1,5 @@
+.venv/
+__pycache__/
+*.pyc
+dist/
+*.egg-info/

giul-0.1.0/PKG-INFO

@@ -0,0 +1,115 @@
+Metadata-Version: 2.5
+Name: giul
+Version: 0.1.0
+Summary: One ruler for what an answer cost in joules, across a GPU fleet.
+Project-URL: Homepage, https://github.com/todd427/giul
+Author: Todd McCaffrey
+License-Expression: Apache-2.0
+Keywords: energy,gpu,joules,nvml,telemetry
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Topic :: System :: Monitoring
+Requires-Python: >=3.11
+Provides-Extra: nvml
+Requires-Dist: nvidia-ml-py>=12.535; extra == 'nvml'
+Provides-Extra: remote
+Requires-Dist: httpx>=0.24; extra == 'remote'
+Description-Content-Type: text/markdown
+
+# giul
+
+*giúl* — Irish for joule. One ruler for "what did this answer cost in energy"
+across the FoxxeLabs fleet: Aigne, Tuiscint, Gléas.
+
+Two rulers make a comparison an argument instead of evidence. This is one,
+and all three pin it.
+
+```bash
+pip install giul              # no dependencies; imports fine with no GPU
+pip install giul[nvml]        # + nvidia-ml-py, for the energy counter
+pip install giul[remote]      # + httpx, for metering a remote node
+```
+
+## What a joule means here
+
+The number a request is charged is **energy above idle**:
+
+```
+joules = measured − idle_w × seconds
+```
+
+A request pays for the power it *caused*, not for the power the box burns
+existing. `idle_w` is supplied by the caller, per node — giul never guesses
+it, because a wrong idle floor silently rewrites every figure that node
+reports.
+
+## Three rules
+
+1. **Charged above idle.** As above.
+2. **No fabricated zeros.** A window that measured ≤ 0 J above idle did not
+   measure a free request, it failed to measure one. It degrades to
+   `estimated` (with a hint) or `unknown` — never a `sampled` zero.
+3. **Estimates are labelled as estimates.** An estimate that reads as a
+   measurement is exactly how an efficiency claim stops being evidence. A
+   sampler that cannot be reached costs a measurement, never the request.
+
+## Use
+
+```python
+from giul import Meter, Node
+
+node = Node(name="iris", gpu_index=0, idle_w=38.0,
+            is_local=True, power_endpoint=None)
+
+# async
+async with Meter.for_node(node, joules_per_1k_hint=4100.0) as m:
+    await retrieve(); m.mark("retrieve")
+    await generate(); m.mark("generate")
+e = m.result(tokens=412)
+# e.joules, e.method, e.backend, e.stages == {"retrieve": Energy, "generate": Energy}
+
+# sync
+with Meter.for_node(node, sync=torch.cuda.synchronize).sync() as m:
+    ...; m.mark("verify")
+e = m.result(tokens=n)
+```
+
+`mark(name)` closes the current stage and opens the next. The total is always
+computed over the whole window, never by summing stages, so a stage the
+sampler was too slow to see cannot corrupt it.
+
+**The caller synchronises the GPU before the closing read.** On the counter
+backend the register only counts work the card has *finished*; pass
+`sync=torch.cuda.synchronize` when the work is local. An HTTP upstream needs
+nothing — the completion returning is the sync point.
+
+## Backends
+
+Chosen at runtime in one place (`Meter.for_node`), needing no per-node
+configuration. Support is probed once per card and cached.
+
+| `backend` | when | how |
+|---|---|---|
+| `nvml_counter` | local card, `pynvml` present, counter answers | `nvmlDeviceGetTotalEnergyConsumption` delta — a true measurement of a sub-second window |
+| `smi_sampler` | local card, `nvidia-smi` on PATH | integral of `power.draw` samples |
+| `remote_agent` | `node.power_endpoint` set | the same integral, sampled by `giul-agent` on that node |
+| `none` | otherwise | `estimated` with a hint, else `unknown` |
+
+`method` stays `sampled` / `estimated` / `unknown`. The counter *is* a
+measurement, so it reports `method="sampled"`; the distinction lives in
+`backend`.
+
+## Tools
+
+```bash
+giul-probe                    # what this host's cards support; counter vs sampler
+giul-agent --port 9402        # expose a node's power draw; stdlib only, read-only
+```
+
+Bind `giul-agent` to the mesh address, not `0.0.0.0`, unless the node is
+otherwise firewalled.
+
+## Not in 0.1
+
+CPU/RAPL, carbon, € cost, storing series, any UI.

giul-0.1.0/README.md

@@ -0,0 +1,96 @@
+# giul
+
+*giúl* — Irish for joule. One ruler for "what did this answer cost in energy"
+across the FoxxeLabs fleet: Aigne, Tuiscint, Gléas.
+
+Two rulers make a comparison an argument instead of evidence. This is one,
+and all three pin it.
+
+```bash
+pip install giul              # no dependencies; imports fine with no GPU
+pip install giul[nvml]        # + nvidia-ml-py, for the energy counter
+pip install giul[remote]      # + httpx, for metering a remote node
+```
+
+## What a joule means here
+
+The number a request is charged is **energy above idle**:
+
+```
+joules = measured − idle_w × seconds
+```
+
+A request pays for the power it *caused*, not for the power the box burns
+existing. `idle_w` is supplied by the caller, per node — giul never guesses
+it, because a wrong idle floor silently rewrites every figure that node
+reports.
+
+## Three rules
+
+1. **Charged above idle.** As above.
+2. **No fabricated zeros.** A window that measured ≤ 0 J above idle did not
+   measure a free request, it failed to measure one. It degrades to
+   `estimated` (with a hint) or `unknown` — never a `sampled` zero.
+3. **Estimates are labelled as estimates.** An estimate that reads as a
+   measurement is exactly how an efficiency claim stops being evidence. A
+   sampler that cannot be reached costs a measurement, never the request.
+
+## Use
+
+```python
+from giul import Meter, Node
+
+node = Node(name="iris", gpu_index=0, idle_w=38.0,
+            is_local=True, power_endpoint=None)
+
+# async
+async with Meter.for_node(node, joules_per_1k_hint=4100.0) as m:
+    await retrieve(); m.mark("retrieve")
+    await generate(); m.mark("generate")
+e = m.result(tokens=412)
+# e.joules, e.method, e.backend, e.stages == {"retrieve": Energy, "generate": Energy}
+
+# sync
+with Meter.for_node(node, sync=torch.cuda.synchronize).sync() as m:
+    ...; m.mark("verify")
+e = m.result(tokens=n)
+```
+
+`mark(name)` closes the current stage and opens the next. The total is always
+computed over the whole window, never by summing stages, so a stage the
+sampler was too slow to see cannot corrupt it.
+
+**The caller synchronises the GPU before the closing read.** On the counter
+backend the register only counts work the card has *finished*; pass
+`sync=torch.cuda.synchronize` when the work is local. An HTTP upstream needs
+nothing — the completion returning is the sync point.
+
+## Backends
+
+Chosen at runtime in one place (`Meter.for_node`), needing no per-node
+configuration. Support is probed once per card and cached.
+
+| `backend` | when | how |
+|---|---|---|
+| `nvml_counter` | local card, `pynvml` present, counter answers | `nvmlDeviceGetTotalEnergyConsumption` delta — a true measurement of a sub-second window |
+| `smi_sampler` | local card, `nvidia-smi` on PATH | integral of `power.draw` samples |
+| `remote_agent` | `node.power_endpoint` set | the same integral, sampled by `giul-agent` on that node |
+| `none` | otherwise | `estimated` with a hint, else `unknown` |
+
+`method` stays `sampled` / `estimated` / `unknown`. The counter *is* a
+measurement, so it reports `method="sampled"`; the distinction lives in
+`backend`.
+
+## Tools
+
+```bash
+giul-probe                    # what this host's cards support; counter vs sampler
+giul-agent --port 9402        # expose a node's power draw; stdlib only, read-only
+```
+
+Bind `giul-agent` to the mesh address, not `0.0.0.0`, unless the node is
+otherwise firewalled.
+
+## Not in 0.1
+
+CPU/RAPL, carbon, € cost, storing series, any UI.

giul-0.1.0/giul/__init__.py

@@ -0,0 +1,41 @@
+"""giul — one ruler for what an answer cost in joules.
+
+*giúl*, Irish for joule. Aigne, Tuiscint and Gléas each need to say what a
+request cost in energy; two rulers make a comparison an argument instead of
+evidence, so there is one, and all three pin it.
+
+    from giul import Meter, Node
+
+    node = Node(name="iris", gpu_index=0, idle_w=38.0, is_local=True)
+    async with Meter.for_node(node, joules_per_1k_hint=4100.0) as m:
+        await retrieve(); m.mark("retrieve")
+        await generate(); m.mark("generate")
+    e = m.result(tokens=412)
+
+Three rules the library will not bend:
+
+1. A request is charged above idle — for the power it caused, not the power
+   the box burns existing. ``idle_w`` comes from the caller; giul never
+   guesses it.
+2. A window that measured <= 0 J above idle failed to measure a request; it
+   did not measure a free one. It degrades to ``estimated`` or ``unknown``,
+   never a ``sampled`` zero.
+3. Estimates are labelled as estimates. A sampler that cannot be reached
+   costs a measurement, never the request.
+"""
+
+from .backends import NvmlCounter, RemoteAgent, SmiSampler
+from .meter import Energy, EnergyMeter, Meter, Node
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "Energy",
+    "EnergyMeter",
+    "Meter",
+    "Node",
+    "NvmlCounter",
+    "RemoteAgent",
+    "SmiSampler",
+    "__version__",
+]

giul-0.1.0/giul/agent.py

@@ -0,0 +1,102 @@
+#!/usr/bin/env python3
+"""Expose a node's GPU power draw over HTTP, so giul can meter remote tiers.
+
+`nvidia-smi` only sees the card in the machine it runs on, which meant every
+node except the gateway's own reported "energy not measured here". That makes
+the fleet's headline metric a single-node metric. This closes it.
+
+Deliberately stdlib-only and dependency-free: it has to run on any fleet node
+without provisioning a venv, and it must not be able to break a serving node.
+It reads power and nothing else — no control surface, no writes.
+
+    giul-agent --port 9402
+    curl localhost:9402/power?gpu=1   -> {"watts": 231.4, "gpu": 1, ...}
+
+Bind to the Féith mesh address, not 0.0.0.0, unless the node is otherwise
+firewalled.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import subprocess
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from urllib.parse import parse_qs, urlparse
+
+SMI = "nvidia-smi"
+QUERY_TIMEOUT_S = 5.0
+
+
+def read_gpu(index: int) -> dict:
+    """Instantaneous board draw for one GPU, plus enough to identify it.
+
+    The name is included so a caller can assert it is metering the card it
+    thinks it is — a wrong `gpu_index` otherwise silently reports an idle
+    neighbour, which reads as "this model is astonishingly efficient".
+    """
+    out = subprocess.run(
+        [SMI, f"--id={index}",
+         "--query-gpu=name,power.draw,memory.used,utilization.gpu",
+         "--format=csv,noheader,nounits"],
+        capture_output=True, text=True, timeout=QUERY_TIMEOUT_S, check=True,
+    ).stdout.strip()
+    name, watts, mem, util = (p.strip() for p in out.split(",", 3))
+    return {
+        "gpu": index,
+        "name": name,
+        "watts": float(watts),
+        "memory_used_mib": int(float(mem)),
+        "utilization_pct": int(float(util)),
+    }
+
+
+class Handler(BaseHTTPRequestHandler):
+    default_gpu = 0
+
+    def _send(self, code: int, payload: dict) -> None:
+        body = json.dumps(payload).encode()
+        self.send_response(code)
+        self.send_header("Content-Type", "application/json")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def do_GET(self) -> None:  # noqa: N802 (stdlib naming)
+        url = urlparse(self.path)
+        if url.path not in ("/power", "/healthz"):
+            return self._send(404, {"error": "try /power or /healthz"})
+
+        params = parse_qs(url.query)
+        try:
+            index = int(params.get("gpu", [self.default_gpu])[0])
+        except ValueError:
+            return self._send(400, {"error": "gpu must be an integer"})
+
+        try:
+            self._send(200, read_gpu(index))
+        except subprocess.CalledProcessError as exc:
+            self._send(404, {"error": f"no gpu {index}", "detail": exc.stderr.strip()[:200]})
+        except (subprocess.SubprocessError, FileNotFoundError, ValueError) as exc:
+            # Never 500 into the gateway's request path: an unreadable sampler
+            # must degrade to "unmeasured", not to a failed completion.
+            self._send(503, {"error": f"{type(exc).__name__}: {exc}"})
+
+    def log_message(self, *args) -> None:
+        """Silent: this is polled several times a second per request."""
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--host", default="0.0.0.0")
+    ap.add_argument("--port", type=int, default=9402)
+    ap.add_argument("--gpu", type=int, default=0, help="default GPU when ?gpu= is omitted")
+    args = ap.parse_args()
+
+    Handler.default_gpu = args.gpu
+    print(f"power-agent on {args.host}:{args.port}, default gpu {args.gpu}", flush=True)
+    ThreadingHTTPServer((args.host, args.port), Handler).serve_forever()
+
+
+if __name__ == "__main__":
+    main()

giul-0.1.0/giul/backends.py

@@ -0,0 +1,232 @@
+"""Three ways to ask a GPU what it drew, and one that cannot ask.
+
+Two shapes of answer, not one:
+
+- A **counter** backend (``nvml_counter``) reads a cumulative millijoule
+  register the driver maintains. Two reads and a subtraction give a true
+  delta over the window, with no sampling resolution to fall through. This
+  is why sub-second requests can be measured at all.
+- A **sampler** backend (``smi_sampler``, ``remote_agent``) reads
+  instantaneous watts and the meter integrates them. Fine for multi-second
+  windows, blind below its poll interval.
+
+Every read returns ``None`` rather than raising. A sampler that cannot be
+reached must degrade the request to unmeasured, never fail it.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import shutil
+import subprocess
+
+SMI = shutil.which("nvidia-smi")
+
+COUNTER = "counter"
+SAMPLER = "sampler"
+
+
+def read_power_w(gpu_index: int = 0) -> float | None:
+    """Instantaneous board draw in watts, or None if nvidia-smi can't say."""
+    if not SMI:
+        return None
+    try:
+        out = subprocess.run(
+            [SMI, f"--id={gpu_index}", "--query-gpu=power.draw",
+             "--format=csv,noheader,nounits"],
+            capture_output=True, text=True, timeout=5, check=True,
+        ).stdout.strip()
+        return float(out.splitlines()[0])
+    except (subprocess.SubprocessError, ValueError, IndexError, OSError):
+        return None
+
+
+class Backend:
+    """Something that can be asked about a card, without saying how."""
+
+    name = "none"
+    kind = "none"
+
+    def read_watts(self) -> float | None:
+        return None
+
+    async def aread_watts(self) -> float | None:
+        loop = asyncio.get_running_loop()
+        return await loop.run_in_executor(None, self.read_watts)
+
+    def read_joules(self) -> float | None:
+        """Cumulative joules since driver load. Counter backends only."""
+        return None
+
+    async def aread_joules(self) -> float | None:
+        loop = asyncio.get_running_loop()
+        return await loop.run_in_executor(None, self.read_joules)
+
+    def close(self) -> None:
+        pass
+
+    async def aclose(self) -> None:
+        self.close()
+
+
+class NvmlCounter(Backend):
+    """``nvmlDeviceGetTotalEnergyConsumption`` — a real energy register.
+
+    Volta and later. Cumulative millijoules since the driver loaded, so the
+    window's joules are one subtraction and carry no sampling error. The
+    caller is responsible for synchronising the GPU before the closing read
+    (see ``Meter``); an unsynchronised read charges the next window for work
+    this one queued.
+    """
+
+    name = "nvml_counter"
+    kind = COUNTER
+
+    def __init__(self, gpu_index: int = 0) -> None:
+        import pynvml  # noqa: PLC0415 — optional, probed at construction
+
+        self._nvml = pynvml
+        self.gpu_index = gpu_index
+        self._handle = pynvml.nvmlDeviceGetHandleByIndex(gpu_index)
+
+    def read_joules(self) -> float | None:
+        try:
+            mj = self._nvml.nvmlDeviceGetTotalEnergyConsumption(self._handle)
+        except Exception:
+            return None
+        return mj / 1000.0
+
+    async def aread_joules(self) -> float | None:
+        # Two register reads; no reason to leave the event loop for it.
+        return self.read_joules()
+
+    def read_watts(self) -> float | None:
+        try:
+            return self._nvml.nvmlDeviceGetPowerUsage(self._handle) / 1000.0
+        except Exception:
+            return None
+
+    async def aread_watts(self) -> float | None:
+        return self.read_watts()
+
+
+class SmiSampler(Backend):
+    """nvidia-smi on this host. Blocking, so async reads go off the loop."""
+
+    name = "smi_sampler"
+    kind = SAMPLER
+
+    def __init__(self, gpu_index: int = 0) -> None:
+        self.gpu_index = gpu_index
+
+    def read_watts(self) -> float | None:
+        return read_power_w(self.gpu_index)
+
+
+class RemoteAgent(Backend):
+    """A fleet node running ``giul-agent``.
+
+    Still a measurement of the real card — only the process running
+    nvidia-smi differs — so it is recorded as ``sampled``, with ``source``
+    distinguishing it from the local path.
+    """
+
+    name = "remote_agent"
+    kind = SAMPLER
+
+    def __init__(self, endpoint: str, gpu_index: int = 0, timeout_s: float = 2.0) -> None:
+        import httpx  # noqa: PLC0415 — optional, only under [remote]
+
+        self._httpx = httpx
+        self.url = endpoint.rstrip("/") + "/power"
+        self.gpu_index = gpu_index
+        self.timeout_s = timeout_s
+        self._aclient: object | None = None
+        self._client: object | None = None
+
+    def read_watts(self) -> float | None:
+        if self._client is None:
+            self._client = self._httpx.Client(timeout=self.timeout_s)
+        try:
+            resp = self._client.get(self.url, params={"gpu": self.gpu_index})
+            if resp.status_code != 200:
+                return None
+            return float(resp.json()["watts"])
+        except (self._httpx.HTTPError, KeyError, ValueError, TypeError):
+            return None
+
+    async def aread_watts(self) -> float | None:
+        if self._aclient is None:
+            self._aclient = self._httpx.AsyncClient(timeout=self.timeout_s)
+        try:
+            resp = await self._aclient.get(self.url, params={"gpu": self.gpu_index})
+            if resp.status_code != 200:
+                return None
+            return float(resp.json()["watts"])
+        except (self._httpx.HTTPError, KeyError, ValueError, TypeError):
+            return None
+
+    def close(self) -> None:
+        if self._client is not None:
+            self._client.close()
+            self._client = None
+
+    async def aclose(self) -> None:
+        self.close()
+        if self._aclient is not None:
+            await self._aclient.aclose()
+            self._aclient = None
+
+
+# --- backend availability, probed once per card -----------------------------
+#
+# Whether a card supports the energy counter is a property of the silicon and
+# the driver, not of the request, so it is asked once and remembered. A card
+# that does not support it (pre-Volta, or a driver that says NotSupported)
+# falls through to the sampler silently — never into a request.
+
+_nvml_started: bool | None = None
+_counter_support: dict[int, NvmlCounter | None] = {}
+
+
+def _nvml_init() -> bool:
+    """Initialise NVML once. False if pynvml is absent or NVML won't start."""
+    global _nvml_started
+    if _nvml_started is None:
+        try:
+            import pynvml
+
+            pynvml.nvmlInit()
+            _nvml_started = True
+        except Exception:
+            _nvml_started = False
+    return _nvml_started
+
+
+def counter_for(gpu_index: int = 0) -> NvmlCounter | None:
+    """An ``NvmlCounter`` for this card, or None if it has no energy register.
+
+    Support is confirmed by actually taking a reading: a handle that exists
+    but whose counter raises ``NVMLError_NotSupported`` is not support.
+    """
+    if gpu_index in _counter_support:
+        return _counter_support[gpu_index]
+
+    backend: NvmlCounter | None = None
+    if _nvml_init():
+        try:
+            candidate = NvmlCounter(gpu_index)
+            if candidate.read_joules() is not None:
+                backend = candidate
+        except Exception:
+            backend = None
+
+    _counter_support[gpu_index] = backend
+    return backend
+
+
+def reset_probe_cache() -> None:
+    """Forget what was probed. For tests, and for a driver reload."""
+    global _nvml_started
+    _nvml_started = None
+    _counter_support.clear()