offpeak 0.2.1__tar.gz → 0.2.2__tar.gz

{offpeak-0.2.1 → offpeak-0.2.2}/.github/workflows/nightly.yml

@@ -20,8 +20,11 @@ on:
 permissions:
   contents: write
 
+# Everything that pushes board-data shares one queue. Two workflows writing
+# different files on the same branch still race at the push, and a rejected
+# push loses a night's mark.
 concurrency:
-  group: night-board
+  group: board-data
   cancel-in-progress: false
 
 jobs:
@@ -60,6 +63,10 @@ jobs:
         run: pip install "gridstatus>=0.36"
 
       - name: Generate
+        # EIA_API_KEY is optional: absent, the US carbon columns are recorded
+        # unavailable and every other leg runs exactly as before.
+        env:
+          EIA_API_KEY: ${{ secrets.EIA_API_KEY }}
         run: |
           python tools/night_report.py \
             --mode "${{ steps.pick.outputs.mode }}" \

offpeak-0.2.2/.github/workflows/settle.yml

@@ -0,0 +1,56 @@
+name: Settled runs
+
+# Publishes the receipts in `receipts/` onto the board-data branch as
+# `nightly/SETTLED.md`. Manual only: a settlement is a deliberate act, and the
+# ledger should move when someone decides it moves, not on every push.
+#
+# board-data is written by CI and never by hand — the same rule the night board
+# follows, for the same reason: main is protected and a bot push there would be
+# rejected, and a ledger anyone can hand-edit is not a ledger.
+
+on:
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+# Shared with the night board: everything that pushes board-data queues here.
+concurrency:
+  group: board-data
+  cancel-in-progress: false
+
+jobs:
+  settle:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out the generator and the receipts (main)
+        uses: actions/checkout@v7
+
+      - uses: actions/setup-python@v7
+        with:
+          python-version: "3.12"
+
+      - name: Check out the ledger (board-data)
+        uses: actions/checkout@v7
+        with:
+          ref: board-data
+          path: board
+
+      - name: Render
+        run: |
+          python tools/settle_report.py \
+            --receipts receipts \
+            --outdir board/nightly
+
+      - name: Commit to board-data
+        working-directory: board
+        run: |
+          git config user.name "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+          git add nightly
+          if git diff --staged --quiet; then
+            echo "nothing changed — the ledger already says this"
+            exit 0
+          fi
+          git commit -m "board: settled runs $(date -u +%Y-%m-%d)"
+          git push

{offpeak-0.2.1 → offpeak-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: offpeak
-Version: 0.2.1
+Version: 0.2.2
 Summary: Deadline-priced inference: give AI jobs a deadline and run them on the cheapest venue — provider batch tiers (−50%) today. Same model, same tokens, a different hour.
 Project-URL: Homepage, https://github.com/offpeak-ai/offpeak
 Project-URL: Repository, https://github.com/offpeak-ai/offpeak

{offpeak-0.2.1 → offpeak-0.2.2}/docs/night-board.md

@@ -28,6 +28,13 @@ night of 2026-08-20 the ERCOT Houston hub marked **3.94x** between its evening
 peak and the trough that followed, against the 4x a venue charges for the same
 hour of impatience.
 
+The two sides of the grid do not move together, which is the point of marking
+both. On 2026-08-18 and 08-19, CAISO's carbon ran **cleaner at the evening peak
+than in the small hours** — spreads of 0.76x and 0.73x — because the sun that
+serves the California evening has set by midnight. Cheap hours are not
+automatically clean hours, and a board that only recorded price would have
+implied otherwise.
+
 **[→ Read the board](https://github.com/offpeak-ai/offpeak/blob/board-data/nightly/BOARD.md)**
 
 ## How it works
@@ -49,6 +56,23 @@ Output lands on the
 `nightly/BOARD.md` plus the raw JSON per night — because `main` is protected and
 would reject a nightly bot push.
 
+## Settled runs are a different ledger
+
+`BOARD.md` observes; it spends nothing at any venue. Runs that actually
+executed and actually billed go in `nightly/SETTLED.md` on the same branch,
+written by
+[`tools/settle_report.py`](https://github.com/offpeak-ai/offpeak/blob/main/tools/settle_report.py)
+from the receipts in
+[`receipts/`](https://github.com/offpeak-ai/offpeak/tree/main/receipts) — never
+by hand — and published by a manual workflow, because a settlement is a
+deliberate act.
+
+Every settled row carries its **scale**, and the column is not decoration. A
+few dozen jobs proving the mechanics end to end and a production book are both
+real settlements and are not the same evidence. A ledger that lets a reader
+confuse them is doing marketing rather than accounting, so the scale is printed
+before the money is.
+
 ## Sources
 
 - **Carbon** — [NESO carbon intensity](https://api.carbonintensity.org.uk),
@@ -57,6 +81,10 @@ would reject a nightly bot push.
   rates, GB region C, keyless.
 - **Power, US** — CAISO SP15 and ERCOT Houston day-ahead hourly, via
   [gridstatus](https://github.com/gridstatus/gridstatus), keyless.
+- **Carbon, US** — [EIA-930](https://www.eia.gov/electricity/gridmonitor/)
+  hourly generation by fuel for the CAISO and ERCOT balancing authorities,
+  through the EIA Hourly Grid Monitor. Needs a free API key; without one the
+  column records itself unavailable and nothing else changes.
 - **Tokens** — the published price sheets, not a measurement:
   [OpenAI](https://developers.openai.com/api/docs/pricing) and
   [Anthropic](https://platform.claude.com/docs/en/about-claude/pricing).
@@ -66,9 +94,21 @@ any venue.
 
 ## Honest limits
 
-- **Four zones, two of them thin.** GB carbon and GB power are half-hourly and
+- **Four zones, unevenly covered.** GB carbon and GB power are half-hourly and
   complete. CAISO SP15 and ERCOT Houston are **day-ahead hourly** prices, not
-  settled real-time ones, and they are power only — no US carbon leg yet.
+  settled real-time ones.
+- **US carbon is derived, GB carbon is measured.** NESO publishes an intensity;
+  EIA does not. The US columns are computed from EIA-930's hourly generation
+  mix times EIA's own CO2 coefficients and fleet heat rates — every input
+  published, the product an estimate, and marked `"basis": "derived"` in the
+  record so it is never confused with a measurement. It counts generation, not
+  consumption: imports and the carbon already stored in a battery are outside
+  what the method can see, and the share of generation EIA files under "other"
+  is reported per night rather than averaged in.
+- **EIA runs about a day behind.** The 06:30Z mark usually lands before EIA has
+  published the night it is marking, so the US carbon columns are often empty
+  at first sight and fill in on a later re-mark. A column that is not there yet
+  is recorded as unavailable, never as zero.
 - **The token column is published, not observed.** The 2.0x and the 4x are read
   off price sheets; only the grid columns are measurements. A published number
   and a marked one are different kinds of claim, and the board should not blur

{offpeak-0.2.1 → offpeak-0.2.2}/docs/quickstart.md

@@ -60,6 +60,22 @@ print(q.spread_usd, q.spread_pct)
 
     `Quote.basis` reports the provenance of every figure.
 
+!!! warning "Reasoning models spend the ceiling before they speak"
+    On models that reason before answering — OpenAI's gpt-5 family, the
+    o-series — `max_tokens` caps **reasoning plus visible output**, and the
+    reasoning goes first. Set it too low and the job bills a full ceiling of
+    reasoning tokens and returns an empty string: a `Result` that is
+    technically ok, costs real money, and says nothing.
+
+    This is not hypothetical. A real batch here ran 24 jobs at
+    `max_tokens=16`, billed 374 output tokens, and returned 24 empty strings.
+    Give a reasoning model room — hundreds of tokens, not dozens — and price
+    the ceiling you actually set, which is what `quote()` does.
+
+    `offpeak` sends the ceiling under whichever name the venue wants
+    (`max_completion_tokens` where the model demands it), but it cannot make a
+    ceiling large enough to answer in.
+
 ### If you know roughly what it will write
 
 A floor is honest but not always useful. When you do have a sense of the output

{offpeak-0.2.1 → offpeak-0.2.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "offpeak"
-version = "0.2.1"
+version = "0.2.2"
 description = "Deadline-priced inference: give AI jobs a deadline and run them on the cheapest venue — provider batch tiers (−50%) today. Same model, same tokens, a different hour."
 readme = "README.md"
 license = "Apache-2.0"

offpeak-0.2.2/receipts/2026-08-22-mechanics-1.json

@@ -0,0 +1,32 @@
+{
+  "run_id": "2026-08-22-mechanics-1",
+  "scale": "mechanics proof (48 jobs, two venues)",
+  "settled_utc": "2026-08-21T21:35:46-07:00",
+  "deadline": "06:00",
+  "price_sheet": "2026-08-21",
+  "offpeak_version": "0.2.2.dev0",
+  "jobs": 48,
+  "ok": 24,
+  "failed": 24,
+  "fell_back": 0,
+  "sla_met": 24,
+  "input_tokens": 787,
+  "output_tokens": 155,
+  "list_usd": 0.001562,
+  "paid_usd": 0.000781,
+  "captured_usd": 0.000781,
+  "captured_pct": 50.0,
+  "left_on_table_usd": 0.0,
+  "by_venue": {
+    "anthropic:batch": 24,
+    "openai:batch": 24
+  },
+  "failure_kinds": [
+    "HTTP 400"
+  ],
+  "notes": [
+    "The first genuine settlement. Twenty-four short public-domain lines, classified in one word each, through both venues' cheapest lanes on a 06:00 deadline.",
+    "The Anthropic leg settled 24/24 at the batch tier. The OpenAI leg was rejected job by job with HTTP 400 'max_tokens is not supported with this model; use max_completion_tokens instead' - no tokens were processed, so nothing was billed there.",
+    "The sync fallback did not rescue those jobs: it rescues jobs that never came back, and these came back as failures. Fixed in the driver; run 2 is the same book re-run."
+  ]
+}

offpeak-0.2.2/receipts/2026-08-22-mechanics-2.json

@@ -0,0 +1,28 @@
+{
+  "run_id": "2026-08-22-mechanics-2",
+  "scale": "mechanics proof (24 jobs, OpenAI, ceiling too low)",
+  "settled_utc": "2026-08-21T21:39:39-07:00",
+  "deadline": "06:00",
+  "price_sheet": "2026-08-21",
+  "offpeak_version": "0.2.2.dev0",
+  "jobs": 24,
+  "ok": 24,
+  "failed": 0,
+  "fell_back": 0,
+  "sla_met": 24,
+  "input_tokens": 728,
+  "output_tokens": 374,
+  "list_usd": 0.0005943999999999998,
+  "paid_usd": 0.0002971999999999999,
+  "captured_usd": 0.0002971999999999999,
+  "captured_pct": 50.0,
+  "left_on_table_usd": 0.0,
+  "by_venue": {
+    "openai:batch": 24
+  },
+  "notes": [
+    "The OpenAI leg re-run after the driver learned to send max_completion_tokens for the gpt-5 family. It settled: 24/24, billed, SLA met.",
+    "It also returned twenty-four empty strings. gpt-5.6 reasons before it answers and the 16-token ceiling was spent entirely on reasoning, leaving nothing to say. The money here bought 374 output tokens of nothing.",
+    "Published rather than quietly dropped: a settlement that billed for no usable output is exactly the kind of thing a receipt exists to make visible."
+  ]
+}

offpeak-0.2.2/receipts/2026-08-22-mechanics-3.json

@@ -0,0 +1,27 @@
+{
+  "run_id": "2026-08-22-mechanics-3",
+  "scale": "mechanics proof (24 jobs, OpenAI, ceiling sized to the model)",
+  "settled_utc": "2026-08-21T21:42:51-07:00",
+  "deadline": "06:00",
+  "price_sheet": "2026-08-21",
+  "offpeak_version": "0.2.2.dev0",
+  "jobs": 24,
+  "ok": 24,
+  "failed": 0,
+  "fell_back": 0,
+  "sla_met": 24,
+  "input_tokens": 728,
+  "output_tokens": 971,
+  "list_usd": 0.0013108,
+  "paid_usd": 0.0006554,
+  "captured_usd": 0.0006554,
+  "captured_pct": 50.0,
+  "left_on_table_usd": 0.0,
+  "by_venue": {
+    "openai:batch": 24
+  },
+  "notes": [
+    "The same twenty-four lines once more, with a 256-token ceiling that leaves a reasoning model room to answer in.",
+    "24/24 settled with real one-word answers, 971 output tokens, zero fallbacks, every SLA met."
+  ]
+}

{offpeak-0.2.1 → offpeak-0.2.2}/src/offpeak/__init__.py

@@ -18,7 +18,7 @@ from .prices import format_usd
 from .quote import Quote, VenueQuote, quote
 from .venues.base import BatchState, Venue
 
-__version__ = "0.2.1"
+__version__ = "0.2.2"
 
 __all__ = [
     "job",

{offpeak-0.2.1 → offpeak-0.2.2}/src/offpeak/venues/openai_batch.py

@@ -11,17 +11,44 @@ import json
 from ..job import Job, Result
 from .base import BatchState, Venue
 
-__all__ = ["OpenAIBatch", "build_jsonl", "parse_output_line"]
+__all__ = ["OpenAIBatch", "build_jsonl", "parse_output_line", "body_params"]
 
 _MODEL_PREFIXES = ("gpt-", "o1", "o3", "o4", "chatgpt-")
 _ENDPOINT = "/v1/chat/completions"
 
+# OpenAI's newer families reject ``max_tokens`` outright — "Unsupported
+# parameter: 'max_tokens' is not supported with this model. Use
+# 'max_completion_tokens' instead." — and a batch of a thousand jobs discovers
+# this one HTTP 400 at a time, hours after submission.
+#
+# Translating is the driver's job. A caller says ``max_tokens`` once and it
+# means the same thing at every venue; the venue that spells it differently is
+# the venue's problem, not the caller's.
+_MAX_COMPLETION_TOKENS_PREFIXES = ("o1", "o3", "o4", "gpt-5")
+
+
+def body_params(job: Job) -> dict:
+    """The job's params as this venue's API spells them.
+
+    An explicit ``max_completion_tokens`` always wins: the caller who reached
+    for the provider's own name for the field meant it, and sending both would
+    be rejected.
+    """
+    params = dict(job.params)
+    if "max_tokens" not in params:
+        return params
+    if not job.model.startswith(_MAX_COMPLETION_TOKENS_PREFIXES):
+        return params
+    ceiling = params.pop("max_tokens")
+    params.setdefault("max_completion_tokens", ceiling)
+    return params
+
 
 def build_jsonl(jobs: list[Job]) -> bytes:
     """Render *jobs* as OpenAI Batch API JSONL (one request per line)."""
     lines = []
     for j in jobs:
-        body = {"model": j.model, "messages": j.messages, **j.params}
+        body = {"model": j.model, "messages": j.messages, **body_params(j)}
         lines.append(
             json.dumps(
                 {"custom_id": j.id, "method": "POST", "url": _ENDPOINT, "body": body},
@@ -129,7 +156,7 @@ class OpenAIBatch(Venue):
     def run_sync(self, job: Job) -> Result:
         try:
             response = self.client.chat.completions.create(
-                model=job.model, messages=job.messages, **job.params
+                model=job.model, messages=job.messages, **body_params(job)
             )
         except Exception as exc:  # noqa: BLE001
             return Result(job=job, error=str(exc))

offpeak-0.2.2/tests/test_mechanics_run.py

@@ -0,0 +1,133 @@
+"""The capped settlement runner — everything that happens before money moves.
+
+These tests never submit anything: `quote()` makes no API calls, and both paths
+exercised here stop before the venues are touched.
+"""
+
+import importlib.util
+import json
+from pathlib import Path
+
+import pytest
+
+_spec = importlib.util.spec_from_file_location(
+    "mechanics_run", Path(__file__).resolve().parent.parent / "tools" / "mechanics_run.py"
+)
+mr = importlib.util.module_from_spec(_spec)
+_spec.loader.exec_module(mr)
+
+
+class TestBook:
+    def test_one_lane_per_model_over_the_same_lines(self):
+        jobs = mr.build_book(["claude-haiku-4-5", "gpt-5.6-luna"])
+        assert len(jobs) == 2 * len(mr.LINES)
+        assert {j.model for j in jobs} == {"claude-haiku-4-5", "gpt-5.6-luna"}
+
+    def test_every_job_carries_a_ceiling(self):
+        jobs = mr.build_book(["gpt-5.6-luna"], max_tokens=64)
+        assert all(j.params["max_tokens"] == 64 for j in jobs)
+
+    def test_the_default_ceiling_leaves_a_reasoning_model_room_to_answer(self):
+        # A ceiling smaller than the reasoning buys a bill and an empty string.
+        assert mr.DEFAULT_MAX_TOKENS >= 128
+
+    def test_the_lines_are_the_work_not_filler(self):
+        jobs = mr.build_book(["gpt-5.6-luna"])
+        assert mr.LINES[0] in jobs[0].messages[0]["content"]
+
+
+class TestCapGate:
+    def _out(self, tmp_path):
+        return str(tmp_path / "run")
+
+    def test_a_book_under_the_cap_passes_and_still_submits_nothing_on_dry_run(
+        self, tmp_path, capsys
+    ):
+        rc = mr.main(["--out", self._out(tmp_path), "--dry-run"])
+        out = capsys.readouterr().out
+        assert rc == 0
+        assert "under cap" in out
+        assert not (tmp_path / "run" / "handles.jsonl").exists()
+
+    def test_a_book_over_the_cap_aborts_before_submitting(self, tmp_path, capsys):
+        rc = mr.main(["--out", self._out(tmp_path), "--cap", "0.0000001"])
+        assert rc == 2
+        assert "ABORT" in capsys.readouterr().out
+        assert not (tmp_path / "run" / "handles.jsonl").exists()
+
+    def test_the_cap_counts_what_the_session_already_exposed(self, tmp_path, capsys):
+        # The cap is on the total, so a small book still trips it once enough
+        # has been spent around it.
+        rc = mr.main(
+            ["--out", self._out(tmp_path), "--models", "gpt-5.6-luna",
+             "--cap", "0.01", "--already-spent", "0.0099"]
+        )
+        assert rc == 2
+        assert "already exposed" in capsys.readouterr().out
+
+    def test_the_gate_prices_the_ceiling_not_a_hope(self, tmp_path, capsys):
+        # The quoted worst case must scale with the ceiling actually set.
+        mr.main(["--out", self._out(tmp_path), "--max-tokens", "16", "--dry-run"])
+        small = capsys.readouterr().out
+        mr.main(["--out", str(tmp_path / "run2"), "--max-tokens", "512", "--dry-run"])
+        large = capsys.readouterr().out
+
+        def worst(text):
+            line = next(ln for ln in text.splitlines() if ln.startswith("cap check"))
+            return float(line.split("$")[1].split()[0])
+
+        assert worst(large) > worst(small)
+
+
+class TestRefusesToResubmit:
+    def test_an_existing_handle_log_stops_the_run(self, tmp_path, capsys):
+        out = tmp_path / "run"
+        out.mkdir()
+        (out / "handles.jsonl").write_text(
+            json.dumps({"venue": "openai:batch", "handle": "batch_x", "jobs": 1}) + "\n"
+        )
+        rc = mr.main(["--out", str(out)])
+        assert rc == 3
+        assert "refusing to re-submit" in capsys.readouterr().out
+
+
+class TestCancelPath:
+    def test_cancelling_with_no_log_is_not_an_error(self, tmp_path, capsys):
+        rc = mr.main(["--out", str(tmp_path / "nothing-here"), "--cancel"])
+        assert rc == 0
+        assert "no handles recorded" in capsys.readouterr().out
+
+    def test_every_recorded_handle_is_cancelled_at_its_own_venue(self, tmp_path, capsys):
+        log = tmp_path / "handles.jsonl"
+        log.write_text(
+            json.dumps({"venue": "openai:batch", "handle": "batch_x", "jobs": 1}) + "\n"
+            + json.dumps({"venue": "anthropic:batch", "handle": "msgbatch_y", "jobs": 1}) + "\n"
+        )
+        cancelled = []
+
+        class FakeVenue:
+            def __init__(self, name):
+                self.name = name
+
+            def cancel(self, handle):
+                cancelled.append((self.name, handle))
+
+        mr.cancel_all(log, [FakeVenue("openai:batch"), FakeVenue("anthropic:batch")])
+        assert cancelled == [("openai:batch", "batch_x"), ("anthropic:batch", "msgbatch_y")]
+
+    def test_a_handle_from_an_unconfigured_venue_is_reported_not_skipped_silently(
+        self, tmp_path, capsys
+    ):
+        log = tmp_path / "handles.jsonl"
+        log.write_text(json.dumps({"venue": "groq:batch", "handle": "g_1", "jobs": 1}) + "\n")
+        mr.cancel_all(log, [])
+        assert "no venue to cancel with" in capsys.readouterr().out
+
+
+@pytest.mark.parametrize("flag,attr,value", [
+    ("--cap", "cap", 0.25),
+    ("--max-tokens", "max_tokens", 32),
+])
+def test_the_guard_rails_are_all_settable(flag, attr, value):
+    args = mr.parse_args([flag, str(value)])
+    assert getattr(args, attr) == value