eggpool 0.6.2__tar.gz → 0.6.4__tar.gz

{eggpool-0.6.2 → eggpool-0.6.4}/AGENTS.md

@@ -143,7 +143,7 @@ CI sets `PYTHONHASHSEED=0` and `TZ=UTC`; reproduce locally for deterministic res
 - **Database invariants**: SQLite WAL, single-connection serialization, `async with db.transaction():` for all DML.
 - **Quota and routing**: tier-based routing via `routing_priority`, `QuotaFairScorer`, upstream-authoritative suppression, same-tier fairness rotor.
 - **Error hierarchy**: `AggregatorError` → `UpstreamError` → specific subclasses. `CapabilityError` for thinking/reasoning capability mismatches. `TranscodeLossError` (HTTP 400) for loss-policy reject. `ProtocolMismatchError` for endpoint/model-protocol mismatches.
-- **Process model**: supervisor + Granian worker (`workers=1`), PID file lifecycle, daemon mode (default for `eggpool serve`; `--verbose` for foreground). Default `runtime_threads=2`, `database_worker_threads=2` (separate read-only stats connection). Startup logs `Granian profile: workers=1 runtime_threads=N database_worker_threads=M access_log=...`.
+- **Process model**: supervisor + Granian worker (`workers=1`), PID file lifecycle, daemon mode (default for `eggpool serve`; `--verbose` for foreground). Default `runtime_threads=4`, `database_worker_threads=2` (separate read-only stats connection). Startup logs `Granian profile: workers=1 runtime_threads=N database_worker_threads=M access_log=...`.
 - **Dashboard**: server-rendered HTML, Chart.js v4, grouped timeseries, CSS tooltips. Dashboard HTML pages use a 30s in-memory cache (`DashboardTelemetry`) for heavy aggregate queries. Telemetry exposes p50/p95 render times and slowest route under `/api/stats/runtime`. Progressive graph hydration (Phase 7) decouples page shell render from chart data: chart pages render a loading shell and hydrate via `data-chart-endpoint` after the shell arrives, with per-namespace/per-period cache TTLs (1h=15s, 24h=30-60s, 7d=120-240s, 30d=240-300s) and rollup-first query behavior suppressing raw fallback for windows >2h. Overview token row uses three explicit cards (`Accounted tokens`, `Fresh tokens`, `Provider cache hit rate`) so cache-heavy workloads (`cache_read > fresh_tokens`) cannot look like an arithmetic bug; the headline `Accounted tokens` card surfaces `input + output + cache_read + cache_write` while `Fresh tokens` keeps the legacy `input + output` semantics. The `Provider cache hit rate` card uses protocol-aware canonical terms: `cache_read_tokens_canonical` (hits), `cache_write_tokens_canonical` (warmup, not hits), and `cache_eligible_input_tokens` (denominator). Per-protocol denominator rules: OpenAI-compatible input is total-billed prompt tokens (`prompt_tokens`); Anthropic denominator is `input_tokens + cache_read + cache_creation` because Anthropic `input_tokens` is fresh-only. Cache writes/creation are never described as hits. Cache read share stays bounded to `cache_read / (input + cache_read + cache_write)`. Stats summary payload exposes `total_tokens` (legacy fresh-token volume, kept for API backward compatibility), `fresh_tokens`, and `accounted_tokens` — the dashboard uses `accounted_tokens` without changing the `total_tokens` API value. Dashboard `/models` page joins canonical model-info summaries to rendered rows via `_normalize_dashboard_model_row()` (splitting provider-suffixed `model_id` into `base_model_id`/`provider_id`/`_model_info_lookup_id`) and surfaces join-failure diagnostics through `ModelInfoDashboardState.matched_row_count` / `unmatched_sample`. `CatalogRowsState` mirrors `ModelInfoDashboardState` for the catalog row builder; the previously-silent `except Exception: return []` blocks now log the traceback and set `degraded_reason="fetch_error"`. The catalog accessor `ModelCatalogCache.get_provider_model_entries()` is the source of truth for the provider-scoped `/models` rows: returns a deterministic `dict[(model_id, provider_id), dict]`, excludes the deprecated placeholder, applies configured capability overrides when `_config` is attached, and emits shallow copies so callers cannot mutate the cache. Unresolved rows (`protocol=None`) are kept so the dashboard can render them as `available=False, catalog_status="unavailable"` rather than silently dropping them.
 - **Model capabilities**: protocol-neutral `ThinkingCapability` / `ModelCapabilities` with deterministic merge. Config overrides at `[model_capabilities."<id>".thinking]` and per-provider scoped.
 - **Catalog refresh**: non-destructive by default; destructive withdrawal gated on `authoritative=True AND allow_withdrawals=True`. `static_models` is the source of truth for provider-specific protocol.

{eggpool-0.6.2 → eggpool-0.6.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: eggpool
-Version: 0.6.2
+Version: 0.6.4
 Summary: A lightweight proxy that aggregates multiple LLM provider accounts behind one OpenAI-compatible endpoint
 Project-URL: Homepage, https://github.com/eggstack/eggpool
 Project-URL: Repository, https://github.com/eggstack/eggpool

{eggpool-0.6.2 → eggpool-0.6.4}/architecture/README.md

@@ -2693,14 +2693,14 @@ Granular dashboard handlers touched: `handle_runtime`, `handle_cache`, `handle_t
 
 #### Granian Runtime Threads
 
-`ServerConfig.threads` defaults to `2` (was `1`). The default config example now exposes the knob:
+`ServerConfig.threads` defaults to `4` (was `2`). The default config example now exposes the knob:
 
 ```toml
 [server]
-threads = 2
+threads = 4
 ```
 
-Granian still passes `workers=1`. Two runtime threads let a single worker multiplex streaming proxy traffic, dashboard requests, and lightweight background work without the single-event-loop starvation that becomes visible when long-lived SSE streams block the loop. `1` remains the documented minimum-footprint override for extremely constrained devices. Startup logs the effective profile:
+Granian still passes `workers=1`. Four runtime threads let a single worker multiplex streaming proxy traffic, dashboard requests, and lightweight background work without the single-event-loop starvation that becomes visible when long-lived SSE streams block the loop. `1` remains the documented minimum-footprint override for extremely constrained devices. Startup logs the effective profile:
 
 ```text
 Granian profile: workers=1 runtime_threads=N database_worker_threads=M access_log=...
@@ -2945,7 +2945,7 @@ CLI reproducer.
 ### Critical rules
 
 - Do not raise `server.threads` or `workers` to "fix" high-concurrency
-  stream instability. Granian runs `workers=1` and `threads=2`; raising
+  stream instability. Granian runs `workers=1` and `threads=4`; raising
   `threads` does not improve HTTPX concurrency and raising `workers`
   multiplies the SQLite connection budget.
 - Keep `database.worker_threads = 2` (default) so the dashboard

{eggpool-0.6.2 → eggpool-0.6.4}/config.example.toml

@@ -5,9 +5,7 @@ port = 11300
 log_level = "INFO"
 access_log = true
 # Granian runtime threads inside the single worker process. Keep workers=1.
-# 1 is minimum footprint; 2 is usually a better default for Raspberry Pi 4/5
-# and other SBC installs that serve dashboard traffic during active proxy use.
-threads = 2
+threads = 4
 
 [upstream]
 base_url = "https://opencode.ai/zen/go/v1"

{eggpool-0.6.2 → eggpool-0.6.4}/docs/deployment.md

@@ -317,7 +317,7 @@ dashboard responsiveness under request load.
 
 ```toml
 [server]
-threads = 2
+threads = 4
 access_log = false
 
 [database]
@@ -403,7 +403,7 @@ and restart. Revert to `"sampled"` or `"off"` when done.
 | Symptom | Likely cause | Knob to adjust |
 |---|---|---|
 | Dashboard loads slowly under request load | DB lock contention | Set `database.worker_threads = 2` |
-| Dashboard still slow | Runtime thread starvation | Set `server.threads = 2` |
+| Dashboard still slow | Runtime thread starvation | Check dashboard telemetry, then tune `server.threads` above 4 only on capable hardware |
 | High write volume / microSD wear | Routing trace writes | Set `routing.trace.mode = "sampled"` |
 | Background tasks cluster at minute boundaries | Task scheduling | Default is staggered; check `initial_delay_s` |
 | Stale dashboard data | Cache TTL | Wait 30s or check dashboard cache settings |
@@ -565,11 +565,11 @@ address for the probe). Either a live PID or a 200 from the probe
 causes the new `serve` to exit non-zero so a stale PID file is
 never silently overwritten.
 
-The primary tuning knob is `[server].threads` (int, default `2`,
+The primary tuning knob is `[server].threads` (int, default `4`,
 min `1`, max `64`), which sets Granian `runtime_threads` — the
-number of event-loop threads in the worker. The default of two
-threads is recommended for Raspberry Pi 4/5 so the single Granian
-worker can multiplex streaming proxy traffic + dashboard requests
+number of event-loop threads in the worker. The default of four
+threads lets the single Granian worker multiplex streaming proxy
+traffic + dashboard requests
 without single-event-loop starvation. Set `threads = 1` for
 extremely constrained devices, or raise it on capable hardware:
 

{eggpool-0.6.2 → eggpool-0.6.4}/docs/opencode-stream-stability.md

@@ -103,10 +103,10 @@ For OpenCode-style workloads, budget these limits:
 - `database.worker_threads`: 2 is sufficient for dashboards; raise
   to 4 if the dashboard itself lags.
 
-Do **not** raise `server.threads` or `workers`. Granian runs one
-worker with a single asyncio loop. Adding threads does not improve
-HTTPX concurrency, and adding workers multiplies the connection
-budget per upstream IP.
+Keep `server.threads` bounded and leave `workers=1`. The default of
+4 helps the single worker multiplex dashboard work and active streams;
+it does not increase HTTPX pool capacity. Adding workers multiplies the
+connection budget per upstream IP.
 
 ## Closure validation
 
@@ -151,4 +151,4 @@ The `/api/stats/runtime` endpoint exposes these stream-stability sections:
 - `stream_diagnostics` — outcome counters and histograms
 - `finalization_retry_queue` — queue depth, drain stats, overflow/duplicate counters
 - `routing_trace_guard` — skip rate and lock-pressure threshold
-- `db.contention` — lock-wait p50/p95/p99/max and sample count
+- `db.contention` — lock-wait p50/p95/p99/max and sample count

{eggpool-0.6.2 → eggpool-0.6.4}/docs/providers.md

@@ -445,7 +445,7 @@ state:
 
 ```toml
 [server]
-threads = 2
+threads = 4
 
 [database]
 worker_threads = 2
@@ -468,7 +468,7 @@ read timeout to absorb slow model first-token latencies:
 
 ```toml
 [server]
-threads = 2
+threads = 4
 
 [database]
 worker_threads = 2
@@ -482,9 +482,10 @@ write_timeout_s = 30
 pool_timeout_s = 60
 ```
 
-Keep `server.threads` low — Granian already serializes the asyncio
-event loop, so adding threads does not improve HTTPX throughput. The
-real lever is `max_connections`.
+Keep `server.threads` bounded — it helps the single worker multiplex
+dashboard work alongside active streams, but it does not raise HTTPX
+pool capacity. The real upstream-concurrency lever is
+`max_connections`.
 
 ### Diagnostic low-noise mode
 

{eggpool-0.6.2 → eggpool-0.6.4}/docs/raspberry-pi.md

@@ -65,7 +65,7 @@ explicitly set the recommended profile:
 
 ```toml
 [server]
-threads = 2          # 2 runtime threads for dashboard responsiveness
+threads = 4          # runtime threads for dashboard responsiveness
 access_log = false   # optional: reduce I/O noise after initial setup
 
 [database]
@@ -100,20 +100,19 @@ include_score_components = false
 ## Process Model
 
 EggPool's default process model is Pi-friendly: one `eggpool serve`
-supervisor process plus one Granian worker, with two event-loop
+supervisor process plus one Granian worker, with four event-loop
 threads in the worker (configurable via `[server].threads`). Both
 processes appear as `eggpool` in `ps` / `top` (no generic `python`
-entry), so the total footprint is two processes and two threads before
+entry), so the total footprint is two processes and four runtime threads before
 considering any upstream outbound connections.
 
 The single tuning knob for per-worker concurrency is `[server].threads`
-(int, default `2`, max `64`), which maps to Granian `runtime_threads`.
-The default is correct for Pi 4 / Pi 5; raise it only if your workload
-genuinely needs more concurrency than two event loops can deliver:
+(int, default `4`, max `64`), which maps to Granian `runtime_threads`.
+Set it to `1` for minimum footprint or raise it on capable hardware:
 
 ```toml
 [server]
-threads = 2
+threads = 4
 ```
 
 The PID file path is resolved by `eggpool.runtime_paths.default_pid_file()` in this precedence: `$EGGPOOL_PID_FILE` → `$XDG_RUNTIME_DIR/eggpool.pid` → `~/.local/state/eggpool/eggpool.pid` → `/tmp/eggpool-<UID>.pid`, and is owned by the supervisor. If `eggpool serve` ever exits non-zero with a message about an existing instance, that is the duplicate-instance guard catching a live PID or a successful `/v1/healthz` probe — check `pgrep -f eggpool` before retrying.

{eggpool-0.6.2 → eggpool-0.6.4}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "eggpool"
-version = "0.6.2"
+version = "0.6.4"
 description = "A lightweight proxy that aggregates multiple LLM provider accounts behind one OpenAI-compatible endpoint"
 readme = "README.md"
 requires-python = ">=3.11"

{eggpool-0.6.2 → eggpool-0.6.4}/src/eggpool/_share/config.example.toml

@@ -5,9 +5,9 @@ port = 11300
 log_level = "INFO"
 access_log = true
 # Granian runtime threads inside the single worker process. Keep workers=1.
-# 1 is minimum footprint; 2 is usually a better default for Raspberry Pi 4/5
-# and other SBC installs that serve dashboard traffic during active proxy use.
-threads = 2
+# 1 is minimum footprint; 4 is the default for installs that serve dashboard
+# traffic during active proxy use. Granian still runs workers=1.
+threads = 4
 
 [upstream]
 base_url = "https://opencode.ai/zen/go/v1"

{eggpool-0.6.2 → eggpool-0.6.4}/src/eggpool/app.py

@@ -1002,16 +1002,22 @@ async def _lifespan_runtime(app: FastAPI) -> AsyncGenerator[None]:
             offset_30d_microdollars=acct_cfg.monthly_offset_microdollars,
         )
 
-    # 20. Metrics write coalescer for buffered analytics
+    # 20. Metrics write coalescer for buffered analytics.  Writes stay on the
+    # primary connection; dashboard rollup reads use the same dedicated
+    # read-only stats connection as the rest of StatsService when available.
     rollup_repo = UsageRollupRepository(db)
+    stats_rollup_repo = UsageRollupRepository(stats_db)
+    stats_account_backoff_repo = (
+        account_backoff_repo if stats_db is db else AccountBackoffRepository(stats_db)
+    )
 
     # 21. Statistics service
     app.state.stats = StatsService(
         stats_db,
         health_manager=health_manager,
         ping_repo=PingRepository(stats_db),
-        account_backoff_repo=account_backoff_repo,
-        rollup_repo=rollup_repo,
+        account_backoff_repo=stats_account_backoff_repo,
+        rollup_repo=stats_rollup_repo,
     )
     if config.dashboard.enabled and stats_db is db:
         logger.warning(

{eggpool-0.6.2 → eggpool-0.6.4}/src/eggpool/dashboard/render.py

@@ -362,6 +362,16 @@ def _format_tooltip_date(day_str: str) -> str:
     return parsed.strftime("%a, %b %-d %Y")
 
 
+def _heatmap_today() -> date:
+    """Return the heatmap's UTC calendar day.
+
+    Request timestamps and the heatmap query range are normalized to UTC;
+    using the host-local date here would shift the final column around local
+    midnight on deployments whose system timezone is not UTC.
+    """
+    return datetime.now(UTC).date()
+
+
 def get_theme_css(theme_name: str, themes_dir: str | None = None) -> str:
     """Load a theme by name and return the CSS :root block, or empty string.
 
@@ -548,8 +558,8 @@ def _render_layout(
     &middot; <span id="dashboard-updated">ready</span>{update_indicator}</small>
 </footer>
 {script_block}
-{chart_script}
 {dashboard_script}
+{chart_script}
 </body>
 </html>"""
 
@@ -1549,34 +1559,38 @@ def _render_bandwidth_heatmap(
     day_requests: dict[str, int] = {}
     if value_field == "bytes":
         day_values: dict[str, int] = {}
+        in_out: dict[str, tuple[int, int]] = {}
         for row in daily_data:
             day_str = str(row.get("day", ""))
-            val = int(row.get("bytes_emitted", 0)) + int(row.get("bytes_received", 0))
-            day_values[day_str] = val
-            day_requests[day_str] = int(row.get("request_count", 0))
-        formatter: Any = format_bytes
-        in_out: dict[str, tuple[int, int]] = {
-            str(row.get("day", "")): (
-                int(row.get("bytes_received", 0)),
-                int(row.get("bytes_emitted", 0)),
+            received = int(row.get("bytes_received", 0))
+            emitted = int(row.get("bytes_emitted", 0))
+            day_values[day_str] = day_values.get(day_str, 0) + received + emitted
+            day_requests[day_str] = day_requests.get(day_str, 0) + int(
+                row.get("request_count", 0)
             )
-            for row in daily_data
-        }
+            previous_received, previous_emitted = in_out.get(day_str, (0, 0))
+            in_out[day_str] = (
+                previous_received + received,
+                previous_emitted + emitted,
+            )
+        formatter: Any = format_bytes
     elif value_field == "total_tokens":
-        day_values = {
-            str(row.get("day", "")): int(row.get("total_tokens", 0))
-            for row in daily_data
-        }
+        day_values = {}
         formatter = format_tokens
         in_out = {}
         for row in daily_data:
             day_str = str(row.get("day", ""))
-            day_requests[day_str] = int(row.get("request_count", 0))
+            day_values[day_str] = day_values.get(day_str, 0) + int(
+                row.get("total_tokens", 0)
+            )
+            day_requests[day_str] = day_requests.get(day_str, 0) + int(
+                row.get("request_count", 0)
+            )
     else:
         raise ValueError(f"unsupported heatmap value_field: {value_field!r}")
 
     # Date range: last 180 days ending today
-    today = date.today()
+    today = _heatmap_today()
     start_date = today - timedelta(days=179)
 
     # Pad start to Sunday (GitHub convention: weeks start on Sunday)
@@ -1587,10 +1601,15 @@ def _render_bandwidth_heatmap(
     # Calculate number of weeks
     num_weeks = ((today - grid_start).days // 7) + 1
 
-    # Find max value for color scaling
-    max_val = max(day_values.values()) if day_values else 1
-    if max_val == 0:
-        max_val = 1
+    # Find max value for color scaling from the rendered window only.  The
+    # service may return a boundary row just outside the calendar window
+    # because its query uses a rolling timestamp range; that row must not
+    # make every visible day look artificially faint.
+    visible_values = [
+        day_values.get((start_date + timedelta(days=offset)).isoformat(), 0)
+        for offset in range((today - start_date).days + 1)
+    ]
+    max_val = max(visible_values, default=0) or 1
 
     # Color scale (theme-aware or default GitHub Primer green)
     colors = heatmap_colors or _DEFAULT_HEATMAP_COLORS
@@ -1671,7 +1690,6 @@ def _render_bandwidth_heatmap(
                 color = _get_color(value)
                 x = left_margin + week * step
                 y = top_margin + day_of_week * step
-                tooltip = f"{day_str}: {formatter(value)}"
                 request_count = day_requests.get(day_str, 0)
                 request_count_text = (
                     f"{request_count:,} request{'s' if request_count != 1 else ''}"
@@ -1691,11 +1709,14 @@ def _render_bandwidth_heatmap(
                         f"{request_count_text}"
                     )
                 tooltip_attr = _stdlib_escape(tooltip_text, quote=True)
+                tooltip_title_text = tooltip_text.replace("\n", " — ")
+                tooltip_title = f"{day_str}: {tooltip_title_text}"
                 cells.append(
                     f'<rect x="{x}" y="{y}" width="{cell_size}" '
                     f'height="{cell_size}" rx="2" fill="{color}" '
                     f'class="heatmap-cell" pointer-events="none">'
-                    f"<title>{_stdlib_escape(tooltip)}</title></rect>"
+                    f"<title>{_stdlib_escape(tooltip_title)}"
+                    f"</title></rect>"
                 )
                 hitboxes.append(
                     f'<div class="heatmap-hitbox" '