cctally 1.74.0 → 1.75.1

package/CHANGELOG.md

@@ -5,6 +5,25 @@ based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+## [1.75.1] - 2026-07-20
+
+### Fixed
+- Database repair now rejects an SQLite shell without functional `.recover` support before creating a forensic copy, explains the required `SQLITE_ENABLE_DBPAGE_VTAB` capability, and runs its public Linux recovery tests against a checksum-pinned official SQLite shell instead of Ubuntu's incompatible distro build.
+- Automatic transcript-space reclamation now drives zero-column `incremental_vacuum` pragmas through SQLite's completion path in bounded, progress-checked chunks, fixing the Python 3.11/Linux case that could leave freed pages on the freelist after retention pruning.
+
+## [1.75.0] - 2026-07-20
+
+### Added
+- Add `cctally codex percent-breakdown`, a cached native seven-day Codex milestone command with the same terminal design as `cctally percent-breakdown`, exact root/limit selection, current or retained cycle selection, query-time pricing, and matching five-hour context. The materialized dashboard projection is read by default; `--sync` explicitly refreshes it.
+- Add guided `stats.db` corruption repair and SQLite-native online backups, with verified recovery, preserved usage history/schema version, safe original-file retention, active-writer and production guards, and actionable malformed-database errors. (#314)
+
+### Fixed
+- Dashboard Codex hero now uses the exact Claude hero composition and metric order. Its native seven-day cycle supplies Week Usage, optional 5-hour usage, reset countdown, cycle spend, `$ / 1%`, forecast-at-reset, prior-cycle `$ / 1%` delta, and snapshot age; unavailable cycle accounting stays explicit instead of falling back to token or budget tiles.
+- Force-refresh and automatic OAuth refresh now serialize their complete 429 backoff transitions under the same selected-state lock. Concurrent rate limits preserve both the consecutive-429 count and furthest deadline, while a completed force-refresh can no longer erase a newer automatic-refresh cooldown. (#309)
+- Dashboard Codex modals now populate the canonical Claude-shaped current-cycle, session, period, project, cache, forecast, alert, and **$/1% Trend** views from native session metadata, quota observations, token accounting, and computed cache/model breakdowns instead of sparse placeholders or renamed semantics. The current-cycle modal reuses the complete durable quota breakdown, preserving every 1% crossing even when the dashboard's bounded observation tail begins later. (#319)
+- The current 5-hour block no longer lingers as an approximate (`~`) window in `cctally blocks` and the dashboard Blocks panel while the statusline already shows the correct 5-hour data. When a new 5-hour window opened while your usage percentage was flat, `record-usage` deduped the unchanged tick and never wrote the new window's canonical anchor, so the active block fell back to its heuristic start until the percentage next moved (a lag of up to several minutes at window start). The dedup self-heal now also materializes the rolled-over window's anchor, so the active block reads as API-anchored right away; the statusline was always correct because it renders the live rate limits rather than the stored snapshot.
+- Conversation-transcript retention now reclaims disk automatically and defaults to a tighter window, so `cache.db` no longer grows to multiple gigabytes of transcript text and search index that slow every dashboard sync tick — which, under multi-process contention (a second dashboard plus status-line hooks), could wedge the sync into a "sync paused" state and leave Codex quota/weekly panels blank. A freshly-created `cache.db` now uses SQLite `INCREMENTAL` auto-vacuum, and the once-a-day retention prune returns the freed pages to the OS (physically reclaimed on the next end-of-sync checkpoint) instead of requiring a manual `cctally db vacuum`; a `cache.db` created by an older version keeps the legacy behavior until a one-time `cctally db vacuum` or `cctally cache-sync --rebuild`, after which it too reclaims automatically. The default `conversation.retention_days` is now 90 (was 180) — set your own window or `off` with `cctally config set conversation.retention_days`.
+
 ## [1.74.0] - 2026-07-19
 
 ### Fixed

package/bin/_cctally_cache.py

@@ -4753,6 +4753,15 @@ def open_cache_db() -> sqlite3.Connection:
     except OSError as exc:
         eprint(f"[cache] could not chmod cache.db 0600 ({exc}); continuing")
 
+    # #313 P3 reclaim: create a fresh cache.db in INCREMENTAL auto-vacuum mode so
+    # the transcript-retention prune can return freed pages to the OS via
+    # `PRAGMA incremental_vacuum` instead of only growing the freelist — the
+    # conversation-transcript store is what bloated cache.db to multiple GB and
+    # slowed every sync tick. This MUST run before the first page is written
+    # (before journal_mode=WAL / any CREATE TABLE) to take effect; it is a
+    # harmless no-op on an already-created DB, which keeps its existing mode
+    # until a full VACUUM (`cctally db vacuum`) or a `cache-sync --rebuild`.
+    conn.execute("PRAGMA auto_vacuum=INCREMENTAL")
     conn.execute("PRAGMA journal_mode=WAL")
     # #297: 15 s (was 5 s) lets a writer wait out a slow-but-normal sync (>5 s)
     # instead of instantly erroring with "database is locked". Covers plain

package/bin/_cctally_config.py

@@ -333,7 +333,7 @@ ALLOWED_CONFIG_KEYS = (
 
 _CODEX_BUDGET_LEAF_PREFIX = "budget.codex."
 
-_DEFAULT_CONVERSATION_RETENTION_DAYS = 180
+_DEFAULT_CONVERSATION_RETENTION_DAYS = 90
 
 
 def _validate_retention_days_value(raw: object) -> int:
@@ -374,9 +374,9 @@ def _validate_retention_days_value(raw: object) -> int:
 def resolve_retention_days(config: dict) -> int:
     """Effective conversation transcript retention in days (F8).
 
-    Default is ``180``; ``0`` disables retention (keep forever). Any malformed
+    Default is ``90``; ``0`` disables retention (keep forever). Any malformed
     persisted value (non-int, boolean, negative, un-parseable string) degrades
-    to the safe 180 default rather than raising.
+    to the safe 90 default rather than raising.
     """
     default = _DEFAULT_CONVERSATION_RETENTION_DAYS
     block = config.get("conversation") if isinstance(config, dict) else None

package/bin/_cctally_core.py

@@ -1098,11 +1098,19 @@ def open_db() -> sqlite3.Connection:
     _log_migration_error = c._log_migration_error
     _clear_migration_error_log_entries = c._clear_migration_error_log_entries
 
+    repair_marker = DB_PATH.with_name("stats.db.repairing")
+    if repair_marker.exists():
+        raise c.StatsDbMaintenanceError()
     ensure_dirs()
+    if repair_marker.exists():
+        raise c.StatsDbMaintenanceError()
     conn = sqlite3.connect(DB_PATH)
+    if repair_marker.exists():
+        conn.close()
+        raise c.StatsDbMaintenanceError()
     conn.row_factory = sqlite3.Row
     # #279 S1 F4: probe connect + initial PRAGMAs so a corrupt stats.db (the
-    # non-re-derivable DB) surfaces as a one-line diagnosis + exit 2 instead of
+    # non-re-derivable DB) surfaces as a one-line diagnosis + staged exit 3 instead of
     # a raw traceback. The catch boundary is DELIBERATELY narrow — ONLY the
     # connect/PRAGMA/probe below. The DDL + `_run_pending_migrations` region
     # further down is NOT wrapped: migration-handler failures have their own
@@ -1133,8 +1141,9 @@ def open_db() -> sqlite3.Connection:
         raise c.StatsDbCorruptError(
             f"stats.db appears corrupt or unreadable ({exc}). path: {DB_PATH}. "
             f"Not auto-recreated — it holds your recorded usage history. "
-            f'Recovery: back up the file, then try `sqlite3 "{DB_PATH}" ".recover"`, '
-            f"or move it aside to start fresh. If this recurs, please file an issue."
+            "Recovery: run `cctally db repair --db stats --yes`; it preserves "
+            "the corrupt original before replacing anything. Do not copy, "
+            "restore, or move the live DB by hand."
         ) from exc
     conn.execute(
         """

package/bin/_cctally_dashboard.py

@@ -483,16 +483,20 @@ def _source_safe_native_detail(row: Mapping, *, source: str,
     common = {"detail_kind": f"{source}_{resource}", "key": key}
     allowed = {
         "session": (
-            "last_activity", "cost_usd", "input_tokens", "cached_input_tokens",
+            "label", "project", "started_at", "duration_min", "last_activity",
+            "cost_usd", "input_tokens", "cached_input_tokens",
             "output_tokens", "reasoning_output_tokens", "total_tokens", "models",
+            "model_breakdowns",
         ),
         "project": (
-            "first_seen", "last_seen", "cost_usd", "input_tokens",
+            "label", "first_seen", "last_seen", "range_start", "range_end",
+            "cost_usd", "input_tokens",
             "cached_input_tokens", "output_tokens", "reasoning_output_tokens",
-            "total_tokens", "session_count", "sessions_count",
+            "total_tokens", "session_count", "sessions_count", "models", "sessions",
         ),
         "block": (
-            "resets_at", "current_percent", "orphaned", "forecast",
+            "label", "start_at", "end_at", "resets_at", "current_percent",
+            "orphaned", "is_active", "cost_usd", "model_breakdowns", "forecast",
             "window_minutes", "observed_slot", "milestones",
         ),
     }[resource]
@@ -570,6 +574,9 @@ def _codex_detail_inputs(snapshot):
             week_start_name=semantics.week_start_name,
             speed=semantics.speed,
             codex_budget=semantics.codex_budget,
+            codex_quota_actual_thresholds=semantics.codex_quota_actual_thresholds,
+            codex_quota_projected_thresholds=semantics.codex_quota_projected_thresholds,
+            cache_report_anomaly_threshold_pp=semantics.cache_report_anomaly_threshold_pp,
         )
         qualified = load_qualified_codex_entries(
             range_start,
@@ -769,16 +776,15 @@ def _codex_partial_source_detail(snapshot, row: Mapping, *, resource: str,
     detail = _source_safe_native_detail(
         row, source="codex", resource=resource, key=key,
     )
-    detail.update({
-        "metadata_availability": "partial",
-        "metadata_reason": (
-            "Codex project metadata is incomplete; available accounting totals "
-            "are preserved."
-        ),
-    })
+    metadata_missing = resource != "session" or not row.get("project")
+    if metadata_missing:
+        detail.update({
+            "metadata_availability": "partial",
+            "metadata_reason": "Project metadata is unavailable for this item.",
+        })
     if resource == "session":
         detail.setdefault("models", [])
-        detail["model_breakdowns"] = []
+        detail.setdefault("model_breakdowns", [])
     elif resource == "project":
         generated_at = getattr(snapshot, "generated_at", None)
         detail.setdefault("range_start", detail.get("first_seen"))
@@ -787,8 +793,8 @@ def _codex_partial_source_detail(snapshot, row: Mapping, *, resource: str,
             generated_at.astimezone(dt.timezone.utc).isoformat()
             if isinstance(generated_at, dt.datetime) else detail.get("last_seen"),
         )
-        detail["models"] = []
-        detail["sessions"] = []
+        detail.setdefault("models", [])
+        detail.setdefault("sessions", [])
     return detail
 
 
@@ -988,7 +994,19 @@ def build_source_detail(*, snapshot, source: str, resource: str,
     if source == "claude":
         return _build_claude_source_detail(snapshot, resource=resource, key=key)
     try:
-        return _build_codex_source_detail(snapshot, resource=resource, key=key)
+        detail = _build_codex_source_detail(snapshot, resource=resource, key=key)
+        if resource == "session":
+            for field in ("label", "project", "started_at", "duration_min"):
+                detail[field] = row.get(field)
+        elif resource == "project":
+            detail["label"] = row.get("label")
+        elif resource == "block":
+            for field in (
+                "label", "start_at", "end_at", "is_active", "cost_usd",
+                "model_breakdowns",
+            ):
+                detail[field] = row.get(field)
+        return detail
     except sys.modules["_cctally_source_analytics"].QualifiedMetadataUnavailable:
         return _codex_partial_source_detail(
             snapshot, row, resource=resource, key=key,

package/bin/_cctally_dashboard_conversation.py

@@ -39,6 +39,7 @@ edits — the forwarders hit ``sys.modules["_cctally_dashboard"]`` at call time
 from __future__ import annotations
 
 import json
+import re
 import socket
 import sqlite3
 import sys
@@ -224,8 +225,7 @@ _QUALIFIED_SEARCH_ACCEPTED = ("source", "q", "kind", "limit", "cursor")
 _QUALIFIED_FACETS_ACCEPTED = ("source",)
 # A raw browse cursor is a conversation key — printable + URL-safe by construction
 # (§2.2). Syntactic-only validation: reject whitespace/control/empty; echo raw.
-import re as _re_qs  # noqa: E402 — module-level compile for the browse-cursor check
-_BROWSE_CURSOR_RE = _re_qs.compile(r"\A[!-~]+\Z")
+_BROWSE_CURSOR_RE = re.compile(r"\A[!-~]+\Z")
 
 
 def _resolve_effective_speed():