cctally 1.87.2 → 1.88.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,12 @@ based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
5
5
 
6
6
  ## [Unreleased]
7
7
 
8
+ ## [1.88.0] - 2026-07-31
9
+
10
+ ### Fixed
11
+ - The Codex hero's Snapshot chip no longer reports stale evidence while the dashboard's own freshness state for the same quota window reads fresh. One `/api/data` response could carry two different capture timestamps for a single quota window, and the chip aged off the older one: the initial build stamped the active row with the timestamp of the last reading whose VALUE changed, while that same row's freshness came from the last reading actually received — and because OpenAI repeats an unchanged percentage while you are idle, the two drift apart in exactly the common case. `active[].captured_at` now always means evidence recency, the same observation behind the row's freshness, and the percentage it reports is unchanged. The idle clock additionally refreshes every per-account view rather than only the merged one, so a focused account no longer reads freshness frozen at page load, and the hero's own copy of the quota summary is re-derived on each tick instead of keeping its build-time value. Retained quota windows and their live rows are now capped together, so an install tracking more than 250 windows can no longer publish a live row whose history was dropped or a summary percentage belonging to a window that was. The source envelope's `source_schema_version` moves to 2, because a published value changed meaning and a dashboard tab left open across an in-place upgrade does meet the new server over its existing event stream. That tab keeps working either way — nothing reads the version yet — so the bump is a correctness signal for future readers rather than the thing that preserves compatibility. (#429)
12
+ - Codex sessions started outside the Desktop app — through the Codex MCP server, the CLI, `exec`, subagents, and older Desktop builds — now appear in the Conversation Viewer and resolve their project in Recent Sessions. Codex is part-way through rolling out the field cctally used to identify a conversation, and until now a session whose log omitted it was given no identity at all: its transcript was never indexed, and the sessions card could only report that project metadata was unavailable. cctally now infers the missing value, so those sessions are ordinary sessions with no visible difference from any other. On upgrade, existing history is re-read once in the background to repair it; a session's spend stays attributed to the account it was always attributed to, and a session whose log carries no usable identifier at all is left alone rather than guessed at. Because the repair re-reads the session logs themselves, totals are unchanged for every session still on disk — but a session whose log Codex has since deleted is no longer counted, so long-run totals can fall slightly. Sessions with no working directory recorded still appear, and still honestly report their project as unavailable. One visible consequence of that re-read: it briefly restores transcripts older than your configured retention window, so the ordinary retention sweep runs straight afterwards and those older conversations leave the Conversation Viewer on upgrade instead of lingering past their window.
13
+
8
14
  ## [1.87.2] - 2026-07-30
9
15
 
10
16
  ### Fixed
@@ -511,6 +511,38 @@ _codex_conversation_fts_full_clear = _cctally_db_sib._codex_conversation_fts_ful
511
511
  # writes (#179) so the ON CONFLICT idiom lives in one place. Caller commits.
512
512
  _set_cache_meta = _cctally_db_sib._set_cache_meta
513
513
 
514
+ # Byte-zero Codex replay markers (spec
515
+ # docs/superpowers/specs/2026-07-30-codex-thread-source-inference-design.md
516
+ # §4.3). Cache migration 035 / conversations migration 002 write them and clear
517
+ # NO table; the sync functions consume them, because only the sync owns the
518
+ # replay semantics that keep the repair safe:
519
+ #
520
+ # * `sync_codex_cache` ORs the cache-side marker into its own `rebuild`, so
521
+ # the rebuild path captures `rebuild_known_identities` before clearing. A
522
+ # migration clearing `codex_session_files` directly would leave the next
523
+ # ordinary sync with an empty snapshot, sending every re-read rollout to the
524
+ # live-`auth.json` branch and re-attributing historical spend (§4.1).
525
+ # * `sync_codex_conversations` defers on the CONVERSATIONS marker until the
526
+ # cache-side one has cleared, because `_recompute_codex_rollups` reads the
527
+ # thread row from cache.db and a missing one stamps a materialized
528
+ # "(unassigned)" project the read path then prefers permanently (§4.2).
529
+ #
530
+ # The conversations key is deliberately DISTINCT from
531
+ # `conversation_rebuild_codex_pending`: `_ensure_codex_conversation_contract`
532
+ # consumes that one by replaying normalization over already-retained events —
533
+ # which preserves their NULL conversation keys — and then deletes it, silently
534
+ # discarding the repair.
535
+ #
536
+ # The keys themselves are defined in the pure kernel `_lib_codex_conversation`
537
+ # and re-exported here, so the read-side authority probe and the doctor shell
538
+ # bind the same names instead of repeating a SQL string literal.
539
+ CODEX_REPLAY_FROM_ZERO_KEY = (
540
+ _lib_codex_conversation.CODEX_REPLAY_FROM_ZERO_KEY)
541
+ CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY = (
542
+ _lib_codex_conversation.CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY)
543
+ CODEX_REPLAY_BLOCKED_KEY = (
544
+ _lib_codex_conversation.CODEX_REPLAY_BLOCKED_KEY)
545
+
514
546
 
515
547
  # cache.db WAL hardening (#297). See
516
548
  # docs/superpowers/specs/2026-07-13-cache-db-wal-hardening-design.md.
@@ -4903,6 +4935,24 @@ def sync_codex_cache(
4903
4935
  # and bypasses every whole-tree operation (orphan prune, root prune,
4904
4936
  # global quota reconcile) — see the guards threaded through below.
4905
4937
  targeted = only_paths is not None
4938
+
4939
+ # A pending byte-zero replay is consumed HERE, not by the migration that
4940
+ # armed it, so the rebuild path below captures `rebuild_known_identities`
4941
+ # before clearing. A migration that cleared `codex_session_files`
4942
+ # directly would leave the next ordinary sync with an empty snapshot,
4943
+ # sending every re-read rollout to the live-auth branch and
4944
+ # re-attributing historical spend to whoever is authenticated now.
4945
+ replay_pending = conn.execute(
4946
+ "SELECT 1 FROM cache_meta WHERE key=?",
4947
+ (CODEX_REPLAY_FROM_ZERO_KEY,),
4948
+ ).fetchone() is not None
4949
+ if replay_pending and targeted:
4950
+ # A live-tail tick must DEFER, never raise through the
4951
+ # `targeted and rebuild` guard below.
4952
+ stats.deferred_reason = "replay_pending"
4953
+ return stats
4954
+ rebuild = rebuild or replay_pending
4955
+
4906
4956
  if targeted and rebuild:
4907
4957
  raise ValueError(
4908
4958
  "sync_codex_cache: only_paths is incompatible with rebuild")
@@ -5742,6 +5792,35 @@ def sync_codex_cache(
5742
5792
  else:
5743
5793
  conn.execute("DELETE FROM cache_meta WHERE key = ?",
5744
5794
  ("codex_torn_auth_deferred",))
5795
+ # Consume the byte-zero replay marker only after a clean full walk,
5796
+ # and only when THIS call observed it. A contended call returned
5797
+ # long before here, and a walk that failed or deferred a file leaves
5798
+ # the marker standing — a surviving marker is what makes the repair
5799
+ # retry on the next sync, and it is also what keeps
5800
+ # `sync_codex_conversations` deferred until the cache side genuinely
5801
+ # holds the replayed thread rows (§4.2). The `replay_pending` guard
5802
+ # is defense in depth: `open_cache_db` and this walk share an
5803
+ # exclusive lock today, so nothing can arm the marker in between —
5804
+ # but the conversations side has no such exclusion, and the two
5805
+ # clears must keep the same shape.
5806
+ if stats.files_failed == 0 and stats.files_deferred_torn == 0:
5807
+ if replay_pending:
5808
+ conn.execute("DELETE FROM cache_meta WHERE key = ?",
5809
+ (CODEX_REPLAY_FROM_ZERO_KEY,))
5810
+ conn.execute("DELETE FROM cache_meta WHERE key = ?",
5811
+ (CODEX_REPLAY_BLOCKED_KEY,))
5812
+ elif replay_pending:
5813
+ # A full walk ran and could NOT consume the marker, so the
5814
+ # replay — and with it every Codex transcript ingest, which
5815
+ # defers behind this marker — is stalled rather than merely
5816
+ # not-yet-run. `doctor` reads this; the deferral itself stays,
5817
+ # because running ahead is what stamps "(unassigned)" (§4.2).
5818
+ _set_cache_meta(conn, CODEX_REPLAY_BLOCKED_KEY, json.dumps({
5819
+ "at": dt.datetime.now(dt.timezone.utc).isoformat(
5820
+ timespec="seconds").replace("+00:00", "Z"),
5821
+ "files_failed": stats.files_failed,
5822
+ "files_deferred_torn": stats.files_deferred_torn,
5823
+ }, sort_keys=True))
5745
5824
  conn.commit()
5746
5825
  # Window-scoped spend adoption (spec
5747
5826
  # docs/superpowers/specs/2026-07-30-codex-window-scoped-spend-adoption.md).
@@ -7747,6 +7826,15 @@ def _ensure_codex_conversation_contract(conn: sqlite3.Connection) -> bool:
7747
7826
  JSONL. They still must remain usable after an upgrade, so replay only the
7748
7827
  already-retained physical events under the provider-local conversation lock.
7749
7828
  Empty stores keep their existing rebuild marker for the next real sync.
7829
+
7830
+ This must NEVER consume ``CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY`` (§4.3).
7831
+ Do not merge the two keys during a tidy-up: this replay runs over
7832
+ already-retained events, which preserves their NULL conversation keys, and
7833
+ then deletes the flag it consumed — so a ``dashboard --no-sync`` or qualified
7834
+ CLI read landing between the migration and the next real sync would silently
7835
+ discard the byte-zero repair. Only a re-read from offset zero can mint the
7836
+ missing identities, which is why that marker belongs to
7837
+ ``sync_codex_conversations`` alone.
7750
7838
  """
7751
7839
  current = _lib_codex_conversation.CODEX_CONVERSATION_CONTRACT_VERSION
7752
7840
 
@@ -8145,6 +8233,24 @@ def sync_claude_conversations(
8145
8233
  return stats
8146
8234
 
8147
8235
 
8236
+ def _cache_side_replay_pending(conn: sqlite3.Connection) -> bool:
8237
+ """Whether cache.db still has a byte-zero Codex replay pending (§4.3).
8238
+
8239
+ Read through the ``cache_db`` attachment conversation connections already
8240
+ carry, and qualified: ``cache_meta`` exists in BOTH stores, so an unqualified
8241
+ name would resolve to conversations.db's own table and never see the
8242
+ cache-side marker. A bare or legacy connection without the attachment
8243
+ reports False rather than raising, so it cannot wedge the conversations sync.
8244
+ """
8245
+ try:
8246
+ return conn.execute(
8247
+ "SELECT 1 FROM cache_db.cache_meta WHERE key=?",
8248
+ (CODEX_REPLAY_FROM_ZERO_KEY,),
8249
+ ).fetchone() is not None
8250
+ except sqlite3.OperationalError:
8251
+ return False
8252
+
8253
+
8148
8254
  def _clear_codex_conversation_store(conn: sqlite3.Connection) -> None:
8149
8255
  """Clear only the re-derivable Codex transcript families."""
8150
8256
  conn.execute("DELETE FROM codex_conversation_events")
@@ -8191,10 +8297,25 @@ def sync_codex_conversations(
8191
8297
  != _lib_codex_conversation.CODEX_CONVERSATION_CONTRACT_VERSION
8192
8298
  )
8193
8299
  )
8194
- if (pending_rebuild or contract_rebuild) and targeted:
8300
+ codex_replay_pending = conn.execute(
8301
+ "SELECT 1 FROM cache_meta WHERE key=?",
8302
+ (CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY,),
8303
+ ).fetchone() is not None
8304
+ # Ordering (§4.2): the cache replay must finish first.
8305
+ # `_recompute_codex_rollups` reads `codex_conversation_threads` from
8306
+ # cache.db, and a missing thread row does not yield NULL — it stamps a
8307
+ # materialized "(unassigned)" project that the read path then PREFERS,
8308
+ # permanently, for any conversation with no later activity. The two
8309
+ # stores are synced by independent paths (the dashboard runs conversation
8310
+ # sync in its own worker), so nothing else orders them.
8311
+ if _cache_side_replay_pending(conn):
8312
+ stats.deferred_reason = "cache_replay_pending"
8313
+ return stats
8314
+ if (pending_rebuild or contract_rebuild or codex_replay_pending) and targeted:
8195
8315
  stats.deferred_reason = "rebuild_pending"
8196
8316
  return stats
8197
- rebuild = rebuild or pending_rebuild or contract_rebuild
8317
+ rebuild = (
8318
+ rebuild or pending_rebuild or contract_rebuild or codex_replay_pending)
8198
8319
  if rebuild:
8199
8320
  conn.execute(
8200
8321
  "INSERT OR REPLACE INTO cache_meta(key,value) VALUES(?,?)",
@@ -8494,6 +8615,17 @@ def sync_codex_conversations(
8494
8615
  "DELETE FROM cache_meta "
8495
8616
  "WHERE key='conversation_rebuild_codex_pending'"
8496
8617
  )
8618
+ # Clear ONLY the marker this call observed. The dispatcher that
8619
+ # arms it holds `CONVERSATIONS_LOCK_MAINTENANCE_PATH` shared while
8620
+ # this walk serializes on `CONVERSATIONS_LOCK_CODEX_PATH`, so a
8621
+ # marker armed after the read above belongs to a replay this walk
8622
+ # never performed — and deleting it would strand the repair for
8623
+ # good, since the migration is stamped and never re-arms.
8624
+ if codex_replay_pending:
8625
+ conn.execute(
8626
+ "DELETE FROM cache_meta WHERE key=?",
8627
+ (CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY,),
8628
+ )
8497
8629
  conn.commit()
8498
8630
  _report_conversation_progress(progress, "checkpoint", stats)
8499
8631
  _harden_conversation_sidecars()
@@ -53,6 +53,7 @@ from _cctally_core import (
53
53
  _get_budget_config,
54
54
  )
55
55
  from _lib_dashboard_sources import (
56
+ SOURCE_SCHEMA_VERSION,
56
57
  dashboard_resource_key as _dashboard_resource_key,
57
58
  )
58
59
  from _lib_display_tz import _compute_display_block, format_display_dt
@@ -720,7 +721,7 @@ def _source_bundle_to_envelope(bundle: object | None) -> dict:
720
721
  unavailable = {source: _unavailable_source_wire() for source in ("claude", "codex", "all")}
721
722
  if bundle is None:
722
723
  return {
723
- "source_schema_version": 1,
724
+ "source_schema_version": SOURCE_SCHEMA_VERSION,
724
725
  "default_source": "claude",
725
726
  "source_order": ["claude", "codex", "all"],
726
727
  "sources": unavailable,
@@ -741,7 +742,7 @@ def _source_bundle_to_envelope(bundle: object | None) -> dict:
741
742
  }
742
743
  except Exception: # Never turn an unexpected snapshot shape into request-thread I/O.
743
744
  return {
744
- "source_schema_version": 1,
745
+ "source_schema_version": SOURCE_SCHEMA_VERSION,
745
746
  "default_source": "claude",
746
747
  "source_order": ["claude", "codex", "all"],
747
748
  "sources": unavailable,
@@ -183,6 +183,64 @@ def _codex_history_row_is_model_scoped(row: object) -> bool:
183
183
  return bool(isinstance(row, Mapping) and row.get("model_scoped"))
184
184
 
185
185
 
186
+ def _active_row_from_history(
187
+ history_row: Mapping[str, object], *, now_utc: dt.datetime,
188
+ ) -> dict[str, object] | None:
189
+ """Project a serialized quota history row onto its ``summary.active[]`` row.
190
+
191
+ #429 §4.1. The ONE home for the active-window predicate and the active-row
192
+ shape, so the initial build and ``refresh_codex_source_clock`` cannot
193
+ disagree about what ``captured_at`` means — the defect this fixes. Callers
194
+ must keep no separate copy of any part of the predicate, including the #373
195
+ model-pool exclusion: a live Spark/foreign-pool window must never reach an
196
+ account-level aggregate.
197
+
198
+ Both call sites see the same serialized shape. The clock's forecast refresh
199
+ rewrites ``status``, ``remaining_seconds`` and ``projected_percent`` but
200
+ never ``current_percent`` or ``resets_at``, so the fields read here are
201
+ identical at build time and at every tick.
202
+
203
+ #428: the liveness predicate and the emitted ``resets_at`` are the SAME
204
+ ``forecast.resets_at``, which is ``baseline.canonical_resets_at``
205
+ (``_lib_quota.forecast_quota``). The client compares ``active[].resets_at``
206
+ against ``hero.cycle.resets_at`` (``activeWeeklyKeys``) to decide which
207
+ weekly history is the live one, so both must carry that one anchor.
208
+ """
209
+ if _codex_history_row_is_model_scoped(history_row):
210
+ return None
211
+ forecast = history_row.get("forecast")
212
+ if not isinstance(forecast, Mapping):
213
+ return None
214
+ current = forecast.get("current_percent")
215
+ if not isinstance(current, (int, float)) or isinstance(current, bool):
216
+ return None
217
+ resets_at = forecast.get("resets_at")
218
+ try:
219
+ reset = dt.datetime.fromisoformat(
220
+ str(resets_at).replace("Z", "+00:00")
221
+ ).astimezone(UTC)
222
+ except (TypeError, ValueError):
223
+ return None
224
+ if reset <= now_utc:
225
+ return None
226
+ row: dict[str, object] = {
227
+ "key": history_row.get("key"),
228
+ "current_percent": current,
229
+ # #429 §3: evidence recency — the newest PHYSICAL observation, the same
230
+ # one that produced this row's `freshness` and `stale_after_seconds`.
231
+ # Not the interpreted baseline, which is a value axis and belongs to
232
+ # `current_percent` alone.
233
+ "captured_at": history_row.get("captured_at"),
234
+ "resets_at": resets_at,
235
+ "freshness": history_row.get("freshness"),
236
+ "stale_after_seconds": history_row.get("stale_after_seconds"),
237
+ }
238
+ account_key = history_row.get("account_key")
239
+ if account_key:
240
+ row["account_key"] = account_key
241
+ return row
242
+
243
+
186
244
  def _resolve_codex_weekly_cycle(
187
245
  observations: Iterable[object],
188
246
  now_utc: dt.datetime,
@@ -1693,6 +1751,7 @@ def _quota_read_model(
1693
1751
  *,
1694
1752
  accounting_entries: Iterable[object] = (),
1695
1753
  account_key: str | None = None,
1754
+ decorated: bool,
1696
1755
  ) -> dict[str, object]:
1697
1756
  """Use S2's pure history/block/forecast kernels over cache evidence.
1698
1757
 
@@ -1714,9 +1773,11 @@ def _quota_read_model(
1714
1773
  cost_entries = tuple(accounting_entries)
1715
1774
  histories = build_history(quota_observations)
1716
1775
  blocks = build_blocks(quota_observations)
1717
- history_rows: list[dict[str, object]] = []
1718
1776
  milestone_rows: list[dict[str, object]] = []
1719
- active_rows: list[dict[str, object]] = []
1777
+ # #429 §4.2: one candidate unit per identity — (ordinal, history row, active
1778
+ # projection or None) — so the cap retains PAIRS instead of capping the two
1779
+ # lists independently and in different orders.
1780
+ candidates: list[tuple[int, dict[str, object], dict[str, object] | None]] = []
1720
1781
  # R8 (#341 Task 4): the per-account `account_key` is serialized onto each
1721
1782
  # history row ONLY when the Codex provider has >1 REAL account, so the
1722
1783
  # dashboard client can scope per-account quota rows instead of merging them.
@@ -1726,13 +1787,12 @@ def _quota_read_model(
1726
1787
  # public history view is LOSSY — capped at `SOURCE_HISTORY_LIMIT` and without
1727
1788
  # `logical_limit_key` — so it cannot resolve the cycle authoritatively. Build
1728
1789
  # time owns resolution; a `clock_data` decision deadline forces the rebuild.
1729
- _codex_decorated = False
1730
- try:
1731
- import _cctally_account
1732
- _codex_decorated = _cctally_account.provider_is_decorated(
1733
- context.stats_conn, "codex")
1734
- except Exception:
1735
- _codex_decorated = False
1790
+ #
1791
+ # #429 §4.4: the caller owns this gate. Re-querying here, per parent AND per
1792
+ # child, let a transient failure emit decorated scopes whose quota rows were
1793
+ # silently unstamped — and the active-row helper cannot project a field the
1794
+ # history row never carried.
1795
+ _codex_decorated = decorated
1736
1796
  for history in histories:
1737
1797
  identity = history.identity
1738
1798
  key_parts = (
@@ -1777,21 +1837,11 @@ def _quota_read_model(
1777
1837
  "confidence": forecast.confidence,
1778
1838
  },
1779
1839
  }
1780
- history_rows.append(row)
1781
- if _codex_history_row_is_model_scoped(row):
1782
- continue
1783
- # #428: the client compares `active[].resets_at` against
1784
- # `hero.cycle.resets_at` (`activeWeeklyKeys`) to decide which weekly
1785
- # history is the live one, so both must carry the SAME anchor.
1786
- if baseline is not None and baseline.canonical_resets_at > context.now_utc:
1787
- active_rows.append({
1788
- "key": dashboard_resource_key("quota", "codex", *key_parts),
1789
- "current_percent": baseline.used_percent,
1790
- "captured_at": baseline.captured_at.astimezone(UTC).isoformat(),
1791
- "resets_at": baseline.canonical_resets_at.astimezone(UTC).isoformat(),
1792
- "freshness": freshness.state,
1793
- "stale_after_seconds": freshness.stale_after_seconds,
1794
- })
1840
+ candidates.append((
1841
+ len(candidates),
1842
+ row,
1843
+ _active_row_from_history(row, now_utc=context.now_utc),
1844
+ ))
1795
1845
  for block in blocks:
1796
1846
  identity = block.identity
1797
1847
  block_parts = (
@@ -1899,20 +1949,21 @@ def _quota_read_model(
1899
1949
  "marginal_usd": max(0.0, cumulative_usd - previous_cumulative),
1900
1950
  })
1901
1951
  previous_cumulative = cumulative_usd
1902
- latest_percent = max(
1903
- (float(row["current_percent"]) for row in active_rows), default=None,
1904
- )
1905
- active_freshness = (
1906
- "fresh" if active_rows and all(row["freshness"] == "fresh" for row in active_rows)
1907
- else ("unavailable" if not active_rows else "stale")
1908
- )
1909
1952
  # Active account identities are presentation-critical. Independent
1910
1953
  # model-scoped pools are also legitimate provider facts, so reserve the
1911
1954
  # remaining cap space for their newest captures before inactive account
1912
1955
  # history. Opaque resource-key order is only a stable tie-breaker.
1913
- active_keys = {str(row["key"]) for row in active_rows}
1956
+ #
1957
+ # #429 §4.2 — retention decides ONCE per (history, active) unit, then each
1958
+ # list is emitted in its own established order: histories in retention
1959
+ # order, actives in identity order. Emitting both in a single order would
1960
+ # reorder active rows below the cap and move bytes for every install.
1961
+ active_keys = {
1962
+ str(active["key"]) for _, _, active in candidates if active is not None
1963
+ }
1914
1964
 
1915
- def _history_retention_key(row):
1965
+ def _history_retention_key(unit):
1966
+ _, row, _ = unit
1916
1967
  key = str(row["key"])
1917
1968
  if key in active_keys:
1918
1969
  return (0, 0.0, key)
@@ -1927,11 +1978,22 @@ def _quota_read_model(
1927
1978
  return (1, -captured_epoch, key)
1928
1979
  return (2, 0.0, key)
1929
1980
 
1930
- history_rows.sort(key=_history_retention_key)
1931
- history_rows = history_rows[:SOURCE_HISTORY_LIMIT]
1981
+ retained = sorted(candidates, key=_history_retention_key)[:SOURCE_HISTORY_LIMIT]
1982
+ history_rows = [row for _, row, _ in retained]
1983
+ active_rows = [
1984
+ active
1985
+ for _, _, active in sorted(retained, key=lambda unit: unit[0])
1986
+ if active is not None
1987
+ ]
1988
+ latest_percent = max(
1989
+ (float(row["current_percent"]) for row in active_rows), default=None,
1990
+ )
1991
+ active_freshness = (
1992
+ "fresh" if active_rows and all(row["freshness"] == "fresh" for row in active_rows)
1993
+ else ("unavailable" if not active_rows else "stale")
1994
+ )
1932
1995
  milestone_rows.sort(key=lambda row: str(row["captured_at"]), reverse=True)
1933
1996
  milestone_rows = milestone_rows[:SOURCE_HISTORY_LIMIT]
1934
- active_rows = active_rows[:SOURCE_HISTORY_LIMIT]
1935
1997
  return {
1936
1998
  "summary": {
1937
1999
  "window_count": len(blocks),
@@ -2035,6 +2097,117 @@ def _refresh_budget_status_clock(
2035
2097
  }
2036
2098
 
2037
2099
 
2100
+ def _scoped_quota_identity(row: Mapping[str, object]) -> tuple[str, str]:
2101
+ """#429 §3.1. `dashboard_resource_key` carries no account, and two accounts
2102
+ sharing one $CODEX_HOME root emit the same key, so bare key is not an
2103
+ identity under decoration. `"unattributed"` is a legitimate account here."""
2104
+ return (str(row.get("account_key") or ""), str(row.get("key")))
2105
+
2106
+
2107
+ def _reclock_quota_domain(
2108
+ quota: Mapping[str, object], *, now_utc: dt.datetime,
2109
+ ) -> dict[str, object]:
2110
+ """Re-evaluate a quota domain's row freshness and summary against ``now``.
2111
+
2112
+ #429 §4.3. Replaces ONLY `histories` and `summary`; `blocks`, `milestones`
2113
+ and `cycle_index` are carried through untouched, because the per-account
2114
+ scopes carry them and a scope that lost them would render empty.
2115
+
2116
+ Emits TUPLES, matching what `_quota_read_model` publishes. Publication
2117
+ freezes lists into tuples anyway, so this is byte-identical on the wire —
2118
+ but it is what lets the caller detect an unchanged domain by comparing the
2119
+ result against the frozen original. A list would never compare equal to the
2120
+ tuple it was frozen from (``[] != ()``), the caller would report a change on
2121
+ every tick, and the retain/degrade paths that assert the EXACT prior ``data``
2122
+ object is handed back would break.
2123
+ """
2124
+ refreshed = dict(quota)
2125
+ refreshed_histories: list[dict[str, object]] = []
2126
+ active_rows: list[dict[str, object]] = []
2127
+ for raw_history in quota.get("histories", ()):
2128
+ if not isinstance(raw_history, Mapping):
2129
+ continue
2130
+ history = dict(raw_history)
2131
+ # #350 spec §3.9: this is a PER-ROW value and must never shadow the
2132
+ # envelope-level `freshness`. It used to, so after the loop the
2133
+ # envelope held the LAST retained history row's freshness — often an
2134
+ # inactive row, and with a single weekly history the active weekly
2135
+ # one, which silently marked the whole provider stale on an idle
2136
+ # stale crossing and tripped idle eligibility on its own.
2137
+ row_freshness = _clock_freshness(
2138
+ history.get("captured_at"), history.get("stale_after_seconds"), now_utc,
2139
+ )
2140
+ history["freshness"] = row_freshness
2141
+ forecast = history.get("forecast")
2142
+ if isinstance(forecast, Mapping):
2143
+ forecast = dict(forecast)
2144
+ resets_at = forecast.get("resets_at")
2145
+ try:
2146
+ reset = dt.datetime.fromisoformat(
2147
+ str(resets_at).replace("Z", "+00:00")
2148
+ ).astimezone(UTC)
2149
+ except (TypeError, ValueError):
2150
+ reset = None
2151
+ remaining = max(0, int((reset - now_utc).total_seconds())) if reset else None
2152
+ forecast["remaining_seconds"] = remaining
2153
+ sample_count = int(forecast.get("sample_count") or 0)
2154
+ if row_freshness == "future":
2155
+ forecast["status"] = "future"
2156
+ elif row_freshness == "stale":
2157
+ forecast["status"] = "stale"
2158
+ elif sample_count == 0:
2159
+ forecast["status"] = "insufficient-history"
2160
+ else:
2161
+ forecast["status"] = "ok"
2162
+ rate = forecast.get("rate_percent_per_hour")
2163
+ current = forecast.get("current_percent")
2164
+ if (
2165
+ isinstance(rate, (int, float)) and not isinstance(rate, bool)
2166
+ and isinstance(current, (int, float)) and not isinstance(current, bool)
2167
+ and remaining is not None
2168
+ ):
2169
+ forecast["projected_percent"] = min(
2170
+ 100.0, max(float(current), float(current) + float(rate) * remaining / 3600),
2171
+ )
2172
+ history["forecast"] = forecast
2173
+ # Called unconditionally, exactly as the build calls it. The helper
2174
+ # already returns None for a row without a usable forecast, and keeping
2175
+ # a caller-side `isinstance(forecast, Mapping)` guard here would put a
2176
+ # fragment of the predicate back on the caller — the split #429 exists
2177
+ # to remove.
2178
+ active = _active_row_from_history(history, now_utc=now_utc)
2179
+ if active is not None:
2180
+ active_rows.append(active)
2181
+ refreshed_histories.append(history)
2182
+ summary = dict(quota.get("summary") or {})
2183
+ prior_active = summary.get("active")
2184
+ if isinstance(prior_active, (tuple, list)):
2185
+ # #429 §3.1: scoped identity, not bare key — two decorated rows can
2186
+ # share one key and would collapse into a single map entry.
2187
+ active_order = {
2188
+ _scoped_quota_identity(row): index
2189
+ for index, row in enumerate(prior_active)
2190
+ if isinstance(row, Mapping)
2191
+ }
2192
+ active_rows.sort(
2193
+ key=lambda row: active_order.get(
2194
+ _scoped_quota_identity(row), len(active_order)),
2195
+ )
2196
+ summary.update({
2197
+ "active_window_count": len(active_rows),
2198
+ "latest_percent": max(
2199
+ (float(row["current_percent"]) for row in active_rows), default=None),
2200
+ "freshness": (
2201
+ "fresh" if active_rows and all(row["freshness"] == "fresh" for row in active_rows)
2202
+ else ("unavailable" if not active_rows else "stale")
2203
+ ),
2204
+ "active": tuple(active_rows),
2205
+ })
2206
+ refreshed["histories"] = tuple(refreshed_histories)
2207
+ refreshed["summary"] = summary
2208
+ return refreshed
2209
+
2210
+
2038
2211
  def refresh_codex_source_clock(
2039
2212
  state: SourceDashboardState,
2040
2213
  *,
@@ -2059,148 +2232,48 @@ def refresh_codex_source_clock(
2059
2232
  freshness = state.freshness
2060
2233
  domain_freshness = dict(state.domain_freshness or {})
2061
2234
  if isinstance(quota, Mapping):
2062
- quota = dict(quota)
2063
- refreshed_histories: list[dict[str, object]] = []
2064
- active_rows: list[dict[str, object]] = []
2065
- for raw_history in quota.get("histories", ()):
2066
- if not isinstance(raw_history, Mapping):
2067
- continue
2068
- history = dict(raw_history)
2069
- # #350 spec §3.9: this is a PER-ROW value and must never shadow the
2070
- # envelope-level `freshness`. It used to, so after the loop the
2071
- # envelope held the LAST retained history row's freshness often an
2072
- # inactive row, and with a single weekly history the active weekly
2073
- # one, which silently marked the whole provider stale on an idle
2074
- # stale crossing and tripped idle eligibility on its own.
2075
- row_freshness = _clock_freshness(
2076
- history.get("captured_at"), history.get("stale_after_seconds"), now_utc,
2077
- )
2078
- history["freshness"] = row_freshness
2079
- forecast = history.get("forecast")
2080
- if isinstance(forecast, Mapping):
2081
- forecast = dict(forecast)
2082
- resets_at = forecast.get("resets_at")
2083
- try:
2084
- reset = dt.datetime.fromisoformat(
2085
- str(resets_at).replace("Z", "+00:00")
2086
- ).astimezone(UTC)
2087
- except (TypeError, ValueError):
2088
- reset = None
2089
- remaining = max(0, int((reset - now_utc).total_seconds())) if reset else None
2090
- forecast["remaining_seconds"] = remaining
2091
- sample_count = int(forecast.get("sample_count") or 0)
2092
- if row_freshness == "future":
2093
- forecast["status"] = "future"
2094
- elif row_freshness == "stale":
2095
- forecast["status"] = "stale"
2096
- elif sample_count == 0:
2097
- forecast["status"] = "insufficient-history"
2098
- else:
2099
- forecast["status"] = "ok"
2100
- rate = forecast.get("rate_percent_per_hour")
2101
- current = forecast.get("current_percent")
2102
- if (
2103
- isinstance(rate, (int, float)) and not isinstance(rate, bool)
2104
- and isinstance(current, (int, float)) and not isinstance(current, bool)
2105
- and remaining is not None
2106
- ):
2107
- forecast["projected_percent"] = min(
2108
- 100.0, max(float(current), float(current) + float(rate) * remaining / 3600),
2109
- )
2110
- history["forecast"] = forecast
2111
- # #373: same rule as the initial build, through the same
2112
- # predicate, so the two paths cannot drift.
2113
- if (
2114
- not _codex_history_row_is_model_scoped(history)
2115
- and reset is not None and reset > now_utc and current is not None
2116
- ):
2117
- active_rows.append({
2118
- "key": history.get("key"),
2119
- "current_percent": current,
2120
- "captured_at": history.get("captured_at"),
2121
- "resets_at": resets_at,
2122
- "freshness": row_freshness,
2123
- "stale_after_seconds": history.get("stale_after_seconds"),
2124
- })
2125
- refreshed_histories.append(history)
2126
- quota["histories"] = refreshed_histories
2127
- latest_percent = max(
2128
- (float(row["current_percent"]) for row in active_rows), default=None,
2129
- )
2130
- summary = dict(quota.get("summary") or {})
2131
- prior_active = summary.get("active")
2132
- if isinstance(prior_active, (tuple, list)):
2133
- active_order = {
2134
- str(row.get("key")): index
2135
- for index, row in enumerate(prior_active)
2136
- if isinstance(row, Mapping)
2137
- }
2138
- active_rows.sort(
2139
- key=lambda row: active_order.get(str(row.get("key")), len(active_order)),
2140
- )
2141
- summary.update({
2142
- "active_window_count": len(active_rows),
2143
- "latest_percent": latest_percent,
2144
- "freshness": (
2145
- "fresh" if active_rows and all(row["freshness"] == "fresh" for row in active_rows)
2146
- else ("unavailable" if not active_rows else "stale")
2147
- ),
2148
- "active": active_rows,
2149
- })
2150
- # Only account-level active histories reach ``active_rows``; the shared
2151
- # model-scoped predicate above excludes foreign pools. An unavailable
2152
- # active set is a capability/data-availability fact, not invented
2153
- # staleness, so only the exact stale verdict moves this axis.
2235
+ reclocked = _reclock_quota_domain(quota, now_utc=now_utc)
2236
+ quota_changed = reclocked != quota
2237
+ quota = reclocked
2238
+ data["quota"] = quota
2239
+ # Only account-level active histories reach the summary; the shared
2240
+ # model-scoped predicate excludes foreign pools. An unavailable active
2241
+ # set is a capability/data-availability fact, not invented staleness,
2242
+ # so only the exact stale verdict moves this axis. #429 §4.3 keeps this
2243
+ # deriving from the TOP-LEVEL summary only the #350 §3.9 rule that a
2244
+ # row-level freshness value never touches `state.freshness` extends to
2245
+ # the per-account scopes clocked below.
2154
2246
  domain_freshness["quota"] = (
2155
- "stale" if summary["freshness"] == "stale" else "fresh"
2247
+ "stale" if quota["summary"]["freshness"] == "stale" else "fresh"
2156
2248
  )
2157
- quota["summary"] = summary
2158
- data["quota"] = quota
2159
- quota_changed = bool(refreshed_histories)
2160
- hero = data.get("hero")
2161
- hero_capability = state.capabilities.get("hero")
2162
- if (
2163
- isinstance(hero, Mapping)
2164
- and isinstance(hero.get("cycle"), Mapping)
2165
- and hero_capability is not None
2166
- and hero_capability.status == "supported"
2167
- ):
2168
- # #350 spec §3.3: the clock no longer RE-DERIVES cycle validity.
2169
- # Its public-history view is lossy (capped, no `logical_limit_key`,
2170
- # no `quota_identity`), so it cannot resolve the cycle correctly —
2171
- # and per §2.2 it cannot simply trust the old verdict forever either,
2172
- # because resolution is time-dependent on frozen evidence. Build time
2173
- # owns resolution and records a decision deadline in `clock_data`; the
2174
- # tick rebuilds authoritatively at the crossing. All the clock keeps
2175
- # is this cheap invariant guard: a cycle that has already RESET cannot
2176
- # bound current accounting, so it degrades exactly as before.
2177
- # Expiry is also deadline candidate #1, so the two paths are disjoint
2178
- # belt-and-suspenders rather than a single mechanism.
2179
- if _clock_cycle_expired(hero.get("cycle"), now_utc):
2180
- hero = dict(hero)
2181
- for field in (
2182
- "cost_usd", "input_tokens", "cached_input_tokens", "output_tokens",
2183
- "reasoning_output_tokens", "total_tokens", "cycle",
2184
- ):
2185
- hero[field] = None
2186
- data["hero"] = hero
2187
- refreshed_capabilities = dict(state.capabilities)
2188
- refreshed_capabilities["hero"] = CapabilityRecord(
2189
- "unavailable", "missing-or-conflicting-native-cycle",
2190
- )
2191
- capabilities = refreshed_capabilities
2192
- warnings = tuple(
2193
- warning for warning in state.warnings
2194
- if warning.code != "codex_cycle_unavailable"
2195
- ) + (SourceDashboardWarning(
2196
- "codex_cycle_unavailable",
2197
- "Codex native reset cycle is unavailable.",
2198
- "hero",
2199
- ),)
2200
- availability = "partial"
2201
- cycle_changed = True
2249
+ # #429 §4.3: the per-account scopes are independent evidence domains and
2250
+ # were never clocked at all, so a focused account read frozen freshness
2251
+ # forever. The published state is recursively frozen (`MappingProxyType`),
2252
+ # so nothing may be mutated in place — copy outward: the scopes mapping,
2253
+ # then the scope, then only that scope's `quota`.
2254
+ scopes = data.get("account_scopes")
2255
+ scopes_changed = False
2256
+ if isinstance(scopes, Mapping):
2257
+ rebuilt_scopes = dict(scopes)
2258
+ for scope_key, scope in scopes.items():
2259
+ if not isinstance(scope, Mapping):
2260
+ continue
2261
+ scope_quota = scope.get("quota")
2262
+ if not isinstance(scope_quota, Mapping):
2263
+ continue
2264
+ reclocked_scope_quota = _reclock_quota_domain(
2265
+ scope_quota, now_utc=now_utc)
2266
+ if reclocked_scope_quota == scope_quota:
2267
+ continue
2268
+ rebuilt_scope = dict(scope)
2269
+ rebuilt_scope["quota"] = reclocked_scope_quota
2270
+ rebuilt_scopes[scope_key] = rebuilt_scope
2271
+ scopes_changed = True
2272
+ if scopes_changed:
2273
+ data["account_scopes"] = rebuilt_scopes
2202
2274
  budget_domain = data.get("budget")
2203
2275
  budget_changed = False
2276
+ refreshed_budget = None
2204
2277
  if isinstance(budget_domain, Mapping):
2205
2278
  budget_domain = dict(budget_domain)
2206
2279
  refreshed_budget = _refresh_budget_status_clock(
@@ -2214,13 +2287,62 @@ def refresh_codex_source_clock(
2214
2287
  if refreshed_budget is not None:
2215
2288
  budget_domain["status"] = refreshed_budget
2216
2289
  data["budget"] = budget_domain
2217
- hero = data.get("hero")
2218
- if isinstance(hero, Mapping):
2219
- hero = dict(hero)
2220
- hero["budget"] = refreshed_budget
2221
- data["hero"] = hero
2222
2290
  budget_changed = True
2223
- if not (quota_changed or budget_changed or cycle_changed):
2291
+ # #429 §4.5: all three hero mutations compose on ONE copy, in a stated
2292
+ # order. Three independent `dict(hero)` copies let the last write win, which
2293
+ # is how `hero["quota"]` stayed frozen at its build-time value while
2294
+ # `quota["summary"]` advanced.
2295
+ hero = data.get("hero")
2296
+ if isinstance(hero, Mapping):
2297
+ hero = dict(hero)
2298
+ # 1. quota first — `_clock_cycle_expired` reads `hero["cycle"]`, never
2299
+ # `hero["quota"]`, so replacing quota cannot affect the predicate.
2300
+ if isinstance(quota, Mapping):
2301
+ hero["quota"] = quota["summary"]
2302
+ # 2. cycle expiry, with its capability/warning consequences.
2303
+ # #350 spec §3.3: the clock no longer RE-DERIVES cycle validity. Its
2304
+ # public-history view is lossy (capped, no `logical_limit_key`, no
2305
+ # `quota_identity`), so it cannot resolve the cycle correctly — and
2306
+ # per §2.2 it cannot simply trust the old verdict forever either,
2307
+ # because resolution is time-dependent on frozen evidence. Build time
2308
+ # owns resolution and records a decision deadline in `clock_data`; the
2309
+ # tick rebuilds authoritatively at the crossing. All the clock keeps
2310
+ # is this cheap invariant guard: a cycle that has already RESET cannot
2311
+ # bound current accounting, so it degrades exactly as before. Expiry
2312
+ # is also deadline candidate #1, so the two paths are disjoint
2313
+ # belt-and-suspenders rather than a single mechanism.
2314
+ hero_capability = state.capabilities.get("hero")
2315
+ if (
2316
+ isinstance(hero.get("cycle"), Mapping)
2317
+ and hero_capability is not None
2318
+ and hero_capability.status == "supported"
2319
+ and _clock_cycle_expired(hero.get("cycle"), now_utc)
2320
+ ):
2321
+ for field in (
2322
+ "cost_usd", "input_tokens", "cached_input_tokens", "output_tokens",
2323
+ "reasoning_output_tokens", "total_tokens", "cycle",
2324
+ ):
2325
+ hero[field] = None
2326
+ refreshed_capabilities = dict(state.capabilities)
2327
+ refreshed_capabilities["hero"] = CapabilityRecord(
2328
+ "unavailable", "missing-or-conflicting-native-cycle",
2329
+ )
2330
+ capabilities = refreshed_capabilities
2331
+ warnings = tuple(
2332
+ warning for warning in state.warnings
2333
+ if warning.code != "codex_cycle_unavailable"
2334
+ ) + (SourceDashboardWarning(
2335
+ "codex_cycle_unavailable",
2336
+ "Codex native reset cycle is unavailable.",
2337
+ "hero",
2338
+ ),)
2339
+ availability = "partial"
2340
+ cycle_changed = True
2341
+ # 3. budget last
2342
+ if refreshed_budget is not None:
2343
+ hero["budget"] = refreshed_budget
2344
+ data["hero"] = hero
2345
+ if not (quota_changed or budget_changed or cycle_changed or scopes_changed):
2224
2346
  return state
2225
2347
  refreshed_state = SourceDashboardState(
2226
2348
  source=state.source,
@@ -2832,6 +2954,9 @@ def _codex_account_scopes_wire(
2832
2954
  quota = _quota_read_model(
2833
2955
  context, account_observations, accounting_entries=rows,
2834
2956
  account_key=key,
2957
+ # #429 §4.4: this wire is only ever reached under decoration — the
2958
+ # caller gates the whole `account_scopes` surface on it.
2959
+ decorated=True,
2835
2960
  )
2836
2961
  # Each decorated child owns the only honest cycle index for that
2837
2962
  # account. Reusing a merged parent index here would render account A's
@@ -3228,6 +3353,7 @@ def build_codex_source_state(
3228
3353
  context,
3229
3354
  quota_observations,
3230
3355
  accounting_entries=visible_accounting_entries,
3356
+ decorated=_codex_decorated,
3231
3357
  )
3232
3358
  # R8 gate, resolved ONCE and threaded (#341 Task 4 / #416 §5.8). Every
3233
3359
  # per-account decoration below — block/alert/budget `account_key`, the
@@ -3925,6 +3925,54 @@ def _conv_001_adopt_schema_version_marker(conn: sqlite3.Connection) -> None:
3925
3925
  conn.commit()
3926
3926
 
3927
3927
 
3928
+ @conversations_migration("002_codex_thread_source_inference_replay")
3929
+ def _conv_002_codex_thread_source_inference_replay(
3930
+ conn: sqlite3.Connection,
3931
+ ) -> None:
3932
+ """Arm the conversations half of the byte-zero Codex replay.
3933
+
3934
+ Spec:
3935
+ ``docs/superpowers/specs/2026-07-30-codex-thread-source-inference-design.md``
3936
+ §4.3.
3937
+
3938
+ Writes a marker ONLY; it clears no table. ``sync_codex_conversations``
3939
+ consumes it, and DEFERS while the cache-side marker is still pending —
3940
+ ``_recompute_codex_rollups`` resolves project attribution from the cache-side
3941
+ thread row, and a missing one stamps a materialized ``"(unassigned)"`` the
3942
+ read path then prefers permanently.
3943
+
3944
+ The key is DISTINCT from ``conversation_rebuild_codex_pending`` on purpose.
3945
+ ``_ensure_codex_conversation_contract`` consumes that one by replaying
3946
+ normalization over already-retained events — which preserves their NULL
3947
+ conversation keys — and then deletes it, so a ``dashboard --no-sync`` or
3948
+ qualified CLI read between this migration and the next real sync would
3949
+ silently discard the repair.
3950
+
3951
+ Takes the Codex conversations provider flock first, the way cache migration
3952
+ ``028_split_conversation_store`` does, and DEFERS on contention. The
3953
+ conversations dispatcher runs inside ``_conversations_open_guarded``, which
3954
+ holds ``CONVERSATIONS_LOCK_MAINTENANCE_PATH`` only SHARED, while
3955
+ ``sync_codex_conversations`` serializes on ``CONVERSATIONS_LOCK_CODEX_PATH``
3956
+ — so without this lock the marker can be armed in the middle of a walk that
3957
+ already read it as absent, and that walk's finalize would clear a replay it
3958
+ never performed. Deferring leaves the migration pending, so it arms cleanly
3959
+ at the next open.
3960
+
3961
+ Idempotent: re-running rewrites the same marker. NO self-stamp — the
3962
+ dispatcher central-stamps on a clean return (#140).
3963
+ """
3964
+ import _cctally_cache
3965
+
3966
+ held = _acquire_conversations_db_codex_provider_flock(
3967
+ conn, migration="conversations 002 thread_source replay")
3968
+ try:
3969
+ _set_cache_meta(
3970
+ conn, _cctally_cache.CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY, "1")
3971
+ conn.commit()
3972
+ finally:
3973
+ _release_cache_db_writer_flocks(held)
3974
+
3975
+
3928
3976
  # #177 S6: the consolidated multi-column external-content FTS5 table that
3929
3977
  # replaces the old conversation_fts(text) + conversation_fts_aux(search_aux)
3930
3978
  # pair. The three column names MUST match the conversation_messages columns BY
@@ -4683,6 +4731,44 @@ def _acquire_cache_db_codex_provider_flock(
4683
4731
  return held
4684
4732
 
4685
4733
 
4734
+ def _acquire_conversations_db_codex_provider_flock(
4735
+ conn: sqlite3.Connection,
4736
+ *,
4737
+ migration: str,
4738
+ ) -> list[int]:
4739
+ """Take the ``<conversations.db>.codex.lock`` sibling, or DEFER.
4740
+
4741
+ The conversations dispatcher runs under a SHARED maintenance flock, so —
4742
+ unlike the cache dispatcher — it does not exclude the provider sync that
4743
+ owns the marker lifecycle. A conversations handler that writes a marker
4744
+ ``sync_codex_conversations`` consumes must therefore hold the same
4745
+ provider lock that sync holds, or it can arm mid-walk and have its marker
4746
+ swallowed by a walk that already read it as absent.
4747
+
4748
+ Derived from the connection (the lock-path helper is store-agnostic: main DB
4749
+ file + ``.codex.lock``), so a migration test never contends on the caller's
4750
+ real conversations lock.
4751
+ """
4752
+ provider_path = _cache_db_codex_lock_path_for_conn(conn)
4753
+ if provider_path is None:
4754
+ return []
4755
+
4756
+ from _lib_cache_writer_lock import acquire_ordered_flocks
4757
+
4758
+ try:
4759
+ held = acquire_ordered_flocks([(provider_path, fcntl.LOCK_EX)])
4760
+ except OSError as exc:
4761
+ raise MigrationGateNotMet(
4762
+ f"conversations.db Codex lock unavailable; deferring {migration}"
4763
+ ) from exc
4764
+ if held is None:
4765
+ raise MigrationGateNotMet(
4766
+ f"conversations.db Codex lock held by a concurrent Codex "
4767
+ f"conversation sync; deferring {migration}"
4768
+ )
4769
+ return held
4770
+
4771
+
4686
4772
  def _release_cache_db_writer_flocks(held: list[int]) -> None:
4687
4773
  from _lib_cache_writer_lock import release_cache_writer_flocks
4688
4774
 
@@ -6135,6 +6221,42 @@ def _034_codex_window_spend_adoption(conn: sqlite3.Connection) -> None:
6135
6221
  _release_cache_db_writer_flocks(held)
6136
6222
 
6137
6223
 
6224
+ @cache_migration("035_codex_thread_source_inference_replay")
6225
+ def _035_codex_thread_source_inference_replay(conn: sqlite3.Connection) -> None:
6226
+ """Arm a byte-zero Codex replay so rollouts whose ``session_meta`` omits
6227
+ ``thread_source`` gain a conversation identity.
6228
+
6229
+ Spec:
6230
+ ``docs/superpowers/specs/2026-07-30-codex-thread-source-inference-design.md``
6231
+ §4.3.
6232
+
6233
+ The repair must re-read the rollout bytes: the retained events carry NULL
6234
+ conversation keys and the in-place normalization replay preserves them.
6235
+
6236
+ Writes a marker ONLY. ``sync_codex_cache`` consumes it and ORs it into its
6237
+ own ``rebuild``, which is what makes the rebuild path capture
6238
+ ``rebuild_known_identities`` BEFORE the clear. Clearing here instead would
6239
+ delete ``codex_session_files`` out of band, leaving the next ordinary sync
6240
+ with an empty snapshot — every re-read rollout would fall through to the
6241
+ live-``auth.json`` branch and pre-mechanism Codex spend would be
6242
+ re-attributed to whoever is authenticated now (#416 spec D1). Migrations 026
6243
+ and 027 predate that snapshot and are already stamped, so this would be the
6244
+ first migration to hit it live.
6245
+
6246
+ No provider flock: unlike handlers 024-027 and 034 this writes one
6247
+ ``cache_meta`` row and touches no Codex-derived table, so a concurrent
6248
+ ``sync_codex_cache`` has nothing to interleave with — and if one is mid-walk,
6249
+ arming the marker simply defers the replay to the following sync.
6250
+
6251
+ Idempotent: re-running rewrites the same marker. NO self-stamp — the
6252
+ dispatcher central-stamps on a clean return (#140).
6253
+ """
6254
+ import _cctally_cache
6255
+
6256
+ _set_cache_meta(conn, _cctally_cache.CODEX_REPLAY_FROM_ZERO_KEY, "1")
6257
+ conn.commit()
6258
+
6259
+
6138
6260
  # === Region 7d: Stats migration 008_recompute_weekly_cost_snapshots_dedup_fix ===
6139
6261
 
6140
6262
  @stats_migration("008_recompute_weekly_cost_snapshots_dedup_fix")
@@ -1125,12 +1125,25 @@ def _doctor_gather_state_impl(
1125
1125
  # #416 review B4: the durable record that a torn Codex `auth.json` halted
1126
1126
  # ingest. Same cache_meta read, same degrade-to-None-on-anything contract.
1127
1127
  codex_torn_deferred = None
1128
+ # The byte-zero Codex replay stall signal. The marker itself is a bare "1";
1129
+ # the sibling `blocked` record is the JSON one, so it is read through the
1130
+ # same loop while the marker gets a plain existence probe. Key names come
1131
+ # from the kernel constants, never inline literals.
1132
+ codex_replay_pending = None
1133
+ codex_replay_blocked = None
1134
+ try:
1135
+ import _lib_codex_conversation as _codex_kern
1136
+ _blocked_key = _codex_kern.CODEX_REPLAY_BLOCKED_KEY
1137
+ _pending_key = _codex_kern.CODEX_REPLAY_FROM_ZERO_KEY
1138
+ except Exception:
1139
+ _blocked_key = "codex_replay_from_zero_blocked"
1140
+ _pending_key = "codex_replay_from_zero_pending"
1128
1141
  try:
1129
1142
  if _cache_probe_allowed and _cctally_core.CACHE_DB_PATH.exists():
1130
1143
  conn = sqlite3.connect(str(_cctally_core.CACHE_DB_PATH))
1131
1144
  try:
1132
1145
  for _key in ("parse_health_claude", "parse_health_codex",
1133
- "codex_torn_auth_deferred"):
1146
+ "codex_torn_auth_deferred", _blocked_key):
1134
1147
  try:
1135
1148
  row = conn.execute(
1136
1149
  "SELECT value FROM cache_meta WHERE key = ?",
@@ -1143,10 +1156,19 @@ def _doctor_gather_state_impl(
1143
1156
  parse_health_claude = _parsed
1144
1157
  elif _key == "parse_health_codex":
1145
1158
  parse_health_codex = _parsed
1159
+ elif _key == _blocked_key:
1160
+ codex_replay_blocked = _parsed
1146
1161
  else:
1147
1162
  codex_torn_deferred = _parsed
1148
1163
  except (sqlite3.OperationalError, ValueError):
1149
1164
  pass
1165
+ try:
1166
+ codex_replay_pending = conn.execute(
1167
+ "SELECT 1 FROM cache_meta WHERE key = ?",
1168
+ (_pending_key,),
1169
+ ).fetchone() is not None
1170
+ except sqlite3.OperationalError:
1171
+ pass
1150
1172
  finally:
1151
1173
  conn.close()
1152
1174
  except Exception:
@@ -1722,6 +1744,8 @@ def _doctor_gather_state_impl(
1722
1744
  parse_health_claude=parse_health_claude,
1723
1745
  parse_health_codex=parse_health_codex,
1724
1746
  codex_torn_deferred=codex_torn_deferred,
1747
+ codex_replay_pending=codex_replay_pending,
1748
+ codex_replay_blocked=codex_replay_blocked,
1725
1749
  stats_db_quick_check=stats_db_quick_check,
1726
1750
  cache_db_quick_check=cache_db_quick_check,
1727
1751
  conversations_db_quick_check=conversations_db_quick_check,
@@ -49,9 +49,13 @@ def cmd_transcript(args) -> int:
49
49
  # ---- export ----------------------------------------------------------------
50
50
 
51
51
  _SPEED_ONLY_CODEX_MSG = "transcript: --speed applies only to Codex conversations"
52
+ # Deliberately generic about WHAT is pending. Three distinct conditions reach
53
+ # this branch — a cache predating migration 025, a store with the contract
54
+ # rebuild marker armed, and a store with a byte-zero Codex replay marker armed —
55
+ # and naming the migration told users to wait for work applied long ago.
52
56
  _PENDING_EXPORT_MSG = (
53
57
  "transcript: Codex conversation is not yet normalized "
54
- "(migration 025 runs on the next cache open) — retry shortly")
58
+ "(the pending work runs on the next cache open) — retry shortly")
55
59
 
56
60
 
57
61
  def _emit_export(md: str, output) -> None:
@@ -213,9 +217,10 @@ def _cmd_transcript_search_claude(args) -> int:
213
217
 
214
218
  # ---- search (Codex) --------------------------------------------------------
215
219
 
220
+ # Same generic wording as `_PENDING_EXPORT_MSG`, for the same reason.
216
221
  _CODEX_SEARCH_PENDING_MSG = (
217
222
  "transcript: Codex conversations are not yet normalized "
218
- "(migration 025 runs on the next cache open); no results yet")
223
+ "(the pending work runs on the next cache open); no results yet")
219
224
 
220
225
 
221
226
  def _cmd_transcript_search_codex(args) -> int:
@@ -271,6 +271,7 @@ from _cctally_dashboard_sources import (
271
271
  resolve_dashboard_source_semantics,
272
272
  )
273
273
  from _lib_dashboard_sources import (
274
+ SOURCE_SCHEMA_VERSION,
274
275
  CapabilityRecord,
275
276
  SourceDashboardBundle,
276
277
  SourceDashboardState,
@@ -2803,7 +2804,7 @@ def _tui_build_source_bundle(
2803
2804
  codex = refresh_codex_source_clock(codex, now_utc=now_utc)
2804
2805
  combined = compose_all_state(claude, codex)
2805
2806
  bundle = SourceDashboardBundle(
2806
- source_schema_version=1,
2807
+ source_schema_version=SOURCE_SCHEMA_VERSION,
2807
2808
  default_source="claude",
2808
2809
  source_order=("claude", "codex", "all"),
2809
2810
  sources={"claude": claude, "codex": codex, "all": combined},
@@ -2877,7 +2878,7 @@ def _tui_hydrating_source_bundle() -> SourceDashboardBundle:
2877
2878
  domain_freshness={"hero": "stale", "quota": "stale", "sessions": "stale"},
2878
2879
  )
2879
2880
  return SourceDashboardBundle(
2880
- source_schema_version=1,
2881
+ source_schema_version=SOURCE_SCHEMA_VERSION,
2881
2882
  default_source="claude",
2882
2883
  source_order=("claude", "codex", "all"),
2883
2884
  sources={"claude": claude, "codex": codex, "all": compose_all_state(claude, codex)},
@@ -74,6 +74,23 @@ CODEX_TITLE_MAX = _TITLE_MAX
74
74
  # migration (the store is wholly re-derivable).
75
75
  CODEX_CONVERSATION_CONTRACT_VERSION = "5"
76
76
 
77
+ # Byte-zero Codex replay markers (spec
78
+ # docs/superpowers/specs/2026-07-30-codex-thread-source-inference-design.md
79
+ # §4.3). One `cache_meta` key per store, armed by cache migration 035 /
80
+ # conversations migration 002 and consumed inside the sync function that owns
81
+ # the correct replay semantics. They live in this kernel because three layers
82
+ # read them — the cache glue that consumes them, the read-side authority probe,
83
+ # and the doctor I/O shell — and a SQL string literal in any one of them would
84
+ # silently outlive a rename of the others.
85
+ CODEX_REPLAY_FROM_ZERO_KEY = "codex_replay_from_zero_pending"
86
+ CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY = (
87
+ "codex_conversation_replay_from_zero_pending"
88
+ )
89
+ # Written when a whole-tree Codex sync completed and still could NOT consume
90
+ # `CODEX_REPLAY_FROM_ZERO_KEY` — the replay is stalled, not merely pending, and
91
+ # Codex transcript ingest is deferred behind it. Read by `doctor`.
92
+ CODEX_REPLAY_BLOCKED_KEY = "codex_replay_from_zero_blocked"
93
+
77
94
  # Structural wrapper prefixes skipped during title selection (§4.3), pinned from
78
95
  # the corpus (title-wrapper-window). Prefix-structural, never content heuristics.
79
96
  CODEX_TITLE_SKIP_PREFIXES: tuple[str, ...] = (
@@ -35,6 +35,13 @@ from _lib_pricing import _calculate_codex_entry_cost
35
35
  # authoritative; a held-lock deferral leaves it pending.
36
36
  CODEX_NORMALIZATION_MIGRATION = "025_codex_conversation_normalization"
37
37
 
38
+ # The provider-local rebuild marker migration 028's byte-zero replay arms. Its
39
+ # sibling — the thread_source-inference replay marker — is
40
+ # ``kern.CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY``; both are named, never
41
+ # inlined as SQL literals, so a rename cannot leave this probe testing a key
42
+ # nothing writes any more.
43
+ CODEX_CONTRACT_REBUILD_MARKER = "conversation_rebuild_codex_pending"
44
+
38
45
  # Domain separations for the opaque item-key encoding (§5.2). The source-path
39
46
  # fingerprint is a domain-separated hash, NEVER a raw path (privacy-safe).
40
47
  CODEX_ITEM_KEY_DOMAIN = b"cctally-codex-item-key-v1\0"
@@ -93,6 +100,12 @@ def codex_normalization_authoritative(conn: sqlite3.Connection) -> bool:
93
100
  Split stores use their provider-local rebuild marker: current schema alone
94
101
  is not authority while migration 028's byte-zero replay is pending. Legacy
95
102
  monolithic/bare connections retain the migration-025 stamp contract.
103
+
104
+ EITHER pending marker withholds authority. The thread_source-inference
105
+ replay (conversations migration 002) is armed by its own key precisely
106
+ because the contract replay must not consume it, so a probe that tested only
107
+ the contract marker would report a not-yet-repaired store as authoritative
108
+ to every ``--no-sync`` read.
96
109
  """
97
110
  try:
98
111
  split = conn.execute(
@@ -101,8 +114,9 @@ def codex_normalization_authoritative(conn: sqlite3.Connection) -> bool:
101
114
  ).fetchone() is not None
102
115
  if split:
103
116
  pending = conn.execute(
104
- "SELECT 1 FROM cache_meta "
105
- "WHERE key='conversation_rebuild_codex_pending'"
117
+ "SELECT 1 FROM cache_meta WHERE key IN (?,?) LIMIT 1",
118
+ (CODEX_CONTRACT_REBUILD_MARKER,
119
+ kern.CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY),
106
120
  ).fetchone() is not None
107
121
  version = conn.execute(
108
122
  "SELECT value FROM cache_meta "
@@ -22,7 +22,14 @@ CapabilityStatus = Literal[
22
22
  "supported", "derived", "unavailable", "deferred", "not_applicable",
23
23
  ]
24
24
 
25
- SOURCE_SCHEMA_VERSION = 1
25
+ # #429 §3.2 — bumped to 2 because `active[].captured_at` changed MEANING (the
26
+ # newest physical observation, not the interpreted baseline). The in-place
27
+ # update flow `execvp`s the server while the already-loaded client reconnects
28
+ # over its existing EventSource without reloading its JS
29
+ # (`UpdateRunningModal.tsx`, `store/sse.ts`), so an old client demonstrably
30
+ # does meet a new server; `docs/cli-contract.md` calls changing a value's
31
+ # meaning breaking. Version-aware client reaction is deliberately deferred.
32
+ SOURCE_SCHEMA_VERSION = 2
26
33
  DEFAULT_SOURCE = "claude"
27
34
  SOURCE_ORDER = ("claude", "codex", "all")
28
35
  SOURCE_FRESHNESS_DOMAINS = ("hero", "quota", "sessions")
@@ -189,6 +189,14 @@ class DoctorState:
189
189
  # by the next whole-tree sync that defers none. None = key absent (the
190
190
  # normal state) or cache unreadable — the check degrades OK.
191
191
  codex_torn_deferred: Optional[dict] = None
192
+ # The byte-zero Codex replay's stall signal. `codex_replay_pending` mirrors
193
+ # the `codex_replay_from_zero_pending` cache_meta marker; `codex_replay_blocked`
194
+ # is `{"at": iso, "files_failed": N, "files_deferred_torn": N}`, written by a
195
+ # whole-tree Codex sync that RAN and still could not consume it, and deleted
196
+ # by the next clean whole-tree sync. Both None = key absent (the normal
197
+ # state) or cache unreadable — the check degrades OK.
198
+ codex_replay_pending: Optional[bool] = None
199
+ codex_replay_blocked: Optional[dict] = None
192
200
  # #279 S2 (F5b): PRAGMA quick_check(1) results, gathered ONLY under
193
201
  # doctor_gather_state(deep=True) (CLI cmd_doctor) — the dashboard
194
202
  # rebuild loop calls the gather every rebuild and quick_check on a
@@ -1107,6 +1115,50 @@ def _check_data_codex_cache(s: DoctorState) -> CheckResult:
1107
1115
  )
1108
1116
 
1109
1117
 
1118
+ def _check_data_codex_replay(s: DoctorState) -> CheckResult:
1119
+ """WARN while a byte-zero Codex replay is STALLED rather than merely pending.
1120
+
1121
+ `sync_codex_conversations` defers on the cache-side replay marker, so as long
1122
+ as that marker stands no Codex transcript is ingested at all. The deferral is
1123
+ protective — running ahead of the replayed thread rows stamps a materialized
1124
+ `"(unassigned)"` project the read path then prefers permanently — so the fix
1125
+ is never to drop it. But a whole-tree sync that runs and still cannot consume
1126
+ the marker (a persistently torn `auth.json`, a repeated per-file DB error)
1127
+ holds that deferral open indefinitely, and `cache-sync` still exits 0. This
1128
+ leg is the only standing signal that Codex transcript ingest has stopped.
1129
+
1130
+ A marker that is merely pending is NOT reported: it is the ordinary state
1131
+ between the migration and the next sync, and it clears on its own. The WARN
1132
+ needs the durable `blocked` record, which only a completed-but-unsuccessful
1133
+ whole-tree walk writes.
1134
+ """
1135
+ blocked = s.codex_replay_blocked or {}
1136
+ at = blocked.get("at")
1137
+ details = {
1138
+ "pending": bool(s.codex_replay_pending),
1139
+ "blocked_at": at,
1140
+ "files_failed": blocked.get("files_failed"),
1141
+ "files_deferred_torn": blocked.get("files_deferred_torn"),
1142
+ }
1143
+ if s.codex_replay_pending and isinstance(at, str) and at:
1144
+ return CheckResult(
1145
+ id="data.codex_replay", title="Codex transcript replay",
1146
+ severity="warn",
1147
+ summary=(f"stalled since {at} — Codex transcript ingest is "
1148
+ "deferred until it completes"),
1149
+ remediation=("Check the Codex auth.json (re-run `codex login` if it "
1150
+ "is truncated), then `cctally cache-sync --source codex`"),
1151
+ details=details,
1152
+ )
1153
+ return CheckResult(
1154
+ id="data.codex_replay", title="Codex transcript replay",
1155
+ severity="ok",
1156
+ summary="pending (clears on the next Codex sync)"
1157
+ if s.codex_replay_pending else "none pending",
1158
+ remediation=None, details=details,
1159
+ )
1160
+
1161
+
1110
1162
  def _check_data_codex_project_metadata(s: DoctorState) -> CheckResult:
1111
1163
  """Report the identity-safe all-history Codex metadata partition."""
1112
1164
  if s.codex_project_metadata_error is not None:
@@ -2680,7 +2732,12 @@ _CATEGORY_DEFINITIONS: tuple[tuple[str, str, tuple[tuple[str, str], ...]], ...]
2680
2732
  ("data.statusline_pipeline", "_check_statusline_pipeline"),
2681
2733
  ("data.cache_sync_state", "_check_data_cache_sync_state"),
2682
2734
  ("data.codex_cache", "_check_data_codex_cache"),
2735
+ # `data.codex_project_metadata` must stay IMMEDIATELY after
2736
+ # `data.codex_cache` (pinned by
2737
+ # tests/test_doctor_codex_project_metadata.py), so the replay leg goes
2738
+ # after the pair rather than between them.
2683
2739
  ("data.codex_project_metadata", "_check_data_codex_project_metadata"),
2740
+ ("data.codex_replay", "_check_data_codex_replay"),
2684
2741
  ("data.codex_quota", "_check_data_codex_quota"),
2685
2742
  ("data.parse_health", "_check_data_parse_health"),
2686
2743
  ("data.forked_buckets", "_check_data_forked_buckets"),
package/bin/_lib_jsonl.py CHANGED
@@ -549,15 +549,48 @@ def _canonical_container(value: object) -> str | None:
549
549
  return _codex_canonical_json(value)
550
550
 
551
551
 
552
+ # Codex is mid-rollout on `thread_source`: Desktop 0.146.0-alpha.3.1 emits it,
553
+ # the CLI 0.146.0 behind the MCP server does not. Abandoning the identity when
554
+ # it is absent costs the rollout its thread row AND every normalized message,
555
+ # so infer the category the provider would most likely emit instead. Inferring
556
+ # rather than minting a null parent keeps one identity across the release that
557
+ # starts emitting the field.
558
+ _CODEX_DEFAULT_THREAD_SOURCE = "user"
559
+
560
+
561
+ def _inferred_codex_thread_source(payload: dict[str, Any]) -> str:
562
+ """The thread-origin category for one ``session_meta`` payload.
563
+
564
+ Per-record and stateless: a later ``session_meta`` never inherits an earlier
565
+ one's category, so a file's key sequence is a function of its own bytes.
566
+ The string form of ``source`` is deliberately never used — ``source:
567
+ "vscode"`` co-occurs with ``thread_source: "user"``, so the client name and
568
+ the origin category are orthogonal vocabularies.
569
+ """
570
+ explicit = _codex_string(payload.get("thread_source"))
571
+ if explicit is not None:
572
+ return explicit
573
+ source = payload.get("source")
574
+ if isinstance(source, dict) and len(source) == 1:
575
+ # A single-key object is the shape Codex uses for `{"subagent": {...}}`.
576
+ # Guard every malformed variant: the identity encoder REJECTS an empty
577
+ # parent key, so an unguarded rule turns bad metadata into an ingest
578
+ # exception rather than a degraded-but-working ingest.
579
+ only_key = _codex_string(next(iter(source)))
580
+ if only_key is not None:
581
+ return only_key
582
+ return _CODEX_DEFAULT_THREAD_SOURCE
583
+
584
+
552
585
  def _thread_metadata_from_session_meta(
553
586
  payload: dict[str, Any], path_str: str, source_root_key: str | None,
554
587
  ) -> CodexThreadMetadata:
555
588
  accounting_id = _codex_string(payload.get("id"))
556
589
  native_thread_id = _codex_string(payload.get("session_id")) or accounting_id
557
- root_thread_id = _codex_string(payload.get("thread_source"))
590
+ root_thread_id = _inferred_codex_thread_source(payload)
558
591
  parent_thread_id = _codex_string(payload.get("forked_from_id"))
559
592
  conversation_key = None
560
- if native_thread_id is not None and root_thread_id is not None:
593
+ if native_thread_id is not None:
561
594
  conversation_key = canonical_identity_from_root_key(
562
595
  "codex", "conversation", source_root_key, native_thread_id, root_thread_id
563
596
  )
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "cctally",
3
- "version": "1.87.2",
3
+ "version": "1.88.0",
4
4
  "description": "Claude Code usage tracker and local dashboard for Pro/Max subscription limits - weekly cost-per-percent trend, quota forecasts, threshold alerts. ccusage-compatible.",
5
5
  "homepage": "https://github.com/omrikais/cctally",
6
6
  "repository": {