cctally 1.87.2 → 1.88.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/bin/_cctally_cache.py +134 -2
- package/bin/_cctally_dashboard_envelope.py +3 -2
- package/bin/_cctally_dashboard_sources.py +306 -180
- package/bin/_cctally_db.py +122 -0
- package/bin/_cctally_doctor.py +25 -1
- package/bin/_cctally_transcript.py +7 -2
- package/bin/_cctally_tui.py +3 -2
- package/bin/_lib_codex_conversation.py +17 -0
- package/bin/_lib_codex_conversation_query.py +16 -2
- package/bin/_lib_dashboard_sources.py +8 -1
- package/bin/_lib_doctor.py +57 -0
- package/bin/_lib_jsonl.py +35 -2
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,12 @@ based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|
|
5
5
|
|
|
6
6
|
## [Unreleased]
|
|
7
7
|
|
|
8
|
+
## [1.88.0] - 2026-07-31
|
|
9
|
+
|
|
10
|
+
### Fixed
|
|
11
|
+
- The Codex hero's Snapshot chip no longer reports stale evidence while the dashboard's own freshness state for the same quota window reads fresh. One `/api/data` response could carry two different capture timestamps for a single quota window, and the chip aged off the older one: the initial build stamped the active row with the timestamp of the last reading whose VALUE changed, while that same row's freshness came from the last reading actually received — and because OpenAI repeats an unchanged percentage while you are idle, the two drift apart in exactly the common case. `active[].captured_at` now always means evidence recency, the same observation behind the row's freshness, and the percentage it reports is unchanged. The idle clock additionally refreshes every per-account view rather than only the merged one, so a focused account no longer reads freshness frozen at page load, and the hero's own copy of the quota summary is re-derived on each tick instead of keeping its build-time value. Retained quota windows and their live rows are now capped together, so an install tracking more than 250 windows can no longer publish a live row whose history was dropped or a summary percentage belonging to a window that was. The source envelope's `source_schema_version` moves to 2, because a published value changed meaning and a dashboard tab left open across an in-place upgrade does meet the new server over its existing event stream. That tab keeps working either way — nothing reads the version yet — so the bump is a correctness signal for future readers rather than the thing that preserves compatibility. (#429)
|
|
12
|
+
- Codex sessions started outside the Desktop app — through the Codex MCP server, the CLI, `exec`, subagents, and older Desktop builds — now appear in the Conversation Viewer and resolve their project in Recent Sessions. Codex is part-way through rolling out the field cctally used to identify a conversation, and until now a session whose log omitted it was given no identity at all: its transcript was never indexed, and the sessions card could only report that project metadata was unavailable. cctally now infers the missing value, so those sessions are ordinary sessions with no visible difference from any other. On upgrade, existing history is re-read once in the background to repair it; a session's spend stays attributed to the account it was always attributed to, and a session whose log carries no usable identifier at all is left alone rather than guessed at. Because the repair re-reads the session logs themselves, totals are unchanged for every session still on disk — but a session whose log Codex has since deleted is no longer counted, so long-run totals can fall slightly. Sessions with no working directory recorded still appear, and still honestly report their project as unavailable. One visible consequence of that re-read: it briefly restores transcripts older than your configured retention window, so the ordinary retention sweep runs straight afterwards and those older conversations leave the Conversation Viewer on upgrade instead of lingering past their window.
|
|
13
|
+
|
|
8
14
|
## [1.87.2] - 2026-07-30
|
|
9
15
|
|
|
10
16
|
### Fixed
|
package/bin/_cctally_cache.py
CHANGED
|
@@ -511,6 +511,38 @@ _codex_conversation_fts_full_clear = _cctally_db_sib._codex_conversation_fts_ful
|
|
|
511
511
|
# writes (#179) so the ON CONFLICT idiom lives in one place. Caller commits.
|
|
512
512
|
_set_cache_meta = _cctally_db_sib._set_cache_meta
|
|
513
513
|
|
|
514
|
+
# Byte-zero Codex replay markers (spec
|
|
515
|
+
# docs/superpowers/specs/2026-07-30-codex-thread-source-inference-design.md
|
|
516
|
+
# §4.3). Cache migration 035 / conversations migration 002 write them and clear
|
|
517
|
+
# NO table; the sync functions consume them, because only the sync owns the
|
|
518
|
+
# replay semantics that keep the repair safe:
|
|
519
|
+
#
|
|
520
|
+
# * `sync_codex_cache` ORs the cache-side marker into its own `rebuild`, so
|
|
521
|
+
# the rebuild path captures `rebuild_known_identities` before clearing. A
|
|
522
|
+
# migration clearing `codex_session_files` directly would leave the next
|
|
523
|
+
# ordinary sync with an empty snapshot, sending every re-read rollout to the
|
|
524
|
+
# live-`auth.json` branch and re-attributing historical spend (§4.1).
|
|
525
|
+
# * `sync_codex_conversations` defers on the CONVERSATIONS marker until the
|
|
526
|
+
# cache-side one has cleared, because `_recompute_codex_rollups` reads the
|
|
527
|
+
# thread row from cache.db and a missing one stamps a materialized
|
|
528
|
+
# "(unassigned)" project the read path then prefers permanently (§4.2).
|
|
529
|
+
#
|
|
530
|
+
# The conversations key is deliberately DISTINCT from
|
|
531
|
+
# `conversation_rebuild_codex_pending`: `_ensure_codex_conversation_contract`
|
|
532
|
+
# consumes that one by replaying normalization over already-retained events —
|
|
533
|
+
# which preserves their NULL conversation keys — and then deletes it, silently
|
|
534
|
+
# discarding the repair.
|
|
535
|
+
#
|
|
536
|
+
# The keys themselves are defined in the pure kernel `_lib_codex_conversation`
|
|
537
|
+
# and re-exported here, so the read-side authority probe and the doctor shell
|
|
538
|
+
# bind the same names instead of repeating a SQL string literal.
|
|
539
|
+
CODEX_REPLAY_FROM_ZERO_KEY = (
|
|
540
|
+
_lib_codex_conversation.CODEX_REPLAY_FROM_ZERO_KEY)
|
|
541
|
+
CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY = (
|
|
542
|
+
_lib_codex_conversation.CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY)
|
|
543
|
+
CODEX_REPLAY_BLOCKED_KEY = (
|
|
544
|
+
_lib_codex_conversation.CODEX_REPLAY_BLOCKED_KEY)
|
|
545
|
+
|
|
514
546
|
|
|
515
547
|
# cache.db WAL hardening (#297). See
|
|
516
548
|
# docs/superpowers/specs/2026-07-13-cache-db-wal-hardening-design.md.
|
|
@@ -4903,6 +4935,24 @@ def sync_codex_cache(
|
|
|
4903
4935
|
# and bypasses every whole-tree operation (orphan prune, root prune,
|
|
4904
4936
|
# global quota reconcile) — see the guards threaded through below.
|
|
4905
4937
|
targeted = only_paths is not None
|
|
4938
|
+
|
|
4939
|
+
# A pending byte-zero replay is consumed HERE, not by the migration that
|
|
4940
|
+
# armed it, so the rebuild path below captures `rebuild_known_identities`
|
|
4941
|
+
# before clearing. A migration that cleared `codex_session_files`
|
|
4942
|
+
# directly would leave the next ordinary sync with an empty snapshot,
|
|
4943
|
+
# sending every re-read rollout to the live-auth branch and
|
|
4944
|
+
# re-attributing historical spend to whoever is authenticated now.
|
|
4945
|
+
replay_pending = conn.execute(
|
|
4946
|
+
"SELECT 1 FROM cache_meta WHERE key=?",
|
|
4947
|
+
(CODEX_REPLAY_FROM_ZERO_KEY,),
|
|
4948
|
+
).fetchone() is not None
|
|
4949
|
+
if replay_pending and targeted:
|
|
4950
|
+
# A live-tail tick must DEFER, never raise through the
|
|
4951
|
+
# `targeted and rebuild` guard below.
|
|
4952
|
+
stats.deferred_reason = "replay_pending"
|
|
4953
|
+
return stats
|
|
4954
|
+
rebuild = rebuild or replay_pending
|
|
4955
|
+
|
|
4906
4956
|
if targeted and rebuild:
|
|
4907
4957
|
raise ValueError(
|
|
4908
4958
|
"sync_codex_cache: only_paths is incompatible with rebuild")
|
|
@@ -5742,6 +5792,35 @@ def sync_codex_cache(
|
|
|
5742
5792
|
else:
|
|
5743
5793
|
conn.execute("DELETE FROM cache_meta WHERE key = ?",
|
|
5744
5794
|
("codex_torn_auth_deferred",))
|
|
5795
|
+
# Consume the byte-zero replay marker only after a clean full walk,
|
|
5796
|
+
# and only when THIS call observed it. A contended call returned
|
|
5797
|
+
# long before here, and a walk that failed or deferred a file leaves
|
|
5798
|
+
# the marker standing — a surviving marker is what makes the repair
|
|
5799
|
+
# retry on the next sync, and it is also what keeps
|
|
5800
|
+
# `sync_codex_conversations` deferred until the cache side genuinely
|
|
5801
|
+
# holds the replayed thread rows (§4.2). The `replay_pending` guard
|
|
5802
|
+
# is defense in depth: `open_cache_db` and this walk share an
|
|
5803
|
+
# exclusive lock today, so nothing can arm the marker in between —
|
|
5804
|
+
# but the conversations side has no such exclusion, and the two
|
|
5805
|
+
# clears must keep the same shape.
|
|
5806
|
+
if stats.files_failed == 0 and stats.files_deferred_torn == 0:
|
|
5807
|
+
if replay_pending:
|
|
5808
|
+
conn.execute("DELETE FROM cache_meta WHERE key = ?",
|
|
5809
|
+
(CODEX_REPLAY_FROM_ZERO_KEY,))
|
|
5810
|
+
conn.execute("DELETE FROM cache_meta WHERE key = ?",
|
|
5811
|
+
(CODEX_REPLAY_BLOCKED_KEY,))
|
|
5812
|
+
elif replay_pending:
|
|
5813
|
+
# A full walk ran and could NOT consume the marker, so the
|
|
5814
|
+
# replay — and with it every Codex transcript ingest, which
|
|
5815
|
+
# defers behind this marker — is stalled rather than merely
|
|
5816
|
+
# not-yet-run. `doctor` reads this; the deferral itself stays,
|
|
5817
|
+
# because running ahead is what stamps "(unassigned)" (§4.2).
|
|
5818
|
+
_set_cache_meta(conn, CODEX_REPLAY_BLOCKED_KEY, json.dumps({
|
|
5819
|
+
"at": dt.datetime.now(dt.timezone.utc).isoformat(
|
|
5820
|
+
timespec="seconds").replace("+00:00", "Z"),
|
|
5821
|
+
"files_failed": stats.files_failed,
|
|
5822
|
+
"files_deferred_torn": stats.files_deferred_torn,
|
|
5823
|
+
}, sort_keys=True))
|
|
5745
5824
|
conn.commit()
|
|
5746
5825
|
# Window-scoped spend adoption (spec
|
|
5747
5826
|
# docs/superpowers/specs/2026-07-30-codex-window-scoped-spend-adoption.md).
|
|
@@ -7747,6 +7826,15 @@ def _ensure_codex_conversation_contract(conn: sqlite3.Connection) -> bool:
|
|
|
7747
7826
|
JSONL. They still must remain usable after an upgrade, so replay only the
|
|
7748
7827
|
already-retained physical events under the provider-local conversation lock.
|
|
7749
7828
|
Empty stores keep their existing rebuild marker for the next real sync.
|
|
7829
|
+
|
|
7830
|
+
This must NEVER consume ``CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY`` (§4.3).
|
|
7831
|
+
Do not merge the two keys during a tidy-up: this replay runs over
|
|
7832
|
+
already-retained events, which preserves their NULL conversation keys, and
|
|
7833
|
+
then deletes the flag it consumed — so a ``dashboard --no-sync`` or qualified
|
|
7834
|
+
CLI read landing between the migration and the next real sync would silently
|
|
7835
|
+
discard the byte-zero repair. Only a re-read from offset zero can mint the
|
|
7836
|
+
missing identities, which is why that marker belongs to
|
|
7837
|
+
``sync_codex_conversations`` alone.
|
|
7750
7838
|
"""
|
|
7751
7839
|
current = _lib_codex_conversation.CODEX_CONVERSATION_CONTRACT_VERSION
|
|
7752
7840
|
|
|
@@ -8145,6 +8233,24 @@ def sync_claude_conversations(
|
|
|
8145
8233
|
return stats
|
|
8146
8234
|
|
|
8147
8235
|
|
|
8236
|
+
def _cache_side_replay_pending(conn: sqlite3.Connection) -> bool:
|
|
8237
|
+
"""Whether cache.db still has a byte-zero Codex replay pending (§4.3).
|
|
8238
|
+
|
|
8239
|
+
Read through the ``cache_db`` attachment conversation connections already
|
|
8240
|
+
carry, and qualified: ``cache_meta`` exists in BOTH stores, so an unqualified
|
|
8241
|
+
name would resolve to conversations.db's own table and never see the
|
|
8242
|
+
cache-side marker. A bare or legacy connection without the attachment
|
|
8243
|
+
reports False rather than raising, so it cannot wedge the conversations sync.
|
|
8244
|
+
"""
|
|
8245
|
+
try:
|
|
8246
|
+
return conn.execute(
|
|
8247
|
+
"SELECT 1 FROM cache_db.cache_meta WHERE key=?",
|
|
8248
|
+
(CODEX_REPLAY_FROM_ZERO_KEY,),
|
|
8249
|
+
).fetchone() is not None
|
|
8250
|
+
except sqlite3.OperationalError:
|
|
8251
|
+
return False
|
|
8252
|
+
|
|
8253
|
+
|
|
8148
8254
|
def _clear_codex_conversation_store(conn: sqlite3.Connection) -> None:
|
|
8149
8255
|
"""Clear only the re-derivable Codex transcript families."""
|
|
8150
8256
|
conn.execute("DELETE FROM codex_conversation_events")
|
|
@@ -8191,10 +8297,25 @@ def sync_codex_conversations(
|
|
|
8191
8297
|
!= _lib_codex_conversation.CODEX_CONVERSATION_CONTRACT_VERSION
|
|
8192
8298
|
)
|
|
8193
8299
|
)
|
|
8194
|
-
|
|
8300
|
+
codex_replay_pending = conn.execute(
|
|
8301
|
+
"SELECT 1 FROM cache_meta WHERE key=?",
|
|
8302
|
+
(CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY,),
|
|
8303
|
+
).fetchone() is not None
|
|
8304
|
+
# Ordering (§4.2): the cache replay must finish first.
|
|
8305
|
+
# `_recompute_codex_rollups` reads `codex_conversation_threads` from
|
|
8306
|
+
# cache.db, and a missing thread row does not yield NULL — it stamps a
|
|
8307
|
+
# materialized "(unassigned)" project that the read path then PREFERS,
|
|
8308
|
+
# permanently, for any conversation with no later activity. The two
|
|
8309
|
+
# stores are synced by independent paths (the dashboard runs conversation
|
|
8310
|
+
# sync in its own worker), so nothing else orders them.
|
|
8311
|
+
if _cache_side_replay_pending(conn):
|
|
8312
|
+
stats.deferred_reason = "cache_replay_pending"
|
|
8313
|
+
return stats
|
|
8314
|
+
if (pending_rebuild or contract_rebuild or codex_replay_pending) and targeted:
|
|
8195
8315
|
stats.deferred_reason = "rebuild_pending"
|
|
8196
8316
|
return stats
|
|
8197
|
-
rebuild =
|
|
8317
|
+
rebuild = (
|
|
8318
|
+
rebuild or pending_rebuild or contract_rebuild or codex_replay_pending)
|
|
8198
8319
|
if rebuild:
|
|
8199
8320
|
conn.execute(
|
|
8200
8321
|
"INSERT OR REPLACE INTO cache_meta(key,value) VALUES(?,?)",
|
|
@@ -8494,6 +8615,17 @@ def sync_codex_conversations(
|
|
|
8494
8615
|
"DELETE FROM cache_meta "
|
|
8495
8616
|
"WHERE key='conversation_rebuild_codex_pending'"
|
|
8496
8617
|
)
|
|
8618
|
+
# Clear ONLY the marker this call observed. The dispatcher that
|
|
8619
|
+
# arms it holds `CONVERSATIONS_LOCK_MAINTENANCE_PATH` shared while
|
|
8620
|
+
# this walk serializes on `CONVERSATIONS_LOCK_CODEX_PATH`, so a
|
|
8621
|
+
# marker armed after the read above belongs to a replay this walk
|
|
8622
|
+
# never performed — and deleting it would strand the repair for
|
|
8623
|
+
# good, since the migration is stamped and never re-arms.
|
|
8624
|
+
if codex_replay_pending:
|
|
8625
|
+
conn.execute(
|
|
8626
|
+
"DELETE FROM cache_meta WHERE key=?",
|
|
8627
|
+
(CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY,),
|
|
8628
|
+
)
|
|
8497
8629
|
conn.commit()
|
|
8498
8630
|
_report_conversation_progress(progress, "checkpoint", stats)
|
|
8499
8631
|
_harden_conversation_sidecars()
|
|
@@ -53,6 +53,7 @@ from _cctally_core import (
|
|
|
53
53
|
_get_budget_config,
|
|
54
54
|
)
|
|
55
55
|
from _lib_dashboard_sources import (
|
|
56
|
+
SOURCE_SCHEMA_VERSION,
|
|
56
57
|
dashboard_resource_key as _dashboard_resource_key,
|
|
57
58
|
)
|
|
58
59
|
from _lib_display_tz import _compute_display_block, format_display_dt
|
|
@@ -720,7 +721,7 @@ def _source_bundle_to_envelope(bundle: object | None) -> dict:
|
|
|
720
721
|
unavailable = {source: _unavailable_source_wire() for source in ("claude", "codex", "all")}
|
|
721
722
|
if bundle is None:
|
|
722
723
|
return {
|
|
723
|
-
"source_schema_version":
|
|
724
|
+
"source_schema_version": SOURCE_SCHEMA_VERSION,
|
|
724
725
|
"default_source": "claude",
|
|
725
726
|
"source_order": ["claude", "codex", "all"],
|
|
726
727
|
"sources": unavailable,
|
|
@@ -741,7 +742,7 @@ def _source_bundle_to_envelope(bundle: object | None) -> dict:
|
|
|
741
742
|
}
|
|
742
743
|
except Exception: # Never turn an unexpected snapshot shape into request-thread I/O.
|
|
743
744
|
return {
|
|
744
|
-
"source_schema_version":
|
|
745
|
+
"source_schema_version": SOURCE_SCHEMA_VERSION,
|
|
745
746
|
"default_source": "claude",
|
|
746
747
|
"source_order": ["claude", "codex", "all"],
|
|
747
748
|
"sources": unavailable,
|
|
@@ -183,6 +183,64 @@ def _codex_history_row_is_model_scoped(row: object) -> bool:
|
|
|
183
183
|
return bool(isinstance(row, Mapping) and row.get("model_scoped"))
|
|
184
184
|
|
|
185
185
|
|
|
186
|
+
def _active_row_from_history(
|
|
187
|
+
history_row: Mapping[str, object], *, now_utc: dt.datetime,
|
|
188
|
+
) -> dict[str, object] | None:
|
|
189
|
+
"""Project a serialized quota history row onto its ``summary.active[]`` row.
|
|
190
|
+
|
|
191
|
+
#429 §4.1. The ONE home for the active-window predicate and the active-row
|
|
192
|
+
shape, so the initial build and ``refresh_codex_source_clock`` cannot
|
|
193
|
+
disagree about what ``captured_at`` means — the defect this fixes. Callers
|
|
194
|
+
must keep no separate copy of any part of the predicate, including the #373
|
|
195
|
+
model-pool exclusion: a live Spark/foreign-pool window must never reach an
|
|
196
|
+
account-level aggregate.
|
|
197
|
+
|
|
198
|
+
Both call sites see the same serialized shape. The clock's forecast refresh
|
|
199
|
+
rewrites ``status``, ``remaining_seconds`` and ``projected_percent`` but
|
|
200
|
+
never ``current_percent`` or ``resets_at``, so the fields read here are
|
|
201
|
+
identical at build time and at every tick.
|
|
202
|
+
|
|
203
|
+
#428: the liveness predicate and the emitted ``resets_at`` are the SAME
|
|
204
|
+
``forecast.resets_at``, which is ``baseline.canonical_resets_at``
|
|
205
|
+
(``_lib_quota.forecast_quota``). The client compares ``active[].resets_at``
|
|
206
|
+
against ``hero.cycle.resets_at`` (``activeWeeklyKeys``) to decide which
|
|
207
|
+
weekly history is the live one, so both must carry that one anchor.
|
|
208
|
+
"""
|
|
209
|
+
if _codex_history_row_is_model_scoped(history_row):
|
|
210
|
+
return None
|
|
211
|
+
forecast = history_row.get("forecast")
|
|
212
|
+
if not isinstance(forecast, Mapping):
|
|
213
|
+
return None
|
|
214
|
+
current = forecast.get("current_percent")
|
|
215
|
+
if not isinstance(current, (int, float)) or isinstance(current, bool):
|
|
216
|
+
return None
|
|
217
|
+
resets_at = forecast.get("resets_at")
|
|
218
|
+
try:
|
|
219
|
+
reset = dt.datetime.fromisoformat(
|
|
220
|
+
str(resets_at).replace("Z", "+00:00")
|
|
221
|
+
).astimezone(UTC)
|
|
222
|
+
except (TypeError, ValueError):
|
|
223
|
+
return None
|
|
224
|
+
if reset <= now_utc:
|
|
225
|
+
return None
|
|
226
|
+
row: dict[str, object] = {
|
|
227
|
+
"key": history_row.get("key"),
|
|
228
|
+
"current_percent": current,
|
|
229
|
+
# #429 §3: evidence recency — the newest PHYSICAL observation, the same
|
|
230
|
+
# one that produced this row's `freshness` and `stale_after_seconds`.
|
|
231
|
+
# Not the interpreted baseline, which is a value axis and belongs to
|
|
232
|
+
# `current_percent` alone.
|
|
233
|
+
"captured_at": history_row.get("captured_at"),
|
|
234
|
+
"resets_at": resets_at,
|
|
235
|
+
"freshness": history_row.get("freshness"),
|
|
236
|
+
"stale_after_seconds": history_row.get("stale_after_seconds"),
|
|
237
|
+
}
|
|
238
|
+
account_key = history_row.get("account_key")
|
|
239
|
+
if account_key:
|
|
240
|
+
row["account_key"] = account_key
|
|
241
|
+
return row
|
|
242
|
+
|
|
243
|
+
|
|
186
244
|
def _resolve_codex_weekly_cycle(
|
|
187
245
|
observations: Iterable[object],
|
|
188
246
|
now_utc: dt.datetime,
|
|
@@ -1693,6 +1751,7 @@ def _quota_read_model(
|
|
|
1693
1751
|
*,
|
|
1694
1752
|
accounting_entries: Iterable[object] = (),
|
|
1695
1753
|
account_key: str | None = None,
|
|
1754
|
+
decorated: bool,
|
|
1696
1755
|
) -> dict[str, object]:
|
|
1697
1756
|
"""Use S2's pure history/block/forecast kernels over cache evidence.
|
|
1698
1757
|
|
|
@@ -1714,9 +1773,11 @@ def _quota_read_model(
|
|
|
1714
1773
|
cost_entries = tuple(accounting_entries)
|
|
1715
1774
|
histories = build_history(quota_observations)
|
|
1716
1775
|
blocks = build_blocks(quota_observations)
|
|
1717
|
-
history_rows: list[dict[str, object]] = []
|
|
1718
1776
|
milestone_rows: list[dict[str, object]] = []
|
|
1719
|
-
|
|
1777
|
+
# #429 §4.2: one candidate unit per identity — (ordinal, history row, active
|
|
1778
|
+
# projection or None) — so the cap retains PAIRS instead of capping the two
|
|
1779
|
+
# lists independently and in different orders.
|
|
1780
|
+
candidates: list[tuple[int, dict[str, object], dict[str, object] | None]] = []
|
|
1720
1781
|
# R8 (#341 Task 4): the per-account `account_key` is serialized onto each
|
|
1721
1782
|
# history row ONLY when the Codex provider has >1 REAL account, so the
|
|
1722
1783
|
# dashboard client can scope per-account quota rows instead of merging them.
|
|
@@ -1726,13 +1787,12 @@ def _quota_read_model(
|
|
|
1726
1787
|
# public history view is LOSSY — capped at `SOURCE_HISTORY_LIMIT` and without
|
|
1727
1788
|
# `logical_limit_key` — so it cannot resolve the cycle authoritatively. Build
|
|
1728
1789
|
# time owns resolution; a `clock_data` decision deadline forces the rebuild.
|
|
1729
|
-
|
|
1730
|
-
|
|
1731
|
-
|
|
1732
|
-
|
|
1733
|
-
|
|
1734
|
-
|
|
1735
|
-
_codex_decorated = False
|
|
1790
|
+
#
|
|
1791
|
+
# #429 §4.4: the caller owns this gate. Re-querying here, per parent AND per
|
|
1792
|
+
# child, let a transient failure emit decorated scopes whose quota rows were
|
|
1793
|
+
# silently unstamped — and the active-row helper cannot project a field the
|
|
1794
|
+
# history row never carried.
|
|
1795
|
+
_codex_decorated = decorated
|
|
1736
1796
|
for history in histories:
|
|
1737
1797
|
identity = history.identity
|
|
1738
1798
|
key_parts = (
|
|
@@ -1777,21 +1837,11 @@ def _quota_read_model(
|
|
|
1777
1837
|
"confidence": forecast.confidence,
|
|
1778
1838
|
},
|
|
1779
1839
|
}
|
|
1780
|
-
|
|
1781
|
-
|
|
1782
|
-
|
|
1783
|
-
|
|
1784
|
-
|
|
1785
|
-
# history is the live one, so both must carry the SAME anchor.
|
|
1786
|
-
if baseline is not None and baseline.canonical_resets_at > context.now_utc:
|
|
1787
|
-
active_rows.append({
|
|
1788
|
-
"key": dashboard_resource_key("quota", "codex", *key_parts),
|
|
1789
|
-
"current_percent": baseline.used_percent,
|
|
1790
|
-
"captured_at": baseline.captured_at.astimezone(UTC).isoformat(),
|
|
1791
|
-
"resets_at": baseline.canonical_resets_at.astimezone(UTC).isoformat(),
|
|
1792
|
-
"freshness": freshness.state,
|
|
1793
|
-
"stale_after_seconds": freshness.stale_after_seconds,
|
|
1794
|
-
})
|
|
1840
|
+
candidates.append((
|
|
1841
|
+
len(candidates),
|
|
1842
|
+
row,
|
|
1843
|
+
_active_row_from_history(row, now_utc=context.now_utc),
|
|
1844
|
+
))
|
|
1795
1845
|
for block in blocks:
|
|
1796
1846
|
identity = block.identity
|
|
1797
1847
|
block_parts = (
|
|
@@ -1899,20 +1949,21 @@ def _quota_read_model(
|
|
|
1899
1949
|
"marginal_usd": max(0.0, cumulative_usd - previous_cumulative),
|
|
1900
1950
|
})
|
|
1901
1951
|
previous_cumulative = cumulative_usd
|
|
1902
|
-
latest_percent = max(
|
|
1903
|
-
(float(row["current_percent"]) for row in active_rows), default=None,
|
|
1904
|
-
)
|
|
1905
|
-
active_freshness = (
|
|
1906
|
-
"fresh" if active_rows and all(row["freshness"] == "fresh" for row in active_rows)
|
|
1907
|
-
else ("unavailable" if not active_rows else "stale")
|
|
1908
|
-
)
|
|
1909
1952
|
# Active account identities are presentation-critical. Independent
|
|
1910
1953
|
# model-scoped pools are also legitimate provider facts, so reserve the
|
|
1911
1954
|
# remaining cap space for their newest captures before inactive account
|
|
1912
1955
|
# history. Opaque resource-key order is only a stable tie-breaker.
|
|
1913
|
-
|
|
1956
|
+
#
|
|
1957
|
+
# #429 §4.2 — retention decides ONCE per (history, active) unit, then each
|
|
1958
|
+
# list is emitted in its own established order: histories in retention
|
|
1959
|
+
# order, actives in identity order. Emitting both in a single order would
|
|
1960
|
+
# reorder active rows below the cap and move bytes for every install.
|
|
1961
|
+
active_keys = {
|
|
1962
|
+
str(active["key"]) for _, _, active in candidates if active is not None
|
|
1963
|
+
}
|
|
1914
1964
|
|
|
1915
|
-
def _history_retention_key(
|
|
1965
|
+
def _history_retention_key(unit):
|
|
1966
|
+
_, row, _ = unit
|
|
1916
1967
|
key = str(row["key"])
|
|
1917
1968
|
if key in active_keys:
|
|
1918
1969
|
return (0, 0.0, key)
|
|
@@ -1927,11 +1978,22 @@ def _quota_read_model(
|
|
|
1927
1978
|
return (1, -captured_epoch, key)
|
|
1928
1979
|
return (2, 0.0, key)
|
|
1929
1980
|
|
|
1930
|
-
|
|
1931
|
-
history_rows =
|
|
1981
|
+
retained = sorted(candidates, key=_history_retention_key)[:SOURCE_HISTORY_LIMIT]
|
|
1982
|
+
history_rows = [row for _, row, _ in retained]
|
|
1983
|
+
active_rows = [
|
|
1984
|
+
active
|
|
1985
|
+
for _, _, active in sorted(retained, key=lambda unit: unit[0])
|
|
1986
|
+
if active is not None
|
|
1987
|
+
]
|
|
1988
|
+
latest_percent = max(
|
|
1989
|
+
(float(row["current_percent"]) for row in active_rows), default=None,
|
|
1990
|
+
)
|
|
1991
|
+
active_freshness = (
|
|
1992
|
+
"fresh" if active_rows and all(row["freshness"] == "fresh" for row in active_rows)
|
|
1993
|
+
else ("unavailable" if not active_rows else "stale")
|
|
1994
|
+
)
|
|
1932
1995
|
milestone_rows.sort(key=lambda row: str(row["captured_at"]), reverse=True)
|
|
1933
1996
|
milestone_rows = milestone_rows[:SOURCE_HISTORY_LIMIT]
|
|
1934
|
-
active_rows = active_rows[:SOURCE_HISTORY_LIMIT]
|
|
1935
1997
|
return {
|
|
1936
1998
|
"summary": {
|
|
1937
1999
|
"window_count": len(blocks),
|
|
@@ -2035,6 +2097,117 @@ def _refresh_budget_status_clock(
|
|
|
2035
2097
|
}
|
|
2036
2098
|
|
|
2037
2099
|
|
|
2100
|
+
def _scoped_quota_identity(row: Mapping[str, object]) -> tuple[str, str]:
|
|
2101
|
+
"""#429 §3.1. `dashboard_resource_key` carries no account, and two accounts
|
|
2102
|
+
sharing one $CODEX_HOME root emit the same key, so bare key is not an
|
|
2103
|
+
identity under decoration. `"unattributed"` is a legitimate account here."""
|
|
2104
|
+
return (str(row.get("account_key") or ""), str(row.get("key")))
|
|
2105
|
+
|
|
2106
|
+
|
|
2107
|
+
def _reclock_quota_domain(
|
|
2108
|
+
quota: Mapping[str, object], *, now_utc: dt.datetime,
|
|
2109
|
+
) -> dict[str, object]:
|
|
2110
|
+
"""Re-evaluate a quota domain's row freshness and summary against ``now``.
|
|
2111
|
+
|
|
2112
|
+
#429 §4.3. Replaces ONLY `histories` and `summary`; `blocks`, `milestones`
|
|
2113
|
+
and `cycle_index` are carried through untouched, because the per-account
|
|
2114
|
+
scopes carry them and a scope that lost them would render empty.
|
|
2115
|
+
|
|
2116
|
+
Emits TUPLES, matching what `_quota_read_model` publishes. Publication
|
|
2117
|
+
freezes lists into tuples anyway, so this is byte-identical on the wire —
|
|
2118
|
+
but it is what lets the caller detect an unchanged domain by comparing the
|
|
2119
|
+
result against the frozen original. A list would never compare equal to the
|
|
2120
|
+
tuple it was frozen from (``[] != ()``), the caller would report a change on
|
|
2121
|
+
every tick, and the retain/degrade paths that assert the EXACT prior ``data``
|
|
2122
|
+
object is handed back would break.
|
|
2123
|
+
"""
|
|
2124
|
+
refreshed = dict(quota)
|
|
2125
|
+
refreshed_histories: list[dict[str, object]] = []
|
|
2126
|
+
active_rows: list[dict[str, object]] = []
|
|
2127
|
+
for raw_history in quota.get("histories", ()):
|
|
2128
|
+
if not isinstance(raw_history, Mapping):
|
|
2129
|
+
continue
|
|
2130
|
+
history = dict(raw_history)
|
|
2131
|
+
# #350 spec §3.9: this is a PER-ROW value and must never shadow the
|
|
2132
|
+
# envelope-level `freshness`. It used to, so after the loop the
|
|
2133
|
+
# envelope held the LAST retained history row's freshness — often an
|
|
2134
|
+
# inactive row, and with a single weekly history the active weekly
|
|
2135
|
+
# one, which silently marked the whole provider stale on an idle
|
|
2136
|
+
# stale crossing and tripped idle eligibility on its own.
|
|
2137
|
+
row_freshness = _clock_freshness(
|
|
2138
|
+
history.get("captured_at"), history.get("stale_after_seconds"), now_utc,
|
|
2139
|
+
)
|
|
2140
|
+
history["freshness"] = row_freshness
|
|
2141
|
+
forecast = history.get("forecast")
|
|
2142
|
+
if isinstance(forecast, Mapping):
|
|
2143
|
+
forecast = dict(forecast)
|
|
2144
|
+
resets_at = forecast.get("resets_at")
|
|
2145
|
+
try:
|
|
2146
|
+
reset = dt.datetime.fromisoformat(
|
|
2147
|
+
str(resets_at).replace("Z", "+00:00")
|
|
2148
|
+
).astimezone(UTC)
|
|
2149
|
+
except (TypeError, ValueError):
|
|
2150
|
+
reset = None
|
|
2151
|
+
remaining = max(0, int((reset - now_utc).total_seconds())) if reset else None
|
|
2152
|
+
forecast["remaining_seconds"] = remaining
|
|
2153
|
+
sample_count = int(forecast.get("sample_count") or 0)
|
|
2154
|
+
if row_freshness == "future":
|
|
2155
|
+
forecast["status"] = "future"
|
|
2156
|
+
elif row_freshness == "stale":
|
|
2157
|
+
forecast["status"] = "stale"
|
|
2158
|
+
elif sample_count == 0:
|
|
2159
|
+
forecast["status"] = "insufficient-history"
|
|
2160
|
+
else:
|
|
2161
|
+
forecast["status"] = "ok"
|
|
2162
|
+
rate = forecast.get("rate_percent_per_hour")
|
|
2163
|
+
current = forecast.get("current_percent")
|
|
2164
|
+
if (
|
|
2165
|
+
isinstance(rate, (int, float)) and not isinstance(rate, bool)
|
|
2166
|
+
and isinstance(current, (int, float)) and not isinstance(current, bool)
|
|
2167
|
+
and remaining is not None
|
|
2168
|
+
):
|
|
2169
|
+
forecast["projected_percent"] = min(
|
|
2170
|
+
100.0, max(float(current), float(current) + float(rate) * remaining / 3600),
|
|
2171
|
+
)
|
|
2172
|
+
history["forecast"] = forecast
|
|
2173
|
+
# Called unconditionally, exactly as the build calls it. The helper
|
|
2174
|
+
# already returns None for a row without a usable forecast, and keeping
|
|
2175
|
+
# a caller-side `isinstance(forecast, Mapping)` guard here would put a
|
|
2176
|
+
# fragment of the predicate back on the caller — the split #429 exists
|
|
2177
|
+
# to remove.
|
|
2178
|
+
active = _active_row_from_history(history, now_utc=now_utc)
|
|
2179
|
+
if active is not None:
|
|
2180
|
+
active_rows.append(active)
|
|
2181
|
+
refreshed_histories.append(history)
|
|
2182
|
+
summary = dict(quota.get("summary") or {})
|
|
2183
|
+
prior_active = summary.get("active")
|
|
2184
|
+
if isinstance(prior_active, (tuple, list)):
|
|
2185
|
+
# #429 §3.1: scoped identity, not bare key — two decorated rows can
|
|
2186
|
+
# share one key and would collapse into a single map entry.
|
|
2187
|
+
active_order = {
|
|
2188
|
+
_scoped_quota_identity(row): index
|
|
2189
|
+
for index, row in enumerate(prior_active)
|
|
2190
|
+
if isinstance(row, Mapping)
|
|
2191
|
+
}
|
|
2192
|
+
active_rows.sort(
|
|
2193
|
+
key=lambda row: active_order.get(
|
|
2194
|
+
_scoped_quota_identity(row), len(active_order)),
|
|
2195
|
+
)
|
|
2196
|
+
summary.update({
|
|
2197
|
+
"active_window_count": len(active_rows),
|
|
2198
|
+
"latest_percent": max(
|
|
2199
|
+
(float(row["current_percent"]) for row in active_rows), default=None),
|
|
2200
|
+
"freshness": (
|
|
2201
|
+
"fresh" if active_rows and all(row["freshness"] == "fresh" for row in active_rows)
|
|
2202
|
+
else ("unavailable" if not active_rows else "stale")
|
|
2203
|
+
),
|
|
2204
|
+
"active": tuple(active_rows),
|
|
2205
|
+
})
|
|
2206
|
+
refreshed["histories"] = tuple(refreshed_histories)
|
|
2207
|
+
refreshed["summary"] = summary
|
|
2208
|
+
return refreshed
|
|
2209
|
+
|
|
2210
|
+
|
|
2038
2211
|
def refresh_codex_source_clock(
|
|
2039
2212
|
state: SourceDashboardState,
|
|
2040
2213
|
*,
|
|
@@ -2059,148 +2232,48 @@ def refresh_codex_source_clock(
|
|
|
2059
2232
|
freshness = state.freshness
|
|
2060
2233
|
domain_freshness = dict(state.domain_freshness or {})
|
|
2061
2234
|
if isinstance(quota, Mapping):
|
|
2062
|
-
|
|
2063
|
-
|
|
2064
|
-
|
|
2065
|
-
|
|
2066
|
-
|
|
2067
|
-
|
|
2068
|
-
|
|
2069
|
-
|
|
2070
|
-
|
|
2071
|
-
|
|
2072
|
-
|
|
2073
|
-
# one, which silently marked the whole provider stale on an idle
|
|
2074
|
-
# stale crossing and tripped idle eligibility on its own.
|
|
2075
|
-
row_freshness = _clock_freshness(
|
|
2076
|
-
history.get("captured_at"), history.get("stale_after_seconds"), now_utc,
|
|
2077
|
-
)
|
|
2078
|
-
history["freshness"] = row_freshness
|
|
2079
|
-
forecast = history.get("forecast")
|
|
2080
|
-
if isinstance(forecast, Mapping):
|
|
2081
|
-
forecast = dict(forecast)
|
|
2082
|
-
resets_at = forecast.get("resets_at")
|
|
2083
|
-
try:
|
|
2084
|
-
reset = dt.datetime.fromisoformat(
|
|
2085
|
-
str(resets_at).replace("Z", "+00:00")
|
|
2086
|
-
).astimezone(UTC)
|
|
2087
|
-
except (TypeError, ValueError):
|
|
2088
|
-
reset = None
|
|
2089
|
-
remaining = max(0, int((reset - now_utc).total_seconds())) if reset else None
|
|
2090
|
-
forecast["remaining_seconds"] = remaining
|
|
2091
|
-
sample_count = int(forecast.get("sample_count") or 0)
|
|
2092
|
-
if row_freshness == "future":
|
|
2093
|
-
forecast["status"] = "future"
|
|
2094
|
-
elif row_freshness == "stale":
|
|
2095
|
-
forecast["status"] = "stale"
|
|
2096
|
-
elif sample_count == 0:
|
|
2097
|
-
forecast["status"] = "insufficient-history"
|
|
2098
|
-
else:
|
|
2099
|
-
forecast["status"] = "ok"
|
|
2100
|
-
rate = forecast.get("rate_percent_per_hour")
|
|
2101
|
-
current = forecast.get("current_percent")
|
|
2102
|
-
if (
|
|
2103
|
-
isinstance(rate, (int, float)) and not isinstance(rate, bool)
|
|
2104
|
-
and isinstance(current, (int, float)) and not isinstance(current, bool)
|
|
2105
|
-
and remaining is not None
|
|
2106
|
-
):
|
|
2107
|
-
forecast["projected_percent"] = min(
|
|
2108
|
-
100.0, max(float(current), float(current) + float(rate) * remaining / 3600),
|
|
2109
|
-
)
|
|
2110
|
-
history["forecast"] = forecast
|
|
2111
|
-
# #373: same rule as the initial build, through the same
|
|
2112
|
-
# predicate, so the two paths cannot drift.
|
|
2113
|
-
if (
|
|
2114
|
-
not _codex_history_row_is_model_scoped(history)
|
|
2115
|
-
and reset is not None and reset > now_utc and current is not None
|
|
2116
|
-
):
|
|
2117
|
-
active_rows.append({
|
|
2118
|
-
"key": history.get("key"),
|
|
2119
|
-
"current_percent": current,
|
|
2120
|
-
"captured_at": history.get("captured_at"),
|
|
2121
|
-
"resets_at": resets_at,
|
|
2122
|
-
"freshness": row_freshness,
|
|
2123
|
-
"stale_after_seconds": history.get("stale_after_seconds"),
|
|
2124
|
-
})
|
|
2125
|
-
refreshed_histories.append(history)
|
|
2126
|
-
quota["histories"] = refreshed_histories
|
|
2127
|
-
latest_percent = max(
|
|
2128
|
-
(float(row["current_percent"]) for row in active_rows), default=None,
|
|
2129
|
-
)
|
|
2130
|
-
summary = dict(quota.get("summary") or {})
|
|
2131
|
-
prior_active = summary.get("active")
|
|
2132
|
-
if isinstance(prior_active, (tuple, list)):
|
|
2133
|
-
active_order = {
|
|
2134
|
-
str(row.get("key")): index
|
|
2135
|
-
for index, row in enumerate(prior_active)
|
|
2136
|
-
if isinstance(row, Mapping)
|
|
2137
|
-
}
|
|
2138
|
-
active_rows.sort(
|
|
2139
|
-
key=lambda row: active_order.get(str(row.get("key")), len(active_order)),
|
|
2140
|
-
)
|
|
2141
|
-
summary.update({
|
|
2142
|
-
"active_window_count": len(active_rows),
|
|
2143
|
-
"latest_percent": latest_percent,
|
|
2144
|
-
"freshness": (
|
|
2145
|
-
"fresh" if active_rows and all(row["freshness"] == "fresh" for row in active_rows)
|
|
2146
|
-
else ("unavailable" if not active_rows else "stale")
|
|
2147
|
-
),
|
|
2148
|
-
"active": active_rows,
|
|
2149
|
-
})
|
|
2150
|
-
# Only account-level active histories reach ``active_rows``; the shared
|
|
2151
|
-
# model-scoped predicate above excludes foreign pools. An unavailable
|
|
2152
|
-
# active set is a capability/data-availability fact, not invented
|
|
2153
|
-
# staleness, so only the exact stale verdict moves this axis.
|
|
2235
|
+
reclocked = _reclock_quota_domain(quota, now_utc=now_utc)
|
|
2236
|
+
quota_changed = reclocked != quota
|
|
2237
|
+
quota = reclocked
|
|
2238
|
+
data["quota"] = quota
|
|
2239
|
+
# Only account-level active histories reach the summary; the shared
|
|
2240
|
+
# model-scoped predicate excludes foreign pools. An unavailable active
|
|
2241
|
+
# set is a capability/data-availability fact, not invented staleness,
|
|
2242
|
+
# so only the exact stale verdict moves this axis. #429 §4.3 keeps this
|
|
2243
|
+
# deriving from the TOP-LEVEL summary only — the #350 §3.9 rule that a
|
|
2244
|
+
# row-level freshness value never touches `state.freshness` extends to
|
|
2245
|
+
# the per-account scopes clocked below.
|
|
2154
2246
|
domain_freshness["quota"] = (
|
|
2155
|
-
"stale" if summary["freshness"] == "stale" else "fresh"
|
|
2247
|
+
"stale" if quota["summary"]["freshness"] == "stale" else "fresh"
|
|
2156
2248
|
)
|
|
2157
|
-
|
|
2158
|
-
|
|
2159
|
-
|
|
2160
|
-
|
|
2161
|
-
|
|
2162
|
-
|
|
2163
|
-
|
|
2164
|
-
|
|
2165
|
-
|
|
2166
|
-
|
|
2167
|
-
|
|
2168
|
-
|
|
2169
|
-
|
|
2170
|
-
|
|
2171
|
-
|
|
2172
|
-
|
|
2173
|
-
|
|
2174
|
-
|
|
2175
|
-
|
|
2176
|
-
|
|
2177
|
-
|
|
2178
|
-
|
|
2179
|
-
|
|
2180
|
-
|
|
2181
|
-
|
|
2182
|
-
"cost_usd", "input_tokens", "cached_input_tokens", "output_tokens",
|
|
2183
|
-
"reasoning_output_tokens", "total_tokens", "cycle",
|
|
2184
|
-
):
|
|
2185
|
-
hero[field] = None
|
|
2186
|
-
data["hero"] = hero
|
|
2187
|
-
refreshed_capabilities = dict(state.capabilities)
|
|
2188
|
-
refreshed_capabilities["hero"] = CapabilityRecord(
|
|
2189
|
-
"unavailable", "missing-or-conflicting-native-cycle",
|
|
2190
|
-
)
|
|
2191
|
-
capabilities = refreshed_capabilities
|
|
2192
|
-
warnings = tuple(
|
|
2193
|
-
warning for warning in state.warnings
|
|
2194
|
-
if warning.code != "codex_cycle_unavailable"
|
|
2195
|
-
) + (SourceDashboardWarning(
|
|
2196
|
-
"codex_cycle_unavailable",
|
|
2197
|
-
"Codex native reset cycle is unavailable.",
|
|
2198
|
-
"hero",
|
|
2199
|
-
),)
|
|
2200
|
-
availability = "partial"
|
|
2201
|
-
cycle_changed = True
|
|
2249
|
+
# #429 §4.3: the per-account scopes are independent evidence domains and
|
|
2250
|
+
# were never clocked at all, so a focused account read frozen freshness
|
|
2251
|
+
# forever. The published state is recursively frozen (`MappingProxyType`),
|
|
2252
|
+
# so nothing may be mutated in place — copy outward: the scopes mapping,
|
|
2253
|
+
# then the scope, then only that scope's `quota`.
|
|
2254
|
+
scopes = data.get("account_scopes")
|
|
2255
|
+
scopes_changed = False
|
|
2256
|
+
if isinstance(scopes, Mapping):
|
|
2257
|
+
rebuilt_scopes = dict(scopes)
|
|
2258
|
+
for scope_key, scope in scopes.items():
|
|
2259
|
+
if not isinstance(scope, Mapping):
|
|
2260
|
+
continue
|
|
2261
|
+
scope_quota = scope.get("quota")
|
|
2262
|
+
if not isinstance(scope_quota, Mapping):
|
|
2263
|
+
continue
|
|
2264
|
+
reclocked_scope_quota = _reclock_quota_domain(
|
|
2265
|
+
scope_quota, now_utc=now_utc)
|
|
2266
|
+
if reclocked_scope_quota == scope_quota:
|
|
2267
|
+
continue
|
|
2268
|
+
rebuilt_scope = dict(scope)
|
|
2269
|
+
rebuilt_scope["quota"] = reclocked_scope_quota
|
|
2270
|
+
rebuilt_scopes[scope_key] = rebuilt_scope
|
|
2271
|
+
scopes_changed = True
|
|
2272
|
+
if scopes_changed:
|
|
2273
|
+
data["account_scopes"] = rebuilt_scopes
|
|
2202
2274
|
budget_domain = data.get("budget")
|
|
2203
2275
|
budget_changed = False
|
|
2276
|
+
refreshed_budget = None
|
|
2204
2277
|
if isinstance(budget_domain, Mapping):
|
|
2205
2278
|
budget_domain = dict(budget_domain)
|
|
2206
2279
|
refreshed_budget = _refresh_budget_status_clock(
|
|
@@ -2214,13 +2287,62 @@ def refresh_codex_source_clock(
|
|
|
2214
2287
|
if refreshed_budget is not None:
|
|
2215
2288
|
budget_domain["status"] = refreshed_budget
|
|
2216
2289
|
data["budget"] = budget_domain
|
|
2217
|
-
hero = data.get("hero")
|
|
2218
|
-
if isinstance(hero, Mapping):
|
|
2219
|
-
hero = dict(hero)
|
|
2220
|
-
hero["budget"] = refreshed_budget
|
|
2221
|
-
data["hero"] = hero
|
|
2222
2290
|
budget_changed = True
|
|
2223
|
-
|
|
2291
|
+
# #429 §4.5: all three hero mutations compose on ONE copy, in a stated
|
|
2292
|
+
# order. Three independent `dict(hero)` copies let the last write win, which
|
|
2293
|
+
# is how `hero["quota"]` stayed frozen at its build-time value while
|
|
2294
|
+
# `quota["summary"]` advanced.
|
|
2295
|
+
hero = data.get("hero")
|
|
2296
|
+
if isinstance(hero, Mapping):
|
|
2297
|
+
hero = dict(hero)
|
|
2298
|
+
# 1. quota first — `_clock_cycle_expired` reads `hero["cycle"]`, never
|
|
2299
|
+
# `hero["quota"]`, so replacing quota cannot affect the predicate.
|
|
2300
|
+
if isinstance(quota, Mapping):
|
|
2301
|
+
hero["quota"] = quota["summary"]
|
|
2302
|
+
# 2. cycle expiry, with its capability/warning consequences.
|
|
2303
|
+
# #350 spec §3.3: the clock no longer RE-DERIVES cycle validity. Its
|
|
2304
|
+
# public-history view is lossy (capped, no `logical_limit_key`, no
|
|
2305
|
+
# `quota_identity`), so it cannot resolve the cycle correctly — and
|
|
2306
|
+
# per §2.2 it cannot simply trust the old verdict forever either,
|
|
2307
|
+
# because resolution is time-dependent on frozen evidence. Build time
|
|
2308
|
+
# owns resolution and records a decision deadline in `clock_data`; the
|
|
2309
|
+
# tick rebuilds authoritatively at the crossing. All the clock keeps
|
|
2310
|
+
# is this cheap invariant guard: a cycle that has already RESET cannot
|
|
2311
|
+
# bound current accounting, so it degrades exactly as before. Expiry
|
|
2312
|
+
# is also deadline candidate #1, so the two paths are disjoint
|
|
2313
|
+
# belt-and-suspenders rather than a single mechanism.
|
|
2314
|
+
hero_capability = state.capabilities.get("hero")
|
|
2315
|
+
if (
|
|
2316
|
+
isinstance(hero.get("cycle"), Mapping)
|
|
2317
|
+
and hero_capability is not None
|
|
2318
|
+
and hero_capability.status == "supported"
|
|
2319
|
+
and _clock_cycle_expired(hero.get("cycle"), now_utc)
|
|
2320
|
+
):
|
|
2321
|
+
for field in (
|
|
2322
|
+
"cost_usd", "input_tokens", "cached_input_tokens", "output_tokens",
|
|
2323
|
+
"reasoning_output_tokens", "total_tokens", "cycle",
|
|
2324
|
+
):
|
|
2325
|
+
hero[field] = None
|
|
2326
|
+
refreshed_capabilities = dict(state.capabilities)
|
|
2327
|
+
refreshed_capabilities["hero"] = CapabilityRecord(
|
|
2328
|
+
"unavailable", "missing-or-conflicting-native-cycle",
|
|
2329
|
+
)
|
|
2330
|
+
capabilities = refreshed_capabilities
|
|
2331
|
+
warnings = tuple(
|
|
2332
|
+
warning for warning in state.warnings
|
|
2333
|
+
if warning.code != "codex_cycle_unavailable"
|
|
2334
|
+
) + (SourceDashboardWarning(
|
|
2335
|
+
"codex_cycle_unavailable",
|
|
2336
|
+
"Codex native reset cycle is unavailable.",
|
|
2337
|
+
"hero",
|
|
2338
|
+
),)
|
|
2339
|
+
availability = "partial"
|
|
2340
|
+
cycle_changed = True
|
|
2341
|
+
# 3. budget last
|
|
2342
|
+
if refreshed_budget is not None:
|
|
2343
|
+
hero["budget"] = refreshed_budget
|
|
2344
|
+
data["hero"] = hero
|
|
2345
|
+
if not (quota_changed or budget_changed or cycle_changed or scopes_changed):
|
|
2224
2346
|
return state
|
|
2225
2347
|
refreshed_state = SourceDashboardState(
|
|
2226
2348
|
source=state.source,
|
|
@@ -2832,6 +2954,9 @@ def _codex_account_scopes_wire(
|
|
|
2832
2954
|
quota = _quota_read_model(
|
|
2833
2955
|
context, account_observations, accounting_entries=rows,
|
|
2834
2956
|
account_key=key,
|
|
2957
|
+
# #429 §4.4: this wire is only ever reached under decoration — the
|
|
2958
|
+
# caller gates the whole `account_scopes` surface on it.
|
|
2959
|
+
decorated=True,
|
|
2835
2960
|
)
|
|
2836
2961
|
# Each decorated child owns the only honest cycle index for that
|
|
2837
2962
|
# account. Reusing a merged parent index here would render account A's
|
|
@@ -3228,6 +3353,7 @@ def build_codex_source_state(
|
|
|
3228
3353
|
context,
|
|
3229
3354
|
quota_observations,
|
|
3230
3355
|
accounting_entries=visible_accounting_entries,
|
|
3356
|
+
decorated=_codex_decorated,
|
|
3231
3357
|
)
|
|
3232
3358
|
# R8 gate, resolved ONCE and threaded (#341 Task 4 / #416 §5.8). Every
|
|
3233
3359
|
# per-account decoration below — block/alert/budget `account_key`, the
|
package/bin/_cctally_db.py
CHANGED
|
@@ -3925,6 +3925,54 @@ def _conv_001_adopt_schema_version_marker(conn: sqlite3.Connection) -> None:
|
|
|
3925
3925
|
conn.commit()
|
|
3926
3926
|
|
|
3927
3927
|
|
|
3928
|
+
@conversations_migration("002_codex_thread_source_inference_replay")
|
|
3929
|
+
def _conv_002_codex_thread_source_inference_replay(
|
|
3930
|
+
conn: sqlite3.Connection,
|
|
3931
|
+
) -> None:
|
|
3932
|
+
"""Arm the conversations half of the byte-zero Codex replay.
|
|
3933
|
+
|
|
3934
|
+
Spec:
|
|
3935
|
+
``docs/superpowers/specs/2026-07-30-codex-thread-source-inference-design.md``
|
|
3936
|
+
§4.3.
|
|
3937
|
+
|
|
3938
|
+
Writes a marker ONLY; it clears no table. ``sync_codex_conversations``
|
|
3939
|
+
consumes it, and DEFERS while the cache-side marker is still pending —
|
|
3940
|
+
``_recompute_codex_rollups`` resolves project attribution from the cache-side
|
|
3941
|
+
thread row, and a missing one stamps a materialized ``"(unassigned)"`` the
|
|
3942
|
+
read path then prefers permanently.
|
|
3943
|
+
|
|
3944
|
+
The key is DISTINCT from ``conversation_rebuild_codex_pending`` on purpose.
|
|
3945
|
+
``_ensure_codex_conversation_contract`` consumes that one by replaying
|
|
3946
|
+
normalization over already-retained events — which preserves their NULL
|
|
3947
|
+
conversation keys — and then deletes it, so a ``dashboard --no-sync`` or
|
|
3948
|
+
qualified CLI read between this migration and the next real sync would
|
|
3949
|
+
silently discard the repair.
|
|
3950
|
+
|
|
3951
|
+
Takes the Codex conversations provider flock first, the way cache migration
|
|
3952
|
+
``028_split_conversation_store`` does, and DEFERS on contention. The
|
|
3953
|
+
conversations dispatcher runs inside ``_conversations_open_guarded``, which
|
|
3954
|
+
holds ``CONVERSATIONS_LOCK_MAINTENANCE_PATH`` only SHARED, while
|
|
3955
|
+
``sync_codex_conversations`` serializes on ``CONVERSATIONS_LOCK_CODEX_PATH``
|
|
3956
|
+
— so without this lock the marker can be armed in the middle of a walk that
|
|
3957
|
+
already read it as absent, and that walk's finalize would clear a replay it
|
|
3958
|
+
never performed. Deferring leaves the migration pending, so it arms cleanly
|
|
3959
|
+
at the next open.
|
|
3960
|
+
|
|
3961
|
+
Idempotent: re-running rewrites the same marker. NO self-stamp — the
|
|
3962
|
+
dispatcher central-stamps on a clean return (#140).
|
|
3963
|
+
"""
|
|
3964
|
+
import _cctally_cache
|
|
3965
|
+
|
|
3966
|
+
held = _acquire_conversations_db_codex_provider_flock(
|
|
3967
|
+
conn, migration="conversations 002 thread_source replay")
|
|
3968
|
+
try:
|
|
3969
|
+
_set_cache_meta(
|
|
3970
|
+
conn, _cctally_cache.CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY, "1")
|
|
3971
|
+
conn.commit()
|
|
3972
|
+
finally:
|
|
3973
|
+
_release_cache_db_writer_flocks(held)
|
|
3974
|
+
|
|
3975
|
+
|
|
3928
3976
|
# #177 S6: the consolidated multi-column external-content FTS5 table that
|
|
3929
3977
|
# replaces the old conversation_fts(text) + conversation_fts_aux(search_aux)
|
|
3930
3978
|
# pair. The three column names MUST match the conversation_messages columns BY
|
|
@@ -4683,6 +4731,44 @@ def _acquire_cache_db_codex_provider_flock(
|
|
|
4683
4731
|
return held
|
|
4684
4732
|
|
|
4685
4733
|
|
|
4734
|
+
def _acquire_conversations_db_codex_provider_flock(
|
|
4735
|
+
conn: sqlite3.Connection,
|
|
4736
|
+
*,
|
|
4737
|
+
migration: str,
|
|
4738
|
+
) -> list[int]:
|
|
4739
|
+
"""Take the ``<conversations.db>.codex.lock`` sibling, or DEFER.
|
|
4740
|
+
|
|
4741
|
+
The conversations dispatcher runs under a SHARED maintenance flock, so —
|
|
4742
|
+
unlike the cache dispatcher — it does not exclude the provider sync that
|
|
4743
|
+
owns the marker lifecycle. A conversations handler that writes a marker
|
|
4744
|
+
``sync_codex_conversations`` consumes must therefore hold the same
|
|
4745
|
+
provider lock that sync holds, or it can arm mid-walk and have its marker
|
|
4746
|
+
swallowed by a walk that already read it as absent.
|
|
4747
|
+
|
|
4748
|
+
Derived from the connection (the lock-path helper is store-agnostic: main DB
|
|
4749
|
+
file + ``.codex.lock``), so a migration test never contends on the caller's
|
|
4750
|
+
real conversations lock.
|
|
4751
|
+
"""
|
|
4752
|
+
provider_path = _cache_db_codex_lock_path_for_conn(conn)
|
|
4753
|
+
if provider_path is None:
|
|
4754
|
+
return []
|
|
4755
|
+
|
|
4756
|
+
from _lib_cache_writer_lock import acquire_ordered_flocks
|
|
4757
|
+
|
|
4758
|
+
try:
|
|
4759
|
+
held = acquire_ordered_flocks([(provider_path, fcntl.LOCK_EX)])
|
|
4760
|
+
except OSError as exc:
|
|
4761
|
+
raise MigrationGateNotMet(
|
|
4762
|
+
f"conversations.db Codex lock unavailable; deferring {migration}"
|
|
4763
|
+
) from exc
|
|
4764
|
+
if held is None:
|
|
4765
|
+
raise MigrationGateNotMet(
|
|
4766
|
+
f"conversations.db Codex lock held by a concurrent Codex "
|
|
4767
|
+
f"conversation sync; deferring {migration}"
|
|
4768
|
+
)
|
|
4769
|
+
return held
|
|
4770
|
+
|
|
4771
|
+
|
|
4686
4772
|
def _release_cache_db_writer_flocks(held: list[int]) -> None:
|
|
4687
4773
|
from _lib_cache_writer_lock import release_cache_writer_flocks
|
|
4688
4774
|
|
|
@@ -6135,6 +6221,42 @@ def _034_codex_window_spend_adoption(conn: sqlite3.Connection) -> None:
|
|
|
6135
6221
|
_release_cache_db_writer_flocks(held)
|
|
6136
6222
|
|
|
6137
6223
|
|
|
6224
|
+
@cache_migration("035_codex_thread_source_inference_replay")
|
|
6225
|
+
def _035_codex_thread_source_inference_replay(conn: sqlite3.Connection) -> None:
|
|
6226
|
+
"""Arm a byte-zero Codex replay so rollouts whose ``session_meta`` omits
|
|
6227
|
+
``thread_source`` gain a conversation identity.
|
|
6228
|
+
|
|
6229
|
+
Spec:
|
|
6230
|
+
``docs/superpowers/specs/2026-07-30-codex-thread-source-inference-design.md``
|
|
6231
|
+
§4.3.
|
|
6232
|
+
|
|
6233
|
+
The repair must re-read the rollout bytes: the retained events carry NULL
|
|
6234
|
+
conversation keys and the in-place normalization replay preserves them.
|
|
6235
|
+
|
|
6236
|
+
Writes a marker ONLY. ``sync_codex_cache`` consumes it and ORs it into its
|
|
6237
|
+
own ``rebuild``, which is what makes the rebuild path capture
|
|
6238
|
+
``rebuild_known_identities`` BEFORE the clear. Clearing here instead would
|
|
6239
|
+
delete ``codex_session_files`` out of band, leaving the next ordinary sync
|
|
6240
|
+
with an empty snapshot — every re-read rollout would fall through to the
|
|
6241
|
+
live-``auth.json`` branch and pre-mechanism Codex spend would be
|
|
6242
|
+
re-attributed to whoever is authenticated now (#416 spec D1). Migrations 026
|
|
6243
|
+
and 027 predate that snapshot and are already stamped, so this would be the
|
|
6244
|
+
first migration to hit it live.
|
|
6245
|
+
|
|
6246
|
+
No provider flock: unlike handlers 024-027 and 034 this writes one
|
|
6247
|
+
``cache_meta`` row and touches no Codex-derived table, so a concurrent
|
|
6248
|
+
``sync_codex_cache`` has nothing to interleave with — and if one is mid-walk,
|
|
6249
|
+
arming the marker simply defers the replay to the following sync.
|
|
6250
|
+
|
|
6251
|
+
Idempotent: re-running rewrites the same marker. NO self-stamp — the
|
|
6252
|
+
dispatcher central-stamps on a clean return (#140).
|
|
6253
|
+
"""
|
|
6254
|
+
import _cctally_cache
|
|
6255
|
+
|
|
6256
|
+
_set_cache_meta(conn, _cctally_cache.CODEX_REPLAY_FROM_ZERO_KEY, "1")
|
|
6257
|
+
conn.commit()
|
|
6258
|
+
|
|
6259
|
+
|
|
6138
6260
|
# === Region 7d: Stats migration 008_recompute_weekly_cost_snapshots_dedup_fix ===
|
|
6139
6261
|
|
|
6140
6262
|
@stats_migration("008_recompute_weekly_cost_snapshots_dedup_fix")
|
package/bin/_cctally_doctor.py
CHANGED
|
@@ -1125,12 +1125,25 @@ def _doctor_gather_state_impl(
|
|
|
1125
1125
|
# #416 review B4: the durable record that a torn Codex `auth.json` halted
|
|
1126
1126
|
# ingest. Same cache_meta read, same degrade-to-None-on-anything contract.
|
|
1127
1127
|
codex_torn_deferred = None
|
|
1128
|
+
# The byte-zero Codex replay stall signal. The marker itself is a bare "1";
|
|
1129
|
+
# the sibling `blocked` record is the JSON one, so it is read through the
|
|
1130
|
+
# same loop while the marker gets a plain existence probe. Key names come
|
|
1131
|
+
# from the kernel constants, never inline literals.
|
|
1132
|
+
codex_replay_pending = None
|
|
1133
|
+
codex_replay_blocked = None
|
|
1134
|
+
try:
|
|
1135
|
+
import _lib_codex_conversation as _codex_kern
|
|
1136
|
+
_blocked_key = _codex_kern.CODEX_REPLAY_BLOCKED_KEY
|
|
1137
|
+
_pending_key = _codex_kern.CODEX_REPLAY_FROM_ZERO_KEY
|
|
1138
|
+
except Exception:
|
|
1139
|
+
_blocked_key = "codex_replay_from_zero_blocked"
|
|
1140
|
+
_pending_key = "codex_replay_from_zero_pending"
|
|
1128
1141
|
try:
|
|
1129
1142
|
if _cache_probe_allowed and _cctally_core.CACHE_DB_PATH.exists():
|
|
1130
1143
|
conn = sqlite3.connect(str(_cctally_core.CACHE_DB_PATH))
|
|
1131
1144
|
try:
|
|
1132
1145
|
for _key in ("parse_health_claude", "parse_health_codex",
|
|
1133
|
-
"codex_torn_auth_deferred"):
|
|
1146
|
+
"codex_torn_auth_deferred", _blocked_key):
|
|
1134
1147
|
try:
|
|
1135
1148
|
row = conn.execute(
|
|
1136
1149
|
"SELECT value FROM cache_meta WHERE key = ?",
|
|
@@ -1143,10 +1156,19 @@ def _doctor_gather_state_impl(
|
|
|
1143
1156
|
parse_health_claude = _parsed
|
|
1144
1157
|
elif _key == "parse_health_codex":
|
|
1145
1158
|
parse_health_codex = _parsed
|
|
1159
|
+
elif _key == _blocked_key:
|
|
1160
|
+
codex_replay_blocked = _parsed
|
|
1146
1161
|
else:
|
|
1147
1162
|
codex_torn_deferred = _parsed
|
|
1148
1163
|
except (sqlite3.OperationalError, ValueError):
|
|
1149
1164
|
pass
|
|
1165
|
+
try:
|
|
1166
|
+
codex_replay_pending = conn.execute(
|
|
1167
|
+
"SELECT 1 FROM cache_meta WHERE key = ?",
|
|
1168
|
+
(_pending_key,),
|
|
1169
|
+
).fetchone() is not None
|
|
1170
|
+
except sqlite3.OperationalError:
|
|
1171
|
+
pass
|
|
1150
1172
|
finally:
|
|
1151
1173
|
conn.close()
|
|
1152
1174
|
except Exception:
|
|
@@ -1722,6 +1744,8 @@ def _doctor_gather_state_impl(
|
|
|
1722
1744
|
parse_health_claude=parse_health_claude,
|
|
1723
1745
|
parse_health_codex=parse_health_codex,
|
|
1724
1746
|
codex_torn_deferred=codex_torn_deferred,
|
|
1747
|
+
codex_replay_pending=codex_replay_pending,
|
|
1748
|
+
codex_replay_blocked=codex_replay_blocked,
|
|
1725
1749
|
stats_db_quick_check=stats_db_quick_check,
|
|
1726
1750
|
cache_db_quick_check=cache_db_quick_check,
|
|
1727
1751
|
conversations_db_quick_check=conversations_db_quick_check,
|
|
@@ -49,9 +49,13 @@ def cmd_transcript(args) -> int:
|
|
|
49
49
|
# ---- export ----------------------------------------------------------------
|
|
50
50
|
|
|
51
51
|
_SPEED_ONLY_CODEX_MSG = "transcript: --speed applies only to Codex conversations"
|
|
52
|
+
# Deliberately generic about WHAT is pending. Three distinct conditions reach
|
|
53
|
+
# this branch — a cache predating migration 025, a store with the contract
|
|
54
|
+
# rebuild marker armed, and a store with a byte-zero Codex replay marker armed —
|
|
55
|
+
# and naming the migration told users to wait for work applied long ago.
|
|
52
56
|
_PENDING_EXPORT_MSG = (
|
|
53
57
|
"transcript: Codex conversation is not yet normalized "
|
|
54
|
-
"(
|
|
58
|
+
"(the pending work runs on the next cache open) — retry shortly")
|
|
55
59
|
|
|
56
60
|
|
|
57
61
|
def _emit_export(md: str, output) -> None:
|
|
@@ -213,9 +217,10 @@ def _cmd_transcript_search_claude(args) -> int:
|
|
|
213
217
|
|
|
214
218
|
# ---- search (Codex) --------------------------------------------------------
|
|
215
219
|
|
|
220
|
+
# Same generic wording as `_PENDING_EXPORT_MSG`, for the same reason.
|
|
216
221
|
_CODEX_SEARCH_PENDING_MSG = (
|
|
217
222
|
"transcript: Codex conversations are not yet normalized "
|
|
218
|
-
"(
|
|
223
|
+
"(the pending work runs on the next cache open); no results yet")
|
|
219
224
|
|
|
220
225
|
|
|
221
226
|
def _cmd_transcript_search_codex(args) -> int:
|
package/bin/_cctally_tui.py
CHANGED
|
@@ -271,6 +271,7 @@ from _cctally_dashboard_sources import (
|
|
|
271
271
|
resolve_dashboard_source_semantics,
|
|
272
272
|
)
|
|
273
273
|
from _lib_dashboard_sources import (
|
|
274
|
+
SOURCE_SCHEMA_VERSION,
|
|
274
275
|
CapabilityRecord,
|
|
275
276
|
SourceDashboardBundle,
|
|
276
277
|
SourceDashboardState,
|
|
@@ -2803,7 +2804,7 @@ def _tui_build_source_bundle(
|
|
|
2803
2804
|
codex = refresh_codex_source_clock(codex, now_utc=now_utc)
|
|
2804
2805
|
combined = compose_all_state(claude, codex)
|
|
2805
2806
|
bundle = SourceDashboardBundle(
|
|
2806
|
-
source_schema_version=
|
|
2807
|
+
source_schema_version=SOURCE_SCHEMA_VERSION,
|
|
2807
2808
|
default_source="claude",
|
|
2808
2809
|
source_order=("claude", "codex", "all"),
|
|
2809
2810
|
sources={"claude": claude, "codex": codex, "all": combined},
|
|
@@ -2877,7 +2878,7 @@ def _tui_hydrating_source_bundle() -> SourceDashboardBundle:
|
|
|
2877
2878
|
domain_freshness={"hero": "stale", "quota": "stale", "sessions": "stale"},
|
|
2878
2879
|
)
|
|
2879
2880
|
return SourceDashboardBundle(
|
|
2880
|
-
source_schema_version=
|
|
2881
|
+
source_schema_version=SOURCE_SCHEMA_VERSION,
|
|
2881
2882
|
default_source="claude",
|
|
2882
2883
|
source_order=("claude", "codex", "all"),
|
|
2883
2884
|
sources={"claude": claude, "codex": codex, "all": compose_all_state(claude, codex)},
|
|
@@ -74,6 +74,23 @@ CODEX_TITLE_MAX = _TITLE_MAX
|
|
|
74
74
|
# migration (the store is wholly re-derivable).
|
|
75
75
|
CODEX_CONVERSATION_CONTRACT_VERSION = "5"
|
|
76
76
|
|
|
77
|
+
# Byte-zero Codex replay markers (spec
|
|
78
|
+
# docs/superpowers/specs/2026-07-30-codex-thread-source-inference-design.md
|
|
79
|
+
# §4.3). One `cache_meta` key per store, armed by cache migration 035 /
|
|
80
|
+
# conversations migration 002 and consumed inside the sync function that owns
|
|
81
|
+
# the correct replay semantics. They live in this kernel because three layers
|
|
82
|
+
# read them — the cache glue that consumes them, the read-side authority probe,
|
|
83
|
+
# and the doctor I/O shell — and a SQL string literal in any one of them would
|
|
84
|
+
# silently outlive a rename of the others.
|
|
85
|
+
CODEX_REPLAY_FROM_ZERO_KEY = "codex_replay_from_zero_pending"
|
|
86
|
+
CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY = (
|
|
87
|
+
"codex_conversation_replay_from_zero_pending"
|
|
88
|
+
)
|
|
89
|
+
# Written when a whole-tree Codex sync completed and still could NOT consume
|
|
90
|
+
# `CODEX_REPLAY_FROM_ZERO_KEY` — the replay is stalled, not merely pending, and
|
|
91
|
+
# Codex transcript ingest is deferred behind it. Read by `doctor`.
|
|
92
|
+
CODEX_REPLAY_BLOCKED_KEY = "codex_replay_from_zero_blocked"
|
|
93
|
+
|
|
77
94
|
# Structural wrapper prefixes skipped during title selection (§4.3), pinned from
|
|
78
95
|
# the corpus (title-wrapper-window). Prefix-structural, never content heuristics.
|
|
79
96
|
CODEX_TITLE_SKIP_PREFIXES: tuple[str, ...] = (
|
|
@@ -35,6 +35,13 @@ from _lib_pricing import _calculate_codex_entry_cost
|
|
|
35
35
|
# authoritative; a held-lock deferral leaves it pending.
|
|
36
36
|
CODEX_NORMALIZATION_MIGRATION = "025_codex_conversation_normalization"
|
|
37
37
|
|
|
38
|
+
# The provider-local rebuild marker migration 028's byte-zero replay arms. Its
|
|
39
|
+
# sibling — the thread_source-inference replay marker — is
|
|
40
|
+
# ``kern.CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY``; both are named, never
|
|
41
|
+
# inlined as SQL literals, so a rename cannot leave this probe testing a key
|
|
42
|
+
# nothing writes any more.
|
|
43
|
+
CODEX_CONTRACT_REBUILD_MARKER = "conversation_rebuild_codex_pending"
|
|
44
|
+
|
|
38
45
|
# Domain separations for the opaque item-key encoding (§5.2). The source-path
|
|
39
46
|
# fingerprint is a domain-separated hash, NEVER a raw path (privacy-safe).
|
|
40
47
|
CODEX_ITEM_KEY_DOMAIN = b"cctally-codex-item-key-v1\0"
|
|
@@ -93,6 +100,12 @@ def codex_normalization_authoritative(conn: sqlite3.Connection) -> bool:
|
|
|
93
100
|
Split stores use their provider-local rebuild marker: current schema alone
|
|
94
101
|
is not authority while migration 028's byte-zero replay is pending. Legacy
|
|
95
102
|
monolithic/bare connections retain the migration-025 stamp contract.
|
|
103
|
+
|
|
104
|
+
EITHER pending marker withholds authority. The thread_source-inference
|
|
105
|
+
replay (conversations migration 002) is armed by its own key precisely
|
|
106
|
+
because the contract replay must not consume it, so a probe that tested only
|
|
107
|
+
the contract marker would report a not-yet-repaired store as authoritative
|
|
108
|
+
to every ``--no-sync`` read.
|
|
96
109
|
"""
|
|
97
110
|
try:
|
|
98
111
|
split = conn.execute(
|
|
@@ -101,8 +114,9 @@ def codex_normalization_authoritative(conn: sqlite3.Connection) -> bool:
|
|
|
101
114
|
).fetchone() is not None
|
|
102
115
|
if split:
|
|
103
116
|
pending = conn.execute(
|
|
104
|
-
"SELECT 1 FROM cache_meta "
|
|
105
|
-
|
|
117
|
+
"SELECT 1 FROM cache_meta WHERE key IN (?,?) LIMIT 1",
|
|
118
|
+
(CODEX_CONTRACT_REBUILD_MARKER,
|
|
119
|
+
kern.CODEX_CONVERSATION_REPLAY_FROM_ZERO_KEY),
|
|
106
120
|
).fetchone() is not None
|
|
107
121
|
version = conn.execute(
|
|
108
122
|
"SELECT value FROM cache_meta "
|
|
@@ -22,7 +22,14 @@ CapabilityStatus = Literal[
|
|
|
22
22
|
"supported", "derived", "unavailable", "deferred", "not_applicable",
|
|
23
23
|
]
|
|
24
24
|
|
|
25
|
-
|
|
25
|
+
# #429 §3.2 — bumped to 2 because `active[].captured_at` changed MEANING (the
|
|
26
|
+
# newest physical observation, not the interpreted baseline). The in-place
|
|
27
|
+
# update flow `execvp`s the server while the already-loaded client reconnects
|
|
28
|
+
# over its existing EventSource without reloading its JS
|
|
29
|
+
# (`UpdateRunningModal.tsx`, `store/sse.ts`), so an old client demonstrably
|
|
30
|
+
# does meet a new server; `docs/cli-contract.md` calls changing a value's
|
|
31
|
+
# meaning breaking. Version-aware client reaction is deliberately deferred.
|
|
32
|
+
SOURCE_SCHEMA_VERSION = 2
|
|
26
33
|
DEFAULT_SOURCE = "claude"
|
|
27
34
|
SOURCE_ORDER = ("claude", "codex", "all")
|
|
28
35
|
SOURCE_FRESHNESS_DOMAINS = ("hero", "quota", "sessions")
|
package/bin/_lib_doctor.py
CHANGED
|
@@ -189,6 +189,14 @@ class DoctorState:
|
|
|
189
189
|
# by the next whole-tree sync that defers none. None = key absent (the
|
|
190
190
|
# normal state) or cache unreadable — the check degrades OK.
|
|
191
191
|
codex_torn_deferred: Optional[dict] = None
|
|
192
|
+
# The byte-zero Codex replay's stall signal. `codex_replay_pending` mirrors
|
|
193
|
+
# the `codex_replay_from_zero_pending` cache_meta marker; `codex_replay_blocked`
|
|
194
|
+
# is `{"at": iso, "files_failed": N, "files_deferred_torn": N}`, written by a
|
|
195
|
+
# whole-tree Codex sync that RAN and still could not consume it, and deleted
|
|
196
|
+
# by the next clean whole-tree sync. Both None = key absent (the normal
|
|
197
|
+
# state) or cache unreadable — the check degrades OK.
|
|
198
|
+
codex_replay_pending: Optional[bool] = None
|
|
199
|
+
codex_replay_blocked: Optional[dict] = None
|
|
192
200
|
# #279 S2 (F5b): PRAGMA quick_check(1) results, gathered ONLY under
|
|
193
201
|
# doctor_gather_state(deep=True) (CLI cmd_doctor) — the dashboard
|
|
194
202
|
# rebuild loop calls the gather every rebuild and quick_check on a
|
|
@@ -1107,6 +1115,50 @@ def _check_data_codex_cache(s: DoctorState) -> CheckResult:
|
|
|
1107
1115
|
)
|
|
1108
1116
|
|
|
1109
1117
|
|
|
1118
|
+
def _check_data_codex_replay(s: DoctorState) -> CheckResult:
|
|
1119
|
+
"""WARN while a byte-zero Codex replay is STALLED rather than merely pending.
|
|
1120
|
+
|
|
1121
|
+
`sync_codex_conversations` defers on the cache-side replay marker, so as long
|
|
1122
|
+
as that marker stands no Codex transcript is ingested at all. The deferral is
|
|
1123
|
+
protective — running ahead of the replayed thread rows stamps a materialized
|
|
1124
|
+
`"(unassigned)"` project the read path then prefers permanently — so the fix
|
|
1125
|
+
is never to drop it. But a whole-tree sync that runs and still cannot consume
|
|
1126
|
+
the marker (a persistently torn `auth.json`, a repeated per-file DB error)
|
|
1127
|
+
holds that deferral open indefinitely, and `cache-sync` still exits 0. This
|
|
1128
|
+
leg is the only standing signal that Codex transcript ingest has stopped.
|
|
1129
|
+
|
|
1130
|
+
A marker that is merely pending is NOT reported: it is the ordinary state
|
|
1131
|
+
between the migration and the next sync, and it clears on its own. The WARN
|
|
1132
|
+
needs the durable `blocked` record, which only a completed-but-unsuccessful
|
|
1133
|
+
whole-tree walk writes.
|
|
1134
|
+
"""
|
|
1135
|
+
blocked = s.codex_replay_blocked or {}
|
|
1136
|
+
at = blocked.get("at")
|
|
1137
|
+
details = {
|
|
1138
|
+
"pending": bool(s.codex_replay_pending),
|
|
1139
|
+
"blocked_at": at,
|
|
1140
|
+
"files_failed": blocked.get("files_failed"),
|
|
1141
|
+
"files_deferred_torn": blocked.get("files_deferred_torn"),
|
|
1142
|
+
}
|
|
1143
|
+
if s.codex_replay_pending and isinstance(at, str) and at:
|
|
1144
|
+
return CheckResult(
|
|
1145
|
+
id="data.codex_replay", title="Codex transcript replay",
|
|
1146
|
+
severity="warn",
|
|
1147
|
+
summary=(f"stalled since {at} — Codex transcript ingest is "
|
|
1148
|
+
"deferred until it completes"),
|
|
1149
|
+
remediation=("Check the Codex auth.json (re-run `codex login` if it "
|
|
1150
|
+
"is truncated), then `cctally cache-sync --source codex`"),
|
|
1151
|
+
details=details,
|
|
1152
|
+
)
|
|
1153
|
+
return CheckResult(
|
|
1154
|
+
id="data.codex_replay", title="Codex transcript replay",
|
|
1155
|
+
severity="ok",
|
|
1156
|
+
summary="pending (clears on the next Codex sync)"
|
|
1157
|
+
if s.codex_replay_pending else "none pending",
|
|
1158
|
+
remediation=None, details=details,
|
|
1159
|
+
)
|
|
1160
|
+
|
|
1161
|
+
|
|
1110
1162
|
def _check_data_codex_project_metadata(s: DoctorState) -> CheckResult:
|
|
1111
1163
|
"""Report the identity-safe all-history Codex metadata partition."""
|
|
1112
1164
|
if s.codex_project_metadata_error is not None:
|
|
@@ -2680,7 +2732,12 @@ _CATEGORY_DEFINITIONS: tuple[tuple[str, str, tuple[tuple[str, str], ...]], ...]
|
|
|
2680
2732
|
("data.statusline_pipeline", "_check_statusline_pipeline"),
|
|
2681
2733
|
("data.cache_sync_state", "_check_data_cache_sync_state"),
|
|
2682
2734
|
("data.codex_cache", "_check_data_codex_cache"),
|
|
2735
|
+
# `data.codex_project_metadata` must stay IMMEDIATELY after
|
|
2736
|
+
# `data.codex_cache` (pinned by
|
|
2737
|
+
# tests/test_doctor_codex_project_metadata.py), so the replay leg goes
|
|
2738
|
+
# after the pair rather than between them.
|
|
2683
2739
|
("data.codex_project_metadata", "_check_data_codex_project_metadata"),
|
|
2740
|
+
("data.codex_replay", "_check_data_codex_replay"),
|
|
2684
2741
|
("data.codex_quota", "_check_data_codex_quota"),
|
|
2685
2742
|
("data.parse_health", "_check_data_parse_health"),
|
|
2686
2743
|
("data.forked_buckets", "_check_data_forked_buckets"),
|
package/bin/_lib_jsonl.py
CHANGED
|
@@ -549,15 +549,48 @@ def _canonical_container(value: object) -> str | None:
|
|
|
549
549
|
return _codex_canonical_json(value)
|
|
550
550
|
|
|
551
551
|
|
|
552
|
+
# Codex is mid-rollout on `thread_source`: Desktop 0.146.0-alpha.3.1 emits it,
|
|
553
|
+
# the CLI 0.146.0 behind the MCP server does not. Abandoning the identity when
|
|
554
|
+
# it is absent costs the rollout its thread row AND every normalized message,
|
|
555
|
+
# so infer the category the provider would most likely emit instead. Inferring
|
|
556
|
+
# rather than minting a null parent keeps one identity across the release that
|
|
557
|
+
# starts emitting the field.
|
|
558
|
+
_CODEX_DEFAULT_THREAD_SOURCE = "user"
|
|
559
|
+
|
|
560
|
+
|
|
561
|
+
def _inferred_codex_thread_source(payload: dict[str, Any]) -> str:
|
|
562
|
+
"""The thread-origin category for one ``session_meta`` payload.
|
|
563
|
+
|
|
564
|
+
Per-record and stateless: a later ``session_meta`` never inherits an earlier
|
|
565
|
+
one's category, so a file's key sequence is a function of its own bytes.
|
|
566
|
+
The string form of ``source`` is deliberately never used — ``source:
|
|
567
|
+
"vscode"`` co-occurs with ``thread_source: "user"``, so the client name and
|
|
568
|
+
the origin category are orthogonal vocabularies.
|
|
569
|
+
"""
|
|
570
|
+
explicit = _codex_string(payload.get("thread_source"))
|
|
571
|
+
if explicit is not None:
|
|
572
|
+
return explicit
|
|
573
|
+
source = payload.get("source")
|
|
574
|
+
if isinstance(source, dict) and len(source) == 1:
|
|
575
|
+
# A single-key object is the shape Codex uses for `{"subagent": {...}}`.
|
|
576
|
+
# Guard every malformed variant: the identity encoder REJECTS an empty
|
|
577
|
+
# parent key, so an unguarded rule turns bad metadata into an ingest
|
|
578
|
+
# exception rather than a degraded-but-working ingest.
|
|
579
|
+
only_key = _codex_string(next(iter(source)))
|
|
580
|
+
if only_key is not None:
|
|
581
|
+
return only_key
|
|
582
|
+
return _CODEX_DEFAULT_THREAD_SOURCE
|
|
583
|
+
|
|
584
|
+
|
|
552
585
|
def _thread_metadata_from_session_meta(
|
|
553
586
|
payload: dict[str, Any], path_str: str, source_root_key: str | None,
|
|
554
587
|
) -> CodexThreadMetadata:
|
|
555
588
|
accounting_id = _codex_string(payload.get("id"))
|
|
556
589
|
native_thread_id = _codex_string(payload.get("session_id")) or accounting_id
|
|
557
|
-
root_thread_id =
|
|
590
|
+
root_thread_id = _inferred_codex_thread_source(payload)
|
|
558
591
|
parent_thread_id = _codex_string(payload.get("forked_from_id"))
|
|
559
592
|
conversation_key = None
|
|
560
|
-
if native_thread_id is not None
|
|
593
|
+
if native_thread_id is not None:
|
|
561
594
|
conversation_key = canonical_identity_from_root_key(
|
|
562
595
|
"codex", "conversation", source_root_key, native_thread_id, root_thread_id
|
|
563
596
|
)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "cctally",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.88.0",
|
|
4
4
|
"description": "Claude Code usage tracker and local dashboard for Pro/Max subscription limits - weekly cost-per-percent trend, quota forecasts, threshold alerts. ccusage-compatible.",
|
|
5
5
|
"homepage": "https://github.com/omrikais/cctally",
|
|
6
6
|
"repository": {
|