cctally 1.85.0 → 1.86.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +16 -0
- package/bin/_cctally_cache.py +136 -35
- package/bin/_cctally_dashboard_sources.py +66 -33
- package/bin/_cctally_db.py +67 -1
- package/bin/_cctally_doctor.py +16 -1
- package/bin/_cctally_journal.py +5 -0
- package/bin/_cctally_quota.py +31 -8
- package/bin/_cctally_record.py +32 -6
- package/bin/_lib_aggregators.py +3 -0
- package/bin/_lib_doctor.py +28 -0
- package/bin/_lib_quota.py +170 -6
- package/dashboard/static/assets/index-B0ZCsoxI.css +1 -0
- package/dashboard/static/assets/index-Bvp8mxtz.js +92 -0
- package/dashboard/static/dashboard.html +2 -2
- package/package.json +1 -1
- package/dashboard/static/assets/index-DlVVJeS4.js +0 -92
- package/dashboard/static/assets/index-OYBkyglj.css +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,22 @@ based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|
|
5
5
|
|
|
6
6
|
## [Unreleased]
|
|
7
7
|
|
|
8
|
+
## [1.86.0] - 2026-07-30
|
|
9
|
+
|
|
10
|
+
### Fixed
|
|
11
|
+
- Each Claude, Codex, or future-provider logical model release now receives a distinct, accessible dashboard color that stays consistent across cards and modals, while dated and capacity-qualified variants of the same release share one color.
|
|
12
|
+
- Codex quota windows whose reported reset drifts through a chain of nearby values now converge into one physical window. The original upgrade grouped each sample only against already-settled anchors, so real history could leave one connected window split across multiple percentages, forecasts, milestone ladders and alert identities even though every adjacent reset remained within the intended ten-minute tolerance. A follow-up cache migration repairs already-upgraded history without changing the raw provider evidence, live ingest applies the same transitive rule, and terminal alert evidence follows exact retained members even when an endpoint is more than ten minutes from the winning first observation. Release cuts now also refuse every newly registered migration until a dated real-production-copy verification is tracked; `--ack-migrations` cannot bypass that gate. (#425)
|
|
13
|
+
- Pooled Codex weekly rows now name every account whose near-identical reset boundary contributed to the row. The Weekly and Trend panels and their expanded tables no longer show one account's maximum percentage without disclosing the other account blended into it; focused and single-account views remain unchanged. Three dashboard tests also now reject malformed trailing text instead of checking only a correct-looking prefix. (#424)
|
|
14
|
+
- The dashboard now keeps account focus consistent across every account-sensitive surface. Claude's merged and All-provider headlines no longer borrow one account's quota, while a focused Claude account uses only the percentage, reset and spend fields its own card actually provides. Focused keyboard shortcuts open scoped sessions and blocks, merged Codex block rows always expose their account in visible and accessible text, and shared-root quota forecasts no longer join through the account-blind parent. Account selection announcements now name the selected state correctly, long account labels remain identifiable on 320px screens, and small non-zero dollar values retain their cents instead of rounding to `$0`. (#423)
|
|
15
|
+
- A weekly percent milestone is no longer recorded carrying the cost of an earlier crossing. Recording a crossing first refreshes the week's cost, but when a concurrent cache ingest made that account-scoped refresh fail closed, cctally kept going and stamped the last stored cost onto the new percentage — so the Current Week per-percent ladder showed two consecutive percentages at an identical cumulative dollar figure, a `$0.00` marginal and a `$/1%` that never happened. Because milestone rows are written once, that was permanent. The crossing is now skipped instead and recorded by the next observation with its real cost, so nothing is lost; the same fail-closed condition on a reset-affected week also no longer aborts the whole ingest cycle over a transient lock.
|
|
16
|
+
- Codex quota percentages now appear only for windows the server still marks live. The Current Cycle modal no longer turns a retained, already-reset history row into a current percentage or `$ / 1%`; the All-accounts cycle table derives each live percentage and reset from that account's own quota window, including a live unattributed bucket whose totals-only card intentionally blanks those fields. Decorated parent envelopes also stop exposing one representative account's cycle-history index, and Forecast panels now preserve the same one-decimal percentage shown by their expanded views. (#422)
|
|
17
|
+
- Codex cache sync now stops before reading rollout files when its durable account-attribution decisions cannot be replayed, preventing a transient journal/cache failure from reassigning usage to the account currently signed in. Doctor also reports retained quota rows that lack their canonical reset anchor and points to the safe Codex cache rebuild. (#421)
|
|
18
|
+
|
|
19
|
+
## [1.85.1] - 2026-07-29
|
|
20
|
+
|
|
21
|
+
### Fixed
|
|
22
|
+
- Internal (maintainer-only): the daily project-filter fixture now selects its target by a unique path segment while deliberately placing the same basename in the shared scratch ancestor, preventing a random temporary-directory collision from failing one Linux release matrix lane.
|
|
23
|
+
|
|
8
24
|
## [1.85.0] - 2026-07-29
|
|
9
25
|
|
|
10
26
|
### Fixed
|
package/bin/_cctally_cache.py
CHANGED
|
@@ -1054,7 +1054,11 @@ class CodexResetAnchorResolver:
|
|
|
1054
1054
|
|
|
1055
1055
|
def __init__(self, conn: sqlite3.Connection):
|
|
1056
1056
|
self._conn = conn
|
|
1057
|
-
self._groups: "dict[tuple[str, str, str, object], _lib_quota.
|
|
1057
|
+
self._groups: "dict[tuple[str, str, str, object], _lib_quota.ResetAnchorComponents]" = {}
|
|
1058
|
+
self._pending_merges: list[
|
|
1059
|
+
tuple[tuple[str, str, str, object], str, dt.datetime,
|
|
1060
|
+
tuple[dt.datetime, ...]]
|
|
1061
|
+
] = []
|
|
1058
1062
|
self._seed_failed = False
|
|
1059
1063
|
|
|
1060
1064
|
@staticmethod
|
|
@@ -1068,34 +1072,31 @@ class CodexResetAnchorResolver:
|
|
|
1068
1072
|
_lib_jsonl.snap_codex_window_minutes(window_minutes),
|
|
1069
1073
|
)
|
|
1070
1074
|
|
|
1071
|
-
def
|
|
1075
|
+
def _components_for(
|
|
1072
1076
|
self, group, logical_limit_key: str,
|
|
1073
|
-
) -> "_lib_quota.
|
|
1074
|
-
"""The group's
|
|
1075
|
-
|
|
1076
|
-
|
|
1077
|
-
|
|
1078
|
-
|
|
1079
|
-
walk by `cache-sync --rebuild`. A linear scan per observation makes both
|
|
1080
|
-
quadratic in the group's anchor count (~1,750 a year for a 5h window).
|
|
1077
|
+
) -> "_lib_quota.ResetAnchorComponents":
|
|
1078
|
+
"""The group's raw-reset components, ordered by deterministic ingest.
|
|
1079
|
+
|
|
1080
|
+
Raw values, not only stored anchors, are required to recover transitive
|
|
1081
|
+
chain membership. Ordering by rollout byte identity preserves the
|
|
1082
|
+
original first-sight winner while making a rebuild reproducible.
|
|
1081
1083
|
"""
|
|
1082
|
-
|
|
1083
|
-
if
|
|
1084
|
-
return
|
|
1085
|
-
|
|
1084
|
+
components = self._groups.get(group)
|
|
1085
|
+
if components is not None:
|
|
1086
|
+
return components
|
|
1087
|
+
components = _lib_quota.ResetAnchorComponents()
|
|
1086
1088
|
if not self._seed_failed:
|
|
1087
1089
|
candidates = _lib_jsonl.codex_snap_equivalent_limit_keys(
|
|
1088
1090
|
str(logical_limit_key))
|
|
1089
1091
|
placeholders = ",".join("?" for _ in candidates)
|
|
1090
1092
|
try:
|
|
1091
1093
|
rows = self._conn.execute(
|
|
1092
|
-
"SELECT
|
|
1094
|
+
"SELECT source_path, line_offset, id, resets_at_utc "
|
|
1093
1095
|
"FROM quota_window_snapshots "
|
|
1094
1096
|
"WHERE source = 'codex' AND source_root_key = ? "
|
|
1095
1097
|
" AND observed_slot = ? "
|
|
1096
1098
|
f" AND logical_limit_key IN ({placeholders}) "
|
|
1097
|
-
"
|
|
1098
|
-
"ORDER BY canonical_resets_at_utc",
|
|
1099
|
+
"ORDER BY source_path, line_offset, id",
|
|
1099
1100
|
(group[0], group[1], *candidates),
|
|
1100
1101
|
).fetchall()
|
|
1101
1102
|
except sqlite3.DatabaseError:
|
|
@@ -1105,15 +1106,66 @@ class CodexResetAnchorResolver:
|
|
|
1105
1106
|
self._seed_failed = True
|
|
1106
1107
|
rows = []
|
|
1107
1108
|
for row in rows:
|
|
1108
|
-
parsed = _parse_anchor_iso(row[
|
|
1109
|
+
parsed = _parse_anchor_iso(row[3])
|
|
1109
1110
|
if parsed is not None:
|
|
1110
|
-
|
|
1111
|
-
|
|
1112
|
-
|
|
1111
|
+
components.add(
|
|
1112
|
+
parsed,
|
|
1113
|
+
order_key=(
|
|
1114
|
+
str(row[0]), int(row[1]), int(row[2])),
|
|
1115
|
+
)
|
|
1116
|
+
self._groups[group] = components
|
|
1117
|
+
return components
|
|
1118
|
+
|
|
1119
|
+
def _merge_stored_anchors(
|
|
1120
|
+
self, group, logical_limit_key: str, winner: dt.datetime,
|
|
1121
|
+
retired: tuple[dt.datetime, ...],
|
|
1122
|
+
) -> None:
|
|
1123
|
+
if not retired:
|
|
1124
|
+
return
|
|
1125
|
+
self._pending_merges.append(
|
|
1126
|
+
(group, str(logical_limit_key), winner, retired))
|
|
1127
|
+
|
|
1128
|
+
def apply_pending_merges(self) -> None:
|
|
1129
|
+
"""Apply queued component merges inside the caller's transaction.
|
|
1130
|
+
|
|
1131
|
+
Resolution happens while the direct walk is still buffering a file.
|
|
1132
|
+
Deferring DML until `_write_codex_file_batch` keeps retired-anchor
|
|
1133
|
+
updates atomic with that file's rows and cursor; its one rollback/retry
|
|
1134
|
+
can safely reapply the unchanged queue.
|
|
1135
|
+
"""
|
|
1136
|
+
for group, logical_limit_key, winner, retired in self._pending_merges:
|
|
1137
|
+
candidates = _lib_jsonl.codex_snap_equivalent_limit_keys(
|
|
1138
|
+
logical_limit_key)
|
|
1139
|
+
key_placeholders = ",".join("?" for _ in candidates)
|
|
1140
|
+
retired_text = tuple(
|
|
1141
|
+
_codex_anchor_iso(value) for value in retired)
|
|
1142
|
+
retired_placeholders = ",".join("?" for _ in retired_text)
|
|
1143
|
+
self._conn.execute(
|
|
1144
|
+
"UPDATE quota_window_snapshots "
|
|
1145
|
+
"SET canonical_resets_at_utc = ? "
|
|
1146
|
+
"WHERE source = 'codex' AND source_root_key = ? "
|
|
1147
|
+
" AND observed_slot = ? "
|
|
1148
|
+
f" AND logical_limit_key IN ({key_placeholders}) "
|
|
1149
|
+
f" AND canonical_resets_at_utc IN ({retired_placeholders})",
|
|
1150
|
+
(
|
|
1151
|
+
_codex_anchor_iso(winner), group[0], group[1],
|
|
1152
|
+
*candidates, *retired_text,
|
|
1153
|
+
),
|
|
1154
|
+
)
|
|
1155
|
+
|
|
1156
|
+
def mark_file_committed(self) -> None:
|
|
1157
|
+
self._pending_merges.clear()
|
|
1158
|
+
|
|
1159
|
+
def discard_uncommitted_file(self) -> None:
|
|
1160
|
+
"""Forget buffered evidence after its file did not commit."""
|
|
1161
|
+
self._groups.clear()
|
|
1162
|
+
self._pending_merges.clear()
|
|
1113
1163
|
|
|
1114
1164
|
def resolve(
|
|
1115
1165
|
self, *, source_root_key: str, observed_slot: str,
|
|
1116
1166
|
logical_limit_key: str, window_minutes: object, resets_at_utc: object,
|
|
1167
|
+
source_path: "str | None" = None,
|
|
1168
|
+
line_offset: "int | None" = None,
|
|
1117
1169
|
) -> "str | None":
|
|
1118
1170
|
"""The canonical anchor for one observation, as stored TEXT.
|
|
1119
1171
|
|
|
@@ -1126,10 +1178,17 @@ class CodexResetAnchorResolver:
|
|
|
1126
1178
|
return None
|
|
1127
1179
|
group = self.group_key(
|
|
1128
1180
|
source_root_key, observed_slot, logical_limit_key, window_minutes)
|
|
1129
|
-
|
|
1130
|
-
|
|
1131
|
-
if
|
|
1132
|
-
|
|
1181
|
+
components = self._components_for(group, logical_limit_key)
|
|
1182
|
+
order_key = None
|
|
1183
|
+
if (
|
|
1184
|
+
isinstance(source_path, str)
|
|
1185
|
+
and isinstance(line_offset, int)
|
|
1186
|
+
and not isinstance(line_offset, bool)
|
|
1187
|
+
):
|
|
1188
|
+
order_key = (source_path, line_offset, -1)
|
|
1189
|
+
chosen, retired = components.add(raw, order_key=order_key)
|
|
1190
|
+
self._merge_stored_anchors(
|
|
1191
|
+
group, logical_limit_key, chosen, retired)
|
|
1133
1192
|
# ALWAYS re-serialized through the canonical UTC form, established or
|
|
1134
1193
|
# joined alike: two spellings of one instant ("…Z" vs "…+00:00") must
|
|
1135
1194
|
# never mint two anchors for one cluster, and a row seeded by anything
|
|
@@ -1137,6 +1196,25 @@ class CodexResetAnchorResolver:
|
|
|
1137
1196
|
# spell it either way.
|
|
1138
1197
|
return _codex_anchor_iso(chosen)
|
|
1139
1198
|
|
|
1199
|
+
def normalize_quota_rows(self, quota_rows: list[tuple[Any, ...]]) -> None:
|
|
1200
|
+
"""Converge buffered direct-walk rows after a later bridge observation.
|
|
1201
|
+
|
|
1202
|
+
The journal applier inserts one row immediately after resolution, so a
|
|
1203
|
+
component merge can update earlier rows in SQLite. The direct rollout
|
|
1204
|
+
walk buffers a whole file before its first DML; this final pass applies
|
|
1205
|
+
the same union result to those not-yet-inserted tuples.
|
|
1206
|
+
"""
|
|
1207
|
+
for index, row in enumerate(quota_rows):
|
|
1208
|
+
raw = _parse_anchor_iso(row[11])
|
|
1209
|
+
if raw is None:
|
|
1210
|
+
continue
|
|
1211
|
+
group = self.group_key(row[1], row[5], row[6], row[9])
|
|
1212
|
+
components = self._groups.get(group)
|
|
1213
|
+
if components is None:
|
|
1214
|
+
continue
|
|
1215
|
+
quota_rows[index] = row[:-1] + (
|
|
1216
|
+
_codex_anchor_iso(components.canonical(raw)),)
|
|
1217
|
+
|
|
1140
1218
|
|
|
1141
1219
|
def _codex_anchor_iso(value: dt.datetime) -> str:
|
|
1142
1220
|
return value.astimezone(dt.timezone.utc).isoformat().replace("+00:00", "Z")
|
|
@@ -1272,7 +1350,9 @@ def _discover_codex_files_with_roots() -> list[CodexDiscoveredFile]:
|
|
|
1272
1350
|
discovered: list[CodexDiscoveredFile] = []
|
|
1273
1351
|
seen: set[pathlib.Path] = set()
|
|
1274
1352
|
for root in _codex_provider_roots():
|
|
1275
|
-
for candidate in
|
|
1353
|
+
for candidate in sorted(
|
|
1354
|
+
root.walk_root.glob("**/*.jsonl"), key=lambda path: str(path)
|
|
1355
|
+
):
|
|
1276
1356
|
if not candidate.is_file():
|
|
1277
1357
|
continue
|
|
1278
1358
|
physical_path = _canonical_codex_path(candidate)
|
|
@@ -1905,6 +1985,7 @@ def _write_codex_file_batch(
|
|
|
1905
1985
|
file_identity: "str | None" = None,
|
|
1906
1986
|
incarnation: "int | None" = None,
|
|
1907
1987
|
file_account_decision: "tuple[int, str | None] | None" = None,
|
|
1988
|
+
anchor_resolver: "CodexResetAnchorResolver | None" = None,
|
|
1908
1989
|
) -> int:
|
|
1909
1990
|
"""Write one fully-buffered Codex file atomically and return entry changes.
|
|
1910
1991
|
|
|
@@ -1960,6 +2041,8 @@ def _write_codex_file_batch(
|
|
|
1960
2041
|
)
|
|
1961
2042
|
rows_changed = conn.total_changes - before
|
|
1962
2043
|
if quota_rows:
|
|
2044
|
+
if anchor_resolver is not None:
|
|
2045
|
+
anchor_resolver.apply_pending_merges()
|
|
1963
2046
|
conn.executemany(
|
|
1964
2047
|
"""INSERT OR IGNORE INTO quota_window_snapshots
|
|
1965
2048
|
(source, source_root_key, source_path, line_offset,
|
|
@@ -2027,7 +2110,7 @@ def _iter_codex_jsonl_paths(roots: list[pathlib.Path]) -> Iterator[pathlib.Path]
|
|
|
2027
2110
|
"""
|
|
2028
2111
|
seen: set[pathlib.Path] = set()
|
|
2029
2112
|
for root in roots:
|
|
2030
|
-
for jp in root.glob("**/*.jsonl"):
|
|
2113
|
+
for jp in sorted(root.glob("**/*.jsonl"), key=lambda path: str(path)):
|
|
2031
2114
|
# Dedup on the RESOLVED path, not the raw spelling. A symlinked
|
|
2032
2115
|
# $CODEX_HOME root or an alias entry (`.../.codex`,
|
|
2033
2116
|
# `.../sub/../.codex`) can glob the same physical file under
|
|
@@ -4845,10 +4928,13 @@ def sync_codex_cache(
|
|
|
4845
4928
|
conn.commit()
|
|
4846
4929
|
eprint("[cache-sync] rebuild: cleared Codex cached entries")
|
|
4847
4930
|
# #416 spec §3.4: rehydrate the attribution map from the journal BEFORE
|
|
4848
|
-
# the walk.
|
|
4849
|
-
#
|
|
4850
|
-
#
|
|
4851
|
-
#
|
|
4931
|
+
# the walk. This cursor is intentionally distinct from the stats ingest
|
|
4932
|
+
# cursor that drives `_cache_applier`: a recreated/rebuilt cache.db may
|
|
4933
|
+
# have an empty map while stats is already at journal high-water, and
|
|
4934
|
+
# invoking stats ingest here would either reverse the total lock order
|
|
4935
|
+
# (inside the cache flocks) or leave an append-before-lock race (outside
|
|
4936
|
+
# them). The private cache-map cursor is therefore the only safe witness
|
|
4937
|
+
# that every durable decision visible to this locked walk was replayed.
|
|
4852
4938
|
#
|
|
4853
4939
|
# Deliberately NOT rebuild-only. Every production Codex call site syncs
|
|
4854
4940
|
# with rebuild=False, and the corruption auto-heal recreates the cache.db
|
|
@@ -4921,9 +5007,11 @@ def sync_codex_cache(
|
|
|
4921
5007
|
_jr._report_file_account_conflicts(_declined)
|
|
4922
5008
|
except Exception as exc:
|
|
4923
5009
|
conn.rollback()
|
|
5010
|
+
stats.deferred_reason = "attribution_rehydration"
|
|
4924
5011
|
eprint(
|
|
4925
5012
|
"[cache-sync] could not rehydrate Codex attribution "
|
|
4926
|
-
f"decisions: {exc};
|
|
5013
|
+
f"decisions: {exc}; deferring the Codex walk")
|
|
5014
|
+
return stats
|
|
4927
5015
|
|
|
4928
5016
|
# Pure read (glob + is_file only); safe to run before the SELECT and
|
|
4929
5017
|
# the per-file loop, where no cache.db write lock may be held. Targeted
|
|
@@ -5186,9 +5274,14 @@ def sync_codex_cache(
|
|
|
5186
5274
|
# The guard is the whole safety argument: `delta_append` means
|
|
5187
5275
|
# `start_offset` is this file's ingest watermark, and the second
|
|
5188
5276
|
# condition means the new range starts strictly beyond every
|
|
5189
|
-
# decided range.
|
|
5190
|
-
#
|
|
5191
|
-
# a
|
|
5277
|
+
# decided range. Today the term is algebraically redundant:
|
|
5278
|
+
# every non-delta branch sets `start_offset = 0`, while every
|
|
5279
|
+
# decided range starts at a non-negative offset, so the strict
|
|
5280
|
+
# comparison alone implies a delta append. Keep the explicit
|
|
5281
|
+
# term as belt-and-suspenders: it pins the semantic permission
|
|
5282
|
+
# to consult auth.json if a future branch changes the offsets.
|
|
5283
|
+
# Auth can therefore mint a range only for bytes NOBODY has
|
|
5284
|
+
# attributed yet; it never re-decides covered bytes.
|
|
5192
5285
|
if delta_append and start_offset > account_ranges[-1][0]:
|
|
5193
5286
|
root_account = _live_root_account()
|
|
5194
5287
|
if root_account.status == "torn":
|
|
@@ -5345,6 +5438,8 @@ def sync_codex_cache(
|
|
|
5345
5438
|
logical_limit_key=quota.logical_limit_key,
|
|
5346
5439
|
window_minutes=quota.window_minutes,
|
|
5347
5440
|
resets_at_utc=quota.resets_at_utc,
|
|
5441
|
+
source_path=quota.source_path,
|
|
5442
|
+
line_offset=quota.line_offset,
|
|
5348
5443
|
),
|
|
5349
5444
|
))
|
|
5350
5445
|
if (thread := emission.thread) is not None and (
|
|
@@ -5394,6 +5489,7 @@ def sync_codex_cache(
|
|
|
5394
5489
|
stats.skip_reasons[_r] = stats.skip_reasons.get(_r, 0) + _n
|
|
5395
5490
|
except OSError as exc:
|
|
5396
5491
|
eprint(f"[codex-cache] could not read {jp}: {exc}")
|
|
5492
|
+
anchor_resolver.discard_uncommitted_file()
|
|
5397
5493
|
if targeted:
|
|
5398
5494
|
stats.files_failed += 1 # §5.1 I/O decline → call dirty
|
|
5399
5495
|
continue
|
|
@@ -5468,6 +5564,7 @@ def sync_codex_cache(
|
|
|
5468
5564
|
f"[codex-cache] attribution decision journal append "
|
|
5469
5565
|
f"failed for {jp}: {exc}; deferring the file")
|
|
5470
5566
|
stats.files_failed += 1
|
|
5567
|
+
anchor_resolver.discard_uncommitted_file()
|
|
5471
5568
|
continue
|
|
5472
5569
|
|
|
5473
5570
|
# Task 7 Item 1: journal the Codex quota observations BEFORE the cache
|
|
@@ -5477,6 +5574,7 @@ def sync_codex_cache(
|
|
|
5477
5574
|
# (idempotent at the QUOTA_APPLIER natural key) rather than losing the
|
|
5478
5575
|
# observation. Appended once here, not inside the retry loop, so a DB
|
|
5479
5576
|
# retry never double-journals.
|
|
5577
|
+
anchor_resolver.normalize_quota_rows(quota_rows)
|
|
5480
5578
|
_append_codex_quota_obs(quota_rows)
|
|
5481
5579
|
|
|
5482
5580
|
# Every derived row above was buffered before the first DML. A
|
|
@@ -5514,6 +5612,7 @@ def sync_codex_cache(
|
|
|
5514
5612
|
file_identity=file_identity,
|
|
5515
5613
|
incarnation=incarnation,
|
|
5516
5614
|
file_account_decision=pending_decision,
|
|
5615
|
+
anchor_resolver=anchor_resolver,
|
|
5517
5616
|
)
|
|
5518
5617
|
except sqlite3.DatabaseError as exc:
|
|
5519
5618
|
conn.rollback()
|
|
@@ -5543,7 +5642,9 @@ def sync_codex_cache(
|
|
|
5543
5642
|
# walk count. Targeted mode already depended on this signal;
|
|
5544
5643
|
# explicit rebuild now uses it to reject partial success too.
|
|
5545
5644
|
stats.files_failed += 1
|
|
5645
|
+
anchor_resolver.discard_uncommitted_file()
|
|
5546
5646
|
continue
|
|
5647
|
+
anchor_resolver.mark_file_committed()
|
|
5547
5648
|
|
|
5548
5649
|
# Private test seam (§5.1 post-preflight late-shrink race): fires
|
|
5549
5650
|
# after each file's successful commit, so a race test can shrink a
|
|
@@ -162,6 +162,7 @@ class CodexWeeklyPeriod:
|
|
|
162
162
|
end_at: dt.datetime
|
|
163
163
|
source_root_keys: tuple[str, ...]
|
|
164
164
|
used_percent: float | None = None
|
|
165
|
+
account_keys: tuple[str, ...] = ()
|
|
165
166
|
|
|
166
167
|
|
|
167
168
|
def _codex_history_row_is_model_scoped(row: object) -> bool:
|
|
@@ -506,8 +507,9 @@ def _codex_weekly_periods(
|
|
|
506
507
|
``max(...)`` hands the focused account the OTHER account's percentage —
|
|
507
508
|
the never-combine violation D6 forbids — while ``end_at = min(resets_at,
|
|
508
509
|
next_start)`` clips one account's week at the other's start. ``None`` keeps
|
|
509
|
-
the merged "All accounts" read
|
|
510
|
-
|
|
510
|
+
the merged "All accounts" read used by the parent. Every merged boundary
|
|
511
|
+
retains the account-key set that contributed its percentage; the parent
|
|
512
|
+
publishes that set only under the existing multi-account decoration gate.
|
|
511
513
|
|
|
512
514
|
The predicate is strict equality, deliberately NOT the one-directional
|
|
513
515
|
``(account, unattributed)`` widening ``_codex_five_hour_rows`` uses: that
|
|
@@ -528,8 +530,8 @@ def _codex_weekly_periods(
|
|
|
528
530
|
account_params: tuple = () if account_key is None else (account_key,)
|
|
529
531
|
try:
|
|
530
532
|
rows = stats_conn.execute(
|
|
531
|
-
"SELECT source_root_key, logical_limit_key, limit_name,
|
|
532
|
-
"nominal_start_at_utc, current_percent "
|
|
533
|
+
"SELECT source_root_key, account_key, logical_limit_key, limit_name, "
|
|
534
|
+
"resets_at_utc, nominal_start_at_utc, current_percent "
|
|
533
535
|
"FROM quota_window_blocks "
|
|
534
536
|
"WHERE source='codex' AND window_minutes=10080 "
|
|
535
537
|
f"AND source_root_key IN ({placeholders}) AND orphaned_at IS NULL "
|
|
@@ -547,10 +549,10 @@ def _codex_weekly_periods(
|
|
|
547
549
|
# live boundary together with its own durable row, so the flag is OR-ed on
|
|
548
550
|
# merge rather than taken from either side.
|
|
549
551
|
raw_boundaries: list[
|
|
550
|
-
tuple[dt.datetime, dt.datetime, set[str], list[float], bool]
|
|
552
|
+
tuple[dt.datetime, dt.datetime, set[str], list[float], set[str], bool]
|
|
551
553
|
] = []
|
|
552
554
|
|
|
553
|
-
for (root_key, logical_limit_key, limit_name, resets_at_raw,
|
|
555
|
+
for (root_key, row_account_key, logical_limit_key, limit_name, resets_at_raw,
|
|
554
556
|
start_at_raw, current_percent) in rows:
|
|
555
557
|
if is_model_scoped_codex_quota(logical_limit_key, limit_name):
|
|
556
558
|
continue
|
|
@@ -568,7 +570,14 @@ def _codex_weekly_periods(
|
|
|
568
570
|
used_values = []
|
|
569
571
|
if isinstance(current_percent, (int, float)) and not isinstance(current_percent, bool):
|
|
570
572
|
used_values.append(float(current_percent))
|
|
571
|
-
raw_boundaries.append((
|
|
573
|
+
raw_boundaries.append((
|
|
574
|
+
start_at,
|
|
575
|
+
resets_at,
|
|
576
|
+
{str(root_key)},
|
|
577
|
+
used_values,
|
|
578
|
+
{str(row_account_key or _lib_accounts.UNATTRIBUTED)},
|
|
579
|
+
False,
|
|
580
|
+
))
|
|
572
581
|
|
|
573
582
|
# `active_cycle is None` is the case §7.4 calls out explicitly: no boundary
|
|
574
583
|
# is live, so nothing is exempt and every period clips exactly as before.
|
|
@@ -578,13 +587,18 @@ def _codex_weekly_periods(
|
|
|
578
587
|
active_cycle.resets_at.astimezone(UTC),
|
|
579
588
|
set(active_cycle.source_root_keys),
|
|
580
589
|
[active_cycle.used_percent] if active_cycle.used_percent is not None else [],
|
|
590
|
+
{
|
|
591
|
+
active_cycle.quota_identity.account_key
|
|
592
|
+
if active_cycle.quota_identity is not None
|
|
593
|
+
else _lib_accounts.UNATTRIBUTED
|
|
594
|
+
},
|
|
581
595
|
True,
|
|
582
596
|
))
|
|
583
597
|
|
|
584
598
|
ordered: list[
|
|
585
|
-
tuple[dt.datetime, dt.datetime, set[str], list[float], bool]
|
|
599
|
+
tuple[dt.datetime, dt.datetime, set[str], list[float], set[str], bool]
|
|
586
600
|
] = []
|
|
587
|
-
for start_at, resets_at, period_roots, used_values, is_live in sorted(
|
|
601
|
+
for start_at, resets_at, period_roots, used_values, period_accounts, is_live in sorted(
|
|
588
602
|
raw_boundaries, key=lambda item: (item[0], item[1]),
|
|
589
603
|
):
|
|
590
604
|
if (
|
|
@@ -592,19 +606,26 @@ def _codex_weekly_periods(
|
|
|
592
606
|
and (start_at - ordered[-1][0]).total_seconds()
|
|
593
607
|
< _FIVE_HOUR_JITTER_FLOOR_SECONDS
|
|
594
608
|
):
|
|
595
|
-
|
|
609
|
+
(
|
|
610
|
+
first_start, latest_reset, existing_roots, existing_used,
|
|
611
|
+
existing_accounts, existing_live,
|
|
612
|
+
) = ordered[-1]
|
|
596
613
|
existing_roots.update(period_roots)
|
|
597
614
|
existing_used.extend(used_values)
|
|
615
|
+
existing_accounts.update(period_accounts)
|
|
598
616
|
ordered[-1] = (
|
|
599
617
|
first_start, max(latest_reset, resets_at), existing_roots, existing_used,
|
|
600
|
-
existing_live or is_live,
|
|
618
|
+
existing_accounts, existing_live or is_live,
|
|
601
619
|
)
|
|
602
620
|
else:
|
|
603
621
|
ordered.append((
|
|
604
|
-
start_at, resets_at, set(period_roots), list(used_values),
|
|
622
|
+
start_at, resets_at, set(period_roots), list(used_values),
|
|
623
|
+
set(period_accounts), is_live,
|
|
605
624
|
))
|
|
606
625
|
periods: list[CodexWeeklyPeriod] = []
|
|
607
|
-
for index, (
|
|
626
|
+
for index, (
|
|
627
|
+
start_at, resets_at, period_roots, used_values, period_accounts, is_live,
|
|
628
|
+
) in enumerate(ordered):
|
|
608
629
|
next_start = ordered[index + 1][0] if index + 1 < len(ordered) else None
|
|
609
630
|
# The live cycle always ends at its own reset (#373 §7.4).
|
|
610
631
|
if is_live:
|
|
@@ -617,6 +638,7 @@ def _codex_weekly_periods(
|
|
|
617
638
|
end_at=end_at,
|
|
618
639
|
source_root_keys=tuple(sorted(period_roots)),
|
|
619
640
|
used_percent=max(used_values) if used_values else None,
|
|
641
|
+
account_keys=tuple(sorted(period_accounts)),
|
|
620
642
|
))
|
|
621
643
|
return tuple(periods)
|
|
622
644
|
|
|
@@ -929,6 +951,9 @@ def _bucket_wire(bucket: Any) -> dict[str, object]:
|
|
|
929
951
|
value = getattr(bucket, name, None)
|
|
930
952
|
if value is not None:
|
|
931
953
|
result[name] = value
|
|
954
|
+
account_keys = getattr(bucket, "account_keys", ())
|
|
955
|
+
if account_keys:
|
|
956
|
+
result["account_keys"] = tuple(account_keys)
|
|
932
957
|
return result
|
|
933
958
|
|
|
934
959
|
|
|
@@ -2474,11 +2499,13 @@ def _build_codex_native_weekly_view(
|
|
|
2474
2499
|
display_tz_name: str | None,
|
|
2475
2500
|
speed: str,
|
|
2476
2501
|
account_key: str | None = None,
|
|
2502
|
+
include_account_keys: bool = False,
|
|
2477
2503
|
) -> CodexWeeklyView:
|
|
2478
2504
|
"""Aggregate Codex cost into observed native quota-cycle segments.
|
|
2479
2505
|
|
|
2480
2506
|
``account_key`` scopes the durable boundary read to one account (#416
|
|
2481
|
-
Slice 3A review B1); ``None`` is the merged parent read
|
|
2507
|
+
Slice 3A review B1); ``None`` is the merged parent read. The additive
|
|
2508
|
+
ownership axis is emitted only when ``include_account_keys`` is true.
|
|
2482
2509
|
"""
|
|
2483
2510
|
periods = _codex_weekly_periods(
|
|
2484
2511
|
stats_conn,
|
|
@@ -2532,6 +2559,10 @@ def _build_codex_native_weekly_view(
|
|
|
2532
2559
|
and periods_by_bucket[row.bucket].used_percent > 0
|
|
2533
2560
|
else None
|
|
2534
2561
|
),
|
|
2562
|
+
account_keys=(
|
|
2563
|
+
periods_by_bucket[row.bucket].account_keys
|
|
2564
|
+
if include_account_keys else ()
|
|
2565
|
+
),
|
|
2535
2566
|
)
|
|
2536
2567
|
for row in rows
|
|
2537
2568
|
)
|
|
@@ -2802,14 +2833,11 @@ def _codex_account_scopes_wire(
|
|
|
2802
2833
|
context, account_observations, accounting_entries=rows,
|
|
2803
2834
|
account_key=key,
|
|
2804
2835
|
)
|
|
2805
|
-
#
|
|
2806
|
-
#
|
|
2807
|
-
#
|
|
2808
|
-
#
|
|
2809
|
-
#
|
|
2810
|
-
# has (its own `CodexCycleBoundary` plus `stats_conn`), so it is built
|
|
2811
|
-
# genuinely rather than declared parent-only. No cycle => `()`, an
|
|
2812
|
-
# honest empty state, never another account's ledger.
|
|
2836
|
+
# Each decorated child owns the only honest cycle index for that
|
|
2837
|
+
# account. Reusing a merged parent index here would render account A's
|
|
2838
|
+
# milestone HISTORY on account B's hero. The index is derivable from
|
|
2839
|
+
# the child's own `CodexCycleBoundary` plus `stats_conn`; no cycle =>
|
|
2840
|
+
# `()`, an honest empty state, never another account's ledger.
|
|
2813
2841
|
cycle_index: tuple = ()
|
|
2814
2842
|
if cycle is not None and not hero_failure:
|
|
2815
2843
|
try:
|
|
@@ -3166,6 +3194,15 @@ def build_codex_source_state(
|
|
|
3166
3194
|
monthly = build_codex_monthly_view(
|
|
3167
3195
|
entries, now_utc=context.now_utc, tz_name=context.display_tz_name, speed=context.speed,
|
|
3168
3196
|
)
|
|
3197
|
+
# R8 gate, resolved once before the parent weekly projection so that only
|
|
3198
|
+
# a decorated merged row gains the additive account axis. Focused children
|
|
3199
|
+
# and <=1-real-account providers remain byte-identical.
|
|
3200
|
+
try:
|
|
3201
|
+
import _cctally_account
|
|
3202
|
+
_codex_decorated = _cctally_account.provider_is_decorated(
|
|
3203
|
+
context.stats_conn, "codex")
|
|
3204
|
+
except Exception:
|
|
3205
|
+
_codex_decorated = False
|
|
3169
3206
|
weekly = _build_codex_native_weekly_view(
|
|
3170
3207
|
context.stats_conn,
|
|
3171
3208
|
visible_accounting_entries,
|
|
@@ -3174,6 +3211,7 @@ def build_codex_source_state(
|
|
|
3174
3211
|
now_utc=context.now_utc,
|
|
3175
3212
|
display_tz_name=context.display_tz_name,
|
|
3176
3213
|
speed=context.speed,
|
|
3214
|
+
include_account_keys=_codex_decorated,
|
|
3177
3215
|
)
|
|
3178
3216
|
sessions = (
|
|
3179
3217
|
build_rooted_codex_session_view(
|
|
@@ -3196,12 +3234,6 @@ def build_codex_source_state(
|
|
|
3196
3234
|
# `accounts[]` cards, `hero.cycles[]`, and the `account_scopes` children —
|
|
3197
3235
|
# hangs off this single boolean, so a <=1-real-account install is provably
|
|
3198
3236
|
# byte-identical by construction rather than by golden observation.
|
|
3199
|
-
try:
|
|
3200
|
-
import _cctally_account
|
|
3201
|
-
_codex_decorated = _cctally_account.provider_is_decorated(
|
|
3202
|
-
context.stats_conn, "codex")
|
|
3203
|
-
except Exception:
|
|
3204
|
-
_codex_decorated = False
|
|
3205
3237
|
quota_blocks = _quota_wire(
|
|
3206
3238
|
context.stats_conn,
|
|
3207
3239
|
accounting_entries=visible_accounting_entries,
|
|
@@ -3210,12 +3242,13 @@ def build_codex_source_state(
|
|
|
3210
3242
|
display_tz_name=context.display_tz_name,
|
|
3211
3243
|
decorated=_codex_decorated,
|
|
3212
3244
|
)
|
|
3213
|
-
# Hero-modal historical-milestone navigation index (spec §1c, §3).
|
|
3214
|
-
#
|
|
3215
|
-
#
|
|
3216
|
-
#
|
|
3245
|
+
# Hero-modal historical-milestone navigation index (spec §1c, §3). A
|
|
3246
|
+
# decorated parent has no single cycle history: its representative
|
|
3247
|
+
# `cycle` belongs to one account, while every child builds its own index
|
|
3248
|
+
# above. Preserve the parent index only for the byte-stable undecorated
|
|
3249
|
+
# shape. Guarded: an index failure must never fail the codex source build.
|
|
3217
3250
|
cycle_index: tuple = ()
|
|
3218
|
-
if cycle is not None and not hero_failure:
|
|
3251
|
+
if not _codex_decorated and cycle is not None and not hero_failure:
|
|
3219
3252
|
try:
|
|
3220
3253
|
cycle_index = tuple(
|
|
3221
3254
|
sys.modules["cctally"].build_codex_cycle_index(
|
package/bin/_cctally_db.py
CHANGED
|
@@ -5997,7 +5997,7 @@ def _032_codex_canonical_reset_anchor(conn: sqlite3.Connection) -> None:
|
|
|
5997
5997
|
updates: list[tuple[str, int]] = []
|
|
5998
5998
|
for row in conn.execute(
|
|
5999
5999
|
"SELECT id, source_root_key, observed_slot, logical_limit_key, "
|
|
6000
|
-
" window_minutes, resets_at_utc "
|
|
6000
|
+
" window_minutes, resets_at_utc, source_path, line_offset "
|
|
6001
6001
|
" FROM quota_window_snapshots "
|
|
6002
6002
|
" WHERE source = 'codex' AND canonical_resets_at_utc IS NULL "
|
|
6003
6003
|
" AND source_root_key IS NOT NULL AND observed_slot IS NOT NULL "
|
|
@@ -6007,6 +6007,7 @@ def _032_codex_canonical_reset_anchor(conn: sqlite3.Connection) -> None:
|
|
|
6007
6007
|
source_root_key=row[1], observed_slot=row[2],
|
|
6008
6008
|
logical_limit_key=row[3], window_minutes=row[4],
|
|
6009
6009
|
resets_at_utc=row[5],
|
|
6010
|
+
source_path=row[6], line_offset=row[7],
|
|
6010
6011
|
)
|
|
6011
6012
|
if anchor is not None:
|
|
6012
6013
|
updates.append((anchor, int(row[0])))
|
|
@@ -6019,6 +6020,71 @@ def _032_codex_canonical_reset_anchor(conn: sqlite3.Connection) -> None:
|
|
|
6019
6020
|
conn.commit()
|
|
6020
6021
|
|
|
6021
6022
|
|
|
6023
|
+
@cache_migration("033_codex_reset_anchor_component_closure")
|
|
6024
|
+
def _033_codex_reset_anchor_component_closure(
|
|
6025
|
+
conn: sqlite3.Connection,
|
|
6026
|
+
) -> None:
|
|
6027
|
+
"""#425: converge migration-032 chain-of-neighbours splits.
|
|
6028
|
+
|
|
6029
|
+
Migration 032 compared each raw reset only with established anchors. Real
|
|
6030
|
+
production history contains tolerance-connected chains whose endpoints are
|
|
6031
|
+
more than 600 seconds apart, leaving one physical window under multiple
|
|
6032
|
+
anchors. Rebuild the component closure over the complete population while
|
|
6033
|
+
retaining the deterministic first observation as the winning anchor.
|
|
6034
|
+
|
|
6035
|
+
Raw ``resets_at_utc`` evidence is never rewritten. Re-running computes the
|
|
6036
|
+
same full-population mapping and therefore changes no rows.
|
|
6037
|
+
"""
|
|
6038
|
+
import _cctally_cache as cache_mod
|
|
6039
|
+
import _lib_quota
|
|
6040
|
+
|
|
6041
|
+
cols = {
|
|
6042
|
+
str(row[1]) for row in conn.execute(
|
|
6043
|
+
"PRAGMA table_info(quota_window_snapshots)")
|
|
6044
|
+
}
|
|
6045
|
+
if "canonical_resets_at_utc" not in cols:
|
|
6046
|
+
return
|
|
6047
|
+
|
|
6048
|
+
groups: dict[tuple, _lib_quota.ResetAnchorComponents] = {}
|
|
6049
|
+
evidence: list[
|
|
6050
|
+
tuple[int, dt.datetime, _lib_quota.ResetAnchorComponents, str | None]
|
|
6051
|
+
] = []
|
|
6052
|
+
for row in conn.execute(
|
|
6053
|
+
"SELECT id, source_root_key, observed_slot, logical_limit_key, "
|
|
6054
|
+
" window_minutes, resets_at_utc, canonical_resets_at_utc, "
|
|
6055
|
+
" source_path, line_offset "
|
|
6056
|
+
" FROM quota_window_snapshots "
|
|
6057
|
+
" WHERE source = 'codex' "
|
|
6058
|
+
" AND source_root_key IS NOT NULL AND observed_slot IS NOT NULL "
|
|
6059
|
+
" ORDER BY source_path, line_offset, id"
|
|
6060
|
+
).fetchall():
|
|
6061
|
+
raw = cache_mod._parse_anchor_iso(row[5])
|
|
6062
|
+
if raw is None:
|
|
6063
|
+
continue
|
|
6064
|
+
group = cache_mod.CodexResetAnchorResolver.group_key(
|
|
6065
|
+
row[1], row[2], row[3], row[4])
|
|
6066
|
+
components = groups.setdefault(
|
|
6067
|
+
group, _lib_quota.ResetAnchorComponents())
|
|
6068
|
+
components.add(
|
|
6069
|
+
raw,
|
|
6070
|
+
order_key=(str(row[7]), int(row[8]), int(row[0])),
|
|
6071
|
+
)
|
|
6072
|
+
evidence.append((int(row[0]), raw, components, row[6]))
|
|
6073
|
+
|
|
6074
|
+
updates: list[tuple[str, int]] = []
|
|
6075
|
+
for row_id, raw, components, stored in evidence:
|
|
6076
|
+
canonical = cache_mod._codex_anchor_iso(components.canonical(raw))
|
|
6077
|
+
if canonical != stored:
|
|
6078
|
+
updates.append((canonical, row_id))
|
|
6079
|
+
if updates:
|
|
6080
|
+
conn.executemany(
|
|
6081
|
+
"UPDATE quota_window_snapshots SET canonical_resets_at_utc = ? "
|
|
6082
|
+
"WHERE id = ?",
|
|
6083
|
+
updates,
|
|
6084
|
+
)
|
|
6085
|
+
conn.commit()
|
|
6086
|
+
|
|
6087
|
+
|
|
6022
6088
|
# === Region 7d: Stats migration 008_recompute_weekly_cost_snapshots_dedup_fix ===
|
|
6023
6089
|
|
|
6024
6090
|
@stats_migration("008_recompute_weekly_cost_snapshots_dedup_fix")
|