cctally 1.94.2 → 1.95.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +29 -0
- package/bin/_cctally_core.py +28 -24
- package/bin/_cctally_dashboard_share.py +6 -0
- package/bin/_cctally_db.py +198 -70
- package/bin/_cctally_journal.py +164 -288
- package/bin/_cctally_record.py +9 -0
- package/bin/_cctally_store.py +460 -292
- package/bin/_lib_doctor.py +2 -2
- package/bin/_lib_pricing.py +26 -7
- package/dashboard/static/assets/index-BvCbpJJA.css +1 -0
- package/dashboard/static/assets/{index-CaUQziq_.js → index-CRZMxeYI.js} +55 -55
- package/dashboard/static/dashboard.html +2 -2
- package/package.json +1 -1
- package/dashboard/static/assets/index-DUQjFSX7.css +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,35 @@ based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|
|
5
5
|
|
|
6
6
|
## [Unreleased]
|
|
7
7
|
|
|
8
|
+
## [1.95.1] - 2026-08-09
|
|
9
|
+
|
|
10
|
+
### Fixed
|
|
11
|
+
- Corruption recovery now preserves both incidents when two database families are quarantined within the same second. The second incident atomically reserves a distinct retention-compatible directory instead of reusing the first incident's path and failing because the old and new `stats.db` generations both exist.
|
|
12
|
+
- The authoritative test gate now runs its rollback-contention wall-clock checks after the parallel pytest estate, alongside the existing serial rebuild benchmark. Whole-estate CPU contention can no longer consume those latency ceilings while isolated regressions still fail the release.
|
|
13
|
+
- Concurrent usage ingestion no longer derives five-hour blocks or milestones from a snapshot that a retained suppression event deliberately removed. A duplicate observation now respects the absent Model-A target instead of passing `None` into the derivation path and repeatedly failing every ingester behind it.
|
|
14
|
+
|
|
15
|
+
## [1.95.0] - 2026-08-09
|
|
16
|
+
|
|
17
|
+
### Changed
|
|
18
|
+
- The test gate refuses to report green while verifying less than it claims. `bin/cctally-test-all` now admits the harness estate from a committed manifest (`tests/authoritative-test-manifest.json`) instead of a two-name list, so a harness that lost its executable bit, went missing, or appeared unexpectedly fails the run instead of quietly leaving the bundle. It also reads each harness's exit status rather than only the counters it printed, requires an audited minimum case count per harness, asserts the runner's capabilities before starting, and refuses a run carrying a golden-regeneration variable (#529).
|
|
19
|
+
- A red run now says which kind of red it was. Every run emits a versioned outcome object whose `failureClass` is `product` (tests genuinely failed), `infrastructure` (the run could not start or the machinery broke), or `incomplete` (the run finished without verifying what it claimed), with every contributing reason retained. The exit band is `0` pass, `1` product failure, `2` usage error, `3` infrastructure or incomplete (#529).
|
|
20
|
+
|
|
21
|
+
### Fixed
|
|
22
|
+
- `stats.db` now uses SQLite rollback journaling (`DELETE` / `FULL`) instead of WAL. The disposable index no longer depends on live `-wal`/`-shm` generations or checkpoints; corrupt-family replacement requires a cold second drain and preserves the complete main/rollback-journal family before publishing the rebuilt index. Existing WAL installs rebuild once at epoch 1010 and refuse with restart guidance while an older dashboard or process still holds the legacy family. `cctally db checkpoint --db stats` is retired with exit 2; cache and conversations remain WAL-backed (#538).
|
|
23
|
+
- Codex Guardian approval-review usage is now priced through its canonical `gpt-5.5` model instead of the generic `gpt-5` unknown-model fallback. Retained `codex-auto-review` rows no longer trigger a false pricing-coverage advisory, fast-tier reports use the canonical 2.5x multiplier, and genuinely unknown identifiers still warn and remain actionable (#535).
|
|
24
|
+
- Differently pinned worktrees can now share the default remote test workdir. Each agentmem revision has its own selected executable, so provisioning one tree no longer replaces or rejects the other tree's pin and takes the runner fleet out of service (#519).
|
|
25
|
+
- Codex share artifacts now disclose provider availability consistently across all nine dashboard panels. Current-week, trend, daily, monthly, weekly, and sessions exports were deriving availability only from their row count, so a real partial or empty provider state could leave both the rendered artifact and its drift indicator unchanged (#533).
|
|
26
|
+
- Corrupt `stats.db` recovery no longer starves behind continuous hook traffic. The first detector now elects one durable background owner before waiting for maintenance, other callers return promptly and coalesce onto it, and that owner captures forensics and rebuilds after readers drain instead of every caller timing out before recovery is admitted (#530).
|
|
27
|
+
- Escape now closes the share modal after you have clicked into the live preview. The preview is a sandboxed frame, so a key pressed inside it belonged to a different document and never reached the shortcut that closes the modal — the only way out was to tab back out of the frame first. It works from the composer's combined preview too, and it still closes one layer at a time: a section's actions first, then the composer, then the dialog beneath it (#503).
|
|
28
|
+
- The share preview frame is no longer announced to a screen reader as "Report preview (decorative)". It was fully exposed to assistive technology while its only name told you not to bother with it — on the one control the documentation tells you to review before sharing. It now names the panel and template it is showing (#503).
|
|
29
|
+
- Manage presets is now a real dialog. It claimed to be one but rendered as a plain block at the bottom of the share modal: Tab walked straight out of it into the template gallery behind, and closing it left focus nowhere, because it tried to return focus to a menu item that had already disappeared. It now covers the screen, keeps Tab inside itself, fits a phone without sideways scrolling, and returns focus to the presets button you opened it from (#503).
|
|
30
|
+
- Escape inside a composer section's actions menu no longer closes the whole composer and discards your unsaved title, theme and format edits. It closes just the menu and puts focus back on the button that opened it; press it again to close the composer. Opening one section's actions also closes any other section's, which previously could all be open at once. The menu no longer describes itself to screen readers as a menu it never behaved like — both actions remain reachable by Tab exactly as before (#503).
|
|
31
|
+
- Share checkboxes and radio buttons are 44 pixels tall in a narrow browser window, not only on a touch device. The touch sizing keyed on the pointer type rather than the width, so a narrow desktop window got none of it. The `Anon on export` label — the privacy control — was also rendering at 12 pixels on a phone despite a rule meant to set it to 16, which is the threshold below which iOS zooms the page when you tap a control (#503).
|
|
32
|
+
- The share preview on a phone now shows the report instead of its frontmatter. It was capped at 128 pixels, which left about four lines once the privacy notice had taken its share, and the source label was printed twice inside that space. The cap is now proportional to the visible screen, and it shrinks correctly when the on-screen keyboard is up rather than being pushed off the bottom (#503).
|
|
33
|
+
- Gallery text no longer appears to bleed through the pinned preview on a phone. Two separate causes: a 12-pixel band above the preview where scrolling content passed through a gap, and a bottom edge where the preview and the tile beneath it were the same colour with the same hairline border, so two half-cut lines of text read as one overlapping line. The preview now has a visible boundary and sits flush against the top of the scroll area (#503).
|
|
34
|
+
- Rotating a phone or resizing the window with the share modal open no longer resets the preview. The two layouts rendered separate previews, so crossing the breakpoint threw away the rendered report and its privacy answer, blanked the preview for about a second, and briefly showed the wrong statement about whether your export contains real project names. There is now one preview, positioned by layout at both sizes (#503).
|
|
35
|
+
- The keyboard shortcuts in the share documentation now name the gestures that work. Every passage that advertised a bare `S` or `B` has been rewritten; both shortcuts have always required Shift, and an unshifted `s` opens Settings instead. Several passages also told you to click a panel to focus it, which does not focus it — it opens the panel's own modal, which then blocks the shortcut — and two quoted a help message the application has never shown. The correct gesture is to Tab to one of the panel's actions and press Shift+S, which appeared nowhere. The share icon's own tooltip said `Share (S)` and now says `Share (Shift+S)` (#503).
|
|
36
|
+
|
|
8
37
|
## [1.94.2] - 2026-08-08
|
|
9
38
|
|
|
10
39
|
### Fixed
|
package/bin/_cctally_core.py
CHANGED
|
@@ -321,14 +321,6 @@ OAUTH_BACKOFF_CAP_SECONDS = 3600.0
|
|
|
321
321
|
_init_paths_from_env()
|
|
322
322
|
|
|
323
323
|
|
|
324
|
-
# stats.db WAL cap (#297). Bounds the persistent WAL file so a resetting
|
|
325
|
-
# checkpoint truncates it back down instead of leaving it at its high-water
|
|
326
|
-
# size. stats.db writes are small (its -wal was observed at 0 bytes even under
|
|
327
|
-
# the contention that bloated cache.db to multi-GB), so a tighter 16 MB cap is
|
|
328
|
-
# ample. See docs/superpowers/specs/2026-07-13-cache-db-wal-hardening-design.md.
|
|
329
|
-
STATS_WAL_SIZE_LIMIT_BYTES = 16 * 1024 * 1024 # 16777216
|
|
330
|
-
|
|
331
|
-
|
|
332
324
|
# === stats.db epoch-rebuild versioning (DB journal redesign §7.1/§8) ==
|
|
333
325
|
#
|
|
334
326
|
# stats.db is no longer a versioned migration target — it is a DISPOSABLE index
|
|
@@ -393,7 +385,12 @@ STATS_WAL_SIZE_LIMIT_BYTES = 16 * 1024 * 1024 # 16777216
|
|
|
393
385
|
# before any schema work, so an `add_column_if_missing` would never run on an
|
|
394
386
|
# upgraded install and the column would simply never appear. The registry stays
|
|
395
387
|
# frozen at 13 and an epoch mismatch resolves by rebuild.
|
|
396
|
-
|
|
388
|
+
# 1009 -> 1010 (#538 Task A): retire WAL for the disposable stats index. Every
|
|
389
|
+
# stats connection now uses DELETE/FULL rollback journaling, structurally
|
|
390
|
+
# removing the shared-memory WAL page map implicated by the retained corruption
|
|
391
|
+
# bundles. The one-time rebuild is the mode transition; no stats migration is
|
|
392
|
+
# added and cache/conversations remain WAL/NORMAL.
|
|
393
|
+
STATS_INDEX_EPOCH = 1010
|
|
397
394
|
LEGACY_STATS_HEAD = 13
|
|
398
395
|
|
|
399
396
|
#: #496 S1 F1. A NEW branch, for a state that cannot occur before the
|
|
@@ -773,8 +770,8 @@ def ensure_dirs() -> None:
|
|
|
773
770
|
# bin/_cctally_store.py _heal_flock_blocking, reached through
|
|
774
771
|
# _acquire_stats_maintenance_reentrant by the epoch
|
|
775
772
|
# resolver
|
|
776
|
-
# bin/_cctally_store.py
|
|
777
|
-
#
|
|
773
|
+
# bin/_cctally_store.py detached corruption-heal worker maintenance acquire
|
|
774
|
+
# (the detector itself never acquires it; #530)
|
|
778
775
|
# bin/_cctally_db.py cmd_db_rebuild, _acquire_db_admin_writer_flocks
|
|
779
776
|
# (db skip / db unskip), _cmd_db_repair_exclusive,
|
|
780
777
|
# _vacuum_one_db
|
|
@@ -1647,11 +1644,12 @@ def open_db(*, _target_path=None) -> sqlite3.Connection:
|
|
|
1647
1644
|
# #453: probe the live index before the maintenance-shared opener. During
|
|
1648
1645
|
# a large detached rebuild that opener can wait for its bounded 5-second
|
|
1649
1646
|
# guard; a statusline arriving mid-replay would otherwise pay that delay
|
|
1650
|
-
# (and can reach more than one stats opener in a single render). The
|
|
1651
|
-
#
|
|
1652
|
-
#
|
|
1653
|
-
# established guarded-open paths.
|
|
1654
|
-
# race-closing defense if the
|
|
1647
|
+
# (and can reach more than one stats opener in a single render). The probe
|
|
1648
|
+
# reads only the fixed user_version field in the main-file header; it never
|
|
1649
|
+
# opens SQLite or bypasses the maintenance opener fence. Missing, invalid-
|
|
1650
|
+
# header and legacy indexes retain their established guarded-open paths.
|
|
1651
|
+
# The post-open epoch gate below remains a race-closing defense if the
|
|
1652
|
+
# version changes after this probe.
|
|
1655
1653
|
if (
|
|
1656
1654
|
_target_path is None
|
|
1657
1655
|
and _cctally_store.stats_epoch_enabled()
|
|
@@ -1682,18 +1680,16 @@ def open_db(*, _target_path=None) -> sqlite3.Connection:
|
|
|
1682
1680
|
# (corruption that only surfaces mid-DDL stays a raw error; the probe
|
|
1683
1681
|
# catches the common case).
|
|
1684
1682
|
try:
|
|
1685
|
-
# §6.1 PRAGMA policy via the shared table (stats:
|
|
1683
|
+
# §6.1 PRAGMA policy via the shared table (stats: DELETE / FULL /
|
|
1686
1684
|
# busy_timeout 15000 / journal_size_limit 16 MiB / auto_vacuum unset).
|
|
1687
1685
|
# #297: busy_timeout 15000 lets a writer wait out a slow-but-normal sync
|
|
1688
|
-
# (>5 s) instead of erroring "database is locked"
|
|
1689
|
-
#
|
|
1690
|
-
# write sites, cctally-dev#87). journal_size_limit bounds the persistent
|
|
1691
|
-
# WAL. The corruption probe (SELECT 1) stays here inside the narrow
|
|
1686
|
+
# (>5 s) instead of erroring "database is locked". The corruption probe
|
|
1687
|
+
# (SELECT 1) stays here inside the narrow
|
|
1692
1688
|
# StatsDbCorruptError boundary.
|
|
1693
1689
|
_cctally_store.apply_policy(conn, "stats")
|
|
1694
1690
|
conn.execute("SELECT 1").fetchone()
|
|
1695
|
-
# §9.2 (#496 S6 F23). AFTER `apply_policy`,
|
|
1696
|
-
#
|
|
1691
|
+
# §9.2 (#496 S6 F23). AFTER `apply_policy`, so any rollback journal
|
|
1692
|
+
# materialized during the transition is included in family hardening.
|
|
1697
1693
|
_cctally_store._harden_stats_family(db_path)
|
|
1698
1694
|
except sqlite3.DatabaseError as exc:
|
|
1699
1695
|
try:
|
|
@@ -1714,7 +1710,15 @@ def open_db(*, _target_path=None) -> sqlite3.Connection:
|
|
|
1714
1710
|
# it — all of which fall through to the original guided
|
|
1715
1711
|
# StatsDbCorruptError so the manual path still applies.
|
|
1716
1712
|
heal = getattr(_cctally_store, "HEAL_HOOK", None)
|
|
1717
|
-
|
|
1713
|
+
heal_enabled = (
|
|
1714
|
+
os.environ.get("CCTALLY_TEST_DISABLE_STATS_AUTO_HEAL") != "1"
|
|
1715
|
+
)
|
|
1716
|
+
if (
|
|
1717
|
+
_target_path is None
|
|
1718
|
+
and heal is not None
|
|
1719
|
+
and heal_enabled
|
|
1720
|
+
and heal("stats", exc)
|
|
1721
|
+
):
|
|
1718
1722
|
# #386: the post-heal retry is an opener too. The heal released the
|
|
1719
1723
|
# maintenance lock before returning, so another maintenance path can
|
|
1720
1724
|
# legitimately own the family by now.
|
|
@@ -1763,9 +1763,15 @@ def _build_codex_source_share_snapshot(ls, *, state, panel: str,
|
|
|
1763
1763
|
display_tz_label=display_tz,
|
|
1764
1764
|
)
|
|
1765
1765
|
codex_module = sys.modules["cctally"]._load_sibling("_cctally_codex")
|
|
1766
|
+
# The canonical CLI adapter derives availability from its row count, but
|
|
1767
|
+
# this source-aware path has an authoritative provider state. Project the
|
|
1768
|
+
# normalized state after adapting the rows so all nine panels carry the
|
|
1769
|
+
# same source chrome and built-snapshot digest signal (#533).
|
|
1766
1770
|
return replace(
|
|
1767
1771
|
codex_module._build_codex_share_snapshot(command, view, normalized_rows),
|
|
1768
1772
|
template_id=template_id,
|
|
1773
|
+
availability=availability,
|
|
1774
|
+
availability_reason=reason,
|
|
1769
1775
|
)
|
|
1770
1776
|
|
|
1771
1777
|
|
package/bin/_cctally_db.py
CHANGED
|
@@ -373,10 +373,11 @@ class StatsHealDeferred(StatsRebuildDeferred):
|
|
|
373
373
|
quarantine and rebuild while the caller waited (#496 S3 §6). The caller
|
|
374
374
|
degrades where it already degrades; it never blocks on the rebuild.
|
|
375
375
|
|
|
376
|
-
``heal_id`` correlates this signal with the durable
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
376
|
+
``heal_id`` correlates this signal with the durable admitted occurrence.
|
|
377
|
+
``forensics_path`` is optional: since #530 the fresh worker captures and
|
|
378
|
+
persists that bundle after maintenance drains, so the detector normally
|
|
379
|
+
raises before a path exists. Both remain attributes rather than parsed
|
|
380
|
+
message text.
|
|
380
381
|
"""
|
|
381
382
|
|
|
382
383
|
def __init__(
|
|
@@ -1042,7 +1043,7 @@ def write_corruption_forensics(
|
|
|
1042
1043
|
"truncated": False,
|
|
1043
1044
|
},
|
|
1044
1045
|
}
|
|
1045
|
-
for suffix in ("", "-wal", "-shm"):
|
|
1046
|
+
for suffix in ("", "-journal", "-wal", "-shm"):
|
|
1046
1047
|
p = pathlib.Path(str(db_path) + suffix)
|
|
1047
1048
|
try:
|
|
1048
1049
|
st = p.stat()
|
|
@@ -1276,6 +1277,7 @@ def _load_pending_quarantine(db_path: pathlib.Path) -> "dict[str, Any] | None":
|
|
|
1276
1277
|
isinstance(name, str)
|
|
1277
1278
|
and name in {
|
|
1278
1279
|
db_path.name,
|
|
1280
|
+
f"{db_path.name}-journal",
|
|
1279
1281
|
f"{db_path.name}-wal",
|
|
1280
1282
|
f"{db_path.name}-shm",
|
|
1281
1283
|
}
|
|
@@ -1298,6 +1300,37 @@ def _load_pending_quarantine(db_path: pathlib.Path) -> "dict[str, Any] | None":
|
|
|
1298
1300
|
_QUARANTINE_MISSING_CONTEXT_WARNED = False # one-shot warn flag
|
|
1299
1301
|
|
|
1300
1302
|
|
|
1303
|
+
def _reserve_quarantine_incident(
|
|
1304
|
+
root: pathlib.Path, db_name: str, timestamp: str
|
|
1305
|
+
) -> pathlib.Path:
|
|
1306
|
+
"""Atomically reserve a retention-compatible incident directory."""
|
|
1307
|
+
root.mkdir(parents=True, exist_ok=True)
|
|
1308
|
+
primary = root / f"{db_name}-{timestamp}"
|
|
1309
|
+
try:
|
|
1310
|
+
primary.mkdir()
|
|
1311
|
+
return primary
|
|
1312
|
+
except FileExistsError:
|
|
1313
|
+
pass
|
|
1314
|
+
|
|
1315
|
+
match = re.fullmatch(r"(\d{8}T\d{6})(?:Z|_(\d{6}))", timestamp)
|
|
1316
|
+
if match is None:
|
|
1317
|
+
stem = dt.datetime.now(dt.timezone.utc).strftime("%Y%m%dT%H%M%S")
|
|
1318
|
+
first = 1
|
|
1319
|
+
else:
|
|
1320
|
+
stem = match.group(1)
|
|
1321
|
+
first = int(match.group(2) or "0") + 1
|
|
1322
|
+
for microsecond in range(first, 1_000_000):
|
|
1323
|
+
candidate = root / f"{db_name}-{stem}_{microsecond:06d}"
|
|
1324
|
+
try:
|
|
1325
|
+
candidate.mkdir()
|
|
1326
|
+
return candidate
|
|
1327
|
+
except FileExistsError:
|
|
1328
|
+
continue
|
|
1329
|
+
raise OSError(
|
|
1330
|
+
f"could not allocate a unique quarantine incident for {db_name}"
|
|
1331
|
+
)
|
|
1332
|
+
|
|
1333
|
+
|
|
1301
1334
|
def _warn_quarantine_created_without_context(incident: pathlib.Path) -> None:
|
|
1302
1335
|
"""Report a creation that supplied no `QuarantineContext` (#496 S6 §4.2).
|
|
1303
1336
|
|
|
@@ -1350,8 +1383,7 @@ def _quarantine_db_family_strict(
|
|
|
1350
1383
|
if state is None:
|
|
1351
1384
|
ts = ts or _db_backup_timestamp()
|
|
1352
1385
|
root = _cctally_core.APP_DIR / "quarantine"
|
|
1353
|
-
incident = root
|
|
1354
|
-
incident.mkdir(parents=True, exist_ok=True)
|
|
1386
|
+
incident = _reserve_quarantine_incident(root, db_path.name, ts)
|
|
1355
1387
|
for directory in (root, incident):
|
|
1356
1388
|
try:
|
|
1357
1389
|
os.chmod(directory, 0o700)
|
|
@@ -1363,6 +1395,7 @@ def _quarantine_db_family_strict(
|
|
|
1363
1395
|
members = [
|
|
1364
1396
|
candidate.name
|
|
1365
1397
|
for candidate in (
|
|
1398
|
+
pathlib.Path(f"{db_path}-journal"),
|
|
1366
1399
|
pathlib.Path(f"{db_path}-wal"),
|
|
1367
1400
|
pathlib.Path(f"{db_path}-shm"),
|
|
1368
1401
|
db_path,
|
|
@@ -1448,7 +1481,7 @@ def quarantine_db_family(
|
|
|
1448
1481
|
strict: bool = False,
|
|
1449
1482
|
context: "QuarantineContext | None" = None,
|
|
1450
1483
|
) -> pathlib.Path:
|
|
1451
|
-
"""Move a damaged DB
|
|
1484
|
+
"""Move a damaged DB and its SQLite journals into a single
|
|
1452
1485
|
timestamped incident directory under ``quarantine/`` with a manifest (spec
|
|
1453
1486
|
§6.3). NEVER deletes evidence — three renames under the caller's exclusion
|
|
1454
1487
|
locks, not pretending to be one atomic op. ``0o700`` dir / ``0o600`` files.
|
|
@@ -1475,7 +1508,7 @@ def quarantine_db_family(
|
|
|
1475
1508
|
except OSError:
|
|
1476
1509
|
pass
|
|
1477
1510
|
moved: list = []
|
|
1478
|
-
for suffix in ("", "-wal", "-shm"):
|
|
1511
|
+
for suffix in ("", "-journal", "-wal", "-shm"):
|
|
1479
1512
|
src = pathlib.Path(str(db_path) + suffix)
|
|
1480
1513
|
if not src.exists():
|
|
1481
1514
|
continue
|
|
@@ -9415,10 +9448,10 @@ def _release_repair_marker(path: pathlib.Path, claim: RepairMarkerClaim) -> None
|
|
|
9415
9448
|
|
|
9416
9449
|
|
|
9417
9450
|
def _db_family_open_pids(path: pathlib.Path) -> "set[int] | None":
|
|
9418
|
-
"""Return processes with
|
|
9451
|
+
"""Return processes with any SQLite family member open; None if unknown."""
|
|
9419
9452
|
family = [
|
|
9420
9453
|
pathlib.Path(str(path) + suffix)
|
|
9421
|
-
for suffix in ("", "-wal", "-shm")
|
|
9454
|
+
for suffix in ("", "-journal", "-wal", "-shm")
|
|
9422
9455
|
if pathlib.Path(str(path) + suffix).exists()
|
|
9423
9456
|
]
|
|
9424
9457
|
if not family:
|
|
@@ -9478,7 +9511,7 @@ def _unique_sibling_path(path: pathlib.Path) -> pathlib.Path:
|
|
|
9478
9511
|
def family_exists(candidate: pathlib.Path) -> bool:
|
|
9479
9512
|
return any(
|
|
9480
9513
|
pathlib.Path(str(candidate) + suffix).exists()
|
|
9481
|
-
for suffix in ("", "-wal", "-shm")
|
|
9514
|
+
for suffix in ("", "-journal", "-wal", "-shm")
|
|
9482
9515
|
)
|
|
9483
9516
|
|
|
9484
9517
|
if not family_exists(path):
|
|
@@ -9510,9 +9543,9 @@ def _copy_db_family(
|
|
|
9510
9543
|
source: pathlib.Path,
|
|
9511
9544
|
destination: pathlib.Path,
|
|
9512
9545
|
*,
|
|
9513
|
-
suffixes: "tuple[str, ...]" = ("", "-wal", "-shm"),
|
|
9546
|
+
suffixes: "tuple[str, ...]" = ("", "-journal", "-wal", "-shm"),
|
|
9514
9547
|
) -> None:
|
|
9515
|
-
"""Copy main
|
|
9548
|
+
"""Copy main and journal bytes while the caller owns the family."""
|
|
9516
9549
|
for suffix in suffixes:
|
|
9517
9550
|
src = pathlib.Path(str(source) + suffix)
|
|
9518
9551
|
if not src.exists():
|
|
@@ -9523,6 +9556,53 @@ def _copy_db_family(
|
|
|
9523
9556
|
_fsync_file(dst)
|
|
9524
9557
|
|
|
9525
9558
|
|
|
9559
|
+
def _reserve_post_drain_evidence_path(
|
|
9560
|
+
backup: pathlib.Path, suffix: str
|
|
9561
|
+
) -> pathlib.Path:
|
|
9562
|
+
"""Reserve a unique sibling without ever overwriting retained evidence."""
|
|
9563
|
+
for number in range(1, 10_000):
|
|
9564
|
+
infix = ".post-drain" if number == 1 else f".post-drain-{number}"
|
|
9565
|
+
candidate = pathlib.Path(f"{backup}{infix}{suffix}")
|
|
9566
|
+
try:
|
|
9567
|
+
fd = os.open(
|
|
9568
|
+
candidate,
|
|
9569
|
+
os.O_WRONLY | os.O_CREAT | os.O_EXCL,
|
|
9570
|
+
0o600,
|
|
9571
|
+
)
|
|
9572
|
+
except FileExistsError:
|
|
9573
|
+
continue
|
|
9574
|
+
os.close(fd)
|
|
9575
|
+
return candidate
|
|
9576
|
+
raise OSError(f"could not allocate post-drain evidence beside {backup}")
|
|
9577
|
+
|
|
9578
|
+
|
|
9579
|
+
def _preserve_post_drain_sidecars(
|
|
9580
|
+
source: pathlib.Path, backup: pathlib.Path
|
|
9581
|
+
) -> "tuple[pathlib.Path, ...]":
|
|
9582
|
+
"""Move late sidecars aside without changing the classified backup."""
|
|
9583
|
+
moved = []
|
|
9584
|
+
for suffix in ("-journal", "-wal", "-shm"):
|
|
9585
|
+
member = pathlib.Path(str(source) + suffix)
|
|
9586
|
+
if not member.exists():
|
|
9587
|
+
continue
|
|
9588
|
+
destination = _reserve_post_drain_evidence_path(backup, suffix)
|
|
9589
|
+
try:
|
|
9590
|
+
# The O_EXCL reservation owns this pathname, so replace cannot
|
|
9591
|
+
# destroy pre-existing user or pre-checkpoint evidence.
|
|
9592
|
+
os.replace(member, destination)
|
|
9593
|
+
except OSError:
|
|
9594
|
+
try:
|
|
9595
|
+
os.unlink(destination)
|
|
9596
|
+
except OSError:
|
|
9597
|
+
pass
|
|
9598
|
+
raise
|
|
9599
|
+
os.chmod(destination, 0o600)
|
|
9600
|
+
_fsync_file(destination)
|
|
9601
|
+
_fsync_directory(destination.parent)
|
|
9602
|
+
moved.append(destination)
|
|
9603
|
+
return tuple(moved)
|
|
9604
|
+
|
|
9605
|
+
|
|
9526
9606
|
#: The classification sidecar's own schema version, matching the incident
|
|
9527
9607
|
#: `classification.json` the correlator writes.
|
|
9528
9608
|
_BACKUP_CLASSIFICATION_SCHEMA_VERSION = 1
|
|
@@ -9554,7 +9634,7 @@ def _write_backup_classification_sidecar(
|
|
|
9554
9634
|
safe direction.
|
|
9555
9635
|
"""
|
|
9556
9636
|
members = []
|
|
9557
|
-
for suffix in ("", "-wal", "-shm"):
|
|
9637
|
+
for suffix in ("", "-journal", "-wal", "-shm"):
|
|
9558
9638
|
member = pathlib.Path(str(stem) + suffix)
|
|
9559
9639
|
try:
|
|
9560
9640
|
info = member.stat()
|
|
@@ -9680,7 +9760,23 @@ def _repair_preflight_and_copy(
|
|
|
9680
9760
|
*,
|
|
9681
9761
|
timeout_ms: int,
|
|
9682
9762
|
) -> "tuple[int, dict[str, int | None], int | None, sqlite3.Connection | None, str]":
|
|
9683
|
-
"""Preserve
|
|
9763
|
+
"""Preserve bytes, drain only proven-coherent legacy WAL, and hold writer."""
|
|
9764
|
+
import _lib_stats_wal
|
|
9765
|
+
|
|
9766
|
+
wal_evidence = _lib_stats_wal.inspect_wal_index_family(path)
|
|
9767
|
+
wal_verdict = wal_evidence.get("verdict")
|
|
9768
|
+
if _lib_stats_wal.is_incoherent_wal_index(wal_evidence):
|
|
9769
|
+
return (
|
|
9770
|
+
3, {}, None, None,
|
|
9771
|
+
"incoherent legacy WAL family; refusing to open or checkpoint it",
|
|
9772
|
+
)
|
|
9773
|
+
if wal_verdict not in {"coherent", "wal_absent", "wal_empty"}:
|
|
9774
|
+
return (
|
|
9775
|
+
3, {}, None, None,
|
|
9776
|
+
"could not prove the legacy WAL family coherent "
|
|
9777
|
+
f"({wal_verdict or 'unknown'}); refusing repair",
|
|
9778
|
+
)
|
|
9779
|
+
checkpoint_legacy_wal = wal_verdict == "coherent"
|
|
9684
9780
|
conn = sqlite3.connect(
|
|
9685
9781
|
f"file:{path}?mode=rw", uri=True, timeout=max(timeout_ms, 0) / 1000
|
|
9686
9782
|
)
|
|
@@ -9717,20 +9813,23 @@ def _repair_preflight_and_copy(
|
|
|
9717
9813
|
_write_backup_classification_sidecar(backup)
|
|
9718
9814
|
conn.rollback()
|
|
9719
9815
|
|
|
9720
|
-
|
|
9721
|
-
|
|
9722
|
-
|
|
9723
|
-
|
|
9724
|
-
|
|
9725
|
-
|
|
9726
|
-
|
|
9727
|
-
|
|
9728
|
-
|
|
9729
|
-
|
|
9816
|
+
if checkpoint_legacy_wal:
|
|
9817
|
+
try:
|
|
9818
|
+
checkpoint = conn.execute(
|
|
9819
|
+
"PRAGMA wal_checkpoint(TRUNCATE)"
|
|
9820
|
+
).fetchone()
|
|
9821
|
+
except sqlite3.DatabaseError as exc:
|
|
9822
|
+
conn.close()
|
|
9823
|
+
return 3, {}, None, None, f"WAL checkpoint failed: {exc}"
|
|
9824
|
+
wal_path = pathlib.Path(str(path) + "-wal")
|
|
9825
|
+
wal_bytes = wal_path.stat().st_size if wal_path.exists() else 0
|
|
9826
|
+
if checkpoint is None or int(checkpoint[0]) != 0 or wal_bytes != 0:
|
|
9827
|
+
conn.close()
|
|
9828
|
+
return 3, {}, None, None, "WAL could not be fully checkpointed"
|
|
9730
9829
|
|
|
9731
|
-
# Hold this
|
|
9732
|
-
#
|
|
9733
|
-
#
|
|
9830
|
+
# Hold this write exclusion through source capture and `.recover`.
|
|
9831
|
+
# Final replacement releases it only to perform the mandatory second
|
|
9832
|
+
# whole-family drain under the repair marker + maintenance fence.
|
|
9734
9833
|
try:
|
|
9735
9834
|
conn.execute("BEGIN IMMEDIATE")
|
|
9736
9835
|
except sqlite3.DatabaseError as exc:
|
|
@@ -10047,18 +10146,34 @@ def _cmd_db_repair_under_retention(
|
|
|
10047
10146
|
try:
|
|
10048
10147
|
os.chmod(recovered_path, 0o600)
|
|
10049
10148
|
_fsync_file(recovered_path)
|
|
10050
|
-
#
|
|
10051
|
-
#
|
|
10052
|
-
#
|
|
10053
|
-
#
|
|
10054
|
-
os.replace(recovered_path, path)
|
|
10055
|
-
_fsync_directory(path.parent)
|
|
10149
|
+
# Release our own SQLite handle before the destructive-edge drain.
|
|
10150
|
+
# The repair marker and maintenance-exclusive hold still fence every
|
|
10151
|
+
# normal cctally opener. A second whole-family scan closes the gap
|
|
10152
|
+
# between the admission scan and this physical replacement.
|
|
10056
10153
|
close_guard()
|
|
10057
|
-
|
|
10058
|
-
|
|
10059
|
-
|
|
10060
|
-
|
|
10061
|
-
|
|
10154
|
+
open_pids = _db_family_open_pids(path)
|
|
10155
|
+
if open_pids is None:
|
|
10156
|
+
eprint(
|
|
10157
|
+
"cctally: final stats.db repair declined: the second cold "
|
|
10158
|
+
"drain could not verify open handles. Nothing was moved."
|
|
10159
|
+
)
|
|
10160
|
+
return 3
|
|
10161
|
+
if open_pids:
|
|
10162
|
+
eprint(
|
|
10163
|
+
"cctally: final stats.db repair declined after a new reader "
|
|
10164
|
+
"appeared in process(es) "
|
|
10165
|
+
+ ", ".join(str(pid) for pid in sorted(open_pids))
|
|
10166
|
+
+ ". Stop the dashboard and older cctally processes, then "
|
|
10167
|
+
"retry; nothing was moved."
|
|
10168
|
+
)
|
|
10169
|
+
return 3
|
|
10170
|
+
|
|
10171
|
+
# The old family was copied, fsync'd, and classified before
|
|
10172
|
+
# recovery. Preserve any later survivors under unique post-drain
|
|
10173
|
+
# names; overwriting the exact pre-checkpoint members would destroy
|
|
10174
|
+
# evidence and make their inode/size classification stale.
|
|
10175
|
+
_preserve_post_drain_sidecars(path, backup)
|
|
10176
|
+
os.replace(recovered_path, path)
|
|
10062
10177
|
_fsync_directory(path.parent)
|
|
10063
10178
|
except OSError as exc:
|
|
10064
10179
|
close_guard()
|
|
@@ -10128,11 +10243,11 @@ def cmd_db_backup(args: argparse.Namespace) -> int:
|
|
|
10128
10243
|
prefix=f".{output.name}.tmp-", dir=output.parent
|
|
10129
10244
|
) as scratch_raw:
|
|
10130
10245
|
temp_path = pathlib.Path(scratch_raw) / output.name
|
|
10131
|
-
# #386: `db backup --db stats` holds this handle across the
|
|
10132
|
-
# `source.backup(destination)` loop
|
|
10133
|
-
#
|
|
10134
|
-
#
|
|
10135
|
-
#
|
|
10246
|
+
# #386/#538: `db backup --db stats` holds this handle across the
|
|
10247
|
+
# entire `source.backup(destination)` loop. In rollback mode that is
|
|
10248
|
+
# a long-lived reader, so it participates in the replacement
|
|
10249
|
+
# protocol. The `mode=ro` open is preserved verbatim through the
|
|
10250
|
+
# `connect` seam.
|
|
10136
10251
|
def _backup_source_connect(_p, _timeout_ms=timeout_ms):
|
|
10137
10252
|
return sqlite3.connect(
|
|
10138
10253
|
f"file:{_p}?mode=ro", uri=True,
|
|
@@ -10183,8 +10298,7 @@ def cmd_db_backup(args: argparse.Namespace) -> int:
|
|
|
10183
10298
|
|
|
10184
10299
|
|
|
10185
10300
|
def cmd_db_checkpoint(args: argparse.Namespace) -> int:
|
|
10186
|
-
"""Fast, non-destructive WAL drain (TRUNCATE checkpoint) for cache.db
|
|
10187
|
-
stats.db (#297).
|
|
10301
|
+
"""Fast, non-destructive WAL drain (TRUNCATE checkpoint) for cache.db.
|
|
10188
10302
|
|
|
10189
10303
|
Opens a RAW existing-file-only connection — NOT open_cache_db()/open_db(),
|
|
10190
10304
|
which apply schema, run the migration dispatcher, can DELETE Codex rows on a
|
|
@@ -10200,17 +10314,29 @@ def cmd_db_checkpoint(args: argparse.Namespace) -> int:
|
|
|
10200
10314
|
was not fully truncated through the timeout — an actionable "something is
|
|
10201
10315
|
still holding it" signal.
|
|
10202
10316
|
"""
|
|
10203
|
-
import _cctally_cache
|
|
10204
10317
|
from _lib_json_envelope import stamp_schema_version
|
|
10205
10318
|
|
|
10206
10319
|
which = args.db # "cache" | "stats"
|
|
10207
|
-
|
|
10208
|
-
|
|
10209
|
-
|
|
10210
|
-
|
|
10320
|
+
as_json = bool(getattr(args, "json", False))
|
|
10321
|
+
if which == "stats":
|
|
10322
|
+
reason = (
|
|
10323
|
+
"stats.db uses rollback journaling; no WAL checkpoint is applicable"
|
|
10324
|
+
)
|
|
10325
|
+
if as_json:
|
|
10326
|
+
print(json.dumps(stamp_schema_version({
|
|
10327
|
+
"db": "stats.db",
|
|
10328
|
+
"status": "notApplicable",
|
|
10329
|
+
"reason": reason,
|
|
10330
|
+
}, version=1)))
|
|
10331
|
+
else:
|
|
10332
|
+
eprint(f"cctally: {reason}.")
|
|
10333
|
+
return 2
|
|
10334
|
+
|
|
10335
|
+
import _cctally_cache
|
|
10336
|
+
|
|
10337
|
+
path, label = _cctally_core.CACHE_DB_PATH, "cache.db"
|
|
10211
10338
|
timeout = int(getattr(args, "busy_timeout_ms", None)
|
|
10212
10339
|
or _cctally_cache.CHECKPOINT_CMD_BUSY_TIMEOUT_MS)
|
|
10213
|
-
as_json = bool(getattr(args, "json", False))
|
|
10214
10340
|
|
|
10215
10341
|
# Absent file → nothing to drain; a missing re-derivable cache is not an
|
|
10216
10342
|
# error (mirrors cmd_db_recover / cmd_db_unskip). Do NOT connect — mode=rw
|
|
@@ -10227,20 +10353,10 @@ def cmd_db_checkpoint(args: argparse.Namespace) -> int:
|
|
|
10227
10353
|
|
|
10228
10354
|
from _lib_cache_writer_lock import acquire_ordered_flocks
|
|
10229
10355
|
|
|
10230
|
-
|
|
10231
|
-
|
|
10232
|
-
|
|
10233
|
-
|
|
10234
|
-
# must not overlap one.
|
|
10235
|
-
lock_plan = {
|
|
10236
|
-
"cache": [
|
|
10237
|
-
(_cctally_core.CACHE_LOCK_MAINTENANCE_PATH, fcntl.LOCK_SH),
|
|
10238
|
-
(_cctally_core.CACHE_LOCK_PATH, fcntl.LOCK_EX),
|
|
10239
|
-
],
|
|
10240
|
-
"stats": [
|
|
10241
|
-
(_cctally_core.STATS_LOCK_MAINTENANCE_PATH, fcntl.LOCK_SH),
|
|
10242
|
-
],
|
|
10243
|
-
}[which]
|
|
10356
|
+
lock_plan = [
|
|
10357
|
+
(_cctally_core.CACHE_LOCK_MAINTENANCE_PATH, fcntl.LOCK_SH),
|
|
10358
|
+
(_cctally_core.CACHE_LOCK_PATH, fcntl.LOCK_EX),
|
|
10359
|
+
]
|
|
10244
10360
|
|
|
10245
10361
|
held: list[int] = []
|
|
10246
10362
|
try:
|
|
@@ -10286,7 +10402,7 @@ def cmd_db_checkpoint(args: argparse.Namespace) -> int:
|
|
|
10286
10402
|
|
|
10287
10403
|
# VACUUM writes a full fresh copy of the database into a temporary file and then
|
|
10288
10404
|
# swaps it in, so it transiently needs roughly the DB's own size on top of the
|
|
10289
|
-
# existing file, plus room for
|
|
10405
|
+
# existing file, plus room for any WAL on WAL-backed stores. A short busy_timeout keeps a
|
|
10290
10406
|
# contended VACUUM from hanging (F13).
|
|
10291
10407
|
_VACUUM_BUSY_TIMEOUT_MS = 250
|
|
10292
10408
|
|
|
@@ -10313,7 +10429,7 @@ def _vacuum_required_free_bytes(path) -> int:
|
|
|
10313
10429
|
|
|
10314
10430
|
|
|
10315
10431
|
def _run_vacuum_exclusive(path, label: str) -> int:
|
|
10316
|
-
"""
|
|
10432
|
+
"""VACUUM ``path`` under EXCLUSIVE; checkpoint WAL-backed stores first.
|
|
10317
10433
|
|
|
10318
10434
|
``locking_mode=EXCLUSIVE`` + a short ``busy_timeout`` make a concurrent
|
|
10319
10435
|
reader/writer FAIL PROMPTLY (no TOCTOU gap — the exclusion is the DB's own
|
|
@@ -10326,7 +10442,8 @@ def _run_vacuum_exclusive(path, label: str) -> int:
|
|
|
10326
10442
|
before = conn.execute("PRAGMA page_count").fetchone()[0]
|
|
10327
10443
|
try:
|
|
10328
10444
|
conn.execute("PRAGMA locking_mode=EXCLUSIVE")
|
|
10329
|
-
|
|
10445
|
+
if label != "stats.db":
|
|
10446
|
+
conn.execute("PRAGMA wal_checkpoint(TRUNCATE)")
|
|
10330
10447
|
conn.execute("VACUUM")
|
|
10331
10448
|
except sqlite3.OperationalError as exc:
|
|
10332
10449
|
if "lock" in str(exc).lower() or "busy" in str(exc).lower():
|
|
@@ -10424,6 +10541,17 @@ def _vacuum_one_db(path, label: str, provider_locked: bool) -> int:
|
|
|
10424
10541
|
)
|
|
10425
10542
|
return 3
|
|
10426
10543
|
held.append(fh)
|
|
10544
|
+
if stats_store:
|
|
10545
|
+
import _cctally_store
|
|
10546
|
+
|
|
10547
|
+
if _cctally_store._stats_legacy_wal_family_present(path):
|
|
10548
|
+
eprint(
|
|
10549
|
+
"cctally: stats.db VACUUM declined: the index still uses "
|
|
10550
|
+
"legacy WAL. Stop and restart older cctally processes, "
|
|
10551
|
+
"then run a normal command to complete the epoch rebuild "
|
|
10552
|
+
"before retrying VACUUM."
|
|
10553
|
+
)
|
|
10554
|
+
return 3
|
|
10427
10555
|
return _run_vacuum_exclusive(path, label)
|
|
10428
10556
|
finally:
|
|
10429
10557
|
for fh in held:
|