cctally 1.47.0 → 1.49.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +16 -0
- package/bin/_cctally_cache.py +62 -4
- package/bin/_cctally_dashboard.py +123 -3
- package/bin/_cctally_db.py +67 -7
- package/bin/_lib_conversation.py +40 -1
- package/bin/_lib_conversation_query.py +317 -19
- package/bin/_lib_dashboard_dates.py +166 -0
- package/dashboard/static/assets/index-0Wm0BajQ.js +68 -0
- package/dashboard/static/assets/{index-CuAtCrys.css → index-BT9RRCjD.css} +1 -1
- package/dashboard/static/dashboard.html +2 -2
- package/package.json +2 -1
- package/dashboard/static/assets/index-BOR4FBvt.js +0 -68
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,22 @@ based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|
|
5
5
|
|
|
6
6
|
## [Unreleased]
|
|
7
7
|
|
|
8
|
+
## [1.49.0] - 2026-06-17
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
- **Dashboard conversation viewer — Browse-list filters.** The Browse rail gains a compact `Filters ▾` popover that narrows the conversation list by four axes: date (This month / Last month / Last 7d presets, or an arbitrary from→to range), project (a multi-select matching any of the chosen projects, with per-project conversation counts), cost (an optional minimum and/or maximum USD with quick `≥$1` / `≥$5` / `≥$10` presets), and number of cache rebuilds (a `≥1` / `≥3` / `≥5` or custom threshold). Axes combine with AND, active filters show as removable chips under the search box, and filtering is server-side so pagination and counts stay correct. The date filter matches each session's last activity — the same instant the rail now sorts and groups by — so the date sections, the recent order, and the filter all agree. When a filter set matches nothing the rail shows a distinct "No conversations match these filters." message with a one-click "Clear filters" button (separate from the generic "No conversations." copy for an empty install). Filters apply to Browse only (disabled while a full-text search is active, which also dismisses the popover so the reader's keyboard navigation stays live) and are session-only (reset on reload). Each row's displayed cost is the live, pricing-immediate recompute (consistent with the detail and report views), while the cost filter compares the stored per-session rollup — the two agree except in the brief window after a pricing-table edit, before the next sync re-derives the stored value.
|
|
12
|
+
- **Dashboard conversation viewer — jump to the latest turn.** A `Latest ↓` action in the reader header (and the `End` key) pages the open conversation forward to its end and lands on the most recent turn with the usual flash, reusing the existing jump pipeline with an uncapped pager so it reaches the last turn even in very long conversations — and now correctly waits out a page-load that happens to be in flight when you trigger it, rather than stopping short. Landing at the bottom parks the reader in its stick-to-bottom position, so a live session keeps following new turns automatically.
|
|
13
|
+
|
|
14
|
+
## [1.48.0] - 2026-06-16
|
|
15
|
+
|
|
16
|
+
### Fixed
|
|
17
|
+
- **Dashboard conversation viewer — Claude Code's new subagent transcript format:** restored correct rendering after Claude Code moved subagent transcripts to `subagents/agent-*.jsonl` with `Agent`-renamed spawns and null cross-file parent links. Background (async) subagent cards now show completion (✓) with token/duration/tool counts — derived from the subagent's own thread and marked with a `~` when Claude Code provided none; nested subagents render as recursive nested cards, each shown exactly once with the spawning tool chip merged in (a grandchild no longer renders as a detached title-only card plus a stray `Agent` chip); and a skill invoked inside a live-watched subagent now appears without a manual page refresh (the live-tail re-fetches a small overlap window and upserts folded-in content). All read-time over the existing cache — no migration, no re-ingest.
|
|
18
|
+
- **Dashboard conversation viewer — a message you typed while the agent was still working now renders.** When you send a message while the main agent (or a subagent) is busy, Claude Code queues it and persists it as a `queued_command` *attachment* — with the text in `attachment.prompt` rather than a normal `user` turn — so the transcript reader silently dropped it: the prompt was right there in the session's JSONL but never appeared in the rendered conversation. The reader now ingests those queued prompts (`commandMode: "prompt"`) as ordinary "you" turns in their chronological place, while harness-injected queued items such as background `task-notification`s stay excluded. New conversations show queued messages immediately; existing history is re-derived once by a flag-only cache migration (`014_conversation_queued_prompt_reingest`) — the cache is fully re-derivable, so there is nothing to do on upgrade.
|
|
19
|
+
- **Dashboard conversation viewer — jumping to a deeply-nested subagent now lands the card aligned to the top.** Clicking an outline-rail entry for a *grandchild* subagent (a thread nested inside another subagent card) scrolled the target to the *center* of the viewport — fine for an ordinary message, but a subagent card is often much taller than the screen, so centering it pushed its head well above the fold (roughly 250px up, and more on a large window). A subagent-card jump now aligns the card's head to the top of the viewport instead of centering it (and re-aims on the next animation frame once the force-opened ancestor cards have settled), so a deeply-nested target lands head-first at the top (#204).
|
|
20
|
+
|
|
21
|
+
### Security
|
|
22
|
+
- **Internal (build/test-time only, no user-facing change): bumped the dashboard's dev dependencies to clear three Dependabot advisories** — `vite` 8.0.10 → 8.0.16 (fixes a `server.fs.deny` bypass and an NTLMv2 hash disclosure, both Windows + dev-server only) and the test-only transitive `form-data` 4.0.5 → 4.0.6 (CRLF injection via unescaped multipart field names). None of these packages ship to installed users — the npm/brew artifacts carry only the Python CLI plus the pre-built `dashboard/static/` bundle, never `node_modules` — so there is no runtime exposure; the bundle was rebuilt against the new `vite` and all dashboard tests pass.
|
|
23
|
+
|
|
8
24
|
## [1.47.0] - 2026-06-16
|
|
9
25
|
|
|
10
26
|
### Added
|
package/bin/_cctally_cache.py
CHANGED
|
@@ -594,6 +594,7 @@ _TARGETED_DECLINE_FLAGS = (
|
|
|
594
594
|
"conversation_search_split_pending",
|
|
595
595
|
"conversation_promote_command_args_pending",
|
|
596
596
|
"conversation_sessions_backfill_pending",
|
|
597
|
+
"conversation_queued_prompt_reingest_pending", # migration 014
|
|
597
598
|
)
|
|
598
599
|
|
|
599
600
|
|
|
@@ -725,12 +726,19 @@ def sync_cache(
|
|
|
725
726
|
# media[] placeholders + user-content media index + web_search/
|
|
726
727
|
# web_fetch captures); the same offset-0 walk re-derives them, so drop
|
|
727
728
|
# that flag here as well.
|
|
729
|
+
# Migration 014 sets the DISTINCT
|
|
730
|
+
# conversation_queued_prompt_reingest_pending flag (to land queued-
|
|
731
|
+
# while-busy user prompts persisted as queued_command attachments); the
|
|
732
|
+
# same offset-0 walk re-derives them through the current parser, so drop
|
|
733
|
+
# that flag here too — MISSING this site re-arms the flag on every
|
|
734
|
+
# cache-sync --rebuild.
|
|
728
735
|
conn.execute(
|
|
729
736
|
"DELETE FROM cache_meta WHERE key IN "
|
|
730
737
|
"('conversation_reingest_pending',"
|
|
731
738
|
" 'conversation_source_tool_use_reingest_pending',"
|
|
732
739
|
" 'conversation_reingest_enrichment_pending',"
|
|
733
740
|
" 'conversation_media_reingest_pending',"
|
|
741
|
+
" 'conversation_queued_prompt_reingest_pending',"
|
|
734
742
|
" 'conversation_reingest_cursor',"
|
|
735
743
|
" 'conversation_reingest_cursor_gen')")
|
|
736
744
|
# #177 S6: a rebuild repopulates search_tool/search_thinking via the
|
|
@@ -874,16 +882,20 @@ def sync_cache(
|
|
|
874
882
|
# ANOTHER distinct flag ``conversation_media_reingest_pending`` to land
|
|
875
883
|
# the tool_result media[] placeholders + user-content media index +
|
|
876
884
|
# web_search/web_fetch captures; same offset-0 re-parse, same reason
|
|
877
|
-
# for a distinct flag.
|
|
878
|
-
#
|
|
879
|
-
#
|
|
885
|
+
# for a distinct flag. Migration 014 uses ANOTHER distinct flag
|
|
886
|
+
# ``conversation_queued_prompt_reingest_pending`` to land queued-while-
|
|
887
|
+
# busy user prompts (queued_command attachments the parser now promotes
|
|
888
|
+
# to HUMAN); same offset-0 re-parse, same distinct-flag reason. We
|
|
889
|
+
# trigger the SAME clear + offset-0 backfill on ANY of these flags and
|
|
890
|
+
# clear them ALL atomically here under the held flock.
|
|
880
891
|
try:
|
|
881
892
|
_reingest = conn.execute(
|
|
882
893
|
"SELECT 1 FROM cache_meta WHERE key IN "
|
|
883
894
|
"('conversation_reingest_pending',"
|
|
884
895
|
" 'conversation_source_tool_use_reingest_pending',"
|
|
885
896
|
" 'conversation_reingest_enrichment_pending',"
|
|
886
|
-
" 'conversation_media_reingest_pending'
|
|
897
|
+
" 'conversation_media_reingest_pending',"
|
|
898
|
+
" 'conversation_queued_prompt_reingest_pending')"
|
|
887
899
|
).fetchone() is not None
|
|
888
900
|
except sqlite3.OperationalError:
|
|
889
901
|
_reingest = False
|
|
@@ -1473,6 +1485,7 @@ _REINGEST_FLAG_KEYS = (
|
|
|
1473
1485
|
"conversation_source_tool_use_reingest_pending",
|
|
1474
1486
|
"conversation_reingest_enrichment_pending",
|
|
1475
1487
|
"conversation_media_reingest_pending", # #177 S4 (migration 009)
|
|
1488
|
+
"conversation_queued_prompt_reingest_pending", # migration 014
|
|
1476
1489
|
)
|
|
1477
1490
|
|
|
1478
1491
|
|
|
@@ -1554,6 +1567,7 @@ def _resumable_reingest_conversation_messages(conn):
|
|
|
1554
1567
|
" 'conversation_source_tool_use_reingest_pending',"
|
|
1555
1568
|
" 'conversation_reingest_enrichment_pending',"
|
|
1556
1569
|
" 'conversation_media_reingest_pending',"
|
|
1570
|
+
" 'conversation_queued_prompt_reingest_pending',"
|
|
1557
1571
|
" 'conversation_reingest_cursor',"
|
|
1558
1572
|
" 'conversation_reingest_cursor_gen')")
|
|
1559
1573
|
conn.commit()
|
|
@@ -1623,6 +1637,7 @@ def _recompute_conversation_sessions(conn, session_ids=None) -> None:
|
|
|
1623
1637
|
"(session_id, msg_count, started_utc, last_activity_utc) "
|
|
1624
1638
|
+ _CONV_SESSIONS_SELECT + " GROUP BY session_id"
|
|
1625
1639
|
)
|
|
1640
|
+
_fill_conversation_sessions_filter_columns(conn, None)
|
|
1626
1641
|
return
|
|
1627
1642
|
ids = [s for s in session_ids if s is not None]
|
|
1628
1643
|
for i in range(0, len(ids), 400):
|
|
@@ -1639,6 +1654,49 @@ def _recompute_conversation_sessions(conn, session_ids=None) -> None:
|
|
|
1639
1654
|
+ f" AND session_id IN ({placeholders}) GROUP BY session_id",
|
|
1640
1655
|
chunk,
|
|
1641
1656
|
)
|
|
1657
|
+
_fill_conversation_sessions_filter_columns(conn, ids)
|
|
1658
|
+
|
|
1659
|
+
|
|
1660
|
+
def _fill_conversation_sessions_filter_columns(conn, session_ids):
|
|
1661
|
+
"""Fill the rollup's browse-FILTER columns (project_label / cost_usd /
|
|
1662
|
+
cache_rebuild_count, migration 015) for the given sessions, or ALL when
|
|
1663
|
+
``session_ids is None``. The structural COUNT/MIN/MAX columns are filled by
|
|
1664
|
+
the INSERT in _recompute_conversation_sessions; this is the second pass that
|
|
1665
|
+
materializes the three filter axes so the rail's date/project/cost/rebuild
|
|
1666
|
+
filters are pure-SQL predicates.
|
|
1667
|
+
|
|
1668
|
+
project_label + cost reuse the query kernel's batch maps (the SAME
|
|
1669
|
+
_project_label / _session_cost_map the rail's per-page Python path used), so
|
|
1670
|
+
a filtered/displayed value equals what the live rail produced. cost is
|
|
1671
|
+
rounded to 6dp to match list_conversations' per-row rounding.
|
|
1672
|
+
cache_rebuild_count is a per-session assemble via the query kernel's
|
|
1673
|
+
single-source-of-truth helper (whole-session property — recompute, never
|
|
1674
|
+
increment).
|
|
1675
|
+
|
|
1676
|
+
No-op when the columns are absent (a pre-migration-015 cache.db being
|
|
1677
|
+
re-derived before its 015 ALTER lands), so an early/partial sync never
|
|
1678
|
+
raises ``no such column``. The CALLER owns the commit (this never commits)."""
|
|
1679
|
+
cols = {r[1] for r in conn.execute("PRAGMA table_info(conversation_sessions)")}
|
|
1680
|
+
if "cache_rebuild_count" not in cols:
|
|
1681
|
+
return
|
|
1682
|
+
lq = _load_lib("_lib_conversation_query")
|
|
1683
|
+
if session_ids is None:
|
|
1684
|
+
ids = [r[0] for r in conn.execute(
|
|
1685
|
+
"SELECT session_id FROM conversation_sessions")]
|
|
1686
|
+
else:
|
|
1687
|
+
ids = [s for s in session_ids if s is not None]
|
|
1688
|
+
if not ids:
|
|
1689
|
+
return
|
|
1690
|
+
cost = lq._session_cost_map(conn, ids)
|
|
1691
|
+
meta = lq._session_latest_meta_map(conn, ids)
|
|
1692
|
+
for sid in ids:
|
|
1693
|
+
proj = lq._project_label(meta.get(sid, (None, None))[0])
|
|
1694
|
+
rebuilds = lq.session_cache_rebuild_count(conn, sid)
|
|
1695
|
+
conn.execute(
|
|
1696
|
+
"UPDATE conversation_sessions SET project_label=?, cost_usd=?, "
|
|
1697
|
+
"cache_rebuild_count=? WHERE session_id=?",
|
|
1698
|
+
(proj, round(cost.get(sid, 0.0), 6), rebuilds, sid),
|
|
1699
|
+
)
|
|
1642
1700
|
|
|
1643
1701
|
|
|
1644
1702
|
def _consume_search_split(conn) -> None:
|
|
@@ -5276,6 +5276,13 @@ def _qs_str(q: dict, key: str, default: str | None) -> str | None:
|
|
|
5276
5276
|
_CONV_SEARCH_KINDS = ("all", "prompts", "assistant", "tools", "thinking")
|
|
5277
5277
|
|
|
5278
5278
|
|
|
5279
|
+
class _BadConversationFilter(Exception):
|
|
5280
|
+
"""Internal sentinel: a browse-filter query param failed validation. The
|
|
5281
|
+
parse helper has ALREADY sent the 400 response when this is raised, so the
|
|
5282
|
+
caller just unwinds and returns (the conversation routes all 400 on bad
|
|
5283
|
+
input, consistent with the search ``kind`` facet). Module-private."""
|
|
5284
|
+
|
|
5285
|
+
|
|
5279
5286
|
def _cached_file_sigs(conn, paths):
|
|
5280
5287
|
"""{path: size_bytes} from session_files for the given paths — the cache's
|
|
5281
5288
|
own view of how far each file is ingested. Size-only by design, matching the
|
|
@@ -5419,6 +5426,8 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
|
|
|
5419
5426
|
self._handle_share_history_get()
|
|
5420
5427
|
elif path == "/api/doctor":
|
|
5421
5428
|
self._handle_get_doctor()
|
|
5429
|
+
elif path == "/api/conversations/facets":
|
|
5430
|
+
self._handle_get_conversations_facets()
|
|
5422
5431
|
elif path == "/api/conversations":
|
|
5423
5432
|
self._handle_get_conversations()
|
|
5424
5433
|
elif path == "/api/conversation/search":
|
|
@@ -7462,12 +7471,90 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
|
|
|
7462
7471
|
conn.close()
|
|
7463
7472
|
return True, body
|
|
7464
7473
|
|
|
7474
|
+
def _parse_conversation_filters(self, q):
|
|
7475
|
+
"""Parse the browse-list filter params (spec §2) from a ``parse_qs``
|
|
7476
|
+
mapping. On any malformed value this sends a **400** and returns
|
|
7477
|
+
``None`` — the caller just ``return``s (the conversation routes all 400
|
|
7478
|
+
on bad input). On success returns a dict of ``list_conversations``
|
|
7479
|
+
kwargs: ``date_from``/``date_to`` (UTC-ISO bounds), ``projects``
|
|
7480
|
+
(list[str] | None), ``cost_min``/``cost_max`` (float | None),
|
|
7481
|
+
``rebuild_min`` (int | None). Empty/blank params drop to ``None``.
|
|
7482
|
+
|
|
7483
|
+
Numeric axes validate strictly (a non-numeric cost / non-integer
|
|
7484
|
+
rebuild threshold is a hard 400). Date bounds route through the pure
|
|
7485
|
+
``_lib_dashboard_dates.parse_filter_date_range`` helper, which resolves
|
|
7486
|
+
naive date-only bounds in ``display.tz`` and raises ``ValueError`` (→
|
|
7487
|
+
400) on a malformed date. Projects accept BOTH repeated
|
|
7488
|
+
``?projects=a&projects=b`` and a single comma-joined ``?projects=a,b``.
|
|
7489
|
+
"""
|
|
7490
|
+
def _float(name):
|
|
7491
|
+
v = _qs_str(q, name, "")
|
|
7492
|
+
if v is None or v == "":
|
|
7493
|
+
return None
|
|
7494
|
+
try:
|
|
7495
|
+
return float(v)
|
|
7496
|
+
except ValueError:
|
|
7497
|
+
self._respond_json(400, {"error": f"bad {name}: {v}"})
|
|
7498
|
+
raise _BadConversationFilter
|
|
7499
|
+
|
|
7500
|
+
def _int(name):
|
|
7501
|
+
v = _qs_str(q, name, "")
|
|
7502
|
+
if v is None or v == "":
|
|
7503
|
+
return None
|
|
7504
|
+
try:
|
|
7505
|
+
return int(v)
|
|
7506
|
+
except ValueError:
|
|
7507
|
+
self._respond_json(400, {"error": f"bad {name}: {v}"})
|
|
7508
|
+
raise _BadConversationFilter
|
|
7509
|
+
|
|
7510
|
+
try:
|
|
7511
|
+
cost_min = _float("cost_min")
|
|
7512
|
+
cost_max = _float("cost_max")
|
|
7513
|
+
rebuild_min = _int("rebuild_min")
|
|
7514
|
+
except _BadConversationFilter:
|
|
7515
|
+
return None # 400 already sent
|
|
7516
|
+
|
|
7517
|
+
projects = [p for p in q.get("projects", []) if p] or None
|
|
7518
|
+
# Single comma-joined value -> split (the client may send either form).
|
|
7519
|
+
if projects and len(projects) == 1 and "," in projects[0]:
|
|
7520
|
+
projects = [s for s in projects[0].split(",") if s] or None
|
|
7521
|
+
|
|
7522
|
+
date_from = _qs_str(q, "date_from", "") or None
|
|
7523
|
+
date_to = _qs_str(q, "date_to", "") or None
|
|
7524
|
+
if date_from or date_to:
|
|
7525
|
+
from importlib import import_module
|
|
7526
|
+
tz = _resolve_display_tz_obj(
|
|
7527
|
+
_apply_display_tz_override(
|
|
7528
|
+
load_config(), type(self).display_tz_pref_override
|
|
7529
|
+
)
|
|
7530
|
+
).key
|
|
7531
|
+
try:
|
|
7532
|
+
df, dtt = import_module(
|
|
7533
|
+
"_lib_dashboard_dates"
|
|
7534
|
+
).parse_filter_date_range(date_from, date_to, tz_name=tz)
|
|
7535
|
+
except ValueError as exc:
|
|
7536
|
+
self._respond_json(400, {"error": str(exc)})
|
|
7537
|
+
return None
|
|
7538
|
+
else:
|
|
7539
|
+
df = dtt = None
|
|
7540
|
+
|
|
7541
|
+
return {
|
|
7542
|
+
"date_from": df,
|
|
7543
|
+
"date_to": dtt,
|
|
7544
|
+
"projects": projects,
|
|
7545
|
+
"cost_min": cost_min,
|
|
7546
|
+
"cost_max": cost_max,
|
|
7547
|
+
"rebuild_min": rebuild_min,
|
|
7548
|
+
}
|
|
7549
|
+
|
|
7465
7550
|
def _handle_get_conversations(self) -> None:
|
|
7466
7551
|
"""``GET /api/conversations`` — the browse rail (spec §3.1).
|
|
7467
7552
|
|
|
7468
7553
|
Gated first (loopback / Host allowlist). ``sort``/``limit``/``offset``
|
|
7469
|
-
are read from the query string; the kernel clamps bounds.
|
|
7470
|
-
|
|
7554
|
+
are read from the query string; the kernel clamps bounds. The browse
|
|
7555
|
+
filters (date/project/cost/rebuild — spec §2) are parsed/validated here
|
|
7556
|
+
(malformed → 400) and threaded into the kernel. Cache-open failures are
|
|
7557
|
+
500s, never 5xx-with-stacktrace.
|
|
7471
7558
|
"""
|
|
7472
7559
|
if not self._require_transcripts_allowed():
|
|
7473
7560
|
return
|
|
@@ -7476,14 +7563,33 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
|
|
|
7476
7563
|
sort = _qs_str(q, "sort", "recent")
|
|
7477
7564
|
limit = _qs_int(q, "limit", 50)
|
|
7478
7565
|
offset = _qs_int(q, "offset", 0)
|
|
7566
|
+
filters = self._parse_conversation_filters(q)
|
|
7567
|
+
if filters is None:
|
|
7568
|
+
return # a 400 has already been sent
|
|
7479
7569
|
ok, body = self._run_conversation_query(
|
|
7480
7570
|
lambda conn: self._conversation_query().list_conversations(
|
|
7481
|
-
conn, sort=sort, limit=limit, offset=offset),
|
|
7571
|
+
conn, sort=sort, limit=limit, offset=offset, **filters),
|
|
7482
7572
|
"/api/conversations")
|
|
7483
7573
|
if not ok:
|
|
7484
7574
|
return
|
|
7485
7575
|
self._respond_json(200, body)
|
|
7486
7576
|
|
|
7577
|
+
def _handle_get_conversations_facets(self) -> None:
|
|
7578
|
+
"""``GET /api/conversations/facets`` — distinct project labels + their
|
|
7579
|
+
conversation counts, for the browse filter's project multi-select (spec
|
|
7580
|
+
§2). Behind the SAME loopback/Host privacy gate as the list route; a
|
|
7581
|
+
cheap indexed GROUP BY over the rollup. The popover loads its options
|
|
7582
|
+
once from here (deriving from a paginated page would be incomplete).
|
|
7583
|
+
"""
|
|
7584
|
+
if not self._require_transcripts_allowed():
|
|
7585
|
+
return
|
|
7586
|
+
ok, body = self._run_conversation_query(
|
|
7587
|
+
lambda conn: self._conversation_query().list_conversation_facets(conn),
|
|
7588
|
+
"/api/conversations/facets")
|
|
7589
|
+
if not ok:
|
|
7590
|
+
return
|
|
7591
|
+
self._respond_json(200, body)
|
|
7592
|
+
|
|
7487
7593
|
def _handle_get_conversation_detail(self, path: str) -> None:
|
|
7488
7594
|
"""``GET /api/conversation/<session-id>`` — the reader (spec §3.2).
|
|
7489
7595
|
|
|
@@ -7594,6 +7700,20 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
|
|
|
7594
7700
|
+ "\n\n").encode("utf-8"))
|
|
7595
7701
|
self.wfile.flush()
|
|
7596
7702
|
idle = 0.0
|
|
7703
|
+
# §6 P2-H — a brand-new subagent file's FIRST content was
|
|
7704
|
+
# just ingested by this emitting cycle, so the session's
|
|
7705
|
+
# source-path set may have grown. Re-resolve it now (vs
|
|
7706
|
+
# waiting up to _LIVE_TAIL_FILE_RESET_EVERY cycles) so the
|
|
7707
|
+
# new thread (incl. a skill invoked inside it) live-tails
|
|
7708
|
+
# promptly. A new path seeds seen=None (cur lacks a row),
|
|
7709
|
+
# so changed_paths flags it next cycle → it ingests + emits.
|
|
7710
|
+
# setdefault never disturbs an existing cursor.
|
|
7711
|
+
new_files = _resolve()
|
|
7712
|
+
if set(new_files) != set(files):
|
|
7713
|
+
files = new_files
|
|
7714
|
+
cur = _cached_file_sigs(conn, files)
|
|
7715
|
+
for p in files:
|
|
7716
|
+
seen.setdefault(p, cur.get(p))
|
|
7597
7717
|
continue
|
|
7598
7718
|
idle += _LIVE_TAIL_POLL_INTERVAL
|
|
7599
7719
|
if idle >= _LIVE_TAIL_KEEPALIVE:
|
package/bin/_cctally_db.py
CHANGED
|
@@ -2360,11 +2360,18 @@ def _apply_cache_schema(conn: sqlite3.Connection) -> None:
|
|
|
2360
2360
|
byte_offset INTEGER NOT NULL
|
|
2361
2361
|
);
|
|
2362
2362
|
|
|
2363
|
-
-- Browse-rail rollup (conversation_sessions). Materializes
|
|
2364
|
-
--
|
|
2363
|
+
-- Browse-rail rollup (conversation_sessions). Materializes the four
|
|
2364
|
+
-- structural aggregates the rail's old live GROUP BY produced
|
|
2365
2365
|
-- (COUNT/MIN/MAX over conversation_messages per session_id) so
|
|
2366
2366
|
-- GET /api/conversations no longer scans the whole message table to
|
|
2367
|
-
-- render a 50-row page
|
|
2367
|
+
-- render a 50-row page, PLUS three filter columns
|
|
2368
|
+
-- (project_label/cost_usd/cache_rebuild_count, migration 015) so the
|
|
2369
|
+
-- Browse list's date/project/cost/cache-rebuild filters are pure-SQL
|
|
2370
|
+
-- predicates. The structural columns are recomputed by a COUNT/MIN/MAX
|
|
2371
|
+
-- GROUP BY; the filter columns are filled per-session by
|
|
2372
|
+
-- _fill_conversation_sessions_filter_columns in the same flock-held
|
|
2373
|
+
-- recompute (cost via the query kernel's batch maps, cache_rebuild_count
|
|
2374
|
+
-- via a per-session assemble). The explicit NOT NULL on the non-INTEGER PK
|
|
2368
2375
|
-- matters (SQLite's legacy NULL-in-PK bug); the rail keys on a concrete
|
|
2369
2376
|
-- session_id and the recompute's GROUP BY already filters nulls. The
|
|
2370
2377
|
-- index lets the only paginated ordering (recent) early-terminate at
|
|
@@ -2373,10 +2380,13 @@ def _apply_cache_schema(conn: sqlite3.Connection) -> None:
|
|
|
2373
2380
|
-- flag-gated full recompute) — migration 013 arms the one-time
|
|
2374
2381
|
-- history backfill.
|
|
2375
2382
|
CREATE TABLE IF NOT EXISTS conversation_sessions (
|
|
2376
|
-
session_id
|
|
2377
|
-
msg_count
|
|
2378
|
-
started_utc
|
|
2379
|
-
last_activity_utc
|
|
2383
|
+
session_id TEXT NOT NULL PRIMARY KEY,
|
|
2384
|
+
msg_count INTEGER NOT NULL DEFAULT 0,
|
|
2385
|
+
started_utc TEXT,
|
|
2386
|
+
last_activity_utc TEXT,
|
|
2387
|
+
project_label TEXT,
|
|
2388
|
+
cost_usd REAL NOT NULL DEFAULT 0,
|
|
2389
|
+
cache_rebuild_count INTEGER NOT NULL DEFAULT 0
|
|
2380
2390
|
);
|
|
2381
2391
|
CREATE INDEX IF NOT EXISTS idx_conv_sessions_recent
|
|
2382
2392
|
ON conversation_sessions(last_activity_utc DESC, session_id DESC);
|
|
@@ -3442,6 +3452,56 @@ def _013_create_conversation_sessions(conn: sqlite3.Connection) -> None:
|
|
|
3442
3452
|
conn.commit()
|
|
3443
3453
|
|
|
3444
3454
|
|
|
3455
|
+
@cache_migration("014_conversation_queued_prompt_reingest")
|
|
3456
|
+
def _014_conversation_queued_prompt_reingest(conn: sqlite3.Connection) -> None:
|
|
3457
|
+
"""Flag-only re-ingest so a message typed while the agent was busy — QUEUED
|
|
3458
|
+
and persisted as an ``attachment`` row (``attachment.type=="queued_command"``,
|
|
3459
|
+
``commandMode=="prompt"``) rather than a ``type:"user"`` turn — lands on
|
|
3460
|
+
existing history. The parser (``_queued_prompt_row``) now promotes those to a
|
|
3461
|
+
synthetic HUMAN turn at ingest, so the offset-0 re-parse re-derives them with
|
|
3462
|
+
zero new consumption code. Sets the DISTINCT
|
|
3463
|
+
``conversation_queued_prompt_reingest_pending`` flag (NOT the shared
|
|
3464
|
+
``conversation_reingest_pending``, which also gates migration 005's read-time
|
|
3465
|
+
human-fallback in the query kernel). Consumption rides the #179 RESUMABLE
|
|
3466
|
+
per-file reingest (_resumable_reingest_conversation_messages) — the flag is
|
|
3467
|
+
wired into _TARGETED_DECLINE_FLAGS + _REINGEST_FLAG_KEYS + both flag SELECTs +
|
|
3468
|
+
the two cleanup DELETE lists in _cctally_cache.py (all five sites; missing one
|
|
3469
|
+
either never triggers or re-arms forever). Central stamp via the dispatcher
|
|
3470
|
+
(#140); a fresh install stamps it WITHOUT running (empty table -> the flag, if
|
|
3471
|
+
ever set, is a harmless no-op). Mirrors 007/009."""
|
|
3472
|
+
_set_cache_meta(conn, "conversation_queued_prompt_reingest_pending", "1")
|
|
3473
|
+
conn.commit()
|
|
3474
|
+
|
|
3475
|
+
|
|
3476
|
+
@cache_migration("015_conversation_sessions_filter_columns")
|
|
3477
|
+
def _015_conversation_sessions_filter_columns(conn: sqlite3.Connection) -> None:
|
|
3478
|
+
"""Add the browse-filter columns to the conversation_sessions rollup
|
|
3479
|
+
(project_label, cost_usd, cache_rebuild_count) so the rail's date/project/
|
|
3480
|
+
cost/cache-rebuild filters are pure-SQL predicates (spec §1). ALTERs are
|
|
3481
|
+
idempotent (duplicate-column tolerated). Arms the SHARED
|
|
3482
|
+
conversation_sessions_backfill_pending flag so the next sync_cache full
|
|
3483
|
+
recompute fills the new columns via the augmented
|
|
3484
|
+
_recompute_conversation_sessions — keeping the heavy per-session assemble
|
|
3485
|
+
(cache_rebuild_count) off the migration's critical path, mirroring 013.
|
|
3486
|
+
Central stamp via the dispatcher (#140); handler does NOT self-stamp.
|
|
3487
|
+
|
|
3488
|
+
A fresh install gets the three columns from _apply_cache_schema's CREATE
|
|
3489
|
+
TABLE and stamps 015 WITHOUT running this handler — the ALTERs no-op there
|
|
3490
|
+
(already present), and the empty rollup needs no backfill (the incremental
|
|
3491
|
+
DELETE+INSERT re-derive fills all columns in lockstep). Mirrors 013."""
|
|
3492
|
+
for ddl in (
|
|
3493
|
+
"ALTER TABLE conversation_sessions ADD COLUMN project_label TEXT",
|
|
3494
|
+
"ALTER TABLE conversation_sessions ADD COLUMN cost_usd REAL NOT NULL DEFAULT 0",
|
|
3495
|
+
"ALTER TABLE conversation_sessions ADD COLUMN cache_rebuild_count INTEGER NOT NULL DEFAULT 0",
|
|
3496
|
+
):
|
|
3497
|
+
try:
|
|
3498
|
+
conn.execute(ddl)
|
|
3499
|
+
except sqlite3.OperationalError:
|
|
3500
|
+
pass # idempotent: column already present
|
|
3501
|
+
_set_cache_meta(conn, "conversation_sessions_backfill_pending", "1")
|
|
3502
|
+
conn.commit()
|
|
3503
|
+
|
|
3504
|
+
|
|
3445
3505
|
# === Region 7d: Stats migration 008_recompute_weekly_cost_snapshots_dedup_fix ===
|
|
3446
3506
|
|
|
3447
3507
|
@stats_migration("008_recompute_weekly_cost_snapshots_dedup_fix")
|
package/bin/_lib_conversation.py
CHANGED
|
@@ -263,12 +263,51 @@ def parse_message_row(obj, offset):
|
|
|
263
263
|
populated by a separate second seek-and-walk over the same byte span."""
|
|
264
264
|
t = obj.get("type")
|
|
265
265
|
if t not in ("user", "assistant"):
|
|
266
|
-
|
|
266
|
+
# A message typed while the agent (the main session OR a subagent) is
|
|
267
|
+
# still working is QUEUED and persisted as an ``attachment`` row, never a
|
|
268
|
+
# ``type:"user"`` turn — so the user/assistant gate above would drop it and
|
|
269
|
+
# it would never reach conversation_messages (the reader bug). Promote the
|
|
270
|
+
# user-typed ones here; everything else stays dropped.
|
|
271
|
+
return _queued_prompt_row(obj, t, offset)
|
|
267
272
|
if not obj.get("uuid"):
|
|
268
273
|
return None
|
|
269
274
|
return _normalize(obj, t, offset)
|
|
270
275
|
|
|
271
276
|
|
|
277
|
+
def _queued_prompt_row(obj, t, offset):
|
|
278
|
+
"""A queued user prompt -> a synthetic HUMAN ``MessageRow``, else ``None``.
|
|
279
|
+
|
|
280
|
+
Claude Code persists a message typed while the agent is busy as
|
|
281
|
+
``{"type":"attachment","attachment":{"type":"queued_command",
|
|
282
|
+
"commandMode":"prompt","prompt":<text>}}`` — carrying its OWN
|
|
283
|
+
uuid/parentUuid/timestamp, with the text in ``attachment.prompt`` rather than
|
|
284
|
+
``message.content``. Only ``commandMode=="prompt"`` is promoted: a queued
|
|
285
|
+
``task-notification`` (``commandMode=="task-notification"``) is harness-injected
|
|
286
|
+
background plumbing — the same ``<task-notification>`` content already
|
|
287
|
+
classifies META when it arrives as a regular line — not something the user
|
|
288
|
+
typed, so it stays dropped."""
|
|
289
|
+
if t != "attachment" or not obj.get("uuid"):
|
|
290
|
+
return None
|
|
291
|
+
att = obj.get("attachment")
|
|
292
|
+
if not isinstance(att, dict) or att.get("type") != "queued_command":
|
|
293
|
+
return None
|
|
294
|
+
if att.get("commandMode") != "prompt":
|
|
295
|
+
return None
|
|
296
|
+
prompt = att.get("prompt")
|
|
297
|
+
if not isinstance(prompt, str) or not prompt.strip():
|
|
298
|
+
return None
|
|
299
|
+
# Route a synthesized user message through _normalize so the queued prompt
|
|
300
|
+
# gets the SAME classification a typed turn would (HUMAN, the #188
|
|
301
|
+
# slash-command-args promotion, system-marker / notification folding, the
|
|
302
|
+
# remote-control prefix strip, and the split search-column derivation). Every
|
|
303
|
+
# top-level field (uuid / parentUuid / sessionId / timestamp / cwd / gitBranch
|
|
304
|
+
# / isSidechain) rides along on the shallow copy; _normalize keys off the ``t``
|
|
305
|
+
# arg, never ``obj["type"]``, so the "attachment" type is inert here.
|
|
306
|
+
synth = dict(obj)
|
|
307
|
+
synth["message"] = {"role": "user", "content": prompt}
|
|
308
|
+
return _normalize(synth, "user", offset)
|
|
309
|
+
|
|
310
|
+
|
|
272
311
|
@dataclass
|
|
273
312
|
class AiTitleRow:
|
|
274
313
|
"""Pure per-line AI-title record (no I/O). Parallels MessageRow but for the
|