cctally 1.52.1 → 1.53.0

package/CHANGELOG.md

@@ -5,6 +5,32 @@ based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+## [1.53.0] - 2026-06-21
+
+### Added
+- **Dashboard conversation outline — per-subagent cost, a resizable column, a nested-subagent tree, and a compaction landmark.** Each subagent thread in the outline now shows its cost beside its label (a display-only, summed-cost-once figure that covers every thread, including older ones with no captured spawn metadata). A thin resize divider between the reading column and the outline lets you drag the outline wider or narrower — keyboard-resizable too (`←`/`→` step, `Home`/`End` to the extremes, with a labeled `separator` accessibility surface) — and the width persists across reloads. When a subagent spawns its own sub-subagents, the outline nests those children indented beneath their parent thread (a session with no such nesting looks exactly as before). And a conversation compaction (the auto-summarize point) is now a navigable outline landmark with its own "Jump to" chip and `m`/`M` reader keys. The outline's "Jump to" chips also change: a primary click now jumps to the most-recent occurrence of that landmark family, while shift-click steps to the previous one.
+- **Dashboard conversation reader — opens at the bottom with two-way scrolling, remembers where you were reading, and jumps straight to the last prompt or error.** A long conversation now opens on its newest turns (single-request tail load, with live-tail already following) instead of at the top; a short conversation that fits one page still opens at the top so it reads from the start. The reader window pages in both directions — scrolling down loads newer turns and scrolling up prepends older ones, anchored so the turn you're reading stays put rather than jumping — and a backward (scroll-up) page never breaks the "follow the live session" state. Switching away from a conversation and returning later restores you to the turn you were last reading (anchored to the turn, so it survives a resized window or a grown transcript; a deep-link or jump target still wins, and a vanished turn falls back to the bottom). Two new reader keys jump directly to the most-recent occurrence: `a` to the last prompt and `L` to the last error. `Latest ↓` / `End` now reset to the newest page in a single request (instant even on a very long session) rather than paging all the way forward.
+- **`cctally doctor` — new conversation-rollup consistency check.** `doctor` (and the dashboard's doctor panel) now run a `data.conversation_sessions_rollup` check that compares the materialized `conversation_sessions` browse-rail rollup against the distinct session count in the conversation cache, warning only when they disagree in a quiescent cache — an in-flight sync, reingest, or backfill is detected (via a non-blocking cache-lock probe and the pending-flag set) and never warns — so a stale or half-built rollup that would skew the conversation list's filters and counts is surfaced instead of passing silently.
+- **Dashboard conversation reader — open-at-bottom and backward pagination on the reader API.** `GET /api/conversation/<id>` now pages with `before=<id>` (the page of turns immediately *before* a cursor) and `tail=1` (the last page in a single request, so a client can land on the newest turns without paging forward through the whole session) alongside the existing forward `after=<id>`; the three cursors are mutually exclusive (more than one is a `400`) and `limit` is clamped `1`–`1000` (default `500`). The `page` object gains additive `prev_before`/`has_prev` (older turns remain to page back to) beside `next_after`/`has_more`, each computed from the page's actual position so a short backward page at the head still reports more forward; a stale cursor yields an empty page rather than re-serving the head or tail. Existing no-cursor and `after` responses are byte-stable except for the two additive `page` keys (backend contract only — the frontend lights this up in a later session).
+- **Dashboard conversation search — `kind=title` title search and server-side filtered search.** `GET /api/conversation/search` adds a `kind=title` facet that searches the AI-generated session titles and returns one session-level hit per matching session (anchored to the session's first turn, with `match_kinds:["title"]` and the matched title as the snippet); its `total` counts only sessions with surviving turns, so the count never lies, and it falls back to a `LIKE` scan when the host lacks FTS5. The search route also now accepts the same browse filters as the conversation list (`date_from`/`date_to`/`projects`/`cost_min`/`cost_max`/`rebuild_min`), applied as a session-scope restriction across every kind, with the same brief project/cost/rebuild degradation (a `filter_degraded` flag, date-axis only) while the per-session rollup is still indexing; a no-filter request is byte-stable with the prior output. The cross-session-only `title` facet is rejected with a `400` (never a `500`) on the in-conversation `find` route. Backend contract only — surfaced in a later frontend session.
+- **Dashboard conversation search — `kind=files` file-path search.** `GET /api/conversation/search` adds a `kind=files` facet that searches the write-class file paths each session edited or created (files touched by an `Edit`/`MultiEdit`/`Write`/`NotebookEdit` tool call — read-only `Read` and shell `Bash` are excluded so the axis answers "which sessions modified this file"), returning one hit per distinct `(session, file path)` anchored to that path's most-recent touch (`match_kinds:["file"]`, snippet = the file path) and a `total` that is the distinct `(session, file path)` count; a query with no leading separator is treated as a case-insensitive path prefix that is genuinely index-assisted — it rides the `COLLATE NOCASE` path index as a key-range search (the index is `NOCASE` precisely because the default `LIKE` is case-insensitive and a binary-collated index can't serve it) — while a leading-separator query is a full substring scan over the modest touch table (a leading wildcard can never use an index), and the facet composes with the same browse filters as the other kinds and is rejected with a `400` on the in-conversation `find` route. The axis is backed by a re-derivable `conversation_file_touches` table (cache migration `019`) backfilled once on the first sync after upgrade. Backend contract only — surfaced in a later frontend session.
+- **Dashboard conversation reader — long Bash output collapses by default, and a key to search the conversation list from inside a reader.** A Bash tool card whose output runs long (more than 20 rendered lines) now opens collapsed with a `show N lines` hint so it no longer buries the next turn — short output stays open, and the `[`/`]` collapse-all and a per-card click still override either way. The new `f` key focuses the conversation-list search input even while a conversation is open, so you can look up another session without pressing `Esc` to leave the reader first (`/` stays reader-aware and opens the in-conversation find bar).
+- **Dashboard conversation find bar — `⌘F` / `Ctrl+F`, regex + case-sensitive search, a focus trap, and live results.** The in-conversation find bar now opens with `⌘F` / `Ctrl+F` as well as `/` (the browser's native find bar is suppressed only inside the Conversations workspace, and only when no modal or text input is up — it still works on the dashboard panels), and two toggles beside the input — `.*` (regular expression) and `Aa` (case-sensitive), both remembered across reloads — drive a real server-side match over the full transcript corpus (prose, tool commands/results/stderr, and thinking, the same content the search facets cover); an invalid regex shows an announced "invalid regex" hint rather than failing silently. The prose underline now honors the case toggle (and is suppressed in regex mode, where the match count and jump still work fully). The bar traps keyboard focus while open (`Tab`/`Shift+Tab` cycle its controls, `Esc` exits to the transcript), and its results now live-refresh as the open conversation grows — keeping your place by jumping back to the same match across the refresh, and resetting to the first match only when the one you were on disappeared.
+- **Dashboard conversation rail — sort the browse list, persist your filters + sort, and apply filters to search.** A new **Sort** control in the rail header orders the conversation list by Recent (default), Oldest, Cost (high→low), Messages (high→low), or Project (A→Z, with un-labeled sessions last); the chosen sort and the browse filters now persist across reloads instead of resetting every session. The Browse filters also apply to full-text **search** now — the same filter set is shared, so the Filters button stays enabled with a search needle active, the active-filter chips stay visible, and the chosen date/project/cost/rebuild axes narrow the search results (file-path and AI-title results included) rather than only the browse list. While a brand-new or `--no-sync` dashboard is still building its per-session rollup, the Cost/Project sorts briefly fall back to Recent order (and the rollup-only filter axes to date-only), each with a muted "while indexing" note, self-correcting after the first full sync. The `GET /api/conversations` route gains a `sort` param (`recent`/`oldest`/`cost`/`messages`/`project`, lenient on unknown values) and an additive `page.sort_degraded` flag.
+- **Dashboard conversation search — search across sessions by AI-title or file path, surfaced in the rail.** The cross-session search chip row gains two structural facets after a subtle separator — **Title** (searches the AI-generated session titles) and **Files** (searches the write-class file paths each session edited or created) — joining the existing All / Prompts / Assistant / Tools / Thinking content kinds; both stay available even while the search index is still backfilling (neither rides the split index). A Title hit shows the matched title and opens that session at its first turn; a Files hit leads with the file path (with the session title secondary) and opens the session at that path's most-recent touch, so you can jump from "which session modified `bin/cctally`" straight into the conversation. The new hits carry `title` / `file` match badges, and both facets narrow to your active browse filters like every other kind. (This lights up the `kind=title` / `kind=files` backend facets shipped earlier in the conversation viewer's search route.)
+
+### Changed
+- **Dashboard conversation viewer — backend performance and tech-debt hardening (no visible change).** The per-sync browse-rail rollup no longer runs a full transcript assembly for every touched session — the cache-rebuild count now comes from a dedicated lightweight query that shares the exact cache-failure rule with the reader — so the rollup maintenance that runs under the cache lock on every sync no longer does work proportional to sessions × full-assembly; the in-conversation find skips assembling a transcript entirely when a query matches nothing; cross-session search batches its badge and snippet lookups into a couple of bounded queries instead of issuing up to ~200 per-result round-trips; and a dead, always-empty `search_aux` column is removed via cache migration `016`. The rendered conversation output is byte-for-byte unchanged.
+
+### Fixed
+- **Dashboard conversation reader — a cluster of small reading and accessibility refinements.** The header `Errors` focus-mode badge now counts error *turns* (matching the "Jump to" error chip and exactly what clicking the filter steps between) rather than the raw total of error events, so it can no longer read higher than the number of places the filter lands. A WebSearch results-count chip no longer shows green when the search errored (it takes a neutral/error style, with the `· error` marker alongside). The browse list's "Load more" button disables and shows a loading label while a page is fetching, matching the search results list. The Find control uses an inline icon (matching the other reader controls), the `· N hidden ·` focus-mode marker carries a screen-reader label naming the hidden-turn count, the load-full affordance disables when there's no addressable session, the diff result disclosure names the snippet's line count instead of a fixed string, and the ExitPlanMode outcome badge no longer mis-reads a free-text user response that merely mentions "approve"/"reject" as an approval or rejection.
+- **Dashboard conversation viewer — a subagent whose result exceeds 16 KB no longer drops its nested subagent into a detached, unlinked card.** When a subagent returned more than ~16 KB of text, the `agentId:` trailer that links a nested (grandchild) subagent back to the thread that spawned it could fall past the point where the result is clipped for storage, so the grandchild rendered as a flat, top-level card disconnected from its parent. The link is now captured at ingest time from the full, pre-clip result (with the read-time parse kept as a fallback), so deeply-nested subagents render correctly regardless of result size; existing transcripts are re-derived once by a flag-only cache migration (`017_arm_nested_agent_reingest`), and because the cache is fully re-derivable there is nothing to do on upgrade.
+- **Dashboard conversation viewer — full-payload reads now bound and flag truncation honestly.** A tool input large enough to exceed the payload cap is now actually clipped to that cap rather than served over-cap with only a truncation flag set (the bound is measured in the same serialization the response uses, so it holds for non-ASCII text too), and a tool result whose `stderr` stream is itself truncated now carries an explicit `stderr_truncated` flag — so a consumer of the read-full-payload route can tell a clipped stream from a complete one. No change to the rendered transcript.
+- **Dashboard — the Recent Sessions table no longer spills past its panel card at narrow two-column widths (≈1100–1600px).** Its columns (Started · Dur · Model · Project · Cost) sum wider than the card at those widths, so the Cost column used to bleed past the card and the viewport edge; the table now scrolls horizontally inside the card instead, keeping every column full-width (no truncation) with the scrollbar shown only when the table can't fit. Wider layouts (≥~1300px, including the centered ≥1600px cap) are unchanged — the table fits with no scrollbar (#208).
+- **Dashboard — the Daily heatmap no longer clips the digits of larger costs at narrow widths.** A full three-digit cost such as `$212.83` overflowed the day cell's clip box and lost its leading/trailing glyphs (rendering as "212.8"); each cell now sizes its cost text to the cell's width given the value's length, so the whole amount (cents included) always fits — wide cells keep the original 16px size and only cells that would otherwise overflow shrink, fluidly, at any width (#208).
+- **`record-credit` — a week with an already-applied credit is now refused before the confirm prompt, not after it.** Re-running `record-credit` on a week that already has a fully-recorded credit is refused by default (exit 2, naming the recorded pre-credit value and pointing at `--force`). Previously, on an interactive terminal, the command printed the full preview and asked `Proceed? [y/N]` first, then refused only after you answered `y` — so you were prompted to confirm an action that was never going to apply. The refusal now fires before the prompt, so the "already recorded — pass `--force`" message appears immediately on both interactive and non-interactive reruns (#212).
+- **Dashboard conversation find bar — `Esc` from a find-bar button no longer closes the whole reader, and a Files search result no longer prints its path twice.** Pressing `Esc` while focus was on one of the find bar's buttons (Close, the `.*`/`Aa` toggles, or the prev/next arrows) closed the find bar *and* tore the entire reader down back to the conversation list — only `Esc` from the text input behaved correctly. `Esc` is now handled at the bar level, so it closes only the find bar from any of its controls and restores keyboard focus to the transcript (instead of dropping it to the page body). Separately, a `Files` search hit in the conversation rail showed the file path both as its prominent primary line and again, redundantly, in the snippet row beneath; the duplicate snippet row is now suppressed for file hits (every other result kind keeps its snippet).
+
 ## [1.52.1] - 2026-06-20
 
 ### Fixed

package/bin/_cctally_cache.py

@@ -185,8 +185,8 @@ _CONV_INSERT_SQL = (
     " timestamp_utc,entry_type,text,blocks_json,model,msg_id,"
     " req_id,cwd,git_branch,is_sidechain,source_tool_use_id,"
     " stop_reason,attribution_skill,attribution_plugin,"
-    " search_tool,search_thinking,search_aux)"
-    " VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)"
+    " search_tool,search_thinking)"
+    " VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)"
 )
 
 # #193: last non-null write wins (ai-title carries no timestamp; see spec S1). NO
@@ -206,19 +206,19 @@ def _conv_row_tuple(m, path_str):
     """Flatten a ``MessageRow`` into the ``_CONV_INSERT_SQL`` column order.
 
     The #177 enrichment fields (stop_reason / attribution_skill /
-    attribution_plugin / search_tool / search_thinking / search_aux) are
-    TAIL-APPENDED after source_tool_use_id — same order as the SQL column list —
-    so both ingest paths (fused per-file walk + backfill_conversation_messages)
-    carry them through this one tuple. #177 S6: ``search_aux`` is always written
-    as ``''`` (documented-dead); the split ``search_tool``/``search_thinking``
-    columns carry the non-prose index."""
+    attribution_plugin / search_tool / search_thinking) are TAIL-APPENDED after
+    source_tool_use_id — same order as the SQL column list — so both ingest paths
+    (fused per-file walk + backfill_conversation_messages) carry them through this
+    one tuple. #217 S1 / U7a: the documented-dead ``search_aux`` column is gone
+    from the live schema (dropped by migration 016); the split
+    ``search_tool``/``search_thinking`` columns carry the non-prose index."""
     return (
         m.session_id, m.uuid, m.parent_uuid, path_str, m.byte_offset,
         m.timestamp_utc, m.entry_type, m.text, m.blocks_json, m.model,
         m.msg_id, m.req_id, m.cwd, m.git_branch, m.is_sidechain,
         m.source_tool_use_id,
         m.stop_reason, m.attribution_skill, m.attribution_plugin,
-        m.search_tool, m.search_thinking, m.search_aux,
+        m.search_tool, m.search_thinking,
     )
 
 
@@ -595,6 +595,9 @@ _TARGETED_DECLINE_FLAGS = (
     "conversation_promote_command_args_pending",
     "conversation_sessions_backfill_pending",
     "conversation_queued_prompt_reingest_pending",   # migration 014
+    "conversation_reingest_nested_agent_pending",    # migration 017
+    "conversation_title_fts_backfill_pending",       # migration 018 (P1-2: HERE ONLY)
+    "conversation_reingest_file_touches_pending",    # migration 019 (P1-2: HERE ONLY)
 )
 
 
@@ -731,7 +734,20 @@ def sync_cache(
             # while-busy user prompts persisted as queued_command attachments); the
             # same offset-0 walk re-derives them through the current parser, so drop
             # that flag here too — MISSING this site re-arms the flag on every
-            # cache-sync --rebuild.
+            # cache-sync --rebuild. #217 S1 migration 017 sets the DISTINCT
+            # conversation_reingest_nested_agent_pending flag (to land the
+            # ingest-time structured agent_id stamp on >16 KB nested-subagent
+            # grandchildren); the same offset-0 walk re-derives it through the
+            # current parser, so drop that flag here as well. #217 S2 migrations
+            # 018/019 set conversation_title_fts_backfill_pending (title FTS) and
+            # conversation_reingest_file_touches_pending (+ its
+            # conversation_file_touches_cursor); the offset-0 walk re-derives both
+            # the title FTS and the file-touch axis, so drop them too (#219 S2.3).
+            # NOTE: unlike the flags above, dropping the 018/019 keys here is
+            # COSMETIC — their consumers run on the just-wiped (empty) tables and
+            # self-clear before the offset-0 walk repopulates, so leaving them
+            # caused no redundant expensive pass and no re-arming. We add them for
+            # consistency with the documented convention only.
             conn.execute(
                 "DELETE FROM cache_meta WHERE key IN "
                 "('conversation_reingest_pending',"
@@ -739,6 +755,10 @@ def sync_cache(
                 " 'conversation_reingest_enrichment_pending',"
                 " 'conversation_media_reingest_pending',"
                 " 'conversation_queued_prompt_reingest_pending',"
+                " 'conversation_reingest_nested_agent_pending',"
+                " 'conversation_title_fts_backfill_pending',"
+                " 'conversation_reingest_file_touches_pending',"
+                " 'conversation_file_touches_cursor',"
                 " 'conversation_reingest_cursor',"
                 " 'conversation_reingest_cursor_gen')")
             # #177 S6: a rebuild repopulates search_tool/search_thinking via the
@@ -885,9 +905,14 @@ def sync_cache(
             # for a distinct flag. Migration 014 uses ANOTHER distinct flag
             # ``conversation_queued_prompt_reingest_pending`` to land queued-while-
             # busy user prompts (queued_command attachments the parser now promotes
-            # to HUMAN); same offset-0 re-parse, same distinct-flag reason. We
-            # trigger the SAME clear + offset-0 backfill on ANY of these flags and
-            # clear them ALL atomically here under the held flock.
+            # to HUMAN); same offset-0 re-parse, same distinct-flag reason. #217 S1
+            # migration 017 uses ANOTHER distinct flag
+            # ``conversation_reingest_nested_agent_pending`` to land the ingest-time
+            # structured agent_id stamp on >16 KB nested-subagent grandchildren
+            # (whose agentId: trailer was clipped past the 16 KB cap); same offset-0
+            # re-parse, same distinct-flag reason. We trigger the SAME clear +
+            # offset-0 backfill on ANY of these flags and clear them ALL atomically
+            # here under the held flock.
             try:
                 _reingest = conn.execute(
                     "SELECT 1 FROM cache_meta WHERE key IN "
@@ -895,7 +920,8 @@ def sync_cache(
                     " 'conversation_source_tool_use_reingest_pending',"
                     " 'conversation_reingest_enrichment_pending',"
                     " 'conversation_media_reingest_pending',"
-                    " 'conversation_queued_prompt_reingest_pending')"
+                    " 'conversation_queued_prompt_reingest_pending',"
+                    " 'conversation_reingest_nested_agent_pending')"
                 ).fetchone() is not None
             except sqlite3.OperationalError:
                 _reingest = False
@@ -931,6 +957,21 @@ def sync_cache(
             # HUMAN(text=args); the split-FTS UPDATE triggers re-index the args.
             _consume_promote_command_args(conn)
 
+            # #217 S2 / E7: consume the migration-018 title-FTS backfill under the
+            # SAME held flock. An FTS5 'rebuild' re-derives the external-content
+            # title index from conversation_ai_titles (P1-7) — idempotent under
+            # the 012-then-018 both-pending ordering, and a cheap clear-only
+            # no-op on a no-FTS5 build (P1-6). Touches ONLY the title index, never
+            # conversation_messages (P1-2).
+            _consume_title_fts(conn)
+
+            # #217 S2 / I-3: consume the migration-019 file-touches backfill under
+            # the SAME held flock. Derives conversation_file_touches from existing
+            # blocks_json history (cursor-resumable; idempotent via INSERT OR
+            # IGNORE). Touches ONLY conversation_file_touches, never
+            # conversation_messages (P1-2).
+            _consume_file_touches(conn)
+
         if targeted:
             paths = [pathlib.Path(p) for p in only_paths if pathlib.Path(p).is_file()]
         else:
@@ -1293,6 +1334,16 @@ def sync_cache(
                 # (parallel to the cost path's lifecycle).
                 if conv_rows:
                     conn.executemany(_CONV_INSERT_SQL, conv_rows)
+                    # #217 S2 / I-3: derive this tick's file touches, scoped to the
+                    # just-ingested rows' PHYSICAL keys (cr[3]=source_path,
+                    # cr[4]=byte_offset per _conv_row_tuple). Cheap (proportional to
+                    # new bytes); decoupled from the INSERT OR IGNORE rowcount —
+                    # _fill_file_touches reads conversation_messages by physical key,
+                    # so an already-present (rowcount-0) row still gets its touches.
+                    # Lands in the SAME per-file write transaction as the message
+                    # rows, so they commit atomically.
+                    _fill_file_touches(
+                        conn, scope=[(cr[3], cr[4]) for cr in conv_rows])
                 # #193: ai-title upserts for this file, in file order (last wins).
                 # Committed atomically with the session_files cursor below.
                 if ai_rows:
@@ -1486,6 +1537,7 @@ _REINGEST_FLAG_KEYS = (
     "conversation_reingest_enrichment_pending",
     "conversation_media_reingest_pending",   # #177 S4 (migration 009)
     "conversation_queued_prompt_reingest_pending",   # migration 014
+    "conversation_reingest_nested_agent_pending",    # #217 S1 (migration 017)
 )
 
 
@@ -1550,10 +1602,25 @@ def _resumable_reingest_conversation_messages(conn):
             conn.commit()
             continue
         try:
+            # #217 S2 / I-3 (P1-4): conversation_file_touches is derived state
+            # keyed by conversation_messages.id, and this per-source reingest
+            # DELETEs + re-inserts the file's message rows (bumping autoincrement
+            # ids). Delete the file's touches BEFORE the message delete (resolving
+            # the ids while they still exist), then refill from the reinserted rows
+            # AFTER — all in this one atomic transaction, so a crash leaves no stale
+            # or duplicate anchors.
+            conn.execute(
+                "DELETE FROM conversation_file_touches WHERE message_id IN "
+                "(SELECT id FROM conversation_messages WHERE source_path=?)",
+                (path_str,))
             conn.execute("DELETE FROM conversation_messages WHERE source_path=?",
                          (path_str,))
             if rows:
                 conn.executemany(_CONV_INSERT_SQL, rows)
+                # Refill scoped to this file's just-reinserted physical keys
+                # (col 3=source_path, col 4=byte_offset per _conv_row_tuple).
+                _fill_file_touches(
+                    conn, scope=[(r[3], r[4]) for r in rows])
             _set_cache_meta(conn, "conversation_reingest_cursor", path_str)
             conn.commit()
         except BaseException:
@@ -1568,6 +1635,7 @@ def _resumable_reingest_conversation_messages(conn):
         " 'conversation_reingest_enrichment_pending',"
         " 'conversation_media_reingest_pending',"
         " 'conversation_queued_prompt_reingest_pending',"
+        " 'conversation_reingest_nested_agent_pending',"
         " 'conversation_reingest_cursor',"
         " 'conversation_reingest_cursor_gen')")
     conn.commit()
@@ -1669,9 +1737,10 @@ def _fill_conversation_sessions_filter_columns(conn, session_ids):
     _project_label / _session_cost_map the rail's per-page Python path used), so
     a filtered/displayed value equals what the live rail produced. cost is
     rounded to 6dp to match list_conversations' per-row rounding.
-    cache_rebuild_count is a per-session assemble via the query kernel's
-    single-source-of-truth helper (whole-session property — recompute, never
-    increment).
+    cache_rebuild_count is a per-session lightweight rebuild-count via the
+    query kernel's single-source-of-truth helper (no full assembly — U1; the
+    rule is shared with the reader path) — a whole-session property
+    (recompute, never increment).
 
     No-op when the columns are absent (a pre-migration-015 cache.db being
     re-derived before its 015 ALTER lands), so an early/partial sync never
@@ -1758,6 +1827,153 @@ def _consume_search_split(conn) -> None:
     conn.commit()
 
 
+def _consume_title_fts(conn) -> None:
+    """#217 S2 / E7: flock-held consumer for ``conversation_title_fts_backfill_pending``
+    (set by cache migration 018). Populates the external-content title FTS over
+    ``conversation_ai_titles`` from existing history.
+
+    Uses the FTS5 ``'rebuild'`` command (the established consumer idiom —
+    ``_apply_cache_schema``'s recovery rebuild + ``_consume_search_split`` —
+    NOT blind row inserts, P1-7): ``'rebuild'`` re-derives the whole index from
+    the content table and is IDEMPOTENT even if migration 012's
+    ``ai_titles_backfill_pending`` ran first and already populated the index via
+    the conv_title_fts_ai trigger (the 012-then-018 both-pending upgrade
+    ordering) — re-running yields the same rows, no duplicates or conflict.
+
+    FTS5-unavailable (``fts5_unavailable`` set, P1-6): there is no usable vtable
+    to rebuild and a ``'rebuild'`` would error on the absent fts5 module, so just
+    clear the flag — ``kind=title`` degrades to a LIKE scan over
+    conversation_ai_titles. Touches ONLY the title index (never
+    conversation_messages — P1-2: this is NOT a message reingest); the flag is
+    dropped LAST so a crash mid-rebuild re-runs cleanly on the next sync. A fresh
+    install never sets the flag, so this is a cheap no-op there."""
+    if conn.execute(
+        "SELECT 1 FROM cache_meta "
+        "WHERE key='conversation_title_fts_backfill_pending'"
+    ).fetchone() is None:
+        return
+    fts_off = conn.execute(
+        "SELECT 1 FROM cache_meta WHERE key='fts5_unavailable'"
+    ).fetchone() is not None
+    if not fts_off:
+        # The title FTS may not exist YET even on an FTS5-capable build: a pre-S6
+        # install whose conversation_fts is still the legacy (text) shape makes
+        # _apply_cache_schema early-return at its legacy_present guard BEFORE it
+        # creates conversation_title_fts. When migrations 010 + 018 are both
+        # pending in that open, _consume_search_split swaps only the MESSAGE FTS
+        # (never the title FTS), so a blind 'rebuild' here would raise
+        # "no such table: conversation_title_fts" — fts5_unavailable is NOT set
+        # (FTS5 IS available), so the fts_off guard above does not protect it.
+        # Swallow that OperationalError and RETURN before the flag-clear so the
+        # flag survives: the NEXT open (message FTS now split → legacy_present
+        # False → _apply_cache_schema creates the title FTS) re-runs this consumer
+        # and completes the backfill. Match the message-FTS path's resilience.
+        try:
+            conn.execute(
+                "INSERT INTO conversation_title_fts(conversation_title_fts) "
+                "VALUES('rebuild')")
+        except sqlite3.OperationalError:
+            return   # title FTS not yet created (legacy-shape pre-swap); leave
+                     # the flag set, retry on the next open
+    conn.execute(
+        "DELETE FROM cache_meta WHERE key='conversation_title_fts_backfill_pending'")
+    conn.commit()
+
+
+_FILE_TOUCH_INSERT_SQL = (
+    "INSERT OR IGNORE INTO conversation_file_touches"
+    "(message_id, session_id, uuid, file_path, tool) VALUES(?,?,?,?,?)")
+
+
+def _fill_file_touches(conn, scope=None) -> None:
+    """#217 S2 / I-3: derive ``conversation_file_touches`` rows from
+    ``conversation_messages.blocks_json`` for the in-scope message rows.
+
+    ``scope`` is an iterable of ``(source_path, byte_offset)`` physical keys, or
+    ``None`` for ALL rows (the backfill). We read FROM ``conversation_messages``
+    (the source of truth) and resolve ``message_id`` from the row's own ``id``.
+
+    P1-3 (load-bearing): scope by the PHYSICAL key ``(source_path, byte_offset)``,
+    NEVER by ``uuid``. ``conversation_messages.uuid`` is NOT unique (only
+    ``(source_path, byte_offset)`` is; the uuid index is ``(session_id, uuid)``),
+    and resume/replay rows legitimately share a ``(session_id, uuid)`` — a
+    ``WHERE uuid=?`` fill would touch unrelated physical rows.
+
+    Decoupled from the message-insert rowcount ("dedup must not gate side
+    effects"): a no-op INSERT OR IGNORE of an already-present message row (rowcount
+    0) still has its touches derived here, because we read the row by physical key
+    rather than from the insert's lastrowid/rowcount.
+
+    Cheap at steady state: scoped to the rows ingested this tick (proportional to
+    new bytes), never re-parsing the whole session per tick. ``INSERT OR IGNORE``
+    on ``UNIQUE(message_id, file_path, tool)`` makes it idempotent, and a row's
+    ``blocks_json`` is immutable, so accumulate-via-IGNORE needs no per-tick DELETE.
+    The caller owns the commit (this helper never commits)."""
+    def _emit(rows):
+        for mid, sid, uuid_, bj in rows:
+            if not sid:
+                continue   # a touch row's session_id is NOT NULL; skip null-session rows
+            try:
+                blocks = json.loads(bj) if bj else []
+            except (TypeError, ValueError):
+                blocks = []
+            for fp, tool in _lib_conversation._derive_file_touches(blocks):
+                conn.execute(_FILE_TOUCH_INSERT_SQL, (mid, sid, uuid_, fp, tool))
+
+    if scope is None:
+        # Backfill: cursor-resumable 500-row batches keyed on the message rowid.
+        # Resume from the stored cursor so an interrupt skips already-derived
+        # batches (the fill is also idempotent via INSERT OR IGNORE, so a restart
+        # from 0 would be correct but redundant).
+        row = conn.execute(
+            "SELECT value FROM cache_meta "
+            "WHERE key='conversation_file_touches_cursor'").fetchone()
+        last_id = int(row[0]) if row and row[0] is not None else 0
+        while True:
+            batch = conn.execute(
+                "SELECT id, session_id, uuid, blocks_json FROM conversation_messages "
+                "WHERE id > ? ORDER BY id LIMIT 500",
+                (last_id,)).fetchall()
+            if not batch:
+                break
+            _emit(batch)
+            last_id = batch[-1][0]
+            _cctally_db_sib._set_cache_meta(
+                conn, "conversation_file_touches_cursor", str(last_id))
+            conn.commit()
+        return
+    for sp, off in scope:
+        rows = conn.execute(
+            "SELECT id, session_id, uuid, blocks_json FROM conversation_messages "
+            "WHERE source_path=? AND byte_offset=?", (sp, off)).fetchall()
+        _emit(rows)
+
+
+def _consume_file_touches(conn) -> None:
+    """#217 S2 / I-3: flock-held consumer for
+    ``conversation_reingest_file_touches_pending`` (set by cache migration 019).
+    Derives ``conversation_file_touches`` from ALL existing ``blocks_json`` history
+    via ``_fill_file_touches(conn, scope=None)`` (cursor-resumable, 500-row
+    batches), then clears the flag + cursor.
+
+    Touches ONLY ``conversation_file_touches`` (never ``conversation_messages`` —
+    P1-2: this is NOT a message reingest). The fill is idempotent (INSERT OR
+    IGNORE on the UNIQUE key), so an interrupted backfill resumes cleanly on the
+    next locked sync. A fresh install never sets the flag, so this is a cheap
+    no-op there."""
+    if conn.execute(
+        "SELECT 1 FROM cache_meta "
+        "WHERE key='conversation_reingest_file_touches_pending'"
+    ).fetchone() is None:
+        return
+    _fill_file_touches(conn, scope=None)
+    conn.execute(
+        "DELETE FROM cache_meta WHERE key IN "
+        "('conversation_reingest_file_touches_pending',"
+        " 'conversation_file_touches_cursor')")
+    conn.commit()
+
+
 def _consume_promote_command_args(conn) -> None:
     """#188 bug 4: flock-held consumer for ``conversation_promote_command_args_pending``
     (set by cache migration 011). Cursor-resumable walk of

package/bin/_cctally_dashboard.py

@@ -147,7 +147,7 @@ What stays in bin/cctally:
   c.ORIGINAL_SYS_ARGV = list(sys.argv)``.
 - ``eprint``, ``now_utc_iso``, ``parse_iso_datetime``, ``open_db``,
   ``load_config``, ``save_config``, ``_now_utc``, ``_command_as_of``,
-  ``format_display_dt``, ``resolve_display_tz``,
+  ``format_display_dt``,
   ``normalize_display_tz_value``, ``_render_migration_error_banner``,
   ``_aggregate_daily``, ``_aggregate_monthly``, ``_aggregate_weekly``,
   ``_calculate_entry_cost``, ``_canonical_5h_window_key``,
@@ -305,7 +305,6 @@ from _cctally_core import (
 )
 from _lib_display_tz import (
     format_display_dt,
-    resolve_display_tz,
     normalize_display_tz_value,
     _compute_display_block,
 )
@@ -5284,10 +5283,22 @@ def _qs_str(q: dict, key: str, default: str | None) -> str | None:
     return vals[0] if vals else default
 
 
-# #177 S6: valid kind facets for the conversation search / find routes. Kept in
-# lockstep with ``_lib_conversation_query._SEARCH_KINDS`` (the kernel re-raises
-# ValueError on an unknown kind; the handlers reject with a 400 before the call).
-_CONV_SEARCH_KINDS = ("all", "prompts", "assistant", "tools", "thinking")
+# #177 S6 / #217 S2: valid kind facets for the conversation routes. Kept in
+# lockstep with the kernel (``_lib_conversation_query._SEARCH_KINDS`` /
+# ``_FIND_KINDS``; the kernel re-raises ValueError on an unknown kind, and the
+# handlers reject with a 400 BEFORE the call — ``_run_conversation_query``
+# collapses every kernel exception to a 500, so a per-route 4xx must be decided
+# in the handler, not via try/except around the kernel).
+#
+# P1-1 (load-bearing kind-validation SPLIT): the cross-session search route
+# accepts ``title`` and ``files``; the in-conversation ``/find`` route does NOT —
+# its kernel (``find_in_conversation``) indexes ``_FIND_KIND_COLUMNS[kind]``,
+# which has no ``title``/``files`` entry, so accepting them there would be a 500
+# KeyError. Two distinct tuples keep ``/find?kind=title`` and ``/find?kind=files``
+# a clean 400.
+_CONV_SEARCH_KINDS = (
+    "all", "prompts", "assistant", "tools", "thinking", "title", "files")
+_CONV_FIND_KINDS = ("all", "prompts", "assistant", "tools", "thinking")
 
 
 class _BadConversationFilter(Exception):
@@ -7451,15 +7462,21 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
         """Lazy-load the pure conversation query kernel (Plan 2, §3)."""
         return sys.modules["cctally"]._load_sibling("_lib_conversation_query")
 
-    def _parse_search_kind(self, q):
-        """Read + validate the ``kind`` facet shared by the conversation search
-        and find routes (#177 S6). Returns the kind on success, or ``None`` after
-        having ALREADY sent a 400 — callers just ``return`` on ``None``. Kept in
-        lockstep with the kernel's ``_SEARCH_KINDS`` via ``_CONV_SEARCH_KINDS``
-        (the kernel module is resolved lazily per-request, so the handler keeps a
-        literal tuple rather than reaching across that import edge for a nit)."""
+    def _parse_search_kind(self, q, valid=_CONV_SEARCH_KINDS):
+        """Read + validate the ``kind`` facet for a conversation route (#177 S6 /
+        #217 S2). Returns the kind on success, or ``None`` after having ALREADY
+        sent a 400 — callers just ``return`` on ``None``.
+
+        ``valid`` is the per-route kind set (P1-1 split): the cross-session search
+        route passes ``_CONV_SEARCH_KINDS`` (includes ``title``), the
+        in-conversation ``/find`` route passes ``_CONV_FIND_KINDS`` (excludes
+        ``title``/``files``), so ``/find?kind=title`` is a 400 here — never a 500
+        KeyError downstream in ``find_in_conversation``. Kept in lockstep with the
+        kernel's ``_SEARCH_KINDS`` / ``_FIND_KINDS`` (the kernel module is
+        resolved lazily per-request, so the handler keeps literal tuples rather
+        than reaching across that import edge for a nit)."""
         kind = _qs_str(q, "kind", "all")
-        if kind not in _CONV_SEARCH_KINDS:
+        if kind not in valid:
             self._respond_json(400, {"error": f"unknown kind: {kind}"})
             return None
         return kind
@@ -7615,15 +7632,19 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
         """``GET /api/conversation/<session-id>`` — the reader (spec §3.2).
 
         The id is percent-decoded so clients that encode reserved chars
-        round-trip. Unknown id → 404. ``after``/``limit`` page the items.
+        round-trip. Unknown id → 404. ``after``/``before``/``tail``/``limit``
+        page the items; ``after``/``before``/``tail`` are mutually exclusive
+        (>1 supplied → 400). ``tail=1`` opens at the bottom; ``before=<id>``
+        pages backward (#217 S2 / U4).
         """
         if not self._require_transcripts_allowed():
             return
         import urllib.parse as _u
         # ``path`` is already query-stripped by ``do_GET`` (``self.path.split("?")``),
-        # so the cursor params (?after=/?limit=) live ONLY on the raw ``self.path``.
-        # Sibling handlers read ``self.path`` directly — the detail route must too,
-        # or every request re-serves the head and pagination is dead.
+        # so the cursor params (?after=/?before=/?tail=/?limit=) live ONLY on the
+        # raw ``self.path``. Sibling handlers read ``self.path`` directly — the
+        # detail route must too, or every request re-serves the head and
+        # pagination is dead.
         query_str = self.path.partition("?")[2]
         session_id = _u.unquote(path[len("/api/conversation/"):])
         if not session_id:
@@ -7631,10 +7652,21 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
             return
         q = _u.parse_qs(query_str)
         after = _qs_str(q, "after", None)
+        before = _qs_str(q, "before", None)
+        tail = _qs_str(q, "tail", None) in ("1", "true", "yes")
         limit = _qs_int(q, "limit", 500)
+        # Mutual-exclusion 400 (#217 S2 / U4). The kernel ALSO raises ValueError
+        # on >1 cursor as its own invariant, but ``_run_conversation_query``
+        # collapses every kernel exception to a 500, so the 400 must be decided
+        # HERE, before the kernel call — this explicit pre-call check is the
+        # authoritative backstop for the handler path.
+        if sum(1 for x in (after is not None, before is not None, tail) if x) > 1:
+            self.send_error(400, "after/before/tail are mutually exclusive")
+            return
         ok, body = self._run_conversation_query(
             lambda conn: self._conversation_query().get_conversation(
-                conn, session_id, after=after, limit=limit),
+                conn, session_id, after=after, before=before, tail=tail,
+                limit=limit),
             "/api/conversation")
         if not ok:
             return
@@ -7755,6 +7787,13 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
         FTS/LIKE search (spec §3.3). Matched BEFORE the ``<id>`` reader in
         ``do_GET``. ``kind`` (#177 S6) is validated to ``_CONV_SEARCH_KINDS``
         (else 400) before the kernel call.
+
+        #217 S2 / Filtered-search: the browse filters (date/project/cost/rebuild)
+        are parsed by the SAME ``_parse_conversation_filters`` the browse rail uses
+        (malformed → 400 already sent) and threaded into the kernel, applied as a
+        session-scope restriction across every kind. The 400s (bad kind, bad
+        filter) are decided HERE, before the kernel call — ``_run_conversation_query``
+        collapses kernel exceptions to a 500.
         """
         if not self._require_transcripts_allowed():
             return
@@ -7766,9 +7805,12 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
         kind = self._parse_search_kind(q)
         if kind is None:
             return
+        filters = self._parse_conversation_filters(q)
+        if filters is None:
+            return  # a 400 has already been sent
         ok, body = self._run_conversation_query(
             lambda conn: self._conversation_query().search_conversations(
-                conn, query, limit=limit, offset=offset, kind=kind),
+                conn, query, limit=limit, offset=offset, kind=kind, **filters),
             "/api/conversation/search")
         if not ok:
             return
@@ -7840,9 +7882,20 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
         find → document-ordered rendered-turn anchors (#177 S6). Same fail-closed
         privacy gate as the sibling routes; unknown id → 404; an invalid ``kind``
         → 400. Matched BEFORE the ``<id>`` reader catch-all in ``do_GET``.
+
+        P1-1: validates against ``_CONV_FIND_KINDS`` (NOT the search set), so the
+        cross-session-only ``kind=title``/``files`` return 400 here, never a 500.
+
+        #217 S4 / I-1.2: ``regex``/``case`` are truthy params. An invalid regex
+        is PRE-COMPILED here, BEFORE dispatching to the kernel — exactly as the
+        detail route pre-validates ``after/before/tail`` — because
+        ``_run_conversation_query`` collapses every kernel exception to a 500, so
+        a ``re.error`` from the kernel's ``re.compile`` would otherwise leak as a
+        500 instead of the actionable 400 the client maps to "invalid regex".
         """
         if not self._require_transcripts_allowed():
             return
+        import re as _re
         import urllib.parse as _u
         session_id = _u.unquote(path[len("/api/conversation/"):-len("/find")])
         if not session_id:
@@ -7850,12 +7903,23 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
             return
         q = _u.parse_qs(self.path.partition("?")[2])
         query = _qs_str(q, "q", "")
-        kind = self._parse_search_kind(q)
+        kind = self._parse_search_kind(q, valid=_CONV_FIND_KINDS)
         if kind is None:
             return
+        regex = _qs_str(q, "regex", None) in ("1", "true", "yes")
+        case = _qs_str(q, "case", None) in ("1", "true", "yes")
+        # Pre-validate the regex HERE (Codex P1): the kernel compiles the same
+        # pattern, but its ``re.error`` would be swallowed into the generic 500
+        # envelope below. Compiling first turns a bad pattern into a clean 400.
+        if regex:
+            try:
+                _re.compile(query, 0 if case else _re.IGNORECASE)
+            except _re.error as e:
+                self._respond_json(400, {"error": f"invalid regex: {e}"})
+                return
         ok, body = self._run_conversation_query(
             lambda conn: self._conversation_query().find_in_conversation(
-                conn, session_id, query, kind=kind),
+                conn, session_id, query, kind=kind, regex=regex, case=case),
             "/api/conversation/find")
         if not ok:
             return
@@ -8472,11 +8536,12 @@ def cmd_dashboard(args: argparse.Namespace) -> int:
     _c_boot.ORIGINAL_ENTRYPOINT = shutil.which("cctally")
     _c_boot._UPDATE_WORKER = UpdateWorker()
 
-    # Resolve display tz (--tz overrides config.display.tz). The dashboard's
-    # envelope-emitted display block (Tasks 11-12) reads this; for now,
-    # stash on args so downstream selectors can pick it up uniformly.
+    # Load config for the bind-host + expose-transcripts resolution below. (#217
+    # S1 / U7b: dropped the dead ``args._resolved_tz = resolve_display_tz(...)``
+    # write — nothing in the dashboard ever read it back; the envelope display
+    # block it was speculatively staged for was never wired. Each report
+    # subcommand resolves + reads its own ``args._resolved_tz`` independently.)
     config = load_config()
-    args._resolved_tz = resolve_display_tz(args, config)
 
     # Resolve bind host: --host flag > config.dashboard.bind > argparse default.
     # `--host` defaults to None (Task 2) so we can distinguish "user explicitly