cctally 1.47.0 → 1.48.0

package/CHANGELOG.md

@@ -5,6 +5,16 @@ based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+## [1.48.0] - 2026-06-16
+
+### Fixed
+- **Dashboard conversation viewer — Claude Code's new subagent transcript format:** restored correct rendering after Claude Code moved subagent transcripts to `subagents/agent-*.jsonl` with `Agent`-renamed spawns and null cross-file parent links. Background (async) subagent cards now show completion (✓) with token/duration/tool counts — derived from the subagent's own thread and marked with a `~` when Claude Code provided none; nested subagents render as recursive nested cards, each shown exactly once with the spawning tool chip merged in (a grandchild no longer renders as a detached title-only card plus a stray `Agent` chip); and a skill invoked inside a live-watched subagent now appears without a manual page refresh (the live-tail re-fetches a small overlap window and upserts folded-in content). All read-time over the existing cache — no migration, no re-ingest.
+- **Dashboard conversation viewer — a message you typed while the agent was still working now renders.** When you send a message while the main agent (or a subagent) is busy, Claude Code queues it and persists it as a `queued_command` *attachment* — with the text in `attachment.prompt` rather than a normal `user` turn — so the transcript reader silently dropped it: the prompt was right there in the session's JSONL but never appeared in the rendered conversation. The reader now ingests those queued prompts (`commandMode: "prompt"`) as ordinary "you" turns in their chronological place, while harness-injected queued items such as background `task-notification`s stay excluded. New conversations show queued messages immediately; existing history is re-derived once by a flag-only cache migration (`014_conversation_queued_prompt_reingest`) — the cache is fully re-derivable, so there is nothing to do on upgrade.
+- **Dashboard conversation viewer — jumping to a deeply-nested subagent now lands the card aligned to the top.** Clicking an outline-rail entry for a *grandchild* subagent (a thread nested inside another subagent card) scrolled the target to the *center* of the viewport — fine for an ordinary message, but a subagent card is often much taller than the screen, so centering it pushed its head well above the fold (roughly 250px up, and more on a large window). A subagent-card jump now aligns the card's head to the top of the viewport instead of centering it (and re-aims on the next animation frame once the force-opened ancestor cards have settled), so a deeply-nested target lands head-first at the top (#204).
+
+### Security
+- **Internal (build/test-time only, no user-facing change): bumped the dashboard's dev dependencies to clear three Dependabot advisories** — `vite` 8.0.10 → 8.0.16 (fixes a `server.fs.deny` bypass and an NTLMv2 hash disclosure, both Windows + dev-server only) and the test-only transitive `form-data` 4.0.5 → 4.0.6 (CRLF injection via unescaped multipart field names). None of these packages ship to installed users — the npm/brew artifacts carry only the Python CLI plus the pre-built `dashboard/static/` bundle, never `node_modules` — so there is no runtime exposure; the bundle was rebuilt against the new `vite` and all dashboard tests pass.
+
 ## [1.47.0] - 2026-06-16
 
 ### Added

package/bin/_cctally_cache.py

@@ -594,6 +594,7 @@ _TARGETED_DECLINE_FLAGS = (
     "conversation_search_split_pending",
     "conversation_promote_command_args_pending",
     "conversation_sessions_backfill_pending",
+    "conversation_queued_prompt_reingest_pending",   # migration 014
 )
 
 
@@ -725,12 +726,19 @@ def sync_cache(
             # media[] placeholders + user-content media index + web_search/
             # web_fetch captures); the same offset-0 walk re-derives them, so drop
             # that flag here as well.
+            # Migration 014 sets the DISTINCT
+            # conversation_queued_prompt_reingest_pending flag (to land queued-
+            # while-busy user prompts persisted as queued_command attachments); the
+            # same offset-0 walk re-derives them through the current parser, so drop
+            # that flag here too — MISSING this site re-arms the flag on every
+            # cache-sync --rebuild.
             conn.execute(
                 "DELETE FROM cache_meta WHERE key IN "
                 "('conversation_reingest_pending',"
                 " 'conversation_source_tool_use_reingest_pending',"
                 " 'conversation_reingest_enrichment_pending',"
                 " 'conversation_media_reingest_pending',"
+                " 'conversation_queued_prompt_reingest_pending',"
                 " 'conversation_reingest_cursor',"
                 " 'conversation_reingest_cursor_gen')")
             # #177 S6: a rebuild repopulates search_tool/search_thinking via the
@@ -874,16 +882,20 @@ def sync_cache(
             # ANOTHER distinct flag ``conversation_media_reingest_pending`` to land
             # the tool_result media[] placeholders + user-content media index +
             # web_search/web_fetch captures; same offset-0 re-parse, same reason
-            # for a distinct flag. We trigger the SAME clear + offset-0 backfill on
-            # ANY of these flags and clear them ALL atomically here under the held
-            # flock.
+            # for a distinct flag. Migration 014 uses ANOTHER distinct flag
+            # ``conversation_queued_prompt_reingest_pending`` to land queued-while-
+            # busy user prompts (queued_command attachments the parser now promotes
+            # to HUMAN); same offset-0 re-parse, same distinct-flag reason. We
+            # trigger the SAME clear + offset-0 backfill on ANY of these flags and
+            # clear them ALL atomically here under the held flock.
             try:
                 _reingest = conn.execute(
                     "SELECT 1 FROM cache_meta WHERE key IN "
                     "('conversation_reingest_pending',"
                     " 'conversation_source_tool_use_reingest_pending',"
                     " 'conversation_reingest_enrichment_pending',"
-                    " 'conversation_media_reingest_pending')"
+                    " 'conversation_media_reingest_pending',"
+                    " 'conversation_queued_prompt_reingest_pending')"
                 ).fetchone() is not None
             except sqlite3.OperationalError:
                 _reingest = False
@@ -1473,6 +1485,7 @@ _REINGEST_FLAG_KEYS = (
     "conversation_source_tool_use_reingest_pending",
     "conversation_reingest_enrichment_pending",
     "conversation_media_reingest_pending",   # #177 S4 (migration 009)
+    "conversation_queued_prompt_reingest_pending",   # migration 014
 )
 
 
@@ -1554,6 +1567,7 @@ def _resumable_reingest_conversation_messages(conn):
         " 'conversation_source_tool_use_reingest_pending',"
         " 'conversation_reingest_enrichment_pending',"
         " 'conversation_media_reingest_pending',"
+        " 'conversation_queued_prompt_reingest_pending',"
         " 'conversation_reingest_cursor',"
         " 'conversation_reingest_cursor_gen')")
     conn.commit()

package/bin/_cctally_dashboard.py

@@ -7594,6 +7594,20 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
                              + "\n\n").encode("utf-8"))
                         self.wfile.flush()
                         idle = 0.0
+                        # §6 P2-H — a brand-new subagent file's FIRST content was
+                        # just ingested by this emitting cycle, so the session's
+                        # source-path set may have grown. Re-resolve it now (vs
+                        # waiting up to _LIVE_TAIL_FILE_RESET_EVERY cycles) so the
+                        # new thread (incl. a skill invoked inside it) live-tails
+                        # promptly. A new path seeds seen=None (cur lacks a row),
+                        # so changed_paths flags it next cycle → it ingests + emits.
+                        # setdefault never disturbs an existing cursor.
+                        new_files = _resolve()
+                        if set(new_files) != set(files):
+                            files = new_files
+                            cur = _cached_file_sigs(conn, files)
+                            for p in files:
+                                seen.setdefault(p, cur.get(p))
                         continue
                 idle += _LIVE_TAIL_POLL_INTERVAL
                 if idle >= _LIVE_TAIL_KEEPALIVE:

package/bin/_cctally_db.py

@@ -3442,6 +3442,27 @@ def _013_create_conversation_sessions(conn: sqlite3.Connection) -> None:
     conn.commit()
 
 
+@cache_migration("014_conversation_queued_prompt_reingest")
+def _014_conversation_queued_prompt_reingest(conn: sqlite3.Connection) -> None:
+    """Flag-only re-ingest so a message typed while the agent was busy — QUEUED
+    and persisted as an ``attachment`` row (``attachment.type=="queued_command"``,
+    ``commandMode=="prompt"``) rather than a ``type:"user"`` turn — lands on
+    existing history. The parser (``_queued_prompt_row``) now promotes those to a
+    synthetic HUMAN turn at ingest, so the offset-0 re-parse re-derives them with
+    zero new consumption code. Sets the DISTINCT
+    ``conversation_queued_prompt_reingest_pending`` flag (NOT the shared
+    ``conversation_reingest_pending``, which also gates migration 005's read-time
+    human-fallback in the query kernel). Consumption rides the #179 RESUMABLE
+    per-file reingest (_resumable_reingest_conversation_messages) — the flag is
+    wired into _TARGETED_DECLINE_FLAGS + _REINGEST_FLAG_KEYS + both flag SELECTs +
+    the two cleanup DELETE lists in _cctally_cache.py (all five sites; missing one
+    either never triggers or re-arms forever). Central stamp via the dispatcher
+    (#140); a fresh install stamps it WITHOUT running (empty table -> the flag, if
+    ever set, is a harmless no-op). Mirrors 007/009."""
+    _set_cache_meta(conn, "conversation_queued_prompt_reingest_pending", "1")
+    conn.commit()
+
+
 # === Region 7d: Stats migration 008_recompute_weekly_cost_snapshots_dedup_fix ===
 
 @stats_migration("008_recompute_weekly_cost_snapshots_dedup_fix")

package/bin/_lib_conversation.py

@@ -263,12 +263,51 @@ def parse_message_row(obj, offset):
     populated by a separate second seek-and-walk over the same byte span."""
     t = obj.get("type")
     if t not in ("user", "assistant"):
-        return None
+        # A message typed while the agent (the main session OR a subagent) is
+        # still working is QUEUED and persisted as an ``attachment`` row, never a
+        # ``type:"user"`` turn — so the user/assistant gate above would drop it and
+        # it would never reach conversation_messages (the reader bug). Promote the
+        # user-typed ones here; everything else stays dropped.
+        return _queued_prompt_row(obj, t, offset)
     if not obj.get("uuid"):
         return None
     return _normalize(obj, t, offset)
 
 
+def _queued_prompt_row(obj, t, offset):
+    """A queued user prompt -> a synthetic HUMAN ``MessageRow``, else ``None``.
+
+    Claude Code persists a message typed while the agent is busy as
+    ``{"type":"attachment","attachment":{"type":"queued_command",
+    "commandMode":"prompt","prompt":<text>}}`` — carrying its OWN
+    uuid/parentUuid/timestamp, with the text in ``attachment.prompt`` rather than
+    ``message.content``. Only ``commandMode=="prompt"`` is promoted: a queued
+    ``task-notification`` (``commandMode=="task-notification"``) is harness-injected
+    background plumbing — the same ``<task-notification>`` content already
+    classifies META when it arrives as a regular line — not something the user
+    typed, so it stays dropped."""
+    if t != "attachment" or not obj.get("uuid"):
+        return None
+    att = obj.get("attachment")
+    if not isinstance(att, dict) or att.get("type") != "queued_command":
+        return None
+    if att.get("commandMode") != "prompt":
+        return None
+    prompt = att.get("prompt")
+    if not isinstance(prompt, str) or not prompt.strip():
+        return None
+    # Route a synthesized user message through _normalize so the queued prompt
+    # gets the SAME classification a typed turn would (HUMAN, the #188
+    # slash-command-args promotion, system-marker / notification folding, the
+    # remote-control prefix strip, and the split search-column derivation). Every
+    # top-level field (uuid / parentUuid / sessionId / timestamp / cwd / gitBranch
+    # / isSidechain) rides along on the shallow copy; _normalize keys off the ``t``
+    # arg, never ``obj["type"]``, so the "attachment" type is inert here.
+    synth = dict(obj)
+    synth["message"] = {"role": "user", "content": prompt}
+    return _normalize(synth, "user", offset)
+
+
 @dataclass
 class AiTitleRow:
     """Pure per-line AI-title record (no I/O). Parallels MessageRow but for the

package/bin/_lib_conversation_query.py

@@ -16,7 +16,7 @@ import json as _json
 import os
 import re
 import sqlite3
-from datetime import datetime as _datetime
+from datetime import datetime as _datetime, timezone as _timezone
 
 # Public surface (Plan 2): shipped in the npm tarball + brew formula + public
 # mirror — imported by the dashboard's conversation endpoints at runtime.
@@ -272,6 +272,55 @@ def _subagent_key(source_path):
     return stem or None
 
 
+# §4 1a — nested (grandchild) subagent result parse. The new Claude Code format
+# emits a grandchild's spawn result as STRING content (no structured
+# `agent_id`/`subagent_meta`), with a trailing `agentId: <hash> (use SendMessage
+# …)` line and an OPTIONAL `<usage>` totals wrapper. Anchored on the literal
+# `<usage>` wrapper; tolerant of whitespace/newline variants (re.DOTALL).
+_NESTED_AGENT_ID_RE = re.compile(r"agentId:\s*([0-9a-f]+)")
+_NESTED_USAGE_RE = re.compile(
+    r"<usage>\s*subagent_tokens:\s*(\d+)\s*tool_uses:\s*(\d+)\s*duration_ms:\s*(\d+)\s*</usage>",
+    re.DOTALL,
+)
+
+
+def _parse_nested_agent_result(text):
+    """Nested (grandchild) subagent result: string-content, no structured
+    `agentId` (spec §4 1a). Parse `agentId` (+ OPTIONAL `<usage>` totals) out of
+    the result text. Returns (agent_id, meta) — meta is {} when `<usage>` is
+    absent/clipped past the 16 KB cap, populated with completed totals when
+    present. Returns None when no `agentId` is present (an ordinary result, or a
+    >16 KB clip past the `agentId:` line itself — degrades to a flat card, no
+    mis-link). Linking on `agentId` ALONE is sufficient (Codex P1-B)."""
+    if not text:
+        return None
+    m = _NESTED_AGENT_ID_RE.search(text)
+    if not m:
+        return None
+    meta = {}
+    u = _NESTED_USAGE_RE.search(text)
+    if u:
+        meta["total_tokens"] = int(u.group(1))
+        meta["total_tool_use_count"] = int(u.group(2))
+        meta["total_duration_ms"] = int(u.group(3))
+        meta["status"] = "completed"
+    return m.group(1), meta
+
+
+def _iso_ms(ts):
+    """Epoch milliseconds for an ISO-8601 timestamp string (tolerant of a
+    trailing 'Z'); None on parse failure. Used by §4 1c derived-duration."""
+    if not ts:
+        return None
+    try:
+        dt = _datetime.fromisoformat(ts.replace("Z", "+00:00"))
+        if dt.tzinfo is None:
+            dt = dt.replace(tzinfo=_timezone.utc)
+        return int(dt.timestamp() * 1000)
+    except (ValueError, TypeError):
+        return None
+
+
 def _entry_cost(model, inp, out, cc, cr, cost_usd_raw) -> float:
     """Cost for one session_entries row via the shared pricing helper. Tokens →
     the helper's usage dict. cost_usd_raw is passed as the optional override the
@@ -758,6 +807,9 @@ def _assemble_session(conn, session_id):
     spawn_kind = {}     # tool_use id -> subagent_type
     spawn_desc = {}     # tool_use id -> spawning Task description (#193)
     agent_link = {}     # tool_use id -> (agent_id, raw_meta)
+    nested_candidates = []  # §4 1a: (tool_use_id, block) — string-content spawn
+                            # results with no structured agent_id, resolved after
+                            # spawn_kind is complete (text still present pre-fold)
     ask_link = {}       # tool_use id -> (answers, annotations)  (#177 S2)
     bash_link = {}      # tool_use id -> (stderr, interrupted)   (#177 S3)
     web_search_link = {}  # tool_use id -> web_search payload    (#177 S4)
@@ -782,6 +834,11 @@ def _assemble_session(conn, session_id):
                 meta = b.pop("subagent_meta", None)
                 if aid and b.get("tool_use_id") is not None:
                     agent_link[b["tool_use_id"]] = (aid, meta or {})
+                elif b.get("tool_use_id") is not None:
+                    # §4 1a candidate — a tool_result with no structured agentId.
+                    # Resolved AFTER spawn_kind is complete (text still present,
+                    # pre-Phase-2-fold). The block's `text` holds the result body.
+                    nested_candidates.append((b["tool_use_id"], b))
                 ans = b.pop("ask_answers", None)            # #177 S2
                 anno = b.pop("ask_annotations", None)
                 if ans is not None and b.get("tool_use_id") is not None:
@@ -800,6 +857,14 @@ def _assemble_session(conn, session_id):
                 tlist_ = b.pop("task_list", None)
                 if b.get("tool_use_id") is not None and (tid_ is not None or tlist_ is not None):
                     task_link[b["tool_use_id"]] = {"task_id": tid_, "task_list": tlist_}
+    # §4 1a — nested grandchild results: gate the text-parse on the owning
+    # tool_use being a spawn (in spawn_kind), so an ordinary tool result that
+    # merely mentions "agentId" never matches.
+    for _tuid, _block in nested_candidates:
+        if _tuid in spawn_kind and _tuid not in agent_link:
+            parsed = _parse_nested_agent_result(_block.get("text"))
+            if parsed is not None:
+                agent_link[_tuid] = parsed
     subagent_meta = {}
     for _tuid, _kind in spawn_kind.items():
         _link = agent_link.get(_tuid)
@@ -814,6 +879,24 @@ def _assemble_session(conn, session_id):
                 _entry[_f] = _raw[_f]
         subagent_meta[_aid] = _entry       # agent_id == subagent_key
 
+    # §4 1b — parent linkage: the item HOLDING each linked spawn is the child's
+    # parent thread + placement anchor. tooluse_index[spawn_id] = (item, block);
+    # the item's subagent_key/anchor.uuid were set at build time (available now,
+    # pre-Phase-3). Works for a main parent (subagent_key=None) AND a grandchild
+    # whose parent is a child subagent. spawn_tool_use_id is required because one
+    # assistant item can hold MORE than one spawn (Codex P1-C).
+    for _tuid, (_aid, _raw) in agent_link.items():
+        _entry = subagent_meta.get(_aid)
+        if _entry is None:
+            continue
+        _hit = tooluse_index.get(_tuid)
+        if _hit is None:
+            continue
+        _owner = _hit[0]
+        _entry["parent_subagent_key"] = _owner["subagent_key"]
+        _entry["spawn_uuid"] = _owner["anchor"]["uuid"]
+        _entry["spawn_tool_use_id"] = _tuid
+
     # ---- Phase 2: fold each tool_result item into its owning assistant item ----
     drop = set()                                 # id() of folded placeholder items
     for tr in tool_result_items:
@@ -987,6 +1070,66 @@ def _assemble_session(conn, session_id):
             it.pop("_has_prose", None)
     header_cost = round(header_cost, 6)
 
+    # §4 1c — async completion. A background subagent's launch result carries
+    # status:"async_launched" and NO totals; completion arrives as a separate
+    # <task-notification> meta row (text populated by Phase 4). Join it back, and
+    # derive any totals Claude Code never provided from the child's own thread.
+    _notif_status = {}                       # spawn tool_use_id -> status
+    for it in items:
+        if it["kind"] == "meta" and it.get("meta_kind") == "notification":
+            body = it.get("text") or ""
+            tu = re.search(r"<tool-use-id>([^<]+)</tool-use-id>", body)
+            stx = re.search(r"<status>([^<]+)</status>", body)
+            if tu and stx:
+                _notif_status[tu.group(1).strip()] = stx.group(1).strip()
+    for _tuid, _status in _notif_status.items():
+        _link = agent_link.get(_tuid)
+        if _link is not None and _link[0] in subagent_meta:
+            subagent_meta[_link[0]]["status"] = _status   # upgrades async_launched -> completed
+
+    # Derived totals: any child still missing a count gets it from its own
+    # subagent_key bucket. tool-count = tool_call/tool_use blocks; duration =
+    # (max_ts - min_ts) ms; tokens = sum of per-turn token totals (now stamped).
+    # Authoritative values always win; only missing keys are filled.
+    # Bucket items by subagent_key in ONE document-ordered pass so the loop
+    # below indexes in O(1) instead of re-walking all items per child.
+    _items_by_subagent_key: dict[str, list] = {}
+    for it in items:
+        _sk = it.get("subagent_key")
+        if _sk is not None:
+            _items_by_subagent_key.setdefault(_sk, []).append(it)
+    for _aid, _entry in subagent_meta.items():
+        if all(_entry.get(_f) is not None
+               for _f in ("total_tokens", "total_duration_ms", "total_tool_use_count")):
+            continue
+        _bucket = _items_by_subagent_key.get(_aid)
+        if not _bucket:
+            continue
+        _derived = False
+        if _entry.get("total_tool_use_count") is None:
+            _entry["total_tool_use_count"] = sum(
+                1 for it in _bucket for b in it["blocks"]
+                if b.get("kind") in ("tool_call", "tool_use"))
+            _derived = True
+        if _entry.get("total_duration_ms") is None:
+            _ms = [m for m in (_iso_ms(it.get("ts")) for it in _bucket) if m is not None]
+            if len(_ms) >= 2:
+                _entry["total_duration_ms"] = max(_ms) - min(_ms)
+                _derived = True
+        if _entry.get("total_tokens") is None:
+            _tok, _any = 0, False
+            for it in _bucket:
+                t = it.get("tokens")
+                if t:
+                    _any = True
+                    _tok += (t.get("input", 0) + t.get("output", 0)
+                             + t.get("cache_creation", 0) + t.get("cache_read", 0))
+            if _any:
+                _entry["total_tokens"] = _tok
+                _derived = True
+        if _derived:
+            _entry["totals_derived"] = True
+
     # Stamp cache-failure markers (spec §1) AFTER tokens are on each item and
     # while `items` is still document-ordered (the running-max walk is
     # order-dependent). Healthy turns get no key (absent, not zero). Shared