cctally 1.38.0 → 1.39.0

package/CHANGELOG.md

@@ -5,6 +5,19 @@ based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+## [1.39.0] - 2026-06-12
+
+### Added
+- Conversation viewer: MCP tool calls now render an action-first chip — the readable action (e.g. `browser_take_screenshot`) leads, with a quiet pill for the friendly server name and a per-server icon (playwright, chrome, computer-use, codex each get a dedicated glyph; any other server gets a generic plug icon and its raw name), instead of the full namespaced `mcp__…` name on a generic box chip; the original name stays in the chip tooltip and expanded request panel (#177 S4).
+- Conversation viewer: WebFetch and WebSearch now render as semantic source cards instead of raw JSON tool chips — WebFetch shows the fetched domain, an HTTP status chip, labeled url/prompt fields, and the Markdown summary (clamped with a show-full reveal); WebSearch shows the quoted query, a result-count chip, and a clickable list of result titles and domains — with only `http`/`https` links made clickable and no outbound requests beyond a link you click yourself (#177 S4).
+- Conversation viewer: images now render inline — a lazy-loaded figure with an `open full size ↗` link and a size/type caption, including screenshots returned inside MCP tool results that were previously dropped at ingest — served on demand by a new privacy-gated `GET /api/conversation/<id>/media` route that re-reads the original session JSONL and decodes the bytes without ever copying pixel data into the cache; PDF documents get an upgraded badge with an `open ↗` link that opens the file in a new tab, and any load failure degrades gracefully back to a byte-count badge (#177 S4).
+- Conversation viewer: a one-time background reingest (cache migration 009) backfills the new MCP/web/media rendering onto your existing transcript history on the first `cache-sync` or dashboard sync after upgrade — resumable, lossless, and re-derivable; until it completes, older rows simply keep their pre-upgrade rendering, and `cache-sync --rebuild` forces it through in one pass (#177 S4).
+
+## [1.38.1] - 2026-06-12
+
+### Fixed
+- Dashboard: stop the steady-state one-core CPU peg on large caches. The background sync thread rebuilds the full snapshot every few seconds, and several panels read wide historical ranges that re-parsed the per-row `usage_extra_json` JSON blob on every tick — on a ~260K-entry cache a single rebuild cost roughly a core-second, so rebuilds ran back-to-back and one core stayed pinned. The only non-token field any reader consumes (`speed`, for the `<model>-fast` label) is now materialized into a real `session_entries.speed` column, so the hot cache read paths reconstruct usage with zero JSON parsing; a one-time cache migration backfills the column from the existing blobs, and new ingests write the column directly (the now-unused blob is written `NULL` and reclaimed on the next `cache-sync --rebuild`). Output is byte-identical everywhere `speed` matters; on a 250K-row synthetic cache the wide read dropped from ~1.6s to ~0.35s per call (#181).
+
 ## [1.38.0] - 2026-06-12
 
 ### Added

package/bin/_cctally_cache.py

@@ -638,12 +638,17 @@ def sync_cache(
             # input / full_length / stop_reason / attribution / search_aux); the
             # same offset-0 walk re-derives those through the enriched parser, so
             # drop that flag here too — MISSING this site re-arms the flag on
-            # every cache-sync --rebuild.
+            # every cache-sync --rebuild. #177 S4 migration 009 sets the DISTINCT
+            # conversation_media_reingest_pending flag (to land tool_result
+            # media[] placeholders + user-content media index + web_search/
+            # web_fetch captures); the same offset-0 walk re-derives them, so drop
+            # that flag here as well.
             conn.execute(
                 "DELETE FROM cache_meta WHERE key IN "
                 "('conversation_reingest_pending',"
                 " 'conversation_source_tool_use_reingest_pending',"
                 " 'conversation_reingest_enrichment_pending',"
+                " 'conversation_media_reingest_pending',"
                 " 'conversation_reingest_cursor',"
                 " 'conversation_reingest_cursor_gen')")
             conn.commit()
@@ -712,15 +717,20 @@ def sync_cache(
             # input_truncated, the raised result cap + full_length, stop_reason /
             # attribution_skill / attribution_plugin, and the search_aux FTS-aux
             # blob); the offset-0 re-parse through the enriched parser lands them
-            # all with zero new consumption code. We trigger the SAME clear +
-            # offset-0 backfill on ANY of these flags and clear them ALL atomically
-            # here under the held flock.
+            # all with zero new consumption code. #177 S4 migration 009 uses yet
+            # ANOTHER distinct flag ``conversation_media_reingest_pending`` to land
+            # the tool_result media[] placeholders + user-content media index +
+            # web_search/web_fetch captures; same offset-0 re-parse, same reason
+            # for a distinct flag. We trigger the SAME clear + offset-0 backfill on
+            # ANY of these flags and clear them ALL atomically here under the held
+            # flock.
             try:
                 _reingest = conn.execute(
                     "SELECT 1 FROM cache_meta WHERE key IN "
                     "('conversation_reingest_pending',"
                     " 'conversation_source_tool_use_reingest_pending',"
-                    " 'conversation_reingest_enrichment_pending')"
+                    " 'conversation_reingest_enrichment_pending',"
+                    " 'conversation_media_reingest_pending')"
                 ).fetchone() is not None
             except sqlite3.OperationalError:
                 _reingest = False
@@ -928,14 +938,12 @@ def sync_cache(
                             out = int(usage.get("output_tokens", 0) or 0)
                             cc = int(usage.get("cache_creation_input_tokens", 0) or 0)
                             cr = int(usage.get("cache_read_input_tokens", 0) or 0)
-                            extras = {
-                                k: v for k, v in usage.items()
-                                if k not in (
-                                    "input_tokens", "output_tokens",
-                                    "cache_creation_input_tokens",
-                                    "cache_read_input_tokens",
-                                )
-                            }
+                            # #181: `speed` is the ONLY non-token usage key any
+                            # consumer reads, so materialize just that scalar and
+                            # write NULL into usage_extra_json — no more
+                            # serializing the deeply-nested blob the read paths
+                            # used to json.loads per row.
+                            speed = usage.get("speed")
                             rows.append((
                                 path_str,
                                 offset,
@@ -944,7 +952,8 @@ def sync_cache(
                                 msg_id,
                                 req_id,
                                 inp, out, cc, cr,
-                                json.dumps(extras, sort_keys=True) if extras else None,
+                                None,    # usage_extra_json — bloat no longer written (#181)
+                                speed,   # materialized speed column
                                 entry.cost_usd,
                             ))
                         if mrow is not None:
@@ -1005,8 +1014,8 @@ def sync_cache(
                            (source_path, line_offset, timestamp_utc, model,
                             msg_id, req_id, input_tokens, output_tokens,
                             cache_create_tokens, cache_read_tokens,
-                            usage_extra_json, cost_usd_raw)
-                           VALUES (?,?,?,?,?,?,?,?,?,?,?,?)
+                            usage_extra_json, speed, cost_usd_raw)
+                           VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)
                            ON CONFLICT(msg_id, req_id)
                            WHERE msg_id IS NOT NULL AND req_id IS NOT NULL
                            DO UPDATE SET
@@ -1017,6 +1026,7 @@ def sync_cache(
                                cache_create_tokens = excluded.cache_create_tokens,
                                cache_read_tokens = excluded.cache_read_tokens,
                                usage_extra_json = excluded.usage_extra_json,
+                               speed = excluded.speed,
                                cost_usd_raw = excluded.cost_usd_raw
                            WHERE
                                (excluded.input_tokens + excluded.output_tokens
@@ -1030,8 +1040,8 @@ def sync_cache(
                                =
                                (session_entries.input_tokens + session_entries.output_tokens
                                 + session_entries.cache_create_tokens + session_entries.cache_read_tokens)
-                               AND json_extract(excluded.usage_extra_json, '$.speed') IS NOT NULL
-                               AND json_extract(session_entries.usage_extra_json, '$.speed') IS NULL
+                               AND excluded.speed IS NOT NULL
+                               AND session_entries.speed IS NULL
                             )""",
                         rows,
                     )
@@ -1163,6 +1173,7 @@ _REINGEST_FLAG_KEYS = (
     "conversation_reingest_pending",
     "conversation_source_tool_use_reingest_pending",
     "conversation_reingest_enrichment_pending",
+    "conversation_media_reingest_pending",   # #177 S4 (migration 009)
 )
 
 
@@ -1189,8 +1200,8 @@ def _resumable_reingest_conversation_messages(conn):
     fingerprint (the set of pending reingest flags) resets the cursor whenever the
     pending-flag set changes, so a newly-armed flag forces a fresh pass. The caller
     (``sync_cache``) already holds the cache.db flock; per-file commits bound only
-    the SQLite write transaction, not the flock. Clears the three flags + cursor +
-    gen atomically on completion."""
+    the SQLite write transaction, not the flock. Clears all _REINGEST_FLAG_KEYS +
+    cursor + gen atomically on completion."""
     # 1. Generation guard: reset the cursor if the live pending-flag set differs.
     set_flags = [k for k in _REINGEST_FLAG_KEYS
                  if conn.execute("SELECT 1 FROM cache_meta WHERE key=?", (k,)).fetchone()]
@@ -1243,6 +1254,7 @@ def _resumable_reingest_conversation_messages(conn):
         "('conversation_reingest_pending',"
         " 'conversation_source_tool_use_reingest_pending',"
         " 'conversation_reingest_enrichment_pending',"
+        " 'conversation_media_reingest_pending',"
         " 'conversation_reingest_cursor',"
         " 'conversation_reingest_cursor_gen')")
     conn.commit()
@@ -1266,7 +1278,7 @@ def iter_entries(
 
     sql = (
         "SELECT timestamp_utc, model, input_tokens, output_tokens, "
-        "cache_create_tokens, cache_read_tokens, usage_extra_json, "
+        "cache_create_tokens, cache_read_tokens, speed, "
         "cost_usd_raw, source_path "
         "FROM session_entries "
         "WHERE timestamp_utc >= ? AND timestamp_utc <= ?"
@@ -1292,12 +1304,12 @@ def iter_entries(
             "cache_creation_input_tokens": row[4],
             "cache_read_input_tokens":     row[5],
         }
-        if row[6]:
-            # Safe because sync_cache strips the four token keys from
-            # extras before storing them in usage_extra_json. If that
-            # write-side invariant ever changes, extras could shadow
-            # the int-normalized token columns.
-            usage.update(json.loads(row[6]))
+        # speed is the only non-token usage key any consumer reads (#181);
+        # materialized into its own column so this hot path never parses JSON.
+        # `is not None` (not truthiness) so an empty-string speed still surfaces,
+        # mirroring the SQL `json_extract(...) IS NOT NULL` parity.
+        if row[6] is not None:
+            usage["speed"] = row[6]
         entries.append(UsageEntry(
             timestamp=dt.datetime.fromisoformat(row[0]),
             model=row[1],
@@ -1430,7 +1442,7 @@ def get_claude_session_entries(
         "  se.cache_create_tokens, se.cache_read_tokens, "
         "  se.source_path, "
         "  sf.session_id, sf.project_path, "
-        "  se.cost_usd_raw, se.usage_extra_json "
+        "  se.cost_usd_raw, se.speed "
         "FROM session_entries se "
         "LEFT JOIN session_files sf ON sf.path = se.source_path "
         "WHERE se.timestamp_utc >= ? AND se.timestamp_utc <= ?"
@@ -1458,7 +1470,10 @@ def get_claude_session_entries(
             session_id=row[7],
             project_path=row[8],
             cost_usd=row[9],
-            usage_extra=(json.loads(row[10]) if row[10] else None),
+            # speed materialized into its own column (#181); reconstruct the
+            # {"speed": …} shape _usage_entry_from_joined already merges, with
+            # zero JSON parsing. `is not None` so an empty-string speed surfaces.
+            usage_extra=({"speed": row[10]} if row[10] is not None else None),
         )
         for row in rows
     ]

package/bin/_cctally_dashboard.py

@@ -5345,6 +5345,9 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
             # #178: on-demand load-full. Matched BEFORE the <id> reader
             # catch-all (same precedence as /api/conversation/search).
             self._handle_get_conversation_payload(path)
+        elif path.startswith("/api/conversation/") and path.endswith("/media"):
+            # #177 S4: on-demand media bytes. Matched BEFORE the <id> reader.
+            self._handle_get_conversation_media(path)
         elif path.startswith("/api/conversation/"):
             self._handle_get_conversation_detail(path)
         else:
@@ -7388,6 +7391,95 @@ class DashboardHTTPHandler(BaseHTTPRequestHandler):
             return
         self._respond_json(200, payload)
 
+    _MEDIA_FETCH_SITE_ALLOWED = ("same-origin", "same-site", "none")
+
+    def _handle_get_conversation_media(self, path: str) -> None:
+        """``GET /api/conversation/<sid>/media?tool_use_id=<id>&index=N`` or
+        ``?uuid=<uuid>&index=N`` (#177 S4) — serves decoded image/PDF bytes by
+        re-reading the source JSONL line (the #178 mechanism). Nothing is ever
+        written to cache.db or disk; no outbound requests.
+
+        Gated FIRST by the transcript privacy predicate (fail-closed 403),
+        then by Fetch-Metadata: unlike the JSON routes, images embed
+        cross-origin (an <img src> on any website the user visits passes the
+        Host/loopback gate and leaks existence + dimensions via
+        onload/naturalWidth), so a PRESENT Sec-Fetch-Site header must be
+        same-origin/same-site/none; an absent header (curl, older browsers)
+        is allowed — defense-in-depth, not the primary gate (Codex F1).
+        Exactly one addressing key (tool_use_id XOR uuid) + a non-negative
+        integer index, else 400. Content-Type is the kernel's allowlist
+        constant; images get CSP default-src 'none'; PDFs get inline
+        Content-Disposition instead (a CSP sandbox would break native PDF
+        viewers)."""
+        if not self._require_transcripts_allowed():
+            return
+        sfs = (self.headers.get("Sec-Fetch-Site") or "").strip().lower()
+        if sfs and sfs not in self._MEDIA_FETCH_SITE_ALLOWED:
+            self._respond_json(403, {"error": "cross-site media fetch not allowed"})
+            return
+        import urllib.parse as _u
+        session_id = _u.unquote(path[len("/api/conversation/"):-len("/media")])
+        q = _u.parse_qs(self.path.partition("?")[2])
+        tool_use_id = _qs_str(q, "tool_use_id", "")
+        uuid = _qs_str(q, "uuid", "")
+        index_raw = _qs_str(q, "index", "")
+        if (not session_id or bool(tool_use_id) == bool(uuid)
+                or not index_raw.isdigit()):
+            self._respond_json(400, {"error": "bad request"})
+            return
+        index = int(index_raw)
+        key = ({"tool_use_id": tool_use_id} if tool_use_id else {"uuid": uuid})
+        cq = self._conversation_query()
+        ok, loc = self._run_conversation_query(
+            lambda conn: cq.locate_media(conn, session_id, index=index, **key),
+            "/api/conversation/media")
+        if not ok:
+            return
+        if loc is None:
+            self._respond_json(404, {"error": "not found"})
+            return
+        # Defensive envelope parity with the sibling byte-serving handlers
+        # (`_handle_get_doctor` / `_serve_static_file`): `locate_media` already
+        # runs inside the `_run_conversation_query` 500-envelope, but the
+        # `read_media_bytes` read + the byte emission did not. `read_media_bytes`
+        # is internally defensive (OSError/ValueError → `gone`), so this guards
+        # only an UNEXPECTED escape — but an unguarded one would kill the handler
+        # thread with no logged 500. `response_started` tracks the commit point:
+        # an exception BEFORE `send_response(200)` sends a clean logged 500; one
+        # AFTER (mid-`wfile.write`, headers already out) can't re-send a status,
+        # so it's logged only — never a silent thread death.
+        response_started = False
+        try:
+            status, media_type, raw = cq.read_media_bytes(
+                loc[0], loc[1], index=index, **key)
+            if status == "unsupported":
+                self._respond_json(404, {"error": "not found"})
+                return
+            if status == "too_large":
+                self._respond_json(413, {"error": "media too large"})
+                return
+            if status != "ok":
+                self._respond_json(410, {"error": "source no longer available"})
+                return
+            self.send_response(200)
+            response_started = True
+            self.send_header("Content-Type", media_type)
+            self.send_header("Content-Length", str(len(raw)))
+            self.send_header("X-Content-Type-Options", "nosniff")
+            self.send_header("Cache-Control", "private, max-age=86400")
+            if media_type == "application/pdf":
+                self.send_header("Content-Disposition",
+                                 f'inline; filename="attachment-{index}.pdf"')
+            else:
+                self.send_header("Content-Security-Policy", "default-src 'none'")
+            self.end_headers()
+            self.wfile.write(raw)
+        except Exception as exc:  # noqa: BLE001
+            self.log_error("/api/conversation/media failed: %r", exc)
+            if not response_started:
+                self._respond_json(
+                    500, {"error": f"{type(exc).__name__}: {exc}"})
+
     def _handle_get_project_detail(self) -> None:
         """Return ProjectDetail JSON for ``GET /api/project/<key>?weeks=N``
         (spec §5.3 / §6.5).

package/bin/_cctally_db.py

@@ -2304,7 +2304,8 @@ def _apply_cache_schema(conn: sqlite3.Connection) -> None:
             cache_create_tokens INTEGER NOT NULL DEFAULT 0,
             cache_read_tokens   INTEGER NOT NULL DEFAULT 0,
             usage_extra_json    TEXT,
-            cost_usd_raw        REAL
+            cost_usd_raw        REAL,
+            speed               TEXT
         );
         CREATE INDEX IF NOT EXISTS idx_entries_timestamp
             ON session_entries(timestamp_utc);
@@ -2384,6 +2385,14 @@ def _apply_cache_schema(conn: sqlite3.Connection) -> None:
     # populated lazily in sync_cache() / _ensure_session_files_row().
     add_column_if_missing(conn, "session_files", "session_id", "TEXT")
     add_column_if_missing(conn, "session_files", "project_path", "TEXT")
+    # #181: materialize the only-ever-consumed extra `usage` key (`speed`) into
+    # a real session_entries column so the hot cache read paths (iter_entries /
+    # get_claude_session_entries) stop json.loads-ing the deeply-nested
+    # usage_extra_json blob per row — that per-tick parse was pegging a core in
+    # the dashboard. Idempotent column-add (no marker, no version), appended
+    # after cost_usd_raw to match the CREATE TABLE order; cache migration 008
+    # then backfills it from the legacy blob on existing rows.
+    add_column_if_missing(conn, "session_entries", "speed", "TEXT")
     # Existing-DB guard for the skill-content fold link (cctally-dev
     # skill-content-nesting): the message-level sourceToolUseID. Idempotent
     # column-add (no marker, no version); cache migration 006 then re-ingests
@@ -3171,6 +3180,51 @@ def _007_conversation_reingest_enrichment(conn: sqlite3.Connection) -> None:
     conn.commit()
 
 
+@cache_migration("008_session_entries_speed_backfill")
+def _008_session_entries_speed_backfill(conn: sqlite3.Connection) -> None:
+    """Backfill the materialized ``session_entries.speed`` column from the
+    legacy ``usage_extra_json`` blob (#181). ``speed`` is the ONLY non-token
+    ``usage`` key any consumer reads (``<model>-fast`` rendering in
+    _lib_aggregators + the _should_replace dedup tiebreak); materializing it
+    lets the hot read paths (iter_entries / get_claude_session_entries) stop
+    ``json.loads``-ing the deeply-nested blob per row — the per-tick dashboard
+    rebuild was pegging a core on a ~261K-row cache.
+
+    The column is added by _apply_cache_schema's add_column_if_missing (which
+    runs before this dispatcher), so it always exists here. This handler only
+    backfills existing rows; new ingests write the column directly and write
+    NULL to usage_extra_json. ``WHERE speed IS NULL`` self-guards re-runs.
+    We do NOT rewrite/NULL usage_extra_json on existing rows or VACUUM — the
+    stale blob has no reader and is reclaimed on the next cache-sync --rebuild.
+    json_extract returns NULL when '$.speed' is absent, so speed-less rows stay
+    NULL. Central stamp via the dispatcher (#140); a fresh install stamps it
+    without effect (empty table)."""
+    conn.execute(
+        "UPDATE session_entries "
+        "SET speed = json_extract(usage_extra_json, '$.speed') "
+        "WHERE speed IS NULL AND usage_extra_json IS NOT NULL"
+    )
+    conn.commit()
+
+
+@cache_migration("009_conversation_media_reingest")
+def _009_conversation_media_reingest(conn: sqlite3.Connection) -> None:
+    """Flag-only re-ingest so the #177 S4 media/web enrichment — tool_result
+    ``media[]`` placeholders (iter_media_items ordinals), user-content media
+    ``index``, and the bounded ``web_search``/``web_fetch`` toolUseResult
+    captures — lands on existing history. Sets the DISTINCT
+    ``conversation_media_reingest_pending`` flag (NOT the shared
+    ``conversation_reingest_pending``, which also gates migration 005's
+    read-time human-fallback in the query kernel). Consumption rides the #179
+    RESUMABLE per-file reingest (_resumable_reingest_conversation_messages) —
+    the flag is wired into _REINGEST_FLAG_KEYS + both flag SELECTs + the two
+    cleanup DELETE lists in _cctally_cache.py (all five sites; missing one
+    either never triggers or re-arms forever). Central stamp via the
+    dispatcher (#140); a fresh install stamps it without running."""
+    _set_cache_meta(conn, "conversation_media_reingest_pending", "1")
+    conn.commit()
+
+
 # === Region 7d: Stats migration 008_recompute_weekly_cost_snapshots_dedup_fix ===
 
 @stats_migration("008_recompute_weekly_cost_snapshots_dedup_fix")
@@ -3323,6 +3377,10 @@ def _008_recompute_weekly_cost_snapshots_dedup_fix(
                         "cache_creation_input_tokens": cc,
                         "cache_read_input_tokens": cr,
                     }
+                    # #181: usage_extra_json is cost-irrelevant (cost is
+                    # token-only); parsed here only for pre-008 rows that may
+                    # still carry the legacy blob — NOT a speed reader, so the
+                    # write-side NULL going forward is safe.
                     if extras_json:
                         usage.update(json.loads(extras_json))
                     total += _calculate_entry_cost(
@@ -3637,6 +3695,10 @@ def _009_recompute_five_hour_blocks_dedup_fix(
                         "cache_creation_input_tokens": cc_t,
                         "cache_read_input_tokens": cr_t,
                     }
+                    # #181: usage_extra_json is cost-irrelevant (cost is
+                    # token-only); parsed here only for pre-008 rows that may
+                    # still carry the legacy blob — NOT a speed reader, so the
+                    # write-side NULL going forward is safe.
                     if extras_json:
                         usage.update(json.loads(extras_json))
                     cost = _calculate_entry_cost(
@@ -3909,6 +3971,10 @@ def _010_recompute_percent_milestones_dedup_fix(
                         "cache_creation_input_tokens": cc,
                         "cache_read_input_tokens": cr,
                     }
+                    # #181: usage_extra_json is cost-irrelevant (cost is
+                    # token-only); parsed here only for pre-008 rows that may
+                    # still carry the legacy blob — NOT a speed reader, so the
+                    # write-side NULL going forward is safe.
                     if extras_json:
                         usage.update(json.loads(extras_json))
                     cumulative += _calculate_entry_cost(

package/bin/_cctally_statusline.py

@@ -345,7 +345,7 @@ def _build_statusline_injections(warn_once):
                 "SELECT se.timestamp_utc, se.model, "
                 "  se.input_tokens, se.output_tokens, "
                 "  se.cache_create_tokens, se.cache_read_tokens, "
-                "  se.cost_usd_raw, se.usage_extra_json "
+                "  se.cost_usd_raw "
                 "FROM session_entries se "
                 "LEFT JOIN session_files sf ON sf.path = se.source_path "
                 "WHERE sf.session_id = ?"
@@ -368,13 +368,10 @@ def _build_statusline_injections(warn_once):
                 "cache_creation_input_tokens": r[4] or 0,
                 "cache_read_input_tokens":     r[5] or 0,
             }
-            try:
-                if r[7]:
-                    extras = json.loads(r[7])
-                    if isinstance(extras, dict):
-                        usage.update(extras)
-            except Exception:
-                pass
+            # #181: cost is token-only (_calculate_entry_cost ignores `speed`),
+            # so the statusline cost path no longer selects or json.loads the
+            # usage_extra_json blob — output is byte-identical. r[6] is still
+            # cost_usd_raw (the dropped column was the trailing slot).
             try:
                 total += c._calculate_entry_cost(
                     r[1], usage, mode="auto", cost_usd=r[6],

package/bin/_lib_conversation.py

@@ -24,6 +24,14 @@ _INPUT_MAX_DEPTH = 12      # max nesting depth before subtree elision (Recursion
 _INPUT_KEY_CAP = 512       # max chars per dict key (else keys are stored verbatim, unbounded)
 _INPUT_ELISION = "…"       # sentinel for elided leaves / subtrees
 
+# #177 S4: WebSearch link-list capture bounds + the media item types whose
+# placeholders the ordinal chokepoint (iter_media_items) addresses.
+_WEB_SEARCH_LINK_CAP = 50
+_WEB_LINK_TITLE_CAP = 300
+_WEB_LINK_URL_CAP = 2000
+_WEB_FETCH_CODE_TEXT_CAP = 100   # HTTP reason phrases are short; generous bound
+_MEDIA_BLOCK_TYPES = ("image", "document")
+
 
 @dataclass
 class MessageRow:
@@ -107,6 +115,8 @@ def _normalize(obj, t, offset):
         _attach_subagent_result(blocks, obj)   # #166: record-level toolUseResult
         _attach_ask_answers(blocks, obj)       # #177 S2: AskUserQuestion answers
         _attach_bash_streams(blocks, obj)      # #177 S3: Bash stderr/interrupted
+        _attach_web_search(blocks, obj)        # #177 S4: WebSearch link list
+        _attach_web_fetch(blocks, obj)         # #177 S4: WebFetch HTTP status
         _attach_task_meta(blocks, obj)         # task checklist identity
         # tool_result rows are stored but NOT indexed as prose (spec §2). A
         # user line that mixes a text block with a tool_result block must not
@@ -174,6 +184,10 @@ def _blocks_and_text(content):
         return (([{"kind": "text", "text": content}] if content else []), content, "")
     blocks, texts, aux_parts = [], [], []
     if isinstance(content, list):
+        # #177 S4: ordinal among media items at THIS list level, keyed by the
+        # object identity of each image/document item, so the placeholder writers
+        # below stamp the same ``index`` the media-route reader recomputes.
+        media_index = {id(item): idx for idx, item in iter_media_items(content)}
         for b in content:
             if not isinstance(b, dict):
                 continue
@@ -202,14 +216,24 @@ def _blocks_and_text(content):
             elif bt == "tool_result":
                 raw = _stringify(b.get("content"))
                 clipped = raw[:_TOOL_RESULT_CAP]
-                blocks.append({"kind": "tool_result", "text": clipped,
-                               "truncated": len(raw) > _TOOL_RESULT_CAP,
-                               "full_length": len(raw),
-                               "is_error": bool(b.get("is_error")),
-                               "tool_use_id": b.get("tool_use_id")})
+                block = {"kind": "tool_result", "text": clipped,
+                         "truncated": len(raw) > _TOOL_RESULT_CAP,
+                         "full_length": len(raw),
+                         "is_error": bool(b.get("is_error")),
+                         "tool_use_id": b.get("tool_use_id")}
+                # #177 S4: media placeholders for image/document items inside the
+                # tool_result content array (where every MCP screenshot lives) —
+                # ordinals from the shared iter_media_items chokepoint.
+                media = [{"kind": item.get("type"), **_media(item.get("source")),
+                          "index": idx}
+                         for idx, item in iter_media_items(b.get("content"))]
+                if media:                      # omitted when empty (additive)
+                    block["media"] = media
+                blocks.append(block)
                 aux_parts.append(clipped)
-            elif bt in ("image", "document"):
-                blocks.append({"kind": bt, **_media(b.get("source"))})
+            elif bt in _MEDIA_BLOCK_TYPES:
+                blocks.append({"kind": bt, **_media(b.get("source")),
+                               "index": media_index[id(b)]})
             elif bt == "tool_reference":
                 blocks.append({"kind": "tool_reference", "name": b.get("name")})
     return (blocks, "\n".join(t for t in texts if t),
@@ -319,6 +343,78 @@ def _attach_bash_streams(blocks, obj):
         results[0]["bash_interrupted"] = True
 
 
+def _attach_web_search(blocks, obj):
+    """Stash a WebSearch toolUseResult's structured link list onto its single
+    tool_result block (#177 S4). Self-identifying: fires only on the WebSearch
+    shape (string ``query`` + list ``results``); the query kernel additionally
+    joins NAME-KEYED (only onto name=='WebSearch'), so a shape-coincident
+    toolUseResult from another tool never decorates the wrong card (Codex F3).
+    Links flatten from results[].content[]; items lacking string title+url are
+    skipped; bounded (<=_WEB_SEARCH_LINK_CAP links, title/url char caps) with
+    ``links_truncated`` when links were dropped. Parser-private key
+    ``web_search`` is popped in the query layer's Phase 1 so it never leaks on
+    orphan blocks. Same exactly-one-result-block guard as
+    _attach_subagent_result."""
+    tur = obj.get("toolUseResult")
+    if not isinstance(tur, dict):
+        return
+    query = tur.get("query")
+    raw_results = tur.get("results")
+    if not isinstance(query, str) or not isinstance(raw_results, list):
+        return
+    results = [b for b in blocks if b.get("kind") == "tool_result"]
+    if len(results) != 1:
+        return
+    links, dropped = [], False
+    for r in raw_results:
+        content = r.get("content") if isinstance(r, dict) else None
+        if not isinstance(content, list):
+            continue
+        for item in content:
+            if not (isinstance(item, dict) and isinstance(item.get("title"), str)
+                    and isinstance(item.get("url"), str)):
+                continue
+            if len(links) >= _WEB_SEARCH_LINK_CAP:
+                dropped = True
+                break
+            links.append({"title": item["title"][:_WEB_LINK_TITLE_CAP],
+                          "url": item["url"][:_WEB_LINK_URL_CAP]})
+        if dropped:
+            break
+    payload = {"query": query[:_WEB_LINK_TITLE_CAP], "links": links}
+    if dropped:
+        payload["links_truncated"] = True
+    results[0]["web_search"] = payload
+
+
+def _attach_web_fetch(blocks, obj):
+    """Stash a WebFetch toolUseResult's HTTP status onto its single tool_result
+    block (#177 S4). Self-identifying on the WebFetch triple — ``code`` +
+    ``codeText`` + ``result`` keys all present and ``code`` an int; the query
+    kernel additionally joins NAME-KEYED (only onto name=='WebFetch'). A bare
+    error-string toolUseResult never matches (the card then renders without a
+    status chip — the documented degrade). Only the status is stored (the
+    summary already IS result.text). Parser-private key ``web_fetch`` is popped
+    in Phase 1. Same exactly-one-result-block guard as
+    _attach_subagent_result."""
+    tur = obj.get("toolUseResult")
+    if not isinstance(tur, dict):
+        return
+    if "code" not in tur or "codeText" not in tur or "result" not in tur:
+        return
+    code = tur.get("code")
+    if not isinstance(code, int):
+        return
+    results = [b for b in blocks if b.get("kind") == "tool_result"]
+    if len(results) != 1:
+        return
+    meta = {"code": code}
+    code_text = tur.get("codeText")
+    if isinstance(code_text, str) and code_text:
+        meta["code_text"] = code_text[:_WEB_FETCH_CODE_TEXT_CAP]
+    results[0]["web_fetch"] = meta
+
+
 # Subagent Task tools record toolUseResult=null and put the identity in the
 # human-readable result text instead; the id is the only thing the fold needs
 # from the result (subject/status come from the call input). Anchored to the
@@ -513,3 +609,20 @@ def _media(source):
         return {"media_type": None, "bytes": 0}
     data = source.get("data") or ""
     return {"media_type": source.get("media_type"), "bytes": len(data)}
+
+
+def iter_media_items(content):
+    """Yield ``(index, item)`` for every image/document item in a content list,
+    in document order. ``index`` is the ordinal AMONG MEDIA ITEMS (not the list
+    position) — the stable address shared by the ingest placeholder writer here
+    and the media-route reader (read_media_bytes), so "media item N" can never
+    mean two different things (the _canonical_5h_window_key lesson applied to
+    media addressing — do NOT write a second walk). Non-list input and
+    non-dict / non-media entries are skipped without consuming an ordinal."""
+    if not isinstance(content, list):
+        return
+    idx = 0
+    for item in content:
+        if isinstance(item, dict) and item.get("type") in _MEDIA_BLOCK_TYPES:
+            yield idx, item
+            idx += 1