cctally 1.41.0 → 1.43.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,24 @@ based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
5
5
 
6
6
  ## [Unreleased]
7
7
 
8
+ ## [1.43.0] - 2026-06-13
9
+
10
+ ### Fixed
11
+ - Conversation viewer: a slash command you typed with a real prompt in its arguments (e.g. `/frontend-design <your task>`) is no longer hidden as a "System marker" and dropped from the outline — its arguments now render as your "You" bubble with a small command badge, drive the conversation title, and are searchable; bare control commands (`/clear`, `/compact`, `/model`, the `local-command-*` echoes) and empty-argument invocations still fold into hidden system-marker pills. Recognized at ingest plus a read-time fallback that fixes already-recorded history, with a cache migration (`011`) that backfills the search index for past commands (#188).
12
+ - Conversation viewer: clicking an outline entry now selects and highlights exactly that entry instead of one a turn or two above it, and the forward jump-to-next chips/keys (`u`/`e`/`b`/`p`) reliably advance to the next landmark instead of re-selecting the same one — the outline tracks an explicit selection you set by clicking or jumping, which takes precedence over the scroll position until you next scroll (#188, #187).
13
+ - Conversation viewer: clicking a subagent in the outline now flashes its collapsed card in place and marks the subagent as the current selection, instead of silently force-expanding the thread, flashing a buried inner message, and highlighting the most-recent prompt above it (#188).
14
+ - Conversation viewer: the floating "↓ N new" live-update pill no longer counts messages streamed into a collapsed subagent thread you have not expanded — it counts only turns that actually become visible (top-level messages, a newly appearing subagent card, or messages inside an expanded thread) (#188).
15
+
16
+ ## [1.42.0] - 2026-06-13
17
+
18
+ ### Fixed
19
+ - Conversation viewer: system and slash-command messages (`/clear`, `/model`, the `local-command-stdout`/`local-command-stderr` echoes, and the caveats) were being attributed to you as "You" turns, and the first such line became the whole conversation's title; they now fold into collapsed "System marker" pills and the title falls back to your real first prompt. Such rows are recognized at ingest (stored as meta and kept out of search and the prompts facet) with a read-time fallback that also cleans already-recorded history, plus a belt-and-suspenders title skip that degrades unknown `command-*`/`local-command-*` tags to "skip" rather than letting them poison the title (#186).
20
+ - Conversation viewer: terminal ANSI escape codes (for example the bold `^[[1m…^[[22m` wrapping "Fable 5") no longer leak into titles, outline labels, or message and command body text — they are stripped at ingest (which also keeps the search index clean for new rows) and again at the render chokepoints (which cleans already-recorded history without a re-ingest), while Bash tool output keeps its intentional terminal colors (#186).
21
+
22
+ ### Changed
23
+ - Conversation viewer: the outline rail is redesigned from a flat wall of ~200 truncated prose lines into a bold prompt spine (your prompts, each with a divider above it) with curated landmarks demoted beneath each prompt — section headings, plans, questions, errors (red), and subagent spawns (magenta) — where a turn's thinking blocks collapse to a single `🧠 ×N` badge on its prompt instead of doubling the list, session-start plumbing rows are gone, role glyphs are color-coded for at-a-glance scanning, and the scroll-sync highlight tracks the enclosing prompt (#186).
24
+ - Conversation viewer: the outline header is merged into one "session at a glance" card — time, tokens, and cost become labeled stat tiles, models/tools/errors become labeled rows, the formerly cryptic unlabeled glyph jump-chips gain text labels and move into the card under a "Jump to" row, and the error count reconciles to one phrase ("14 errors in 13 turns") so the previously-disagreeing 13 and 14 no longer read as two competing numbers (#186).
25
+
8
26
  ## [1.41.0] - 2026-06-13
9
27
 
10
28
  ### Added
@@ -681,6 +681,16 @@ def sync_cache(
681
681
  "DELETE FROM cache_meta WHERE key IN "
682
682
  "('conversation_search_split_pending',"
683
683
  " 'conversation_search_split_cursor')")
684
+ # #188 bug 4: a rebuild repopulates conversation_messages via the
685
+ # offset-0 walk through the parser, which now classifies a
686
+ # command-args invocation as entry_type='human' at INGEST (A2) — so
687
+ # the migration-011 backfill is redundant. Drop its flag + cursor so
688
+ # the post-rebuild sync runs no redundant promotion pass. MISSING
689
+ # this site re-arms the flag on every cache-sync --rebuild.
690
+ conn.execute(
691
+ "DELETE FROM cache_meta WHERE key IN "
692
+ "('conversation_promote_command_args_pending',"
693
+ " 'conversation_promote_command_args_cursor')")
684
694
  conn.commit()
685
695
  eprint("[cache-sync] rebuild: cleared Claude cached entries")
686
696
 
@@ -780,6 +790,15 @@ def sync_cache(
780
790
  # old search keeps working until the final swap.
781
791
  _consume_search_split(conn)
782
792
 
793
+ # #188 bug 4: consume the migration-011 command-args promotion under
794
+ # the SAME held flock, AFTER the search split so a row flipped to
795
+ # entry_type='human' here keeps the fresh search_tool/search_thinking
796
+ # the split just wrote (the consumer recomputes them anyway, but
797
+ # ordering keeps the two passes independent + idempotent). Flips
798
+ # legacy META command rows carrying a real <command-args> prompt to
799
+ # HUMAN(text=args); the split-FTS UPDATE triggers re-index the args.
800
+ _consume_promote_command_args(conn)
801
+
783
802
  paths: list[pathlib.Path] = list(_iter_claude_jsonl_files())
784
803
  stats.files_total = len(paths)
785
804
 
@@ -1357,6 +1376,68 @@ def _consume_search_split(conn) -> None:
1357
1376
  conn.commit()
1358
1377
 
1359
1378
 
1379
+ def _consume_promote_command_args(conn) -> None:
1380
+ """#188 bug 4: flock-held consumer for ``conversation_promote_command_args_pending``
1381
+ (set by cache migration 011). Cursor-resumable walk of
1382
+ ``conversation_messages WHERE entry_type='meta'``: a row whose ``blocks_json``
1383
+ is a pure slash-command marker with a NON-EMPTY ``<command-args>`` is a real
1384
+ user turn, so flip it to ``entry_type='human'`` with ``text=args`` and
1385
+ recompute ``search_tool``/``search_thinking`` via the SHARED
1386
+ ``_lib_conversation._derive_search_columns`` chokepoint (byte-identical to
1387
+ live ingest). ``/clear`` and stdout-only markers (``_extract_command_invocation``
1388
+ returns None) stay META untouched.
1389
+
1390
+ The split-FTS ``AFTER UPDATE OF text, search_tool, search_thinking`` triggers
1391
+ keep the external-content index in sync, so we never hand-write FTS rows.
1392
+ FTS5-unavailable (``fts5_unavailable`` set): no triggers exist, so the
1393
+ base-column UPDATE alone is correct (the index lands later via the
1394
+ rebuild-on-availability path). Checkpoints
1395
+ ``conversation_promote_command_args_cursor`` per 500-row batch; clears both
1396
+ keys when the cursor is exhausted. Interrupted ⇒ resumes from the cursor on
1397
+ the next locked sync; a fresh install never sets the flag → cheap no-op."""
1398
+ if conn.execute(
1399
+ "SELECT 1 FROM cache_meta "
1400
+ "WHERE key='conversation_promote_command_args_pending'"
1401
+ ).fetchone() is None:
1402
+ return
1403
+ row = conn.execute(
1404
+ "SELECT value FROM cache_meta "
1405
+ "WHERE key='conversation_promote_command_args_cursor'").fetchone()
1406
+ last_id = int(row[0]) if row else 0
1407
+ while True:
1408
+ batch = conn.execute(
1409
+ "SELECT id, blocks_json FROM conversation_messages "
1410
+ "WHERE id > ? AND entry_type='meta' ORDER BY id LIMIT 500",
1411
+ (last_id,)).fetchall()
1412
+ if not batch:
1413
+ break
1414
+ ups = []
1415
+ for rid, bj in batch:
1416
+ last_id = rid
1417
+ try:
1418
+ blocks = json.loads(bj) if bj else []
1419
+ except (TypeError, ValueError):
1420
+ blocks = []
1421
+ inv = _lib_conversation._extract_command_invocation(
1422
+ blocks, _lib_conversation._join_text_blocks(blocks))
1423
+ if inv is None:
1424
+ continue
1425
+ st, sth = _lib_conversation._derive_search_columns(blocks)
1426
+ ups.append((inv["args"], st, sth, rid))
1427
+ if ups:
1428
+ conn.executemany(
1429
+ "UPDATE conversation_messages SET entry_type='human', text=?, "
1430
+ "search_tool=?, search_thinking=? WHERE id=?", ups)
1431
+ _cctally_db_sib._set_cache_meta(
1432
+ conn, "conversation_promote_command_args_cursor", str(last_id))
1433
+ conn.commit()
1434
+ conn.execute(
1435
+ "DELETE FROM cache_meta WHERE key IN "
1436
+ "('conversation_promote_command_args_pending',"
1437
+ " 'conversation_promote_command_args_cursor')")
1438
+ conn.commit()
1439
+
1440
+
1360
1441
  def iter_entries(
1361
1442
  conn: sqlite3.Connection,
1362
1443
  range_start: dt.datetime,
@@ -821,7 +821,12 @@ def _run_pending_migrations(
821
821
  "projected_milestones",
822
822
  "codex_budget_milestones",
823
823
  ),
824
- "cache.db": ("session_entries",),
824
+ # conversation_messages joins session_entries as a fresh-install
825
+ # signal (#188): a transcript-populated cache whose cost rows are
826
+ # absent must still be treated as non-fresh so a flag-only
827
+ # conversation migration's consumer actually runs (e.g. 011's
828
+ # command-args promotion) instead of being stamped without it.
829
+ "cache.db": ("session_entries", "conversation_messages"),
825
830
  }.get(db_label, ())
826
831
  for probe_table in probe_tables:
827
832
  # _probe_table_nonempty centralizes the "is there data here?"
@@ -3359,6 +3364,25 @@ def conversation_search_depth(conn: sqlite3.Connection) -> str:
3359
3364
  return "prose-only" if pending else "full"
3360
3365
 
3361
3366
 
3367
+ @cache_migration("011_conversation_promote_command_args")
3368
+ def _011_conversation_promote_command_args(conn: sqlite3.Connection) -> None:
3369
+ """Flag-only arm for #188 bug 4. Sets
3370
+ ``conversation_promote_command_args_pending``; sync_cache consumes it under
3371
+ the cache.db.lock flock (cursor-resumable: flip legacy ``entry_type='META'``
3372
+ command-marker rows whose ``<command-args>`` carry a real user prompt to
3373
+ ``entry_type='human'`` with ``text=args``, recomputing the split search
3374
+ columns so the args enter the entry_type='human' list-title/prompts facet and
3375
+ the FTS index — see _cctally_cache._consume_promote_command_args). The
3376
+ handler does NO data work so the dispatcher's central stamp (#140) marks a
3377
+ genuinely-complete handler; a fresh install stamps it WITHOUT running (its
3378
+ rows are already promoted at ingest, so the consumer finds nothing to flip).
3379
+ DISPLAY is already fixed read-time (no migration needed); the migration's
3380
+ sole job is FTS-searchability + the list-title facet on legacy data. Mirrors
3381
+ the flag-only pattern of 002/003/007/009/010."""
3382
+ _set_cache_meta(conn, "conversation_promote_command_args_pending", "1")
3383
+ conn.commit()
3384
+
3385
+
3362
3386
  # === Region 7d: Stats migration 008_recompute_weekly_cost_snapshots_dedup_fix ===
3363
3387
 
3364
3388
  @stats_migration("008_recompute_weekly_cost_snapshots_dedup_fix")
@@ -16,6 +16,102 @@ ASSISTANT = "assistant"
16
16
  TOOL_RESULT = "tool_result"
17
17
  META = "meta"
18
18
 
19
+ # Mirror of dashboard/web/src/conversations/systemMarkers.ts::MARKER_RE — anchored
20
+ # whole-string (fullmatch), unrolled-lazy body for linear time (no ReDoS), \1
21
+ # backref forces each close tag to match its open tag. Used to SKIP slash-command
22
+ # plumbing when deriving a conversation title (#165 Q2) AND — at the parser layer
23
+ # (#186) — to classify a slash-command echo carried as a plain user line as META
24
+ # at ingest. MUST stay equivalent to the TS predicate over ASCII whitespace
25
+ # (parity-tested); exotic Unicode/control whitespace is an explicit non-goal.
26
+ # The query kernel re-exports these names for back-compat. See
27
+ # docs/dashboard-gotchas.md.
28
+ _MARKER_TAGS = ("command-name", "command-message", "command-args",
29
+ "local-command-caveat", "local-command-stdout",
30
+ "local-command-stderr")
31
+ _MARKER_RE = re.compile(
32
+ r"\s*(?:<(" + "|".join(_MARKER_TAGS) + r")>(?:(?!</\1>)[\s\S])*</\1>\s*)+"
33
+ )
34
+
35
+
36
+ def _is_system_marker(text) -> bool:
37
+ """True iff `text` is ONLY concatenated command-marker wrappers (slash-command
38
+ plumbing) — the title-derivation skip predicate AND the parser-layer ingest
39
+ classifier (#186). `fullmatch` reproduces the TS `^\\s*…\\s*$` anchor (no
40
+ `$`-before-trailing-`\\n` foot-gun)."""
41
+ return bool(text) and _MARKER_RE.fullmatch(text) is not None
42
+
43
+
44
+ # #188: a slash-command invocation carries the user's real prompt inside
45
+ # <command-args>. The <command-name>/<command-message> wrappers are plumbing.
46
+ # Anchored mid-string is fine — _is_system_marker already proved the whole text
47
+ # is ONLY markers before these run, so the first match is the lone occurrence.
48
+ _CMD_NAME_RE = re.compile(r"<command-name>([\s\S]*?)</command-name>")
49
+ _CMD_ARGS_RE = re.compile(r"<command-args>([\s\S]*?)</command-args>")
50
+
51
+
52
+ def _join_text_blocks(blocks):
53
+ """'\\n'-join the text-block bodies of a normalized blocks list (mirrors
54
+ _blocks_and_text's prose join + the query kernel's _join_text_blocks). The
55
+ migration-011 consumer rebuilds the marker text from blocks_json to feed
56
+ _extract_command_invocation, so this lives in the parser kernel (the one
57
+ _cctally_cache already imports) to keep the two derivations identical."""
58
+ if not blocks:
59
+ return ""
60
+ return "\n".join(b.get("text", "") or ""
61
+ for b in blocks if b.get("kind") == "text")
62
+
63
+
64
+ def _extract_command_invocation(blocks, text):
65
+ """If a row is a pure slash-command marker whose <command-args> is non-empty
66
+ (after strip), return ``{"name": <command-name or "">, "args": <args>}``;
67
+ else ``None`` (#188 bug 4).
68
+
69
+ Block-aware: promotes ONLY when every block is text (mirrors the all-text
70
+ guard in ``_normalize`` / ``_meta_classify`` so a marker text block PLUS an
71
+ attachment — e.g. an image — is never promoted) AND ``text`` is a pure
72
+ command-marker block (``_is_system_marker``) AND ``<command-args>`` holds a
73
+ real prompt. ``/clear``, ``/exit``, ``/compact``, ``/model`` (empty args) and
74
+ stdout-only markers all return ``None`` so they stay hidden as system
75
+ markers. ``name`` is taken from ``<command-name>`` for the reader's command
76
+ badge; it is purely cosmetic (``""`` when the marker omits the name tag)."""
77
+ if not blocks or not all(b.get("kind") == "text" for b in blocks):
78
+ return None
79
+ if not _is_system_marker(text):
80
+ return None
81
+ am = _CMD_ARGS_RE.search(text)
82
+ if am is None:
83
+ return None
84
+ args = am.group(1).strip()
85
+ if not args:
86
+ return None
87
+ nm = _CMD_NAME_RE.search(text)
88
+ return {"name": (nm.group(1).strip() if nm else ""), "args": args}
89
+
90
+
91
+ # #186: strip ANSI terminal control sequences from captured prose/thinking so a
92
+ # slash-command stdout echo (which keeps its SGR styling, e.g. `\x1b[1mFable
93
+ # 5\x1b[22m`) never leaks literal `^[[1m` control codes into a title, an outline
94
+ # label, or body text — and so FTS indexes clean tokens (`Fable`, not
95
+ # `1mFable`). Three alternatives: CSI (covers all SGR `…m`), OSC (BEL- or
96
+ # ST-terminated, or unterminated), and a lone/truncated ESC. Ordinary `[`,
97
+ # digits, `m` in prose are untouched — they match only as part of a real
98
+ # `\x1b[`-led sequence. Conceptually mirrors the client `parseAnsi` cleanup.
99
+ # Deliberately NOT applied in `_stringify` / tool_result text: `AnsiText` (#177
100
+ # S3) renders Bash stdout/stderr SGR colors, so that path keeps raw ANSI.
101
+ _ANSI_RE = re.compile(
102
+ r"\x1b\[[0-9;:?]*[ -/]*[@-~]" # CSI (covers all SGR …m)
103
+ r"|\x1b\][^\x07\x1b]*(?:\x07|\x1b\\)?" # OSC (BEL/ST terminated or unterminated)
104
+ r"|\x1b" # lone/truncated ESC
105
+ )
106
+
107
+
108
+ def _strip_ansi(text):
109
+ """Remove ANSI control sequences (terminal SGR/CSI/OSC). Ordinary '[',
110
+ digits, 'm' in prose are untouched — they match only inside a real \\x1b[
111
+ run. Empty / None passes through unchanged. (#186)"""
112
+ return _ANSI_RE.sub("", text) if text else text
113
+
114
+
19
115
  _TOOL_RESULT_CAP = 16000 # was 4000; full text always re-derivable from JSONL
20
116
  _INPUT_LEAF_CAP = 8000 # max chars per string leaf in a bounded tool input
21
117
  _INPUT_TOTAL_CAP = 32000 # honesty backstop on the serialized bounded input
@@ -143,6 +239,33 @@ def _normalize(obj, t, offset):
143
239
  # tool_result block still folds as a result.
144
240
  entry_type = META
145
241
  text = ""
242
+ elif (blocks and all(b["kind"] == "text" for b in blocks)
243
+ and (_inv := _extract_command_invocation(blocks, text)) is not None):
244
+ # #188: a slash-command invocation carrying a real user prompt in
245
+ # <command-args> IS a user turn — the wrapper is plumbing but the args
246
+ # are what the user typed. Promote it (entry_type=HUMAN, text=args) so it
247
+ # enters FTS / title derivation / the entry_type='human' prompts facet.
248
+ # blocks_json is UNTOUCHED (still the raw <command-name>…), so the read
249
+ # path derives the command-name badge from the blocks. Ordered BEFORE the
250
+ # empty-args/stdout-only system-marker fold below — /clear, /exit,
251
+ # /compact and friends (empty args) fall through to META there. The
252
+ # all-text guard mirrors the marker fold so a marker+attachment row is
253
+ # never folded NOR promoted (it stays a plain HUMAN turn in the else).
254
+ entry_type = HUMAN
255
+ text = _inv["args"]
256
+ elif blocks and all(b["kind"] == "text" for b in blocks) and _is_system_marker(text):
257
+ # A slash-command echo carried as a plain user line (NOT isMeta): the
258
+ # user did not type it. `<local-command-stdout>…</local-command-stdout>`
259
+ # & friends arrive as ordinary user content, so without this branch they
260
+ # render as a "YOU" prompt and poison the conversation title (#186).
261
+ # Classify META + text="" exactly like the isMeta branch above so the
262
+ # body stays out of FTS, title derivation, and the entry_type='human'
263
+ # prompts facet; the body survives in blocks_json for rendering as a
264
+ # "System marker" pill. The all-text guard mirrors the tool_result
265
+ # block-shape gate (Codex P1b) so an attachment-bearing row — a marker
266
+ # text block PLUS an image — is never folded.
267
+ entry_type = META
268
+ text = ""
146
269
  else:
147
270
  entry_type = HUMAN
148
271
  is_asst = t == "assistant"
@@ -196,6 +319,9 @@ def _blocks_and_text(content):
196
319
  ingest==backfill parity invariant, so the derivation moved to the
197
320
  post-augment chokepoint."""
198
321
  if isinstance(content, str):
322
+ # #186: strip ANSI on the str-content path (the slash-command-stdout
323
+ # carrier) so neither the indexed prose nor the rendered block leaks SGR.
324
+ content = _strip_ansi(content)
199
325
  return (([{"kind": "text", "text": content}] if content else []), content)
200
326
  blocks, texts = [], []
201
327
  if isinstance(content, list):
@@ -208,11 +334,11 @@ def _blocks_and_text(content):
208
334
  continue
209
335
  bt = b.get("type")
210
336
  if bt == "text":
211
- txt = b.get("text", "") or ""
337
+ txt = _strip_ansi(b.get("text", "") or "") # #186: strip SGR
212
338
  blocks.append({"kind": "text", "text": txt})
213
339
  texts.append(txt)
214
340
  elif bt == "thinking":
215
- think = b.get("thinking", "") or ""
341
+ think = _strip_ansi(b.get("thinking", "") or "") # #186: strip SGR
216
342
  blocks.append({"kind": "thinking", "text": think}) # FULL text for render
217
343
  elif bt == "tool_use":
218
344
  bounded, input_trunc = _bound_input(b.get("input"))
@@ -29,25 +29,20 @@ from _lib_conversation import _stringify
29
29
  # #177 S4: the media-route reader walks a content array with the SAME ordinal
30
30
  # generator the ingest placeholders used, so "media item N" addresses one item.
31
31
  from _lib_conversation import iter_media_items
32
-
33
-
34
- # Mirror of dashboard/web/src/conversations/systemMarkers.ts::MARKER_RE anchored
35
- # whole-string (fullmatch), unrolled-lazy body for linear time (no ReDoS), \1
36
- # backref forces each close tag to match its open tag. Used to SKIP slash-command
37
- # plumbing when deriving a conversation title (#165 Q2). MUST stay equivalent to
38
- # the TS predicate over ASCII whitespace (parity-tested); exotic Unicode/control
39
- # whitespace is an explicit non-goal. See docs/dashboard-gotchas.md.
40
- _MARKER_TAGS = ("command-name", "command-message", "command-args", "local-command-caveat")
41
- _MARKER_RE = re.compile(
42
- r"\s*(?:<(" + "|".join(_MARKER_TAGS) + r")>(?:(?!</\1>)[\s\S])*</\1>\s*)+"
43
- )
44
-
45
-
46
- def _is_system_marker(text) -> bool:
47
- """True iff `text` is ONLY concatenated command-marker wrappers (slash-command
48
- plumbing) — the title-derivation skip predicate. `fullmatch` reproduces the TS
49
- `^\\s*…\\s*$` anchor (no `$`-before-trailing-`\\n` foot-gun)."""
50
- return bool(text) and _MARKER_RE.fullmatch(text) is not None
32
+ # #186: the marker predicate moved DOWN to the parser layer (the parser now
33
+ # classifies command-marker user rows as META at ingest). Re-export the names
34
+ # here for back-compat so existing `from _lib_conversation_query import
35
+ # _is_system_marker` importers (and the title-skip path below) keep resolving.
36
+ from _lib_conversation import _MARKER_TAGS, _MARKER_RE, _is_system_marker
37
+ # #188: the slash-command-with-args promotion helper. Used at read time to
38
+ # present a legacy/ingested command-marker row carrying a real <command-args>
39
+ # prompt as a "You" turn (text=args, command_name badge). Re-exported for the
40
+ # back-compat import surface + the consumer in _cctally_cache.py.
41
+ from _lib_conversation import _extract_command_invocation
42
+ # #186: read-time ANSI strip for rows already indexed with raw SGR (no forced
43
+ # re-ingest). Scoped to prose/thinking/title/label — NEVER tool_result (Bash
44
+ # AnsiText boundary). Shares the parser's regex so ingest and read-time agree.
45
+ from _lib_conversation import _strip_ansi
51
46
 
52
47
 
53
48
  _TITLE_MAX = 120
@@ -58,7 +53,7 @@ def _title_from_text(text) -> str:
58
53
  trailing '…' ONLY when truncated (rstrip before the ellipsis). '' if none.
59
54
  Semantics IDENTICAL to the client deriveReaderTitle (#165 P2.5)."""
60
55
  for line in (text or "").split("\n"):
61
- s = line.strip()
56
+ s = _strip_ansi(line).strip() # #186: strip SGR from pre-fix dirty rows
62
57
  if s:
63
58
  return (s[:_TITLE_MAX].rstrip() + "…") if len(s) > _TITLE_MAX else s
64
59
  return ""
@@ -126,22 +121,56 @@ def _meta_classify(item, allow_human_fallback):
126
121
  pre-reingest window — see _reingest_pending). After the reingest a 'human'
127
122
  row keeping the preamble is a real user prompt, so it stays a "You" turn
128
123
  rather than being hidden in a collapsed skill pill (Codex code-review P1).
129
- - command/context: ONLY for a true 'meta' row (slash-command plumbing vs the
130
- rest). A 'human' row that is not a skill body stays human generic injected
131
- context can't be recovered read-time without isMeta; it lands on the next
124
+ - command: a true 'meta' row ALWAYS, plus — the #186 read-time fallback a
125
+ 'human' row whose body is command plumbing AND whose blocks are all text.
126
+ The marker regex is self-identifying (no read-time-recovery hazard, unlike
127
+ generic injected context), so command recovery for a pre-fix human row is
128
+ ungated by ``allow_human_fallback``. The all-text guard mirrors the ingest
129
+ branch (Codex P1b) so an attachment-bearing row is never folded.
130
+ - context: ONLY for a true 'meta' row (the remaining injected-content case).
131
+ A non-skill, non-command 'human' row stays human — generic injected context
132
+ can't be recovered read-time without isMeta; it lands on the next
132
133
  sync-triggered reingest."""
133
134
  is_meta = item["kind"] == "meta"
134
135
  body = item.get("text") or _join_text_blocks(item.get("blocks"))
135
136
  first = _first_nonblank_line(body)
136
137
  if first.startswith(_SKILL_PREAMBLE) and (is_meta or allow_human_fallback):
137
138
  return ("skill", _skill_name_from_preamble(first), body)
139
+ # Command plumbing: self-identifying, so safe to recover for a pre-fix human
140
+ # row too — but only when ALL blocks are text (mirror the ingest all-text
141
+ # guard so an attachment-bearing row is never folded). Runs ABOVE the
142
+ # not-is_meta guard precisely so a stale entry_type='human' command echo
143
+ # reclassifies read-time (#186).
144
+ all_text = all(b.get("kind") == "text" for b in (item.get("blocks") or []))
145
+ if _is_system_marker(body) and (is_meta or all_text):
146
+ return ("command", None, body)
138
147
  if not is_meta:
139
148
  return None
140
- if _is_system_marker(body):
141
- return ("command", None, body)
142
149
  return ("context", None, body)
143
150
 
144
151
 
152
+ # #186 belt-and-suspenders, title-only: a deliberately-broader skip predicate
153
+ # that drops a title candidate wrapped entirely in `command-*` / `local-command-*`
154
+ # plumbing — a tag-name PREFIX shape, NOT the strict known-tag list. The \1
155
+ # backref forces each close tag to match its open tag; the unrolled-lazy body is
156
+ # linear-time (no ReDoS). Used ONLY in title selection, where being liberal is
157
+ # safe: the worst case is the title falls back to the next line or the project
158
+ # label — never hiding content (that fold-to-pill decision keeps strict
159
+ # `_is_system_marker`, where a false positive WOULD hide real user text). A
160
+ # future unrecognized `local-command-foo` tag thus degrades to "skip the title"
161
+ # rather than "poison the title."
162
+ _CMD_FAMILY_RE = re.compile(
163
+ r"\s*(?:<((?:local-)?command-[a-z-]+)>(?:(?!</\1>)[\s\S])*</\1>\s*)+"
164
+ )
165
+
166
+
167
+ def _looks_like_command_plumbing(text) -> bool:
168
+ """Title-only liberal skip: the whole text is one or more
169
+ command-*/local-command-* wrappers (prefix shape). `fullmatch` anchors the
170
+ whole string. See `_CMD_FAMILY_RE`."""
171
+ return bool(text) and _CMD_FAMILY_RE.fullmatch(text) is not None
172
+
173
+
145
174
  def _session_titles_map(conn, session_ids):
146
175
  """{sid: title} for the first non-marker, non-blank MAIN-session human line
147
176
  per session (read-time, no migration). Windowed to the earliest 12 human
@@ -178,7 +207,7 @@ def _session_titles_map(conn, session_ids):
178
207
  for sid, text in rows:
179
208
  if sid in titles:
180
209
  continue # already resolved to the first non-marker
181
- if _is_system_marker(text):
210
+ if _is_system_marker(text) or _looks_like_command_plumbing(text):
182
211
  continue
183
212
  if skip_skill_titles and _first_nonblank_line(text).startswith(_SKILL_PREAMBLE):
184
213
  continue
@@ -622,6 +651,23 @@ def _assemble_session(conn, session_id):
622
651
  allow_human_fallback = _reingest_pending(conn)
623
652
  for it in items:
624
653
  if it["kind"] in ("meta", "human"):
654
+ # #188: a slash-command invocation carrying a real prompt in
655
+ # <command-args> is a USER turn. Promote it BEFORE _meta_classify so
656
+ # it never folds into a command-marker pill. Run on the BLOCK-joined
657
+ # text (NOT it["text"]: '' for a legacy META row, args post-migration
658
+ # — neither is the raw marker the command_name parses from). When it
659
+ # yields non-empty args, present kind='human', text=args, and attach
660
+ # the command_name badge (derived from the blocks). Idempotent: a
661
+ # migrated entry_type='human' row re-derives the same badge. /clear &
662
+ # empty-args/stdout markers yield None and fall through to
663
+ # _meta_classify's command/skill/context fold unchanged.
664
+ inv = _extract_command_invocation(
665
+ it.get("blocks"), _join_text_blocks(it.get("blocks")))
666
+ if inv is not None:
667
+ it["kind"] = "human"
668
+ it["text"] = inv["args"]
669
+ it["command_name"] = inv["name"] or None
670
+ continue
625
671
  cls = _meta_classify(it, allow_human_fallback)
626
672
  if cls is not None:
627
673
  meta_kind, skill_name, body = cls
@@ -745,8 +791,17 @@ def get_conversation(conn, session_id, *, after=None, limit=500):
745
791
  # Stamp the session_id into each anchor (spec anchor is (session_id, uuid);
746
792
  # the dict literals are built session-agnostic, so fill it here where the
747
793
  # session id is known). NOT a no-op — the endpoint/clients rely on it.
794
+ # #186: ALSO strip ANSI from the displayed prose/thinking text of each page
795
+ # item before emit, so a pre-fix row already indexed with raw SGR renders
796
+ # clean (the read-time half of the no-forced-reingest contract). tool_result
797
+ # blocks are EXCLUDED — Bash AnsiText (#177 S3) renders their SGR colors.
748
798
  for it in page:
749
799
  it["anchor"]["session_id"] = session_id
800
+ if it.get("text"):
801
+ it["text"] = _strip_ansi(it["text"])
802
+ for b in it["blocks"]:
803
+ if b.get("kind") in ("text", "thinking") and b.get("text"):
804
+ b["text"] = _strip_ansi(b["text"])
750
805
 
751
806
  first = logical[0]
752
807
  last = logical[-1]
@@ -769,9 +824,10 @@ _OUTLINE_LABEL_CAP = 120
769
824
 
770
825
 
771
826
  def _outline_label(text):
772
- """First non-blank line, capped at _OUTLINE_LABEL_CAP chars ('' when none)."""
827
+ """First non-blank line, capped at _OUTLINE_LABEL_CAP chars ('' when none).
828
+ Read-time ANSI strip (#186) so a pre-fix dirty row's label is clean."""
773
829
  for ln in (text or "").splitlines():
774
- s = ln.strip()
830
+ s = _strip_ansi(ln).strip()
775
831
  if s:
776
832
  return s[:_OUTLINE_LABEL_CAP]
777
833
  return ""