create-walle 0.9.13 → 0.9.15

package/template/claude-task-manager/docs/session-tooltip-freshness-design.md

@@ -0,0 +1,224 @@
+# Session Tooltip Freshness Design
+
+## Problem
+
+The active-session tooltip can show an AI summary that is technically cached
+correctly but no longer represents the current task. The most visible failure is
+a running session whose tooltip headline still describes an older task while the
+latest prompt and progress have already moved on.
+
+This is worse than an empty tooltip because it gives false confidence. The
+operator uses the tooltip to decide which session needs attention, so stale
+intent text can send the user to the wrong session or hide a current blocker.
+
+## Existing Behavior
+
+Current flow:
+
+1. `stream-view.js` opens the tooltip after hover and fetches
+   `/api/sessions/:id/summary?turns=3`.
+2. `SessionStream` stores cleaned user prompts in `userPromptCache`.
+3. A new user prompt debounces AI summary generation by 2 seconds.
+4. The configured provider generates a 10-15 word summary over the cached
+   prompt list.
+5. `getSummary()` returns `summary`, `intent`, `displayPrompt`, `lastPrompt`,
+   and `progress`.
+
+Current failure modes:
+
+- Cached `intent.source = ai-summary` is treated as authoritative even when its
+  timestamp is older than the latest prompt.
+- The tooltip does not refetch or rerender while it is already open for the same
+  session.
+- The AI summary input is a flat list of recent prompts, so older work can
+  dominate the summary after a task switch.
+- The UI label `Intent` does not explain freshness or distinguish current
+  prompt evidence from slower AI synthesis.
+
+## Design Principles
+
+- Latest user intent is the primary truth. AI is a compression layer, not a
+  source of freshness.
+- Stale AI should be visible as context, never promoted as the current task.
+- Refresh should be scoped to visible UI. Do not add broad polling across every
+  session.
+- The tooltip should be copyable, stable, dense, and operational.
+- Existing API fields must stay backward compatible for Session Overview and
+  older clients.
+
+## API Contract
+
+`GET /api/sessions/:id/summary` keeps existing fields and adds:
+
+```js
+{
+  currentTask: {
+    text: string | null,
+    source: 'latest-prompt' | 'ai-summary' | 'prompt-fallback' | 'title-fallback' | 'missing',
+    freshness: 'fresh' | 'updating' | 'stale' | 'missing',
+    updatedAt: number,
+    promptTimestamp: number,
+    staleReason?: string
+  },
+  latestPrompt: {
+    text: string | null,
+    timestamp: number
+  },
+  aiSummary: {
+    text: string | null,
+    source: string | null,
+    status: 'fresh' | 'updating' | 'stale' | 'fallback' | 'unavailable',
+    updatedAt: number,
+    promptTimestamp: number,
+    promptCount: number,
+    staleReason?: string
+  }
+}
+```
+
+Compatibility mapping:
+
+- `intent` remains present.
+- `summary` remains present.
+- When AI is stale, `intent.text` should follow `currentTask.text` so existing
+  clients do not keep rendering stale AI as the primary task.
+- `aiSummary.text` retains the older AI text for secondary display.
+
+Freshness rules:
+
+- If usable AI summary exists and `aiSummary.updatedAt >= latestPrompt.timestamp`,
+  `currentTask.source = ai-summary` and `freshness = fresh`.
+- If usable AI summary exists but is older than the latest prompt,
+  `currentTask.source = latest-prompt`, `freshness = updating`, and
+  `aiSummary.status = stale` or `updating`.
+- If no AI summary exists, use the most recent content-rich prompt and mark the
+  AI summary as `unavailable` or `fallback`.
+- If a fallback summary is raw prompt text, keep current task on prompt evidence
+  and mark `aiSummary.status = fallback`.
+
+## Summary Generation
+
+Change the AI prompt from a flat prompt list to latest-prompt-weighted input.
+
+Recommended input:
+
+```text
+Latest prompt:
+<most recent cleaned prompt>
+
+Recent context:
+1. <older prompt>
+2. <older prompt>
+3. <older prompt>
+```
+
+Recommended system prompt:
+
+```text
+Summarize the user's current task. Prioritize the latest prompt. Use older
+prompts only as context. Return 8-14 words. Return only the summary, no quotes
+or prefix.
+```
+
+The cached summary should record the prompt timestamp and prompt count used to
+generate it. This makes freshness testable without depending on provider speed.
+
+## Tooltip UX
+
+Replace the main `Intent` section with `Current Task`.
+
+Fresh state:
+
+```text
+CURRENT TASK
+[AI SUMMARY] [FRESH] [now]
+Fixing Wall-E session history restore and tooltip freshness.
+```
+
+Updating state:
+
+```text
+CURRENT TASK
+[LATEST PROMPT] [UPDATING] [now]
+Does WallE coding agent have auto compact logic like OpenCode or Claude?
+
+AI SUMMARY
+[STALE] [2m ago]
+Previous: Fixing session tab drag behavior and search issues.
+```
+
+Unavailable state:
+
+```text
+CURRENT TASK
+[LATEST PROMPT] [FALLBACK] [now]
+Fix the session tooltip freshness behavior.
+```
+
+Progress remains separate and should keep using assistant-event evidence.
+
+Interaction behavior:
+
+- Opening a tooltip fetches summary immediately.
+- If the tooltip is already open for that session, new stream events rerender it.
+- While visible, perform a lightweight refresh every 10 seconds.
+- Stop the refresh timer when the tooltip is hidden.
+- Clicking inside the tooltip continues to preserve it for copy/select.
+- Activating a different session tooltip replaces the current tooltip.
+
+## Implementation Plan
+
+Phase 1: documentation
+
+- Add this design note.
+- Commit documentation by itself after TL review.
+
+Phase 2: backend freshness
+
+- Extend `SessionStream` cached summary metadata with prompt timestamp/count.
+- Add `latestPrompt`, `aiSummary`, and `currentTask` to `getSummary()`.
+- Ensure `intent` maps to `currentTask`, not stale AI.
+- Update summary generation prompt to prioritize latest prompt.
+- Add unit tests for fresh, stale, fallback, and provider-lag cases.
+
+Phase 3: tooltip UI
+
+- Render `Current Task` from `currentTask`.
+- Render stale AI only as secondary context.
+- Add freshness/status pills.
+- Add visible-tooltip refresh and stream-event rerender.
+- Add browser/render coverage for a tooltip that updates while open.
+
+Phase 4: dev validation
+
+- Start isolated CTM via `ctm-dev`.
+- Verify service health on the dev port pair.
+- Exercise stale summary behavior through a real browser.
+- Confirm no browser console errors for the tooltip flow.
+- Commit implementation after TL review.
+
+## Test Matrix
+
+Backend:
+
+- AI summary generated after latest prompt: primary task is AI summary.
+- AI summary timestamp older than latest prompt: primary task is latest prompt.
+- AI provider slow or failed: primary task remains latest prompt.
+- Fallback summary raw prompt: primary task remains prompt evidence.
+- Summary generation input puts latest prompt before recent context.
+
+Frontend:
+
+- Tooltip opens and labels `Current Task`.
+- Stale AI is demoted into `AI Summary`.
+- New user stream event updates an already-open tooltip.
+- Fresh summary event promotes AI summary.
+- Tooltip remains copyable and dismisses only on outside click or another
+  tooltip activation.
+
+Dev validation:
+
+- CTM dev server starts on a random non-primary port.
+- `/api/services/status` succeeds on the dev port.
+- Browser test uses the dev port, never `3456` or `3457`.
+- Screenshot or DOM assertions prove the freshness state.

package/template/claude-task-manager/docs/session-ux-issue-review-2026-05-01.md

@@ -0,0 +1,369 @@
+# Claude/Codex Session UX Issue Review
+
+Date: 2026-05-01
+Scope: CTM session terminal UX, restore/restart behavior, status accuracy,
+search/review history, and Wall-E coding session integration.
+
+## Evidence Used
+
+- Wall-E MCP status check: the Wall-E MCP server is installed and exposes the
+  CTM session tools, but the live CTM DB health path currently reports
+  `ctm_db_schema_read_failed` with a `disk I/O error`.
+- CTM cached DB fallback: `~/.walle/data/task-manager.db` has the expected CTM
+  session tables and conversation blobs, but `session_messages` and
+  `session_messages_fts` are empty in the sampled cache.
+- Session corpus scan: recent Claude and Codex session JSONLs plus CTM
+  `session_conversations.messages` were scanned for direct user reports from
+  April 15 onward. The largest clusters were search/review, Wall-E/model, blank
+  or restore behavior, identity/restart, malformed output, input/approval, and
+  status accuracy.
+- Git history: recent CTM fixes show repeated work around blank tabs, Codex
+  terminal restore, output budget handling, status projection, search identity,
+  Wall-E default model hydration, and Codex rollout recovery.
+- Regression tests: the render and session tests now cover blank-tab retry,
+  worker saturation, Codex panel restore gaps, TUI redraw, terminal query
+  handling, output freeze, status typing, search dedupe, title preservation, and
+  Wall-E default model behavior.
+- Code review: current review focused on `public/index.html`, `server.js`,
+  `lib/session-history.js`, Wall-E MCP/session context code, and the Playwright
+  regression tests.
+
+## Issue Clusters
+
+### 1. Blank tabs, lost output, and restore gaps
+
+Observed symptoms:
+
+- Switching back to a long-running tab could show a blank terminal.
+- Restart or panel restore could lose visible output or show a large blank tail.
+- Codex tabs sometimes opened at the top or with the prompt far below the useful
+  output.
+- Review or live tab state could be overwritten by a stale shorter snapshot.
+
+Likely causes:
+
+- Snapshot cache was consumed too early, so a retry had no usable fallback.
+- Headless worker saturation caused soft timeouts; late worker responses were
+  dropped instead of cached.
+- Hidden terminals restored with stale dimensions, then painted an invalid
+  snapshot.
+- Browser WebGL state and xterm state could diverge after tab switches or
+  context loss.
+- Codex emits full-screen terminal clear sequences and can leave a viewport-sized
+  blank tail after ratatui redraws.
+
+Fixes already landed:
+
+- Preserve cached snapshots across attach/retry and keep late worker responses
+  usable after soft timeout.
+- Recreate WebGL renderers, force xterm refresh, and reflow on activation.
+- Reject dimension-mismatched snapshots until resize/reflow catches up.
+- Preserve browser-visible output when a stale server snapshot is shorter.
+- Compact Codex internal blank gaps and anchor follow behavior to the useful
+  prompt viewport.
+
+Representative coverage:
+
+- `blank-tab-retry-recovery.spec.js`
+- `blank-tab-worker-saturation.spec.js`
+- `codex-panel-restore-gap.spec.js`
+- `snapshot-restore-ordering.spec.js`
+- `dimension-mismatch.spec.js`
+
+### 2. Malformed TUI output and terminal freezes
+
+Observed symptoms:
+
+- Claude Code sticky status blocks duplicated or drifted.
+- Codex output could become visually "messed up" while typing.
+- Codex skill picker and helper input alignment could break.
+- Output sometimes froze after high-volume bursts or after a TUI exit.
+- Startup terminal queries could leak into the UI or back into the PTY.
+
+Likely causes:
+
+- The output-budget path split stateful ANSI frames across throttled writes.
+- Raw post-snapshot bytes could be double-applied after a snapshot restore.
+- Terminal query responses such as DSR/DA mode probes were not always consumed
+  silently.
+- Helper textarea position was not consistently aligned to the current xterm
+  viewport.
+- Blank-gap compaction was unsafe while Codex interactive picker UI was visible.
+
+Fixes already landed:
+
+- Treat headless xterm snapshots as the source of truth during output-budget
+  suppression.
+- Add a short post-snapshot grace/buffer so late raw bytes are applied in order.
+- Restore via reset plus FIFO writes to avoid stale state after snapshot reset.
+- Consume known terminal queries and add a real Codex DSR regression.
+- Align the Codex helper textarea to the viewport and avoid compacting during
+  active skill picker UI.
+
+Representative coverage:
+
+- `claude-tui-sticky-block.spec.js`
+- `tui-redraw.spec.js`
+- `tui-rerender.spec.js`
+- `output-freeze.spec.js`
+- `real-codex-dsr-query.spec.js`
+- `codex-clear-scrollback.spec.js`
+
+### 3. Running/idle/waiting status inaccuracies
+
+Observed symptoms:
+
+- Idle Claude and Codex sessions were shown as Running.
+- Active session list, overview, and command tab disagreed.
+- Switching tabs could cause an idle session to become Running.
+- Typing or UI-induced resize/reflow could be treated as agent activity.
+
+Likely causes:
+
+- TUI redraws and status-only terminal output were interpreted as fresh work.
+- Multiple status sources competed: server session payloads, stream status,
+  client PTY detection, standup projection, and overview cards.
+- Codex running holds were renewed from the wrong signal in some cases.
+- Freshness windows differed across UI surfaces.
+
+Fixes already landed:
+
+- Add status hooks/state bus and make live server status the authoritative
+  source while it is fresh.
+- Filter status-only ANSI/TUI noise and suppress UI-refresh output as activity.
+- Renew Codex running holds from detector receipts rather than incidental tab
+  redraws.
+- Align standup and active-session projections on the same live status payload.
+
+Representative coverage:
+
+- `status-detector.spec.js`
+- `session-status-typing.spec.js`
+- overview and standup browser checks from the recent fix passes
+
+### 4. Identity, title, context, and restart recovery
+
+Observed symptoms:
+
+- Sessions lost title or context after restart.
+- Some Codex sessions disappeared after CTM crashes.
+- Review tabs resolved to old prompts or wrong titles.
+- Duplicate titles such as repeated `task-manager` appeared.
+- Codex rollouts were found by the wrong file when IDs overlapped.
+
+Likely causes:
+
+- CTM tab IDs and provider session IDs are separate identities and were not
+  linked consistently on every path.
+- Codex `state_5.sqlite` could be corrupt, missing, or incomplete.
+- JSONL fallback did not always replay the full rollout history.
+- Rollout lookup used weak matching instead of exact UUID resolution.
+- Auto-title refresh could overwrite user or session-owned titles.
+
+Fixes already landed:
+
+- Use dual CTM/agent session IDs and backfill/migrate identities.
+- Resolve Codex rollouts by exact UUID and fall back to JSONL when state DB is
+  unavailable.
+- Replay full Codex rollout history instead of a partial latest fragment.
+- Harden title ownership, preserve explicit renames, and clean branch/title UI.
+
+Representative coverage:
+
+- `codex-title-preservation.spec.js`
+- `session-title-branch-badge.spec.js`
+- recent commits around Codex exact UUID resolution, JSONL fallback, and full
+  rollout replay
+
+### 5. Search, review, and session summary reliability
+
+Observed symptoms:
+
+- Search missed obvious terms such as `$publish`.
+- Search returned duplicate sessions for the same underlying agent run.
+- A summary could say "1 msg" even when the session had many messages.
+- Review panes could be empty while the live terminal had content.
+- Prompt history or prompt dropdown state could be stale.
+
+Likely causes:
+
+- Raw grep over Codex rollouts included developer context and skill files, which
+  produced false positives and hid user-message semantics.
+- Session dedupe did not consistently collapse CTM tab IDs, provider IDs,
+  provisional IDs, and JSONL paths.
+- Search filters and active-state filters could hide relevant historical rows.
+- In the sampled Wall-E cache, `session_conversations.messages` has content but
+  `session_messages` and `session_messages_fts` are empty, so any MCP/search path
+  relying only on message FTS will miss conversation content.
+- Prompt preview caches were not refreshed on every relevant session update.
+
+Fixes already landed:
+
+- Parse Codex sessions through the session-history layer instead of raw text
+  grep for user-visible search semantics.
+- Normalize command prefixes and dedupe session identities more aggressively.
+- Stabilize overview/session search state and live prompt preview refresh.
+
+Remaining gap:
+
+- Wall-E/CTM DB session search should either backfill `session_messages` and FTS
+  from `session_conversations.messages` or query the conversation JSON blobs as a
+  fallback. The cache sample shows this is still a real blind spot.
+
+### 6. Wall-E coding session model and MCP setup
+
+Observed symptoms:
+
+- New Wall-E coding sessions could have no default model.
+- Sending a command with no model selected could enter a busy loop.
+- Wall-E MCP showed as unsupported or unavailable for some users.
+- Npx installs did not reliably make the MCP usable in downstream agents.
+- MCP health did not clearly distinguish "installed" from "DB usable".
+
+Likely causes:
+
+- Model hydration happened too late or was not persisted into the session UI
+  state.
+- Duplicate sends were not blocked while the model handle was unresolved.
+- MCP install/setup checks focused on config presence, not full tool and DB
+  health.
+- The live CTM DB path can be unreadable from Wall-E even when the cached Wall-E
+  DB is healthy.
+
+Fixes already landed:
+
+- Hydrate a default model for Wall-E sessions and ignore duplicate busy sends.
+- Add browser coverage for Wall-E default model behavior.
+- Improve MCP health reporting so DB schema failures are visible.
+
+Remaining gap:
+
+- Make Wall-E MCP degrade gracefully when the configured CTM DB cannot be read:
+  fall back to the healthy Wall-E cache or CTM HTTP APIs, and surface the
+  degraded source in `walle_memory_status`.
+
+### 7. Input, focus, and approval UX
+
+Observed symptoms:
+
+- Input lag became severe under high CPU or heavy output.
+- Input could freeze after websocket reconnect.
+- Renaming, queue input, and toolbar focus could be interrupted by session
+  events.
+- Auto-approval sometimes missed Codex approval prompts.
+
+Likely causes:
+
+- High-volume output saturated the UI path and delayed input handling.
+- Reconnect and toolbar code did not always restore focus and local echo state.
+- Approval prompt parsing was too dependent on provider-specific terminal text.
+
+Fixes already landed:
+
+- Add input-freeze regressions and reconnect-focused fixes.
+- Improve approval transition snapshots so prompts and decisions do not corrupt
+  the terminal state.
+- Expand approval/provider parsing in the recent fix passes.
+
+## Codebase Improvement Opportunities
+
+### P0: Fix DB-first Wall-E session retrieval
+
+The current MCP surface is configured, but live CTM DB health can fail and the
+cached DB has conversation blobs without populated message/FTS rows. This is the
+largest remaining observability hole because it affects future debugging and
+agent memory retrieval.
+
+Recommended change:
+
+- In Wall-E session context search, if `session_messages_fts` is empty or the
+  configured CTM DB fails schema checks, fall back to:
+  - `session_conversations.messages` JSON search in the healthy cache, then
+  - CTM HTTP/session APIs if available.
+- Add a backfill/repair command to populate `session_messages` and
+  `session_messages_fts` from existing `session_conversations`.
+- Make `walle_memory_status` report the active source: live CTM DB, Wall-E cache,
+  or API fallback.
+
+### P1: Extract terminal restore and status state machines
+
+`public/index.html` now owns a large amount of terminal restore, blank watchdog,
+WebGL recovery, scroll anchoring, Codex compaction, and status precedence logic.
+The recent fixes are directionally correct, but the code is hard to reason about
+because the state machine is distributed across inline UI handlers.
+
+Recommended change:
+
+- Extract terminal restore logic into a small module with explicit states:
+  `inactive`, `attaching`, `snapshotRequested`, `restoring`, `live`,
+  `retrying`, `reflowing`, and `failed`.
+- Extract status precedence into a shared module or generated fixture used by
+  server/client tests.
+- Keep the UI layer as a renderer of state transitions, not the owner of every
+  transition rule.
+
+### P1: Add per-session diagnostic ring buffers
+
+Many past reports required reconstructing what happened from screenshots,
+terminal state, DB state, and git history.
+
+Recommended change:
+
+- Add a lightweight debug endpoint per session that records the last N restore
+  and status decisions:
+  - snapshot source and dimensions,
+  - cache hit/miss and timeout path,
+  - blank watchdog attempts,
+  - WebGL recreate/reflow events,
+  - status source and freshness,
+  - identity mapping and title owner.
+- Expose it in dev/debug mode so future agent fixes start from facts instead of
+  screenshots alone.
+
+### P2: Reconcile stale restore invariants in tests and comments
+
+One reviewed test comment says activation flushes the writer queue, while the
+current activation path clears queued writes before requesting an authoritative
+snapshot. That may be an intentional newer invariant, but it should be made
+explicit.
+
+Recommended change:
+
+- Update the test comments if the new invariant is correct.
+- Add a narrow regression proving post-snapshot buffered bytes are not lost when
+  activation clears stale queued bytes.
+
+### P2: Promote the session UX test pack into a standard smoke suite
+
+The project now has good focused tests, but the failure modes recur when fixes
+touch adjacent code.
+
+Recommended change:
+
+- Define a small "session UX smoke" command covering:
+  - blank tab restore,
+  - worker saturation retry,
+  - Codex panel restore gap,
+  - Codex clear scrollback,
+  - status typing,
+  - search dedupe,
+  - Wall-E default model.
+- Run it before CTM releases and after changes to `public/index.html`,
+  `server.js`, `lib/session-history.js`, Wall-E MCP code, or DB migrations.
+
+## Bottom Line
+
+Most of the user-visible regressions were not caused by a single missing check.
+They came from CTM treating live terminal bytes, headless snapshots, structured
+provider logs, persisted DB rows, and client-only UI state as interchangeable.
+The recent fixes work because they make one source authoritative for each job:
+
+- headless snapshots for visual terminal recovery,
+- structured session history for search/review semantics,
+- fresh server live status for status badges,
+- explicit CTM/agent IDs for identity,
+- hydrated model state for Wall-E command sends.
+
+The next quality jump is to make those authority boundaries explicit in code and
+diagnostics, especially for Wall-E DB-first retrieval and the terminal
+restore/status state machines.

package/template/claude-task-manager/fuzzy-utils.js

@@ -384,8 +384,16 @@ function keyboardProximityScore(a, b) {
 
 // --- Query Expansion Orchestrator ---
 
+function normalizeFuzzySearchValue(value) {
+  return String(value || '')
+    .trim()
+    .toLowerCase()
+    .replace(/(^|[\s([{])[$/]+(?=[a-z0-9_-])/g, '$1')
+    .replace(/\s+/g, ' ');
+}
+
 function expandQueryFuzzy(query) {
-  const terms = (query || '').trim().split(/\s+/).filter(t => t.length >= 2);
+  const terms = normalizeFuzzySearchValue(query).split(/\s+/).filter(t => t.length >= 2);
   if (terms.length === 0) return { ftsQuery: '', expansions: [] };
 
   const MAX_VARIANTS_PER_TERM = 8;
@@ -451,5 +459,5 @@ module.exports = {
   levenshtein, generate1EditVariants,
   doubleMetaphone, phoneticMatch,
   keyboardProximityScore, QWERTY_NEIGHBORS,
-  expandQueryFuzzy,
+  expandQueryFuzzy, normalizeFuzzySearchValue,
 };