@rubytech/create-sitedesk-code 0.1.524 → 0.1.526

package/payload/platform/docs/superpowers/plans/2026-07-26-task-2023-sidebar-sessions-async-reads.md

@@ -1,394 +0,0 @@
-# Task 2023 — sidebar-sessions async reads Implementation Plan
-
-> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
-
-**Goal:** Stop `computeSessionList` from blocking the one event loop for the whole duration of its filesystem walk, without changing what the endpoint returns.
-
-**Architecture:** The route already sits behind the Task 1986 per-account cache and its compute is already `async` and awaited. The block is not the awaiting; it is that every read inside the compute is synchronous, so the loop cannot run anything else from the first `readFileSync` to the last. Replacing the per-row reads with `node:fs/promises` gives the loop a yield point per row. Directory enumeration is left alone on measurement, not preference (see Global Constraints).
-
-**Tech Stack:** TypeScript, Node 22, Hono, vitest.
-
-## Global Constraints
-
-- **Measured cost split, maxy 2026-07-26, 52 files / 133.9 MB / 38,042 lines, two identical runs:** `enumMs=1`, `readMs=284.5` then `277.4`, `parseMs=212.3` then `216.6`.
-- **The four `readdirSync` calls cost 1 ms and are NOT converted.** `enumerateJsonls` is imported by six other server modules; forking it for 1 ms would trade a measurable regression risk for no measurable gain. This is a deliberate deviation from the task file's problem statement, which names lines 216, 227, 240 and 255 alongside the reads that do cost.
-- **JSON parsing is ~215 ms of pure CPU and no async change removes it.** The deliverable is that it stops arriving as one 215 ms block and arrives as ~52 chunks separated by I/O yields. Do not claim the parse cost is eliminated.
-- **Reads stay sequential.** The current loop holds one `body` at a time and lets it go out of scope each iteration. `Promise.all` over 52 files would hold 133.9 MB at once. Sequential `await` yields the loop without changing the memory profile.
-- **No behaviour change.** Response shape, field values, ordering, census log lines and cache states are all byte-identical.
-- **Existing sync exports keep their signatures.** `enumerateJsonls`, `loadUserTitles`, `readSidecarMeta` and `readBridgeIds` are consumed by `server/index.ts`, `public-reader.ts`, `webchat.ts`, `whatsapp-reader.ts`, `files.ts` and two test files. None of those are in scope.
-- **One parser per format.** Where a sync and an async reader both exist, they share a single exported pure parse function. Two copies of a parse body is a drift registry and is not acceptable.
-
----
-
-## File Structure
-
-- `server/routes/admin/sidebar-sessions.ts` — the only production file modified. Gains `parseUserTitles`, `parseSidecarMeta`, `loadUserTitlesAsync`, `readSidecarMetaAsync`; `computeSessionList`'s row loop switches to `fs/promises`.
-- `server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts` — new, holds this task's own tests.
-
----
-
-### Task 1: Split the two parsers out of their sync readers
-
-**Files:**
-- Modify: `platform/ui/server/routes/admin/sidebar-sessions.ts:308-335` (`loadUserTitles`), `:371-402` (`readSidecarMeta`)
-- Test: `platform/ui/server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts`
-
-**Interfaces:**
-- Produces: `parseUserTitles(raw: string): Map<string, string>` and `parseSidecarMeta(raw: string): SidecarMeta`. Both take the file's raw text and never touch the filesystem. Both degrade to empty/null on a parse failure or wrong shape, exactly as the current readers do.
-- Consumes: nothing.
-
-- [ ] **Step 1: Write the failing test**
-
-```ts
-import { describe, it, expect } from 'vitest'
-import { parseUserTitles, parseSidecarMeta } from '../sidebar-sessions'
-
-describe('parseUserTitles', () => {
-  it('keeps a trimmed string title and drops every malformed entry', () => {
-    const raw = JSON.stringify({
-      good: { title: '  renamed  ' },
-      blankTitle: { title: '   ' },
-      wrongType: { title: 42 },
-      notAnObject: 'nope',
-      '': { title: 'no id' },
-    })
-    const out = parseUserTitles(raw)
-    expect(out.get('good')).toBe('renamed')
-    expect([...out.keys()]).toEqual(['good'])
-  })
-
-  it('returns an empty map for unparseable or non-object JSON', () => {
-    expect(parseUserTitles('{not json').size).toBe(0)
-    expect(parseUserTitles('[1,2,3]').size).toBe(0)
-    expect(parseUserTitles('null').size).toBe(0)
-  })
-})
-
-describe('parseSidecarMeta', () => {
-  it('reads every field and filters non-string bridgeIds', () => {
-    const raw = JSON.stringify({
-      bridgeIds: ['a', '', 7, 'b'],
-      role: 'admin',
-      channel: 'webchat',
-      senderId: '+447700900000',
-      startedAt: '2026-07-26T00:00:00.000Z',
-      personId: 'p1',
-      adminUserId: 'u1',
-      visitorId: 'v1',
-      agentSlug: 'slug',
-      accountId: 'acct',
-    })
-    expect(parseSidecarMeta(raw)).toEqual({
-      bridgeIds: ['a', 'b'],
-      role: 'admin',
-      channel: 'webchat',
-      senderId: '+447700900000',
-      startedAt: '2026-07-26T00:00:00.000Z',
-      personId: 'p1',
-      adminUserId: 'u1',
-      visitorId: 'v1',
-      agentSlug: 'slug',
-      accountId: 'acct',
-    })
-  })
-
-  it('returns the empty shape for unparseable JSON', () => {
-    expect(parseSidecarMeta('{not json')).toEqual({
-      bridgeIds: [], role: null, channel: null, senderId: null, startedAt: null,
-      personId: null, adminUserId: null, visitorId: null, agentSlug: null, accountId: null,
-    })
-  })
-})
-```
-
-- [ ] **Step 2: Run test to verify it fails**
-
-Run: `cd maxy-code/platform/ui && npx vitest run server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts`
-Expected: FAIL — `parseUserTitles is not a function` (the export does not exist yet).
-
-- [ ] **Step 3: Write minimal implementation**
-
-Replace the body of `loadUserTitles` from the `JSON.parse` onward with a call to a new exported pure function, and do the same for `readSidecarMeta`. The sync readers keep their exact signatures and their `readFileSync` + catch.
-
-```ts
-/** Pure half of {@link loadUserTitles}: parse the on-disk `session-titles.json`
- *  text. Shared with the async reader so one format has one parser. */
-export function parseUserTitles(raw: string): Map<string, string> {
-  const out = new Map<string, string>()
-  let parsed: unknown
-  try { parsed = JSON.parse(raw) } catch { return out }
-  if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return out
-  for (const [sessionId, value] of Object.entries(parsed as Record<string, unknown>)) {
-    if (!sessionId) continue
-    if (!value || typeof value !== 'object') continue
-    const v = value as { title?: unknown }
-    if (typeof v.title !== 'string') continue
-    const trimmed = v.title.trim()
-    if (!trimmed) continue
-    out.set(sessionId, trimmed)
-  }
-  return out
-}
-
-export function loadUserTitles(accountDir: string | null): Map<string, string> {
-  if (!accountDir) return new Map()
-  let raw: string
-  try { raw = readFileSync(join(accountDir, 'session-titles.json'), 'utf8') } catch { return new Map() }
-  return parseUserTitles(raw)
-}
-```
-
-```ts
-// A factory, not a shared constant. `{ ...CONST }` would copy the REFERENCE to
-// one `bridgeIds` array into every empty result, so a single caller mutating it
-// would corrupt every future empty sidecar in the process.
-function emptySidecarMeta(): SidecarMeta {
-  return {
-    bridgeIds: [], role: null, channel: null, senderId: null, startedAt: null,
-    personId: null, adminUserId: null, visitorId: null, agentSlug: null, accountId: null,
-  }
-}
-
-/** Pure half of {@link readSidecarMeta}: parse a `.meta.json` sidecar's text.
- *  Shared with the async reader so one format has one parser. */
-export function parseSidecarMeta(raw: string): SidecarMeta {
-  let parsed: unknown
-  try { parsed = JSON.parse(raw) } catch { return emptySidecarMeta() }
-  if (!parsed || typeof parsed !== 'object') return emptySidecarMeta()
-  const r = parsed as { bridgeIds?: unknown; role?: unknown; channel?: unknown; senderId?: unknown; startedAt?: unknown; personId?: unknown; adminUserId?: unknown; visitorId?: unknown; agentSlug?: unknown; accountId?: unknown }
-  const bridgeIds = Array.isArray(r.bridgeIds)
-    ? r.bridgeIds.filter((b): b is string => typeof b === 'string' && b.length > 0)
-    : []
-  return {
-    bridgeIds,
-    role: typeof r.role === 'string' ? r.role : null,
-    channel: typeof r.channel === 'string' ? r.channel : null,
-    senderId: typeof r.senderId === 'string' ? r.senderId : null,
-    startedAt: typeof r.startedAt === 'string' ? r.startedAt : null,
-    personId: typeof r.personId === 'string' ? r.personId : null,
-    adminUserId: typeof r.adminUserId === 'string' ? r.adminUserId : null,
-    visitorId: typeof r.visitorId === 'string' ? r.visitorId : null,
-    agentSlug: typeof r.agentSlug === 'string' ? r.agentSlug : null,
-    accountId: typeof r.accountId === 'string' ? r.accountId : null,
-  }
-}
-
-export function readSidecarMeta(metaPath: string): SidecarMeta {
-  let raw: string
-  try { raw = readFileSync(metaPath, 'utf8') } catch { return emptySidecarMeta() }
-  return parseSidecarMeta(raw)
-}
-```
-
-- [ ] **Step 4: Run tests to verify they pass**
-
-Run: `cd maxy-code/platform/ui && npx vitest run server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts server/routes/admin/__tests__/sidebar-sessions.test.ts server/routes/admin/__tests__/resolve-title.test.ts`
-Expected: PASS on all three files. The two pre-existing files are the regression guard on the sync readers whose bodies just moved.
-
-- [ ] **Step 5: Commit**
-
-```bash
-git add maxy-code/platform/ui/server/routes/admin/sidebar-sessions.ts maxy-code/platform/ui/server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts
-git commit -m "refactor(2023): split the sidecar and user-title parsers out of their sync readers"
-```
-
----
-
-### Task 2: Add the async readers and switch the row loop onto them
-
-**Files:**
-- Modify: `platform/ui/server/routes/admin/sidebar-sessions.ts:70` (imports), `:572` (`loadUserTitles` call site), `:642-655` (row loop reads)
-- Test: `platform/ui/server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts`
-
-**Interfaces:**
-- Consumes: `parseUserTitles`, `parseSidecarMeta`, `emptySidecarMeta` from Task 1.
-- Produces: `loadUserTitlesAsync(accountDir: string | null): Promise<Map<string, string>>` and `readSidecarMetaAsync(metaPath: string): Promise<SidecarMeta>`. Same degradation rules as their sync twins.
-
-- [ ] **Step 1: Write the failing test**
-
-The load-bearing assertion is that the loop yields. A test that only checks the returned rows would pass just as well against the synchronous version, which is the dead-signal case of Task 1730. So the test races a timer against the compute: with synchronous reads the timer cannot fire until the walk finishes; with async reads it fires during it.
-
-```ts
-import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
-import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'
-import { tmpdir } from 'node:os'
-import { join } from 'node:path'
-import { loadUserTitlesAsync, readSidecarMetaAsync } from '../sidebar-sessions'
-
-let dir: string
-beforeEach(() => { dir = mkdtempSync(join(tmpdir(), 'sbs-async-')) })
-afterEach(() => { rmSync(dir, { recursive: true, force: true }) })
-
-describe('loadUserTitlesAsync', () => {
-  it('reads and parses the same file as the sync twin', async () => {
-    writeFileSync(join(dir, 'session-titles.json'), JSON.stringify({ s1: { title: 'renamed' } }))
-    expect((await loadUserTitlesAsync(dir)).get('s1')).toBe('renamed')
-  })
-
-  it('returns an empty map for a null accountDir and for a missing file', async () => {
-    expect((await loadUserTitlesAsync(null)).size).toBe(0)
-    expect((await loadUserTitlesAsync(join(dir, 'absent'))).size).toBe(0)
-  })
-})
-
-describe('readSidecarMetaAsync', () => {
-  it('reads and parses the same file as the sync twin', async () => {
-    const p = join(dir, 'x.meta.json')
-    writeFileSync(p, JSON.stringify({ role: 'admin', channel: 'webchat' }))
-    const meta = await readSidecarMetaAsync(p)
-    expect(meta.role).toBe('admin')
-    expect(meta.channel).toBe('webchat')
-  })
-
-  it('returns the empty shape for a missing file', async () => {
-    expect((await readSidecarMetaAsync(join(dir, 'absent.meta.json'))).bridgeIds).toEqual([])
-  })
-})
-
-describe('the row loop yields to the event loop', () => {
-  it('lets a pending timer fire while the walk is still running', async () => {
-    const root = join(dir, 'projects', 'slug')
-    mkdirSync(root, { recursive: true })
-    // Bodies large enough that the walk cannot complete inside one tick.
-    const line = JSON.stringify({ type: 'user', message: { role: 'user', content: 'x'.repeat(400) } })
-    for (let i = 0; i < 40; i++) {
-      const id = `0000000${i.toString().padStart(2, '0')}-0000-4000-8000-000000000000`
-      writeFileSync(join(root, `${id}.jsonl`), `${line}\n`.repeat(500))
-    }
-
-    let firedDuringWalk = false
-    const walk = readAllBodiesForTest(join(dir, 'projects'))
-    setTimeout(() => { firedDuringWalk = true }, 0)
-    await walk
-    expect(firedDuringWalk).toBe(true)
-  })
-})
-```
-
-- [ ] **Step 2: Run test to verify it fails**
-
-Run: `cd maxy-code/platform/ui && npx vitest run server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts`
-Expected: FAIL — `loadUserTitlesAsync is not a function`, and `readAllBodiesForTest is not defined`.
-
-- [ ] **Step 3: Write minimal implementation**
-
-Add the promises import beside the existing sync one, add the two async readers, and export the loop's read half as `readAllBodiesForTest` so the yield test drives the shipped code rather than a hand-copied twin.
-
-```ts
-import { readdirSync, readFileSync, statSync } from 'node:fs'
-import { readFile, stat } from 'node:fs/promises'
-```
-
-```ts
-/** Async twin of {@link loadUserTitles}. Same parser, same degradation. Used by
- *  computeSessionList so the request path never blocks the loop on this read. */
-export async function loadUserTitlesAsync(accountDir: string | null): Promise<Map<string, string>> {
-  if (!accountDir) return new Map()
-  let raw: string
-  try { raw = await readFile(join(accountDir, 'session-titles.json'), 'utf8') } catch { return new Map() }
-  return parseUserTitles(raw)
-}
-
-/** Async twin of {@link readSidecarMeta}. Same parser, same degradation. */
-export async function readSidecarMetaAsync(metaPath: string): Promise<SidecarMeta> {
-  let raw: string
-  try { raw = await readFile(metaPath, 'utf8') } catch { return emptySidecarMeta() }
-  return parseSidecarMeta(raw)
-}
-
-/** The row loop's read half, exported only so the yield test drives the shipped
- *  code. Sequential by design: `Promise.all` here would hold every body in
- *  memory at once (133.9 MB measured on maxy 2026-07-26). */
-export async function readAllBodiesForTest(projectsRoot: string): Promise<number> {
-  let n = 0
-  for (const { path } of enumerateJsonls(projectsRoot)) {
-    try { await readFile(path, 'utf8'); await stat(path); n++ } catch { continue }
-  }
-  return n
-}
-```
-
-In `computeSessionList`, change line 572 and the row loop's reads:
-
-```ts
-  const userTitles = await loadUserTitlesAsync(accountDir)
-```
-
-```ts
-    let body: string
-    let mtimeMs: number
-    try {
-      body = await readFile(path, 'utf8')
-      mtimeMs = (await stat(path)).mtimeMs
-    } catch {
-      continue
-    }
-```
-
-```ts
-    const { bridgeIds, role, channel: sidecarChannel, senderId, personId, adminUserId, accountId: rowAccountId } = await readSidecarMetaAsync(metaPath)
-```
-
-- [ ] **Step 4: Run tests to verify they pass**
-
-Run: `cd maxy-code/platform/ui && npx vitest run server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts`
-Expected: PASS, including the yield test.
-
-- [ ] **Step 5: Mutation-check the yield test**
-
-Temporarily revert `readAllBodiesForTest` to `readFileSync` / `statSync` and re-run. Expected: the yield test FAILS with `firedDuringWalk` false. Restore the async version. A yield test that passes against synchronous reads is measuring nothing.
-
-- [ ] **Step 6: Typecheck and run the full change surface**
-
-Run: `cd maxy-code/platform/ui && npx tsc --noEmit -p tsconfig.json`
-Expected: 0 errors.
-
-Run: `cd maxy-code/platform/ui && npx vitest run server/routes/admin/__tests__/ server/lib/__tests__/route-cache.test.ts server/__tests__/edge-request-timing.test.ts`
-Expected: PASS. `route-cache-watchers.test.ts` fails only when batched with its siblings and passes 4/4 alone; that is Task 2014 and predates this change.
-
-- [ ] **Step 7: Commit**
-
-```bash
-git add maxy-code/platform/ui/server/routes/admin/sidebar-sessions.ts maxy-code/platform/ui/server/routes/admin/__tests__/sidebar-sessions-async-reads.test.ts
-git commit -m "perf(2023): read session bodies and sidecars asynchronously so the walk yields the loop"
-```
-
----
-
-### Task 3: Correct the task file's own criterion and figures
-
-**Files:**
-- Modify: `maxy-code/.tasks/2023-sidebar-sessions-sync-fs-blocks-event-loop.md`
-
-**Interfaces:** none.
-
-- [ ] **Step 1: Replace the p99 verification criterion**
-
-The task says a p99 approaching the request durations confirms and a low p99 refutes. Fourteen consecutive windows measured on maxy read `p99=51` or `52` including one with `max=663`, because one blocked tick out of ~590 samples sits nowhere near the 99th percentile. Rewrite the Observability and Verification sections to turn on `max`, which is the statistic that sees a single sub-second block and which Task 2024 also shipped.
-
-- [ ] **Step 2: Replace the 620 ms figure**
-
-The `p50 620 ms across 27,280 requests` sample is undatable and mostly predates the Task 1986 route cache, which landed 2026-07-25. State the datable post-cache distribution instead: `n=38 p50=2 p90=603 p99=678 max=1161`, and that the cost is one full walk per 30 s poll because the poll interval equals `BACKSTOP_MS`.
-
-- [ ] **Step 3: Record the enumeration deviation**
-
-State that the four `readdirSync` calls were measured at 1 ms and deliberately left synchronous, with the six other importers of `enumerateJsonls` as the reason.
-
-- [ ] **Step 4: Commit**
-
-```bash
-git add maxy-code/.tasks/2023-sidebar-sessions-sync-fs-blocks-event-loop.md
-git commit -m "docs(2023): correct the p99 criterion and the pre-cache 620ms figure"
-```
-
----
-
-## Verification (Phase 4, on-device)
-
-The unit tests prove the loop yields. They cannot prove the lag falls on real data, so the sprint's claim rests on a device reading taken the same way as the before reading:
-
-- Before, already recorded: fourteen windows, `max` 310 to 708 ms in every window containing a `cache=stale-backstop` recompute, `max` 52 to 53 ms in every window without one.
-- After: the same grep across a window of real dashboard traffic. A window containing a `stale-backstop` recompute should no longer show a `max` near the recompute's own `ms=`.
-- Refutation case: if `max` still tracks the recompute duration after the change, the remaining block is the ~215 ms of parsing and the change did not achieve its goal. That is a reportable result, not a failed change.
-
-This step needs an installer publish to reach the device and is therefore operator-gated.