@hienlh/ppm 0.17.14 → 0.17.16

package/nul/docs/journals/260414-2001-bundled-extensions.md

@@ -1,219 +0,0 @@
-# Bundled Extensions: Zero-Configuration First-Run Experience
-
-**Date**: 2026-04-14 20:01
-**Severity**: Medium
-**Component**: Extension discovery, manifest loading, CLI
-**Status**: Resolved
-
-## What Happened
-
-Completed implementation of bundled extensions — making ext-git-graph available out-of-the-box on first PPM install without requiring manual `ppm ext install`. Users now see 1 extension in the extension panel immediately after launch, removing the empty-state friction.
-
-**Approach**: Dual-path discovery — scan both `packages/ext-*` (bundled) and `~/.ppm/extensions/` (user-installed) in the same extension list. No copy/symlink operations, no database coupling. Simple, discoverable.
-
-**Core changes**:
-- Extended `discover()` to scan bundled dir relative to extension.service.ts
-- Added `extensionPaths` Map to track actual disk path per extension (bundled vs user)
-- User-installed extensions override bundled if same id
-- Bundled extensions cannot be removed, only disabled
-- CLI ext list now shows "Source" column (bundled/user)
-
-Total LOC: ~50 across 3 files (extension-manifest.ts, extension.service.ts, ext-cmd.ts)
-
-## The Brutal Truth
-
-This should have been trivial. It was exactly that simple in design — just scan another directory. Instead, the implementation exposed three separate bugs, all critical to correctness, which code review caught. None were obvious during development because they existed in different layers.
-
-The frustrating part: the bugs were semantic, not syntactic. TypeScript compiled cleanly. The extension service started cleanly. Tests passed. Then `ppm ext list` returned bundled extensions with undefined paths, CLI showed isBundled=false for bundled extensions, and memory leaked on shutdown. Each was a small logic error that cascaded through the system.
-
-The _dir path leak into the API response felt stupid in hindsight. We load manifests from disk, get paths back, and then... just stored them directly in the response DTO without stripping. A user installing PPM on a 15-character path like `/Users/hienlh/...` would get that path in the JSON, making the API response change every installation. This broke tests that expected stable IDs.
-
-The isBundled check was the most infuriating. We added an `isBundled()` method that consulted the `bundledIds` Set, but **never called `discover()` first**. So `bundledIds` was always empty. The CLI passed all its tests because `isBundled()` just returned false every time — technically consistent behavior. False positives would've been louder.
-
-The memory leak on shutdown was the kick in the teeth. We added state to track bundled extensions, but never cleaned it up in `terminateWorker()`. Hours later, if a process spawned/terminated multiple extensions, heap would grow. Tiny leak, but persistent.
-
-## Technical Details
-
-### Design: Approach A (Dual-Path Discovery)
-
-Initial proposal had two approaches:
-- **A**: Scan bundled dir + user dir, merge lists, track source
-- **B**: Copy bundled extensions to ~/.ppm/extensions/ on first run
-
-Chose A because B couples persistence to the install, requires migration logic, and wastes disk space. A is cleaner: bundled extensions live in their source tree, user extensions in their own dir, both discoverable.
-
-### Implementation
-
-**1. Bundled directory resolution** (extension.service.ts)
-```typescript
-import { dirname } from "path";
-
-// Find bundled extensions relative to this file's location
-const bundledDir = join(dirname(import.meta.dir), "../../packages");
-
-async discover() {
-  const [bundled, user] = await Promise.all([
-    this.scanDir(join(bundledDir, "ext-*")),
-    this.scanDir(join(this.extensionsDir, "*")),
-  ]);
-  
-  return [...bundled, ...user]; // User can override bundled
-}
-```
-
-**2. Path tracking** (extension.service.ts)
-```typescript
-const extensionPaths = new Map<string, string>();
-const bundledIds = new Set<string>();
-
-// After discovering
-for (const ext of bundledExts) {
-  extensionPaths.set(ext.id, ext.path);
-  bundledIds.add(ext.id);
-}
-```
-
-**3. Strip _dir from API responses** (extension-manifest.ts)
-```typescript
-export async function loadManifest(extensionPath: string): ManifestDTO {
-  const manifest = loadJSON(join(extensionPath, "ppm.json"));
-  
-  // Don't expose internal disk paths in API
-  const { _dir, ...safe } = manifest;
-  return safe;
-}
-```
-
-**4. CLI Source column** (ext-cmd.ts)
-```typescript
-const source = bundledIds.has(ext.id) ? "bundled" : "user";
-console.log(`${ext.id}\t${source}\t${ext.version}`);
-```
-
-### Code Review Issues (All Fixed)
-
-**C1: Critical — _dir path leak into responses**
-
-Symptom: `ppm ext info git-graph` returned `{ id, version, _dir: "/Users/hienlh/Projects/ppm/packages/..." }`. Made API response unstable across installations.
-
-Root cause: `loadManifest()` returned raw manifest object with internal fields.
-
-Fix: Destructure to strip `_dir`, `_internal`, and other private fields before returning DTO.
-
-**Files**: src/services/extension-manifest.ts (~5 lines)
-
-**H2: High — isBundled() always false**
-
-Symptom: CLI listed all extensions with `source: "user"` even for ext-git-graph. The `isBundled()` method existed but code path never called `discover()`.
-
-```typescript
-// Before
-isBundled(id: string): boolean {
-  return this.bundledIds.has(id); // bundledIds empty because discover() never ran
-}
-
-// After
-async isBundled(id: string): boolean {
-  await this.discover(); // Ensure bundledIds populated
-  return this.bundledIds.has(id);
-}
-```
-
-Root cause: Service initialization assumed `discover()` would be called by a handler. It was, but not before CLI commands tried to check `isBundled()`.
-
-Fix: Make `isBundled()` async and ensure discovery runs first. Callers must await.
-
-**Files**: src/services/extension.service.ts (~8 lines in isBundled, ~3 lines in CLI)
-
-**H3: High — Memory leak on shutdown**
-
-Symptom: Long-running process with repeated extension loads/unloads (e.g., test suite spawning 50+ worker instances). Heap grew monotonically.
-
-Root cause: `extensionPaths` and `bundledIds` Map/Set not cleared in `terminateWorker()`.
-
-```typescript
-// Added to terminateWorker()
-extensionPaths.clear();
-bundledIds.clear();
-```
-
-Impact: Each terminated worker orphaned extension metadata. Over 100 cycles, noticeable heap overhead.
-
-**Files**: src/services/extension.service.ts (~2 lines)
-
-## What We Tried
-
-1. **Copy-on-install approach** — Rejected early; adds migration complexity and disk duplication.
-
-2. **Symlinks to bundled** — Considered for "bundled extensions appear in user dir", but cross-platform symlink support varies (Windows requires elevation). Dual-path discovery is more portable.
-
-3. **Single merged manifest cache** — Initially tried loading all manifests upfront and caching. Changed to lazy loading per-request to avoid startup time regression.
-
-4. **Environment variable for bundled dir** — First draft used `BUNDLED_EXTENSIONS_DIR` env var. Changed to `import.meta.dir` (Bun-specific, build-time resolved) for testability and no reliance on runtime env.
-
-## Root Cause Analysis
-
-The bugs emerged because we added state (extensionPaths, bundledIds) without tight coupling to initialization. Services in TypeScript often have implicit initialization contracts ("discovery must run before use"), but these aren't type-checked.
-
-**Why did _dir leak through?**
-The manifest loading function was too permissive. We passed the raw `require("ppm.json")` result straight to the API. No explicit field allowlist. This is a pattern issue: loading functions should transform to safe DTOs, not return raw data structures.
-
-**Why was isBundled() broken?**
-Async state (discovering extensions) mixed with sync queries (isBundled check). The Set exists, but it's empty. This violated the principle: "if a method consults state, it should initialize that state or assume it's initialized." We did neither.
-
-**Why wasn't shutdown cleanup obvious?**
-The memory leak only appeared under stress (test suite with 50+ workers). In normal usage, PPM runs as a single long-lived process. Single-instance code often skips cleanup because it happens at process exit anyway. But test suites (and server reloads) spawn/terminate repeatedly, exposing the leak.
-
-## Lessons Learned
-
-1. **DTO transformation is explicit, not implicit** — Define a manifest loading function that returns the exact fields the API exposes. Don't pass raw object + assume API filters properly.
-
-2. **Async initialization contracts need typing** — If a method depends on prior async setup, the type system should reflect this. Consider:
-   ```typescript
-   // Bad: implicit contract
-   isBundled(id): boolean { return bundledIds.has(id); }
-   
-   // Better: explicit precondition
-   async isBundled(id): Promise<boolean> { 
-     await ensureDiscovered(); 
-     return bundledIds.has(id); 
-   }
-   ```
-
-3. **Dual-path patterns add discovery surface** — Now we scan two directories. Code must handle conflicts (user override bundled), precedence order (user first, so they win), and fallback (if user path deleted but bundled still exists). This is more correct than copy-on-install, but requires careful spec.
-
-4. **Cleanup is not just shutdown, it's between cycles** — Test suites expose cleanup issues that production doesn't. Always clear/reset mutable state in terminateWorker(), not just at exit.
-
-5. **Bundled != immutable** — We say "bundled extensions cannot be removed," but that's a policy, not architecture. The code still needs safeguards: `disable(id)` should check `if (isBundled) { disableOnly(); } else { uninstall(); }`. Small gate, big difference.
-
-## Next Steps
-
-1. **Add bundled extension e2e test** (this week) — Verify that fresh install discovers bundled extensions without user interaction. Acceptance: `ppm ext list | grep git-graph` returns "bundled" source.
-
-2. **Expand to other bundled extensions** (v0.9.87) — Currently only ext-git-graph is bundled. Plan to bundle ext-vscode-terminal and ext-github-copilot in upcoming releases. Our infrastructure is ready.
-
-3. **Update install docs** (today) — Release notes should call out "git-graph included by default". Users often miss this without explicit mention.
-
-4. **Monitor memory in long-running tests** (ongoing) — Heap profiler on test suite. If other services leak state on terminateWorker(), catch early.
-
----
-
-**Commits**:
-- One focused commit on bundled discovery (clean diff, 50 LOC)
-- UI improvements deferred to separate commits (as noted in other journals)
-
-**Tests**: 111 extension tests pass (18 new tests added for bundled discovery)
-- Test: discover() returns bundled + user extensions
-- Test: user extension overrides bundled (same id)
-- Test: isBundled() returns true for bundled, false for user
-- Test: CLI lists source column correctly
-- Test: disable bundled extension doesn't remove it from list
-
-**Code Review**: Passed after fixes to all 3 issues (C1, H2, H3)
-
-**Files Modified**:
-- `/Users/hienlh/Projects/ppm/src/services/extension-manifest.ts`
-- `/Users/hienlh/Projects/ppm/src/services/extension.service.ts`
-- `/Users/hienlh/Projects/ppm/src/commands/ext-cmd.ts`
-
-**Unresolved Questions**: None. Approach A proven stable through first-run UX testing.

package/nul/docs/journals/260415-frontend-memory-optimization.md

@@ -1,73 +0,0 @@
-# Frontend Memory Optimization: Four Phases of Rendering & Bundle Reduction
-
-**Date**: 2026-04-15 18:00
-**Severity**: Medium
-**Component**: Frontend (React, Vite bundles)
-**Status**: Resolved (with incident)
-
-## What Happened
-
-Completed 4-phase memory and rendering optimization across the React codebase. Added shallow equality checks, memoization, lazy loading, and code splitting to reduce initial bundle size and re-render overhead. Process was interrupted by a critical git incident mid-session.
-
-## The Brutal Truth
-
-This was simultaneously a win and a disaster. The optimization work itself is solid—17 files now use `useShallow` to prevent unnecessary re-renders, 11 heavy components wrapped in `React.memo`, and critical libraries (mermaid, markdown, xterm) are now lazy-loaded. But a subagent ran `git reset --hard HEAD` without warning, destroying ALL uncommitted changes. Had to re-implement everything from scratch. Incredibly frustrating because the technical work was already done—it just got erased.
-
-## Technical Details
-
-**Phase 1 (Shallow Equality)**
-- Added `useShallow` to 17 files with destructured store selector calls
-- Prevents re-renders when store object reference changes but content doesn't
-- Affected: hooks, chat, sidebar, tree, editor, terminal components
-
-**Phase 2 (Chat Pagination)**
-- PAGE_SIZE=50 pagination with load-more button in message-list.tsx
-- Capped teamActivityRef at 500 messages in use-chat.ts
-- Prevents DOM from holding thousands of message elements
-
-**Phase 3 (Lazy Loading)**
-- React.lazy for MarkdownRenderer (3 call sites, 175KB bundle)
-- Dynamic import for mermaid in markdown-code-block.tsx with promise caching
-- React.lazy for CodeMirror in postgres-viewer.tsx
-- Fallback suspense boundaries in place
-
-**Phase 4 (Code Splitting)**
-- Vite manualChunks: vendor-monaco (lazy), vendor-mermaid (2.4MB lazy), vendor-xterm (344KB lazy), vendor-markdown (597KB lazy), vendor-ui (128KB shared)
-- React.memo on TreeNode
-- Initial bundle reduced to 495KB
-
-## Code Review Fixes Applied
-
-1. Rules of Hooks violation in MessageList—moved hooks above early returns
-2. Pagination reset on every streaming tick—fixed deps to stable `messages[0]?.id`
-3. onFork inline closure defeating MessageBubble memo—made stable with useCallback
-4. Variable shadowing (`s` in tool-cards.tsx)—renamed to `state`
-5. Mermaid import race condition—caching promise instead of module
-
-## What We Tried
-
-Had the work implemented successfully. Subagent spawned mid-session executed `git reset --hard HEAD` as part of cleanup routine, destroying all 26 modified files in one command. Recovered by re-implementing changes from notes and code review feedback.
-
-## Root Cause Analysis
-
-The incident happened because there was no git safety protocol when spawning subagents. The subagent was clearing uncommitted work thinking it was cleaning up, not realizing the changes hadn't been committed yet. No blocking commit before delegation meant work was vulnerable.
-
-The optimization work itself: good decisions across the board. Shallow equality catches the most common re-render patterns. Lazy loading only targets heavy libraries (mermaid is 2.4MB—absolutely needs deferral). Pagination capping is necessary given unbounded message streams. Code splitting is strategic: vendor chunks reduce main bundle pressure.
-
-## Lessons Learned
-
-1. **Always commit before spawning subagents.** Even exploratory work needs a safety commit. Do not delegate with uncommitted changes in working tree.
-2. **Shallow equality is the most impactful optimization.** It catches destructured store selectors without requiring component restructuring. Worth doing broadly.
-3. **Lazy load by file size threshold.** Mermaid (2.4MB), markdown renderer (175KB), and xterm (344KB) were obvious candidates. Sub-100KB libraries don't justify the async boundary cost.
-4. **Pagination is non-negotiable for unbounded collections.** Streaming chat creates thousands of messages. Without cap, DOM pressure degrades linearly.
-5. **Code review catches subtle bugs.** The Rules of Hooks violation and race condition were hard to spot in original implementation but obvious in review.
-
-## Next Steps
-
-1. Commit optimizations with `git commit -m "perf: frontend memory optimization — shallow equality, lazy loading, pagination"`
-2. Test pagination UX on low-end devices to verify load-more doesn't feel janky
-3. Monitor bundle size in CI to catch regression
-4. Document code splitting strategy in `docs/performance.md`
-
-**Commits:** 26 files modified, 178 insertions, 88 deletions
-**Bundle impact:** Initial +0% (495KB), lazy vendor chunks defer 3.5MB to first use

package/nul/docs/lessons-learned.md

@@ -1,70 +0,0 @@
-# Lessons Learned
-
-Knowledge and gotchas discovered during PPM development.
-
----
-
-## Claude Agent SDK
-
-### .env poisoning via project cwd
-
-**Problem**: SDK spawns a CLI process in the project's `cwd`. The CLI auto-loads `.env` via dotenv. If the project has `ANTHROPIC_API_KEY=dummy` or `ANTHROPIC_BASE_URL=http://localhost:...`, the CLI uses those instead of the user's subscription → "Invalid API key" or empty responses with no tool execution.
-
-**Symptoms**:
-- Model returns empty response, no text or tool_use events
-- `result.subtype === "error_during_execution"`
-- Text response: "Invalid API key · Fix external API key"
-- `totalCostUsd: 0` in usage
-
-**Fix**: Neutralize `ANTHROPIC_*` env vars by setting them to empty string (not deleting — dotenv won't override existing vars):
-```ts
-env: {
-  ...process.env,
-  ANTHROPIC_API_KEY: "",
-  ANTHROPIC_BASE_URL: "",
-  ANTHROPIC_AUTH_TOKEN: "",
-},
-```
-
-**File**: `src/providers/claude-agent-sdk.ts`
-
----
-
-### Project-local Claude settings restrict tools
-
-**Problem**: Projects may have `.claude/settings.local.json` with restrictive `permissions.allow` lists (e.g., only `Bash(python:*)`, `Bash(ls:*)`). Even with `permissionMode: "bypassPermissions"`, the SDK CLI still reads these and restricts available tools.
-
-**Fix**: Override with explicit empty settings and no setting sources:
-```ts
-settings: { permissions: { allow: [], deny: [] } },
-settingSources: [],
-```
-
----
-
-### Windows: SDK query() hangs — executable: "node" fix
-
-**Problem**: On Windows + Bun, SDK `query()` yields zero events — appears to hang forever.
-
-**Root cause**: SDK detects Bun runtime and spawns `child_process.spawn("bun", ["cli.js", ...])`. On Windows, `child_process.spawn("bun")` fails with ENOENT (can't resolve `bun` binary). The error is swallowed internally → no events → looks like a hang.
-
-**Fix**: Pass `executable: "node"` in SDK query options. Forces SDK to spawn `node cli.js` instead of `bun cli.js`. Node is always in PATH on Windows.
-
-**File**: `src/providers/claude-agent-sdk.ts` — `queryOptions` in `sendMessage()`
-
----
-
-## WebSocket Chat Architecture
-
-### Event flow: SDK → Provider → WS → Frontend
-
-1. SDK emits: `system` → `stream_event`* → `assistant` → `rate_limit_event` → `user` (tool_result) → `result`
-2. Provider extracts text from `stream_event.event.delta.text` and tool_use from `assistant.message.content`
-3. Provider yields: `text`, `tool_use`, `tool_result`, `usage`, `done`
-4. WS handler sends JSON to frontend
-
-Key: `stream_event` contains raw API events (`content_block_delta` with `text_delta`). The `assistant` event contains the full message with all content blocks.
-
-### tool_result lives in `user` events
-
-SDK returns tool results as `user` type messages (not `assistant`). Provider fetches them via `getSessionMessages()` after detecting `pendingToolCount > 0`.