groundswell 0.0.2 → 1.0.0

package/plan/001_d3bb02af4886/bugfix/001_e8e04329daf3/P1M3T2S2/research/external_research.md

@@ -1,788 +0,0 @@
-# External Research: Incremental Tree Mutation Handling
-
-**Research Date:** 2026-01-12
-**Purpose:** P1M3T2S2 - Implement incremental node map updates for WorkflowTreeDebugger
-**Focus:** BFS/DFS traversal, Map performance, event dispatching, and tree mutation patterns
-
----
-
-## 1. React's Reconciliation & Virtual DOM Patterns
-
-### 1.1 Key Concepts from React's Diffing Algorithm
-
-React's reconciliation algorithm provides excellent patterns for incremental tree updates:
-
-**O(n) Algorithm for Tree Diffing:**
-- React assumes two trees of different elements will produce different trees
-- Uses heuristics to reduce O(n³) problem to O(n)
-- Key heuristic: If elements have different types, entire subtrees are different
-
-**Source:** [React Documentation - Reconciliation](https://react.dev/learn/render-and-commit)
-- **Section:** "React's Diffing Algorithm"
-- **Key Pattern:** Component type comparison for subtree replacement decisions
-
-### 1.2 Applicable Patterns for Tree Debugger
-
-**Pattern 1: Key-based Identification**
-```typescript
-// React uses keys to identify nodes across renders
-// Similarly, nodeMap uses node.id for O(1) lookups
-
-// Current implementation (good):
-private nodeMap: Map<string, WorkflowNode> = new Map();
-
-// Pattern: Always use stable identifiers for tree nodes
-// Avoid using array indices as identifiers (breaks on insertions/deletions)
-```
-
-**Pattern 2: Structural Change Detection**
-```typescript
-// React detects structural changes by comparing element types
-// For tree debugger, detect structural changes by event type:
-
-onEvent(event: WorkflowEvent): void {
-  switch (event.type) {
-    case 'childAttached':
-      // Incremental: Add only new subtree
-      this.addSubtreeToMap(event.child);
-      break;
-    case 'childDetached':
-      // Incremental: Remove only detached subtree
-      this.removeSubtreeFromMap(event.child);
-      break;
-    default:
-      // Non-structural: No map update needed
-      break;
-  }
-}
-```
-
-**Pattern 3: Batching Updates**
-```typescript
-// React batches multiple state updates
-// Apply similar pattern for rapid tree changes:
-
-private pendingUpdates: Array<() => void> = [];
-private updateScheduled = false;
-
-private scheduleUpdate(fn: () => void) {
-  this.pendingUpdates.push(fn);
-  if (!this.updateScheduled) {
-    this.updateScheduled = true;
-    Promise.resolve().then(() => {
-      this.pendingUpdates.forEach(fn => fn());
-      this.pendingUpdates = [];
-      this.updateScheduled = false;
-    });
-  }
-}
-```
-
-### 1.3 Virtual DOM Memory Efficiency
-
-**Insights:**
-- Virtual DOM uses object pooling for frequently created objects
-- Consider object pooling for WorkflowEvent if many events are created
-- Use structural sharing: Only copy changed branches
-
----
-
-## 2. JavaScript Map Manipulation & Performance
-
-### 2.1 MDN Map Best Practices
-
-**Source:** [MDN Web Docs - Map](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map)
-
-**Key Performance Characteristics:**
-- **Operations:** `set()`, `get()`, `has()`, `delete()` are O(1) average case
-- **Iteration:** `forEach()` and `for...of` are optimized by V8
-- **Memory:** Maps have slightly more overhead than Objects, but better for频繁添加/删除
-
-### 2.2 V8 Engine Optimizations
-
-**Source:** [V8 Blog - Map Performance](https://v8.dev/blog/elements-kinds) (Related concepts)
-
-**Optimization Patterns:**
-
-**Pattern 1: Preserve Map Reference**
-```typescript
-// BAD: Creates new Map frequently
-function updateNodeMap(nodes: WorkflowNode[]): Map<string, WorkflowNode> {
-  const newMap = new Map();
-  nodes.forEach(node => newMap.set(node.id, node));
-  return newMap; // Forces garbage collection of old Map
-}
-
-// GOOD: Mutate existing Map
-function addNodesToMap(map: Map<string, WorkflowNode>, nodes: WorkflowNode[]): void {
-  nodes.forEach(node => map.set(node.id, node));
-}
-```
-
-**Pattern 2: Batch Deletions**
-```typescript
-// GOOD: Delete multiple keys in single iteration
-function removeNodes(map: Map<string, WorkflowNode>, idsToRemove: string[]): void {
-  for (const id of idsToRemove) {
-    map.delete(id);
-  }
-  // V8 optimizes consecutive deletions
-}
-
-// BETTER: Collect keys first, then delete
-function removeSubtree(map: Map<string, WorkflowNode>, rootId: string): void {
-  const keysToDelete: string[] = [];
-
-  // First pass: collect all keys in subtree
-  const collectKeys = (node: WorkflowNode) => {
-    keysToDelete.push(node.id);
-    node.children.forEach(child => collectKeys(child));
-  };
-
-  const rootNode = map.get(rootId);
-  if (rootNode) {
-    collectKeys(rootNode);
-  }
-
-  // Second pass: delete all collected keys
-  for (const key of keysToDelete) {
-    map.delete(key);
-  }
-}
-```
-
-**Pattern 3: Use `has()` for Conditional Access**
-```typescript
-// GOOD: Check before accessing
-if (this.nodeMap.has(nodeId)) {
-  const node = this.nodeMap.get(nodeId);
-  // Use node
-}
-
-// ACCEPTABLE: Direct access with undefined check
-const node = this.nodeMap.get(nodeId);
-if (node) {
-  // Use node
-}
-```
-
-### 2.3 Map vs Object for Tree Nodes
-
-**Use Map when:**
-- Keys are not strings/symbols
-- Need to iterate in insertion order
-- Frequent additions/removals
-- Need to know size efficiently
-
-**Use Object when:**
-- Keys are strings/symbols
-- Need JSON serialization
-- Static set of keys
-- Prototype safety concerns
-
-**Decision:** Map is correct choice for nodeMap due to:
-- Frequent additions/removals (childAttached/childDetached)
-- O(1) size property
-- No prototype pollution concerns
-
----
-
-## 3. BFS vs DFS Traversal for Tree Operations
-
-### 3.1 Algorithm Comparison
-
-| Aspect | BFS (Breadth-First) | DFS (Depth-First) |
-|--------|---------------------|-------------------|
-| **Strategy** | Level-by-level | Go deep, then backtrack |
-| **Memory** | O(w) - max width | O(h) - max height |
-| **Time** | O(V + E) | O(V + E) |
-| **Implementation** | Queue | Stack (or recursion) |
-| **Use Case** | Shortest path, level-order | Memory efficient for deep trees |
-
-### 3.2 When to Use BFS for Tree Operations
-
-**Use BFS when:**
-1. **Removing subtrees** - Process parent before children (natural for removal)
-2. **Level-order statistics** - Need to process nodes by depth
-3. **Finding nearest nodes** - Shortest path in unweighted tree
-4. **Wide trees** - When width << height
-
-**BFS Implementation for Subtree Removal:**
-```typescript
-/**
- * Remove subtree using BFS traversal
- * Good for: Processing parent before children, level-by-level cleanup
- */
-private removeSubtreeBFS(rootId: string): void {
-  const queue: string[] = [rootId];
-  const keysToRemove: string[] = [];
-
-  // Collect all node IDs in subtree
-  while (queue.length > 0) {
-    const currentId = queue.shift()!;
-    const node = this.nodeMap.get(currentId);
-
-    if (node) {
-      keysToRemove.push(currentId);
-      // Add children to queue (BFS order)
-      node.children.forEach(child => queue.push(child.id));
-    }
-  }
-
-  // Remove all collected keys
-  for (const key of keysToRemove) {
-    this.nodeMap.delete(key);
-  }
-}
-```
-
-### 3.3 When to Use DFS for Tree Operations
-
-**Use DFS when:**
-1. **Adding subtrees** - Recursively add children
-2. **Deep trees** - When height << width (memory efficient)
-3. **Path finding** - Need specific path between nodes
-4. **Ancestry checks** - Bottom-up processing
-
-**DFS Implementation for Subtree Addition:**
-```typescript
-/**
- * Add subtree using DFS traversal
- * Good for: Memory efficient, natural recursion
- */
-private addSubtreeDFS(node: WorkflowNode): void {
-  this.nodeMap.set(node.id, node);
-
-  // Recursively add children (DFS - depth-first)
-  for (const child of node.children) {
-    this.addSubtreeDFS(child);
-  }
-}
-```
-
-### 3.4 Recommended Pattern for Tree Debugger
-
-**Hybrid Approach:**
-
-```typescript
-/**
- * Incremental update based on event type
- * Uses optimal traversal for each operation
- */
-private updateNodeMapIncremental(event: WorkflowEvent): void {
-  switch (event.type) {
-    case 'childAttached':
-      // DFS for addition: Natural recursion, memory efficient
-      this.addSubtreeDFS(event.child);
-      break;
-
-    case 'childDetached':
-      // BFS for removal: Parent before children, clear ordering
-      this.removeSubtreeBFS(event.child.id);
-      break;
-
-    default:
-      // Other events don't affect structure
-      break;
-  }
-}
-```
-
-### 3.5 Performance Comparison
-
-**Benchmark Scenario:** Tree with 1000 nodes, removing 100-node subtree
-
-| Method | Time Complexity | Memory | Notes |
-|--------|----------------|--------|-------|
-| Full Rebuild | O(n) | O(n) | Clears all, re-adds all |
-| BFS Incremental | O(m) | O(w) | m=subtree size, w=max width |
-| DFS Incremental | O(m) | O(h) | m=subtree size, h=max height |
-
-**Recommendation:** BFS for removal, DFS for addition
-
----
-
-## 4. Memory Management Patterns for Tree Structures
-
-### 4.1 Weak References for Auxiliary Data
-
-**Source:** [MDN - WeakMap](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WeakMap)
-
-**Pattern:** Use WeakMap for metadata that shouldn't prevent garbage collection
-
-```typescript
-// Good pattern: WeakMap for node metadata
-const nodeMetadata = new WeakMap<WorkflowNode, {
-  lastAccessed: number;
-  accessCount: number;
-}>();
-
-// When node is garbage collected, metadata is automatically cleaned up
-// No memory leaks from detached nodes
-```
-
-**Not Applicable to nodeMap:** nodeMap needs strong references (nodes must stay in memory)
-
-### 4.2 Circular Reference Prevention
-
-**Common Pitfall:** Parent <-> Child circular references prevent garbage collection
-
-```typescript
-// BAD: Circular references prevent GC
-class TreeNode {
-  parent: TreeNode | null = null;
-  children: TreeNode[] = [];
-}
-
-// Even after removing from tree, nodes may stay in memory
-// due to circular parent/child references
-
-// Solution: Explicit cleanup
-class TreeNode {
-  dispose() {
-    this.children.forEach(child => {
-      child.parent = null; // Break circular reference
-      child.dispose();
-    });
-    this.children = [];
-  }
-}
-```
-
-**For Tree Debugger:** Ensure detached nodes are removed from nodeMap
-
-```typescript
-private removeSubtreeWithCleanup(rootId: string): void {
-  const node = this.nodeMap.get(rootId);
-  if (!node) return;
-
-  // Break references if needed (implementation dependent)
-  // For WorkflowNode, just remove from map
-  this.removeSubtreeBFS(rootId);
-}
-```
-
-### 4.3 Object Pooling for Frequent Creation
-
-**Pattern:** Reuse objects instead of creating/destroying
-
-```typescript
-// Object pool for frequently created WorkflowEvent objects
-class EventPool {
-  private pool: WorkflowEvent[] = [];
-
-  acquire(): WorkflowEvent {
-    return this.pool.pop() || this.createEvent();
-  }
-
-  release(event: WorkflowEvent): void {
-    // Reset event state
-    this.pool.push(event);
-  }
-
-  private createEvent(): WorkflowEvent {
-    // Create new event
-    return {} as WorkflowEvent;
-  }
-}
-```
-
-**When to Use:**
-- High-frequency events (1000+ per second)
-- Events have consistent structure
-- Memory pressure is a concern
-
-**For Tree Debugger:** Probably not needed unless event frequency is very high
-
-### 4.4 Memory Profiling Patterns
-
-**Chrome DevTools Memory Profiling:**
-```typescript
-// Add memory tracking for debugging
-class MemoryTracker {
-  private snapshots: Map<string, number> = new Map();
-
-  snapshot(label: string): void {
-    if (performance.memory) {
-      this.snapshots.set(label, performance.memory.usedJSHeapSize);
-    }
-  }
-
-  report(): void {
-    console.table(this.snapshots);
-  }
-}
-
-// Usage in tree debugger
-constructor(workflow: Workflow) {
-  this.memoryTracker = new MemoryTracker();
-  this.memoryTracker.snapshot('constructor');
-
-  // ... build node map
-
-  this.memoryTracker.snapshot('after-build');
-}
-```
-
----
-
-## 5. Common Pitfalls in Incremental Tree Updates
-
-### 5.1 Pitfall 1: Missing Descendants
-
-**Problem:** Only removing/adding root node, not entire subtree
-
-```typescript
-// BAD: Only removes root
-private removeSubtreeBad(rootId: string): void {
-  this.nodeMap.delete(rootId); // Children remain in map!
-}
-
-// GOOD: Removes entire subtree
-private removeSubtreeGood(rootId: string): void {
-  const keysToDelete = this.collectSubtreeKeys(rootId);
-  for (const key of keysToDelete) {
-    this.nodeMap.delete(key);
-  }
-}
-
-private collectSubtreeKeys(rootId: string): string[] {
-  const keys: string[] = [];
-  const queue = [rootId];
-
-  while (queue.length > 0) {
-    const currentId = queue.shift()!;
-    const node = this.nodeMap.get(currentId);
-
-    if (node) {
-      keys.push(currentId);
-      node.children.forEach(child => queue.push(child.id));
-    }
-  }
-
-  return keys;
-}
-```
-
-### 5.2 Pitfall 2: Inconsistent State During Update
-
-**Problem:** Map is partially updated when accessed by other code
-
-```typescript
-// BAD: Map is inconsistent during iteration
-private removeSubtreeBad(rootId: string): void {
-  const node = this.nodeMap.get(rootId);
-  if (node) {
-    this.nodeMap.delete(rootId); // Map now missing root
-    // If another thread calls getNode() here, returns undefined
-    node.children.forEach(child => {
-      this.removeSubtreeBad(child.id); // Recursion leaves map inconsistent
-    });
-  }
-}
-
-// GOOD: Collect all keys first, then update atomically
-private removeSubtreeGood(rootId: string): void {
-  const keysToDelete = this.collectSubtreeKeys(rootId);
-
-  // Single atomic update
-  for (const key of keysToDelete) {
-    this.nodeMap.delete(key);
-  }
-}
-```
-
-### 5.3 Pitfall 3: Event Type Confusion
-
-**Problem:** Treating all events as structural changes
-
-```typescript
-// BAD: Rebuilds on every event
-onEvent(event: WorkflowEvent): void {
-  this.nodeMap.clear();
-  this.buildNodeMap(this.root); // Unnecessary for non-structural events
-}
-
-// GOOD: Only update on structural changes
-onEvent(event: WorkflowEvent): void {
-  switch (event.type) {
-    case 'childAttached':
-      this.addSubtreeDFS(event.child);
-      break;
-    case 'childDetached':
-      this.removeSubtreeBFS(event.child.id);
-      break;
-    // Other events don't affect structure
-  }
-}
-```
-
-### 5.4 Pitfall 4: Stack Overflow on Deep Recursion
-
-**Problem:** DFS recursion on very deep trees causes stack overflow
-
-```typescript
-// BAD: Recursion may overflow stack on deep trees
-private addSubtreeDFS(node: WorkflowNode): void {
-  this.nodeMap.set(node.id, node);
-  node.children.forEach(child => this.addSubtreeDFS(child)); // Recursion depth = tree height
-}
-
-// GOOD: Use iterative DFS for deep trees
-private addSubtreeDFSIterative(node: WorkflowNode): void {
-  const stack = [node];
-
-  while (stack.length > 0) {
-    const current = stack.pop()!;
-    this.nodeMap.set(current.id, current);
-
-    // Push children in reverse order for correct traversal
-    for (let i = current.children.length - 1; i >= 0; i--) {
-      stack.push(current.children[i]);
-    }
-  }
-}
-```
-
-**Recommendation:** Use iterative approach for production code with unknown tree depth
-
-### 5.5 Pitfall 5: Memory Leaks from Event Listeners
-
-**Problem:** Observers not removed when workflow is destroyed
-
-```typescript
-// BAD: Observer never removed
-class WorkflowTreeDebugger {
-  constructor(workflow: Workflow) {
-    workflow.addObserver(this); // Never removed
-  }
-}
-
-// GOOD: Implement cleanup
-class WorkflowTreeDebugger {
-  constructor(private workflow: Workflow) {
-    this.workflow.addObserver(this);
-  }
-
-  dispose(): void {
-    this.workflow.removeObserver(this);
-    this.nodeMap.clear();
-  }
-}
-```
-
----
-
-## 6. Actionable Implementation Guide
-
-### 6.1 Implement removeSubtreeNodes() with BFS
-
-```typescript
-/**
- * Remove subtree nodes from nodeMap using BFS traversal
- * @param rootId ID of root node to remove
- *
- * Why BFS:
- * - Processes parent before children (natural removal order)
- * - Level-by-level traversal predictable
- * - No recursion (no stack overflow risk)
- */
-private removeSubtreeNodes(rootId: string): void {
-  const queue: string[] = [rootId];
-  const keysToRemove: string[] = [];
-
-  // BFS traversal to collect all keys in subtree
-  while (queue.length > 0) {
-    const currentId = queue.shift()!;
-    const node = this.nodeMap.get(currentId);
-
-    if (node) {
-      keysToRemove.push(currentId);
-      // Add children to queue for BFS traversal
-      node.children.forEach(child => queue.push(child.id));
-    }
-  }
-
-  // Batch delete all collected keys
-  for (const key of keysToRemove) {
-    this.nodeMap.delete(key);
-  }
-}
-```
-
-### 6.2 Implement Event-Type Dispatch for onEvent()
-
-```typescript
-/**
- * Handle workflow events with type-based dispatch
- * @param event Workflow event to process
- *
- * Performance considerations:
- * - Only updates nodeMap on structural changes
- * - Non-structural events bypass map updates
- * - O(1) switch statement dispatch
- */
-onEvent(event: WorkflowEvent): void {
-  // Early return for non-structural events (fast path)
-  if (event.type !== 'childAttached' && event.type !== 'childDetached') {
-    this.events.next(event);
-    return;
-  }
-
-  // Type-based dispatch for structural events
-  switch (event.type) {
-    case 'childAttached': {
-      // Incremental add: DFS for memory efficiency
-      this.addSubtreeDFS(event.child);
-      break;
-    }
-
-    case 'childDetached': {
-      // Incremental remove: BFS for level-by-level cleanup
-      this.removeSubtreeNodes(event.child.id);
-      break;
-    }
-
-    default: {
-      // Exhaustive check for type safety
-      const _exhaustiveCheck: never = event;
-      break;
-    }
-  }
-
-  // Forward all events to event stream
-  this.events.next(event);
-}
-
-/**
- * Add subtree using DFS traversal
- * Good for: Memory efficient on deep trees, natural recursion
- */
-private addSubtreeDFS(node: WorkflowNode): void {
-  this.nodeMap.set(node.id, node);
-  node.children.forEach(child => this.addSubtreeDFS(child));
-}
-```
-
-### 6.3 Performance Optimization Checklist
-
-**Optimizations Applied:**
-- [x] Use Map instead of Object for O(1) operations
-- [x] Batch deletions (collect keys, then delete)
-- [x] Event-type dispatch (avoid unnecessary updates)
-- [x] BFS for removal (parent before children)
-- [x] DFS for addition (memory efficient)
-- [ ] Consider iterative DFS for very deep trees
-- [ ] Add memory tracking for debugging
-- [ ] Consider batching rapid successive updates
-
-**Future Optimizations:**
-- Object pooling for high-frequency events
-- WeakMap for auxiliary metadata
-- Update batching/debouncing for rapid changes
-- Lazy nodeMap rebuild after N incremental updates
-
-### 6.4 Testing Strategy
-
-```typescript
-describe('Incremental Node Map Updates', () => {
-  it('should correctly remove subtree using BFS', () => {
-    const debugger = new WorkflowTreeDebugger(rootWorkflow);
-
-    // Build tree with 100 nodes
-    const initialSize = debugger.getStats().totalNodes;
-    expect(initialSize).toBe(100);
-
-    // Detach subtree with 20 nodes
-    detachChild(childId);
-
-    // Verify only 20 nodes removed
-    const finalSize = debugger.getStats().totalNodes;
-    expect(finalSize).toBe(80);
-
-    // Verify detached nodes not accessible
-    expect(debugger.getNode(detachedChildId)).toBeUndefined();
-    expect(debugger.getNode(detachedGrandchildId)).toBeUndefined();
-  });
-
-  it('should correctly add subtree using DFS', () => {
-    const debugger = new WorkflowTreeDebugger(rootWorkflow);
-
-    // Attach subtree with 20 nodes
-    attachChild(parentId, newChild);
-
-    // Verify all 20 nodes added
-    const stats = debugger.getStats();
-    expect(stats.totalNodes).toBe(120);
-
-    // Verify all nodes accessible
-    expect(debugger.getNode(newChild.id)).toBeDefined();
-    expect(debugger.getNode(grandchild.id)).toBeDefined();
-  });
-
-  it('should handle non-structural events efficiently', () => {
-    const debugger = new WorkflowTreeDebugger(rootWorkflow);
-    const initialMapSize = (debugger as any).nodeMap.size;
-
-    // Trigger 100 non-structural events
-    for (let i = 0; i < 100; i++) {
-      triggerNonStructuralEvent();
-    }
-
-    // Verify nodeMap not rebuilt
-    expect((debugger as any).nodeMap.size).toBe(initialMapSize);
-  });
-});
-```
-
----
-
-## 7. Summary & Recommendations
-
-### 7.1 Key Takeaways
-
-1. **Use BFS for subtree removal** - Parent before children, predictable ordering
-2. **Use DFS for subtree addition** - Memory efficient, natural recursion
-3. **Event-type dispatch** - Only update map on structural changes
-4. **Batch operations** - Collect keys, then update atomically
-5. **Consider iterative DFS** - Avoid stack overflow on deep trees
-
-### 7.2 Performance Improvements
-
-**Before (Full Rebuild):**
-- Time: O(n) for every tree change
-- Memory: O(n) - entire tree in memory during rebuild
-- Scalability: Poor for large trees
-
-**After (Incremental):**
-- Time: O(m) where m = subtree size
-- Memory: O(w) for BFS, O(h) for DFS
-- Scalability: Excellent for large trees
-
-**Expected Speedup:** 10-100x for large trees when modifying small subtrees
-
-### 7.3 References
-
-1. **React Reconciliation:** https://react.dev/learn/render-and-commit
-   - Section: "React's Diffing Algorithm"
-   - Key concepts: O(n) heuristics, key-based identification
-
-2. **MDN Map Documentation:** https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map
-   - Operations: O(1) get/set/has/delete
-   - Iteration: forEach and for...of performance
-
-3. **V8 Performance Blog:** https://v8.dev/blog
-   - Map optimization patterns
-   - Hidden classes and inline caching
-
-4. **WeakMap Documentation:** https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WeakMap
-   - Use case: Auxiliary data that shouldn't prevent GC
-
-5. **Chrome DevTools Memory:** https://developer.chrome.com/docs/devtools/memory-problems
-   - Heap snapshots, memory profiling
-
----
-
-**Document Status:** Research Complete
-**Next Step:** Implement incremental updates in WorkflowTreeDebugger
-**Estimated Impact:** 10-100x performance improvement for large tree operations