groundswell 0.0.1 → 0.0.2

package/plan/001_d3bb02af4886/bugfix/001_e8e04329daf3/P1M3T2S2/research/external_research.md

@@ -0,0 +1,788 @@
+# External Research: Incremental Tree Mutation Handling
+
+**Research Date:** 2026-01-12
+**Purpose:** P1M3T2S2 - Implement incremental node map updates for WorkflowTreeDebugger
+**Focus:** BFS/DFS traversal, Map performance, event dispatching, and tree mutation patterns
+
+---
+
+## 1. React's Reconciliation & Virtual DOM Patterns
+
+### 1.1 Key Concepts from React's Diffing Algorithm
+
+React's reconciliation algorithm provides excellent patterns for incremental tree updates:
+
+**O(n) Algorithm for Tree Diffing:**
+- React assumes two trees of different elements will produce different trees
+- Uses heuristics to reduce O(n³) problem to O(n)
+- Key heuristic: If elements have different types, entire subtrees are different
+
+**Source:** [React Documentation - Reconciliation](https://react.dev/learn/render-and-commit)
+- **Section:** "React's Diffing Algorithm"
+- **Key Pattern:** Component type comparison for subtree replacement decisions
+
+### 1.2 Applicable Patterns for Tree Debugger
+
+**Pattern 1: Key-based Identification**
+```typescript
+// React uses keys to identify nodes across renders
+// Similarly, nodeMap uses node.id for O(1) lookups
+
+// Current implementation (good):
+private nodeMap: Map<string, WorkflowNode> = new Map();
+
+// Pattern: Always use stable identifiers for tree nodes
+// Avoid using array indices as identifiers (breaks on insertions/deletions)
+```
+
+**Pattern 2: Structural Change Detection**
+```typescript
+// React detects structural changes by comparing element types
+// For tree debugger, detect structural changes by event type:
+
+onEvent(event: WorkflowEvent): void {
+  switch (event.type) {
+    case 'childAttached':
+      // Incremental: Add only new subtree
+      this.addSubtreeToMap(event.child);
+      break;
+    case 'childDetached':
+      // Incremental: Remove only detached subtree
+      this.removeSubtreeFromMap(event.child);
+      break;
+    default:
+      // Non-structural: No map update needed
+      break;
+  }
+}
+```
+
+**Pattern 3: Batching Updates**
+```typescript
+// React batches multiple state updates
+// Apply similar pattern for rapid tree changes:
+
+private pendingUpdates: Array<() => void> = [];
+private updateScheduled = false;
+
+private scheduleUpdate(fn: () => void) {
+  this.pendingUpdates.push(fn);
+  if (!this.updateScheduled) {
+    this.updateScheduled = true;
+    Promise.resolve().then(() => {
+      this.pendingUpdates.forEach(fn => fn());
+      this.pendingUpdates = [];
+      this.updateScheduled = false;
+    });
+  }
+}
+```
+
+### 1.3 Virtual DOM Memory Efficiency
+
+**Insights:**
+- Virtual DOM uses object pooling for frequently created objects
+- Consider object pooling for WorkflowEvent if many events are created
+- Use structural sharing: Only copy changed branches
+
+---
+
+## 2. JavaScript Map Manipulation & Performance
+
+### 2.1 MDN Map Best Practices
+
+**Source:** [MDN Web Docs - Map](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map)
+
+**Key Performance Characteristics:**
+- **Operations:** `set()`, `get()`, `has()`, `delete()` are O(1) average case
+- **Iteration:** `forEach()` and `for...of` are optimized by V8
+- **Memory:** Maps have slightly more overhead than Objects, but better for频繁添加/删除
+
+### 2.2 V8 Engine Optimizations
+
+**Source:** [V8 Blog - Map Performance](https://v8.dev/blog/elements-kinds) (Related concepts)
+
+**Optimization Patterns:**
+
+**Pattern 1: Preserve Map Reference**
+```typescript
+// BAD: Creates new Map frequently
+function updateNodeMap(nodes: WorkflowNode[]): Map<string, WorkflowNode> {
+  const newMap = new Map();
+  nodes.forEach(node => newMap.set(node.id, node));
+  return newMap; // Forces garbage collection of old Map
+}
+
+// GOOD: Mutate existing Map
+function addNodesToMap(map: Map<string, WorkflowNode>, nodes: WorkflowNode[]): void {
+  nodes.forEach(node => map.set(node.id, node));
+}
+```
+
+**Pattern 2: Batch Deletions**
+```typescript
+// GOOD: Delete multiple keys in single iteration
+function removeNodes(map: Map<string, WorkflowNode>, idsToRemove: string[]): void {
+  for (const id of idsToRemove) {
+    map.delete(id);
+  }
+  // V8 optimizes consecutive deletions
+}
+
+// BETTER: Collect keys first, then delete
+function removeSubtree(map: Map<string, WorkflowNode>, rootId: string): void {
+  const keysToDelete: string[] = [];
+
+  // First pass: collect all keys in subtree
+  const collectKeys = (node: WorkflowNode) => {
+    keysToDelete.push(node.id);
+    node.children.forEach(child => collectKeys(child));
+  };
+
+  const rootNode = map.get(rootId);
+  if (rootNode) {
+    collectKeys(rootNode);
+  }
+
+  // Second pass: delete all collected keys
+  for (const key of keysToDelete) {
+    map.delete(key);
+  }
+}
+```
+
+**Pattern 3: Use `has()` for Conditional Access**
+```typescript
+// GOOD: Check before accessing
+if (this.nodeMap.has(nodeId)) {
+  const node = this.nodeMap.get(nodeId);
+  // Use node
+}
+
+// ACCEPTABLE: Direct access with undefined check
+const node = this.nodeMap.get(nodeId);
+if (node) {
+  // Use node
+}
+```
+
+### 2.3 Map vs Object for Tree Nodes
+
+**Use Map when:**
+- Keys are not strings/symbols
+- Need to iterate in insertion order
+- Frequent additions/removals
+- Need to know size efficiently
+
+**Use Object when:**
+- Keys are strings/symbols
+- Need JSON serialization
+- Static set of keys
+- Prototype safety concerns
+
+**Decision:** Map is correct choice for nodeMap due to:
+- Frequent additions/removals (childAttached/childDetached)
+- O(1) size property
+- No prototype pollution concerns
+
+---
+
+## 3. BFS vs DFS Traversal for Tree Operations
+
+### 3.1 Algorithm Comparison
+
+| Aspect | BFS (Breadth-First) | DFS (Depth-First) |
+|--------|---------------------|-------------------|
+| **Strategy** | Level-by-level | Go deep, then backtrack |
+| **Memory** | O(w) - max width | O(h) - max height |
+| **Time** | O(V + E) | O(V + E) |
+| **Implementation** | Queue | Stack (or recursion) |
+| **Use Case** | Shortest path, level-order | Memory efficient for deep trees |
+
+### 3.2 When to Use BFS for Tree Operations
+
+**Use BFS when:**
+1. **Removing subtrees** - Process parent before children (natural for removal)
+2. **Level-order statistics** - Need to process nodes by depth
+3. **Finding nearest nodes** - Shortest path in unweighted tree
+4. **Wide trees** - When width << height
+
+**BFS Implementation for Subtree Removal:**
+```typescript
+/**
+ * Remove subtree using BFS traversal
+ * Good for: Processing parent before children, level-by-level cleanup
+ */
+private removeSubtreeBFS(rootId: string): void {
+  const queue: string[] = [rootId];
+  const keysToRemove: string[] = [];
+
+  // Collect all node IDs in subtree
+  while (queue.length > 0) {
+    const currentId = queue.shift()!;
+    const node = this.nodeMap.get(currentId);
+
+    if (node) {
+      keysToRemove.push(currentId);
+      // Add children to queue (BFS order)
+      node.children.forEach(child => queue.push(child.id));
+    }
+  }
+
+  // Remove all collected keys
+  for (const key of keysToRemove) {
+    this.nodeMap.delete(key);
+  }
+}
+```
+
+### 3.3 When to Use DFS for Tree Operations
+
+**Use DFS when:**
+1. **Adding subtrees** - Recursively add children
+2. **Deep trees** - When height << width (memory efficient)
+3. **Path finding** - Need specific path between nodes
+4. **Ancestry checks** - Bottom-up processing
+
+**DFS Implementation for Subtree Addition:**
+```typescript
+/**
+ * Add subtree using DFS traversal
+ * Good for: Memory efficient, natural recursion
+ */
+private addSubtreeDFS(node: WorkflowNode): void {
+  this.nodeMap.set(node.id, node);
+
+  // Recursively add children (DFS - depth-first)
+  for (const child of node.children) {
+    this.addSubtreeDFS(child);
+  }
+}
+```
+
+### 3.4 Recommended Pattern for Tree Debugger
+
+**Hybrid Approach:**
+
+```typescript
+/**
+ * Incremental update based on event type
+ * Uses optimal traversal for each operation
+ */
+private updateNodeMapIncremental(event: WorkflowEvent): void {
+  switch (event.type) {
+    case 'childAttached':
+      // DFS for addition: Natural recursion, memory efficient
+      this.addSubtreeDFS(event.child);
+      break;
+
+    case 'childDetached':
+      // BFS for removal: Parent before children, clear ordering
+      this.removeSubtreeBFS(event.child.id);
+      break;
+
+    default:
+      // Other events don't affect structure
+      break;
+  }
+}
+```
+
+### 3.5 Performance Comparison
+
+**Benchmark Scenario:** Tree with 1000 nodes, removing 100-node subtree
+
+| Method | Time Complexity | Memory | Notes |
+|--------|----------------|--------|-------|
+| Full Rebuild | O(n) | O(n) | Clears all, re-adds all |
+| BFS Incremental | O(m) | O(w) | m=subtree size, w=max width |
+| DFS Incremental | O(m) | O(h) | m=subtree size, h=max height |
+
+**Recommendation:** BFS for removal, DFS for addition
+
+---
+
+## 4. Memory Management Patterns for Tree Structures
+
+### 4.1 Weak References for Auxiliary Data
+
+**Source:** [MDN - WeakMap](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WeakMap)
+
+**Pattern:** Use WeakMap for metadata that shouldn't prevent garbage collection
+
+```typescript
+// Good pattern: WeakMap for node metadata
+const nodeMetadata = new WeakMap<WorkflowNode, {
+  lastAccessed: number;
+  accessCount: number;
+}>();
+
+// When node is garbage collected, metadata is automatically cleaned up
+// No memory leaks from detached nodes
+```
+
+**Not Applicable to nodeMap:** nodeMap needs strong references (nodes must stay in memory)
+
+### 4.2 Circular Reference Prevention
+
+**Common Pitfall:** Parent <-> Child circular references prevent garbage collection
+
+```typescript
+// BAD: Circular references prevent GC
+class TreeNode {
+  parent: TreeNode | null = null;
+  children: TreeNode[] = [];
+}
+
+// Even after removing from tree, nodes may stay in memory
+// due to circular parent/child references
+
+// Solution: Explicit cleanup
+class TreeNode {
+  dispose() {
+    this.children.forEach(child => {
+      child.parent = null; // Break circular reference
+      child.dispose();
+    });
+    this.children = [];
+  }
+}
+```
+
+**For Tree Debugger:** Ensure detached nodes are removed from nodeMap
+
+```typescript
+private removeSubtreeWithCleanup(rootId: string): void {
+  const node = this.nodeMap.get(rootId);
+  if (!node) return;
+
+  // Break references if needed (implementation dependent)
+  // For WorkflowNode, just remove from map
+  this.removeSubtreeBFS(rootId);
+}
+```
+
+### 4.3 Object Pooling for Frequent Creation
+
+**Pattern:** Reuse objects instead of creating/destroying
+
+```typescript
+// Object pool for frequently created WorkflowEvent objects
+class EventPool {
+  private pool: WorkflowEvent[] = [];
+
+  acquire(): WorkflowEvent {
+    return this.pool.pop() || this.createEvent();
+  }
+
+  release(event: WorkflowEvent): void {
+    // Reset event state
+    this.pool.push(event);
+  }
+
+  private createEvent(): WorkflowEvent {
+    // Create new event
+    return {} as WorkflowEvent;
+  }
+}
+```
+
+**When to Use:**
+- High-frequency events (1000+ per second)
+- Events have consistent structure
+- Memory pressure is a concern
+
+**For Tree Debugger:** Probably not needed unless event frequency is very high
+
+### 4.4 Memory Profiling Patterns
+
+**Chrome DevTools Memory Profiling:**
+```typescript
+// Add memory tracking for debugging
+class MemoryTracker {
+  private snapshots: Map<string, number> = new Map();
+
+  snapshot(label: string): void {
+    if (performance.memory) {
+      this.snapshots.set(label, performance.memory.usedJSHeapSize);
+    }
+  }
+
+  report(): void {
+    console.table(this.snapshots);
+  }
+}
+
+// Usage in tree debugger
+constructor(workflow: Workflow) {
+  this.memoryTracker = new MemoryTracker();
+  this.memoryTracker.snapshot('constructor');
+
+  // ... build node map
+
+  this.memoryTracker.snapshot('after-build');
+}
+```
+
+---
+
+## 5. Common Pitfalls in Incremental Tree Updates
+
+### 5.1 Pitfall 1: Missing Descendants
+
+**Problem:** Only removing/adding root node, not entire subtree
+
+```typescript
+// BAD: Only removes root
+private removeSubtreeBad(rootId: string): void {
+  this.nodeMap.delete(rootId); // Children remain in map!
+}
+
+// GOOD: Removes entire subtree
+private removeSubtreeGood(rootId: string): void {
+  const keysToDelete = this.collectSubtreeKeys(rootId);
+  for (const key of keysToDelete) {
+    this.nodeMap.delete(key);
+  }
+}
+
+private collectSubtreeKeys(rootId: string): string[] {
+  const keys: string[] = [];
+  const queue = [rootId];
+
+  while (queue.length > 0) {
+    const currentId = queue.shift()!;
+    const node = this.nodeMap.get(currentId);
+
+    if (node) {
+      keys.push(currentId);
+      node.children.forEach(child => queue.push(child.id));
+    }
+  }
+
+  return keys;
+}
+```
+
+### 5.2 Pitfall 2: Inconsistent State During Update
+
+**Problem:** Map is partially updated when accessed by other code
+
+```typescript
+// BAD: Map is inconsistent during iteration
+private removeSubtreeBad(rootId: string): void {
+  const node = this.nodeMap.get(rootId);
+  if (node) {
+    this.nodeMap.delete(rootId); // Map now missing root
+    // If another thread calls getNode() here, returns undefined
+    node.children.forEach(child => {
+      this.removeSubtreeBad(child.id); // Recursion leaves map inconsistent
+    });
+  }
+}
+
+// GOOD: Collect all keys first, then update atomically
+private removeSubtreeGood(rootId: string): void {
+  const keysToDelete = this.collectSubtreeKeys(rootId);
+
+  // Single atomic update
+  for (const key of keysToDelete) {
+    this.nodeMap.delete(key);
+  }
+}
+```
+
+### 5.3 Pitfall 3: Event Type Confusion
+
+**Problem:** Treating all events as structural changes
+
+```typescript
+// BAD: Rebuilds on every event
+onEvent(event: WorkflowEvent): void {
+  this.nodeMap.clear();
+  this.buildNodeMap(this.root); // Unnecessary for non-structural events
+}
+
+// GOOD: Only update on structural changes
+onEvent(event: WorkflowEvent): void {
+  switch (event.type) {
+    case 'childAttached':
+      this.addSubtreeDFS(event.child);
+      break;
+    case 'childDetached':
+      this.removeSubtreeBFS(event.child.id);
+      break;
+    // Other events don't affect structure
+  }
+}
+```
+
+### 5.4 Pitfall 4: Stack Overflow on Deep Recursion
+
+**Problem:** DFS recursion on very deep trees causes stack overflow
+
+```typescript
+// BAD: Recursion may overflow stack on deep trees
+private addSubtreeDFS(node: WorkflowNode): void {
+  this.nodeMap.set(node.id, node);
+  node.children.forEach(child => this.addSubtreeDFS(child)); // Recursion depth = tree height
+}
+
+// GOOD: Use iterative DFS for deep trees
+private addSubtreeDFSIterative(node: WorkflowNode): void {
+  const stack = [node];
+
+  while (stack.length > 0) {
+    const current = stack.pop()!;
+    this.nodeMap.set(current.id, current);
+
+    // Push children in reverse order for correct traversal
+    for (let i = current.children.length - 1; i >= 0; i--) {
+      stack.push(current.children[i]);
+    }
+  }
+}
+```
+
+**Recommendation:** Use iterative approach for production code with unknown tree depth
+
+### 5.5 Pitfall 5: Memory Leaks from Event Listeners
+
+**Problem:** Observers not removed when workflow is destroyed
+
+```typescript
+// BAD: Observer never removed
+class WorkflowTreeDebugger {
+  constructor(workflow: Workflow) {
+    workflow.addObserver(this); // Never removed
+  }
+}
+
+// GOOD: Implement cleanup
+class WorkflowTreeDebugger {
+  constructor(private workflow: Workflow) {
+    this.workflow.addObserver(this);
+  }
+
+  dispose(): void {
+    this.workflow.removeObserver(this);
+    this.nodeMap.clear();
+  }
+}
+```
+
+---
+
+## 6. Actionable Implementation Guide
+
+### 6.1 Implement removeSubtreeNodes() with BFS
+
+```typescript
+/**
+ * Remove subtree nodes from nodeMap using BFS traversal
+ * @param rootId ID of root node to remove
+ *
+ * Why BFS:
+ * - Processes parent before children (natural removal order)
+ * - Level-by-level traversal predictable
+ * - No recursion (no stack overflow risk)
+ */
+private removeSubtreeNodes(rootId: string): void {
+  const queue: string[] = [rootId];
+  const keysToRemove: string[] = [];
+
+  // BFS traversal to collect all keys in subtree
+  while (queue.length > 0) {
+    const currentId = queue.shift()!;
+    const node = this.nodeMap.get(currentId);
+
+    if (node) {
+      keysToRemove.push(currentId);
+      // Add children to queue for BFS traversal
+      node.children.forEach(child => queue.push(child.id));
+    }
+  }
+
+  // Batch delete all collected keys
+  for (const key of keysToRemove) {
+    this.nodeMap.delete(key);
+  }
+}
+```
+
+### 6.2 Implement Event-Type Dispatch for onEvent()
+
+```typescript
+/**
+ * Handle workflow events with type-based dispatch
+ * @param event Workflow event to process
+ *
+ * Performance considerations:
+ * - Only updates nodeMap on structural changes
+ * - Non-structural events bypass map updates
+ * - O(1) switch statement dispatch
+ */
+onEvent(event: WorkflowEvent): void {
+  // Early return for non-structural events (fast path)
+  if (event.type !== 'childAttached' && event.type !== 'childDetached') {
+    this.events.next(event);
+    return;
+  }
+
+  // Type-based dispatch for structural events
+  switch (event.type) {
+    case 'childAttached': {
+      // Incremental add: DFS for memory efficiency
+      this.addSubtreeDFS(event.child);
+      break;
+    }
+
+    case 'childDetached': {
+      // Incremental remove: BFS for level-by-level cleanup
+      this.removeSubtreeNodes(event.child.id);
+      break;
+    }
+
+    default: {
+      // Exhaustive check for type safety
+      const _exhaustiveCheck: never = event;
+      break;
+    }
+  }
+
+  // Forward all events to event stream
+  this.events.next(event);
+}
+
+/**
+ * Add subtree using DFS traversal
+ * Good for: Memory efficient on deep trees, natural recursion
+ */
+private addSubtreeDFS(node: WorkflowNode): void {
+  this.nodeMap.set(node.id, node);
+  node.children.forEach(child => this.addSubtreeDFS(child));
+}
+```
+
+### 6.3 Performance Optimization Checklist
+
+**Optimizations Applied:**
+- [x] Use Map instead of Object for O(1) operations
+- [x] Batch deletions (collect keys, then delete)
+- [x] Event-type dispatch (avoid unnecessary updates)
+- [x] BFS for removal (parent before children)
+- [x] DFS for addition (memory efficient)
+- [ ] Consider iterative DFS for very deep trees
+- [ ] Add memory tracking for debugging
+- [ ] Consider batching rapid successive updates
+
+**Future Optimizations:**
+- Object pooling for high-frequency events
+- WeakMap for auxiliary metadata
+- Update batching/debouncing for rapid changes
+- Lazy nodeMap rebuild after N incremental updates
+
+### 6.4 Testing Strategy
+
+```typescript
+describe('Incremental Node Map Updates', () => {
+  it('should correctly remove subtree using BFS', () => {
+    const debugger = new WorkflowTreeDebugger(rootWorkflow);
+
+    // Build tree with 100 nodes
+    const initialSize = debugger.getStats().totalNodes;
+    expect(initialSize).toBe(100);
+
+    // Detach subtree with 20 nodes
+    detachChild(childId);
+
+    // Verify only 20 nodes removed
+    const finalSize = debugger.getStats().totalNodes;
+    expect(finalSize).toBe(80);
+
+    // Verify detached nodes not accessible
+    expect(debugger.getNode(detachedChildId)).toBeUndefined();
+    expect(debugger.getNode(detachedGrandchildId)).toBeUndefined();
+  });
+
+  it('should correctly add subtree using DFS', () => {
+    const debugger = new WorkflowTreeDebugger(rootWorkflow);
+
+    // Attach subtree with 20 nodes
+    attachChild(parentId, newChild);
+
+    // Verify all 20 nodes added
+    const stats = debugger.getStats();
+    expect(stats.totalNodes).toBe(120);
+
+    // Verify all nodes accessible
+    expect(debugger.getNode(newChild.id)).toBeDefined();
+    expect(debugger.getNode(grandchild.id)).toBeDefined();
+  });
+
+  it('should handle non-structural events efficiently', () => {
+    const debugger = new WorkflowTreeDebugger(rootWorkflow);
+    const initialMapSize = (debugger as any).nodeMap.size;
+
+    // Trigger 100 non-structural events
+    for (let i = 0; i < 100; i++) {
+      triggerNonStructuralEvent();
+    }
+
+    // Verify nodeMap not rebuilt
+    expect((debugger as any).nodeMap.size).toBe(initialMapSize);
+  });
+});
+```
+
+---
+
+## 7. Summary & Recommendations
+
+### 7.1 Key Takeaways
+
+1. **Use BFS for subtree removal** - Parent before children, predictable ordering
+2. **Use DFS for subtree addition** - Memory efficient, natural recursion
+3. **Event-type dispatch** - Only update map on structural changes
+4. **Batch operations** - Collect keys, then update atomically
+5. **Consider iterative DFS** - Avoid stack overflow on deep trees
+
+### 7.2 Performance Improvements
+
+**Before (Full Rebuild):**
+- Time: O(n) for every tree change
+- Memory: O(n) - entire tree in memory during rebuild
+- Scalability: Poor for large trees
+
+**After (Incremental):**
+- Time: O(m) where m = subtree size
+- Memory: O(w) for BFS, O(h) for DFS
+- Scalability: Excellent for large trees
+
+**Expected Speedup:** 10-100x for large trees when modifying small subtrees
+
+### 7.3 References
+
+1. **React Reconciliation:** https://react.dev/learn/render-and-commit
+   - Section: "React's Diffing Algorithm"
+   - Key concepts: O(n) heuristics, key-based identification
+
+2. **MDN Map Documentation:** https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map
+   - Operations: O(1) get/set/has/delete
+   - Iteration: forEach and for...of performance
+
+3. **V8 Performance Blog:** https://v8.dev/blog
+   - Map optimization patterns
+   - Hidden classes and inline caching
+
+4. **WeakMap Documentation:** https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WeakMap
+   - Use case: Auxiliary data that shouldn't prevent GC
+
+5. **Chrome DevTools Memory:** https://developer.chrome.com/docs/devtools/memory-problems
+   - Heap snapshots, memory profiling
+
+---
+
+**Document Status:** Research Complete
+**Next Step:** Implement incremental updates in WorkflowTreeDebugger
+**Estimated Impact:** 10-100x performance improvement for large tree operations