groundswell 0.0.1 → 0.0.2

package/plan/001_d3bb02af4886/docs/bugfix-architecture/implementation_patterns.md

@@ -0,0 +1,489 @@
+# Implementation Patterns: Tree Integrity Best Practices
+
+## Overview
+
+This document outlines the proven patterns and best practices for implementing tree integrity in hierarchical data structures, specifically applied to the Workflow engine bug fix.
+
+## Core Patterns for attachChild() Fix
+
+### Pattern 1: Parent State Validation
+
+**Principle**: Always validate the child's current parent state before attaching.
+
+```typescript
+public attachChild(child: Workflow): void {
+  // Validation 1: Prevent duplicate attachment
+  if (this.children.includes(child)) {
+    throw new Error(
+      `Child '${child.node.name}' is already attached to workflow '${this.node.name}'`
+    );
+  }
+
+  // Validation 2: CRITICAL - Check if child already has a different parent
+  if (child.parent !== null && child.parent !== this) {
+    throw new Error(
+      `Child '${child.node.name}' already has parent '${child.parent.node.name}'. ` +
+      `A workflow can only have one parent. ` +
+      `Use detachChild() on the current parent first if you need to reparent.`
+    );
+  }
+
+  // Update child's parent if it's currently null
+  if (child.parent === null) {
+    child.parent = this;
+  }
+
+  // Add to both trees
+  this.children.push(child);
+  this.node.children.push(child.node);
+
+  // Emit event
+  this.emitEvent({
+    type: 'childAttached',
+    parentId: this.id,
+    child: child.node,
+  });
+}
+```
+
+**Key Points**:
+- Check `child.parent !== null` before allowing attachment
+- Check `child.parent !== this` to allow re-attaching to same parent (idempotent)
+- Provide clear error message with current parent's name
+- Suggest solution (use detachChild first)
+
+### Pattern 2: Circular Reference Detection
+
+**Principle**: Prevent attaching an ancestor as a child (would create cycle).
+
+```typescript
+public attachChild(child: Workflow): void {
+  // ... other validations ...
+
+  // Validation 3: Prevent circular references
+  if (this.isDescendantOf(child)) {
+    throw new Error(
+      `Cannot attach child '${child.node.name}' - it is an ancestor of '${this.node.name}'. ` +
+      `This would create a circular reference.`
+    );
+  }
+
+  // ... rest of method ...
+}
+
+/**
+ * Check if this workflow is a descendant of another workflow
+ * Used to prevent circular references in attachChild()
+ */
+private isDescendantOf(ancestor: Workflow): boolean {
+  let current: Workflow | null = this;
+  const visited = new Set<Workflow>();
+
+  while (current !== null) {
+    // Cycle detection during traversal itself
+    if (visited.has(current)) {
+      throw new Error('Circular reference detected in tree structure');
+    }
+    visited.add(current);
+
+    if (current === ancestor) {
+      return true;
+    }
+    current = current.parent;
+  }
+
+  return false;
+}
+```
+
+**Key Points**:
+- Traverse parent chain from `this` upwards
+- If we encounter `ancestor`, it means `ancestor` is above `this` in tree
+- Attaching it would create a cycle
+- Include cycle detection in traversal itself (defensive)
+
+### Pattern 3: Bidirectional Tree Synchronization
+
+**Principle**: Always update both the workflow tree and node tree together.
+
+```typescript
+public attachChild(child: Workflow): void {
+  // ... validations ...
+
+  // Update workflow tree
+  if (child.parent === null) {
+    child.parent = this;
+  }
+  this.children.push(child);
+
+  // Update node tree (must mirror workflow tree)
+  this.node.children.push(child.node);
+
+  // Emit event for observers
+  this.emitEvent({
+    type: 'childAttached',
+    parentId: this.id,
+    child: child.node,
+  });
+}
+```
+
+**Key Points**:
+- Both trees must be updated atomically (in same method)
+- Order matters: update child.parent, then this.children, then this.node.children
+- Event emission happens after both trees are updated
+- This ensures "1:1 tree mirror" requirement from PRD
+
+### Pattern 4: Detach Child for Reparenting
+
+**Principle**: Provide a method to properly remove children for reparenting scenarios.
+
+```typescript
+/**
+ * Detach a child workflow from this parent
+ * @throws {Error} If child is not attached to this workflow
+ */
+public detachChild(child: Workflow): void {
+  const index = this.children.indexOf(child);
+  if (index === -1) {
+    throw new Error(
+      `Child '${child.node.name}' is not attached to workflow '${this.node.name}'`
+    );
+  }
+
+  // Remove from workflow children array
+  this.children.splice(index, 1);
+
+  // Remove from node children array
+  const nodeIndex = this.node.children.indexOf(child.node);
+  if (nodeIndex !== -1) {
+    this.node.children.splice(nodeIndex, 1);
+  }
+
+  // Clear child's parent reference
+  child.parent = null;
+
+  // Emit detached event for observers
+  this.emitEvent({
+    type: 'childDetached',
+    parentId: this.id,
+    childId: child.id,
+  });
+}
+```
+
+**Key Points**:
+- Validate child is actually attached before removing
+- Remove from both trees (workflow and node)
+- Clear child.parent reference
+- Emit event so observers can update
+- Enables reparenting workflow: `oldParent.detachChild(child); newParent.attachChild(child);`
+
+## Observer Pattern Patterns
+
+### Pattern 5: Root-Following with Validation
+
+**Principle**: Observer propagation relies on getRoot() working correctly.
+
+```typescript
+/**
+ * Get the root workflow with robust cycle detection
+ */
+protected getRoot(): Workflow {
+  const visited = new Set<Workflow>();
+  let current: Workflow | null = this;
+
+  while (current) {
+    // Cycle detection - prevents infinite loops
+    if (visited.has(current)) {
+      throw new Error(
+        `Circular parent-child relationship detected at workflow '${current.node.name}'. ` +
+        `Tree integrity violated.`
+      );
+    }
+    visited.add(current);
+
+    // Move to parent
+    current = current.parent;
+  }
+
+  // Return the last non-null node (the root)
+  return Array.from(visited).pop()!;
+}
+```
+
+**Key Points**:
+- With attachChild() fix, this method now works correctly
+- Child can only have one parent, so parent chain is deterministic
+- Cycle detection is still important for edge cases
+- Returns the root workflow that has observers array
+
+### Pattern 6: Observer Error Isolation
+
+**Principle**: Observer errors should not crash the workflow execution.
+
+```typescript
+public emitEvent(event: WorkflowEvent): void {
+  // Add to this node's event log
+  this.node.events.push(event);
+
+  // Get root with validation
+  const root = this.getRoot();
+
+  // Notify all observers with error isolation
+  for (const obs of root.observers) {
+    try {
+      obs.onEvent(event);
+
+      // Also notify tree changed for tree update events
+      if (event.type === 'treeUpdated' || event.type === 'childAttached' || event.type === 'childDetached') {
+        obs.onTreeChanged(root.node);
+      }
+    } catch (err) {
+      // Isolate observer errors to prevent cascading failures
+      console.error('Observer onEvent error:', err);
+
+      // Optionally emit error event
+      this.emitEvent({
+        type: 'error',
+        node: this.node,
+        error: {
+          message: err instanceof Error ? err.message : 'Observer error',
+          original: err,
+          workflowId: this.id,
+          stack: err instanceof Error ? err.stack : undefined,
+          state: null,
+          logs: [],
+        },
+      });
+    }
+  }
+}
+```
+
+**Key Points**:
+- Wrap each observer callback in try-catch
+- Log errors but continue processing other observers
+- This pattern is already implemented and should be preserved
+- Add childDetached to tree update events
+
+## Testing Patterns
+
+### Pattern 7: Tree Integrity Test Suite
+
+```typescript
+describe('Tree Integrity Tests', () => {
+  describe('attachChild() Validation', () => {
+    it('should prevent attaching child with existing parent', () => {
+      const parent1 = new Workflow('Parent1');
+      const parent2 = new Workflow('Parent2');
+      const child = new Workflow('Child', parent1);
+
+      // Should throw
+      expect(() => parent2.attachChild(child)).toThrow(/already has a parent/);
+    });
+
+    it('should allow attaching child with null parent', () => {
+      const parent = new Workflow('Parent');
+      const child = new Workflow('Child'); // No parent
+
+      parent.attachChild(child);
+
+      expect(child.parent).toBe(parent);
+      expect(parent.children).toContain(child);
+    });
+
+    it('should prevent circular references', () => {
+      const root = new Workflow('Root');
+      const child = new Workflow('Child', root);
+
+      // Try to create cycle
+      expect(() => {
+        child.attachChild(root as any);
+      }).toThrow(/circular|cycle/i);
+    });
+
+    it('should prevent duplicate children', () => {
+      const parent = new Workflow('Parent');
+      const child = new Workflow('Child', parent);
+
+      // Try to attach again
+      expect(() => parent.attachChild(child)).toThrow(/already attached/);
+    });
+  });
+
+  describe('Observer Propagation', () => {
+    it('should propagate events from child to root observers', async () => {
+      const events: WorkflowEvent[] = [];
+
+      const root = new Workflow('Root');
+      root.addObserver({
+        onLog: () => {},
+        onEvent: (e) => events.push(e),
+        onStateUpdated: () => {},
+        onTreeChanged: () => {},
+      });
+
+      const child = new Workflow('Child', root);
+      child.emitEvent({ type: 'stepStart', node: child.node, step: 'test' });
+
+      // Event should be received by root observer
+      const stepEvents = events.filter(e => e.type === 'stepStart');
+      expect(stepEvents.length).toBe(1);
+    });
+
+    it('should not propagate events after reparenting', () => {
+      const events1: WorkflowEvent[] = [];
+      const events2: WorkflowEvent[] = [];
+
+      const parent1 = new Workflow('Parent1');
+      const parent2 = new Workflow('Parent2');
+
+      parent1.addObserver({
+        onLog: () => {},
+        onEvent: (e) => events1.push(e),
+        onStateUpdated: () => {},
+        onTreeChanged: () => {},
+      });
+
+      parent2.addObserver({
+        onLog: () => {},
+        onEvent: (e) => events2.push(e),
+        onStateUpdated: () => {},
+        onTreeChanged: () => {},
+      });
+
+      const child = new Workflow('Child', parent1);
+
+      // Reparent
+      parent1.detachChild(child);
+      parent2.attachChild(child);
+
+      // Emit event from child
+      child.emitEvent({ type: 'stepStart', node: child.node, step: 'test' });
+
+      // Should only reach parent2's observers
+      expect(events1.filter(e => e.type === 'stepStart').length).toBe(0);
+      expect(events2.filter(e => e.type === 'stepStart').length).toBe(1);
+    });
+  });
+
+  describe('Bidirectional Consistency', () => {
+    it('should maintain consistency between workflow and node trees', () => {
+      const parent = new Workflow('Parent');
+      const child = new Workflow('Child', parent);
+
+      // Workflow tree
+      expect(child.parent).toBe(parent);
+      expect(parent.children).toContain(child);
+
+      // Node tree
+      expect(child.node.parent).toBe(parent.node);
+      expect(parent.node.children).toContain(child.node);
+    });
+
+    it('should maintain consistency after detach', () => {
+      const parent = new Workflow('Parent');
+      const child = new Workflow('Child', parent);
+
+      parent.detachChild(child);
+
+      // Workflow tree
+      expect(child.parent).toBeNull();
+      expect(parent.children).toContain(child).toBe(false);
+
+      // Node tree
+      expect(parent.node.children).toContain(child.node).toBe(false);
+    });
+  });
+});
+```
+
+### Pattern 8: Adversarial Testing
+
+```typescript
+describe('Adversarial Tree Tests', () => {
+  it('should prevent manual parent mutation creating inconsistency', () => {
+    const parent1 = new Workflow('Parent1');
+    const parent2 = new Workflow('Parent2');
+    const child = new Workflow('Child', parent1);
+
+    // Try to manually mutate (TypeScript should prevent this at compile time)
+    // But at runtime, this could happen with 'as any'
+    expect(() => {
+      (child as any).parent = parent2;
+    }).not.toThrow();
+
+    // Even if mutation succeeds, tree should be validated
+    // This is why we need defensive checks in attachChild()
+  });
+
+  it('should handle deep hierarchies without stack overflow', () => {
+    const root = new Workflow('Root');
+    let current = root;
+
+    // Create 1000 levels deep
+    for (let i = 0; i < 1000; i++) {
+      const child = new Workflow(`Child-${i}`);
+      current.attachChild(child);
+      current = child;
+    }
+
+    // getRoot() should work without stack overflow
+    expect(current.getRoot()).toBe(root);
+  });
+
+  it('should detect complex circular references', () => {
+    const root = new Workflow('Root');
+    const child1 = new Workflow('Child1', root);
+    const child2 = new Workflow('Child2', child1);
+    const child3 = new Workflow('Child3', child2);
+
+    // Try to create cycle: child3 -> root
+    expect(() => {
+      child3.attachChild(root as any);
+    }).toThrow(/circular|cycle/i);
+  });
+});
+```
+
+## Error Message Patterns
+
+### Pattern 9: Clear, Actionable Error Messages
+
+```typescript
+// Bad error message
+throw new Error('Invalid parent');
+
+// Good error message
+throw new Error(
+  `Child '${child.node.name}' already has parent '${child.parent.node.name}'. ` +
+  `A workflow can only have one parent. ` +
+  `Use detachChild() on '${child.parent.node.name}' first if you need to reparent.`
+);
+```
+
+**Components of a Good Error Message**:
+1. **What happened**: Child already has a parent
+2. **Who is involved**: Names of the workflows
+3. **Why it's a problem**: A workflow can only have one parent
+4. **How to fix it**: Use detachChild() first
+
+## Implementation Checklist
+
+- [ ] Add parent validation to attachChild()
+- [ ] Add circular reference detection to attachChild()
+- [ ] Implement isDescendantOf() helper method
+- [ ] Implement detachChild() method
+- [ ] Update emitEvent() to handle childDetached events
+- [ ] Add tree integrity tests
+- [ ] Add observer propagation tests
+- [ ] Add adversarial tests
+- [ ] Verify all 241 existing tests still pass
+- [ ] Update documentation if needed
+
+## References
+
+- DOM Tree API: https://dom.spec.whatwg.org/#concept-tree-parent
+- React Fiber Tree: https://github.com/facebook/react/blob/main/packages/react-reconciler/src/ReactFiber.js
+- TypeScript Private Fields: https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-8.html#ecmascript-private-fields

package/plan/001_d3bb02af4886/docs/bugfix-architecture/system_context.md

@@ -0,0 +1,218 @@
+# System Context: Hierarchical Workflow Engine
+
+## Project Overview
+
+**Project Name**: Hierarchical Workflow Engine
+**Technology Stack**: TypeScript, Vitest
+**Current Status**: Production-ready with critical bug
+**Test Coverage**: 241 tests across 16 test files (all passing)
+
+## Core Architecture
+
+### 1. Dual Tree Representation
+
+The workflow engine maintains two synchronized tree structures:
+
+1. **Workflow Instance Tree**: Runtime objects with parent/child relationships
+2. **Node Tree**: Serializable data structure for logs and events
+
+```
+Workflow Instance Tree          Node Tree
+======================          =========
+Workflow (parent)              WorkflowNode (parent)
+  ├─ children: Workflow[]        ├─ children: WorkflowNode[]
+  └─ parent: Workflow            └─ parent: WorkflowNode
+```
+
+**Critical Invariant**: Both trees must maintain perfect 1:1 mirroring as specified in PRD Section 12.2.
+
+### 2. Core Components
+
+#### Workflow Class (`src/core/workflow.ts`)
+
+**Key Properties**:
+- `parent: Workflow | null` - Reference to parent workflow (line 49)
+- `children: Workflow[]` - Array of child workflows (line 52)
+- `node: WorkflowNode` - Serializable node representation (line 102)
+- `id: string` - Unique workflow identifier (immutable)
+- `status: WorkflowStatus` - Current workflow status
+- `observers: WorkflowObserver[]` - Array of observers (root-only)
+
+**Key Methods**:
+- `constructor(name: string, parent?: Workflow)` - Creates workflow, auto-attaches if parent provided (line 115)
+- `attachChild(child: Workflow): void` - **BUGGY METHOD** (lines 187-201)
+- `detachChild(child: Workflow): void` - Does not exist (needs to be added)
+- `getRoot(): Workflow` - Traverses parent chain with cycle detection
+- `emitEvent(event: WorkflowEvent): void` - Centralized event emission
+- `run(): Promise<any>` - Execute workflow with @Step decorated methods
+
+#### Observer Pattern (`src/types/observer.ts`)
+
+**Observer Interface**:
+```typescript
+interface WorkflowObserver {
+  onEvent(event: WorkflowEvent): void;
+  onLog(log: WorkflowLog): void;
+  onStateUpdated(state: Record<string, unknown>): void;
+  onTreeChanged(root: WorkflowNode): void;
+}
+```
+
+**Key Constraints**:
+- Observers can **only** be attached to root workflows
+- Events propagate via `getRootObservers()` which follows parent chain
+- Observer errors are isolated (logged to console, don't crash execution)
+
+#### Event System (`src/types/events.ts`)
+
+**7 Event Types** (discriminated union):
+1. `stepStart` - Step execution started
+2. `stepFinish` - Step execution completed
+3. `stateUpdated` - @ObservedState property changed
+4. `childAttached` - Child workflow added to tree
+5. `childDetached` - Child workflow removed from tree
+6. `treeUpdated` - Tree structure changed
+7. `error` - Error occurred with state snapshot
+
+### 3. Decorator System
+
+#### @Step Decorator (`src/decorators/step.ts`)
+- Wraps method execution with error handling
+- Captures state snapshots via @ObservedState
+- Emits stepStart/stepFinish events
+- Options: name, snapshotState, trackTiming, logStart, logFinish
+
+#### @Task Decorator (`src/decorators/task.ts`)
+- Decorates methods for concurrent task execution
+- Supports parallel task execution with Promise.all()
+
+#### @ObservedState Decorator (`src/decorators/observed-state.ts`)
+- Tracks property changes automatically
+- Supports redaction (redact option)
+- Supports hiding from logs (hidden option)
+- Emits stateUpdated events on change
+
+### 4. Testing Infrastructure
+
+**Test Organization**:
+```
+src/__tests__/
+├── unit/           - 12 files, core functionality
+├── integration/    - 2 files, component interaction
+└── adversarial/    - 4 files, edge cases and bugs
+    ├── edge-case.test.ts          - 23 tests
+    ├── deep-analysis.test.ts      - 18 tests
+    ├── prd-compliance.test.ts     - 17 tests
+    └── e2e-prd-validation.test.ts - End-to-end validation
+```
+
+**Test Framework**: Vitest
+**Total Tests**: 241 test cases
+**Current Status**: All passing (including the buggy behavior - tests don't catch this bug)
+
+### 5. Current Bug: attachChild() Tree Integrity Violation
+
+**Location**: `src/core/workflow.ts:187-201`
+
+**Current Implementation**:
+```typescript
+public attachChild(child: Workflow): void {
+  if (this.children.includes(child)) {
+    throw new Error('Child already attached to this workflow');
+  }
+
+  this.children.push(child);
+  this.node.children.push(child.node);
+
+  this.emitEvent({
+    type: 'childAttached',
+    parentId: this.id,
+    child: child.node,
+  });
+}
+```
+
+**Problem**: Does not validate if `child.parent` is already set to a different workflow.
+
+**Impact**:
+1. Child appears in multiple parents' children arrays
+2. child.parent only points to original parent
+3. Observer events only propagate to original parent
+4. getRoot() returns wrong root
+5. Violates PRD's "1:1 tree mirror" requirement
+
+## Architecture Constraints & Requirements
+
+### Must Maintain
+
+1. **Dual Tree Synchronization**: Any update must modify both workflow tree and node tree
+2. **Observer Event Propagation**: Events must reach all root observers via parent chain
+3. **Circular Reference Detection**: getRoot() must detect and prevent infinite loops
+4. **Type Safety**: All TypeScript types must be maintained
+5. **Backward Compatibility**: Existing API must not break (except buggy behavior)
+6. **Event Emission**: All tree operations must emit appropriate events
+7. **Immutability**: workflow.id and node references must remain immutable
+
+### Must Add
+
+1. **Parent Validation**: Check if child already has a different parent before attaching
+2. **Circular Reference Prevention**: Check if child is an ancestor before attaching
+3. **detachChild() Method**: New method to properly remove children from tree
+4. **Comprehensive Tests**: Test the new validation and edge cases
+
+## File Locations
+
+**Core Files**:
+- `/home/dustin/projects/groundswell/src/core/workflow.ts` - Main Workflow class
+- `/home/dustin/projects/groundswell/src/core/event-tree.ts` - Tree querying
+- `/home/dustin/projects/groundswell/src/core/logger.ts` - Logging
+- `/home/dustin/projects/groundswell/src/core/context.ts` - Execution context
+
+**Decorator Files**:
+- `/home/dustin/projects/groundswell/src/decorators/step.ts`
+- `/home/dustin/projects/groundswell/src/decorators/task.ts`
+- `/home/dustin/projects/groundswell/src/decorators/observed-state.ts`
+
+**Type Definition Files**:
+- `/home/dustin/projects/groundswell/src/types/workflow.ts`
+- `/home/dustin/projects/groundswell/src/types/observer.ts`
+- `/home/dustin/projects/groundswell/src/types/events.ts`
+
+**Test Files**:
+- `/home/dustin/projects/groundswell/src/__tests__/adversarial/edge-case.test.ts` - Add new tests here
+- All other test files in `src/__tests__/` directory
+
+## PRD Requirements (Reference)
+
+The implementation must comply with PRD Section 12.2 - Workflow Base Class:
+
+> "The workflow tree should maintain consistency:
+> - A child workflow should have exactly one parent
+> - The `child.parent` property should always match the parent that contains it
+> - A child should only appear in one parent's `children` array"
+
+Additionally:
+- PRD Section 7: Observer pattern and event propagation
+- PRD Section 11: Tree debugger API
+- PRD Section 4: Logging and events model (7 event types)
+
+## Development Workflow
+
+1. **Write Failing Test First**: Follow TDD approach
+2. **Implement Fix**: Update attachChild() method
+3. **Add detachChild() Method**: New method for reparenting support
+4. **Update Tests**: Add comprehensive edge case coverage
+5. **Verify Observer Propagation**: Ensure events reach all observers
+6. **Check Tree Consistency**: Verify 1:1 mirror is maintained
+7. **Run All Tests**: Ensure no regressions (241 tests)
+
+## Success Criteria
+
+- [ ] attachChild() validates child.parent before attaching
+- [ ] attachChild() prevents circular references
+- [ ] attachChild() throws clear, actionable error messages
+- [ ] detachChild() method properly removes children
+- [ ] All observer events propagate correctly
+- [ ] Tree debugger shows consistent tree structure
+- [ ] All 241+ tests pass
+- [ ] New tests cover adversarial scenarios