agentic 0.1.0 → 0.2.0

data/.architecture/planning/self_implementation_exercise.md

@@ -0,0 +1,295 @@
+# Self-Implementation Exercise: Building Agentic with Agentic
+
+## Overview
+
+This document explores the meta-approach of using the Agentic framework to implement itself. This exercise demonstrates the framework's capabilities, serves as a practical test case, and showcases the power of the architecture.
+
+## The Goal
+
+Our meta-goal would be:
+
+> "Design and implement the Agentic framework according to the architectural specifications in the documentation."
+
+## Task Planning Phase
+
+Using the TaskPlanner, we would break down the implementation into tasks:
+
+```ruby
+planner = Agentic::TaskPlanner.new("Design and implement the Agentic framework according to the architectural specifications")
+plan = planner.plan
+```
+
+The plan might generate tasks like:
+
+1. **Analyze Architecture Documents**
+   - Agent: ResearchAnalyst
+   - Description: Review all architecture documents and extract key components, interfaces, and dependencies
+
+2. **Design Class Hierarchy**
+   - Agent: SoftwareArchitect
+   - Description: Create a hierarchical representation of all classes, modules, and their relationships
+
+3. **Implement Foundation Layer**
+   - Agent: CoreDeveloper
+   - Description: Implement the base components of the framework including Registry patterns
+
+4. **Implement Runtime Layer**
+   - Agent: SystemDeveloper
+   - Description: Implement Task, TaskFactory, PlanOrchestrator, and ExecutionContext
+
+5. **Implement Verification Layer**
+   - Agent: QualityEngineer
+   - Description: Implement verification strategies and the VerificationHub
+
+6. **Implement Extension System**
+   - Agent: IntegrationSpecialist
+   - Description: Create the plugin architecture and extension points
+
+7. **Implement Learning System**
+   - Agent: MLEngineer
+   - Description: Develop the components for storing and learning from execution history
+
+8. **Create Unit Tests**
+   - Agent: TestEngineer
+   - Description: Develop comprehensive test suite for all components
+
+9. **Document API**
+   - Agent: TechnicalWriter
+   - Description: Create API documentation for all public interfaces
+
+10. **Create Example Implementation**
+    - Agent: ApplicationDeveloper
+    - Description: Build a sample application using the framework
+
+## Executing the Plan
+
+### Task 1: Analyze Architecture Documents
+
+```ruby
+# Task created by the TaskPlanner
+analyze_task = plan.tasks.first
+
+# Creating the Research Analyst agent
+research_analyst = Agentic::Agent.build do |agent|
+  agent.role = "Research Analyst"
+  agent.purpose = "To extract structured information from documentation"
+  agent.backstory = "You are an expert at analyzing technical documents and identifying key components, patterns, and relationships."
+  agent.tools = [:document_analysis, :structure_extraction]
+end
+
+# Execute the task
+architecture_analysis = analyze_task.perform(research_analyst)
+```
+
+The output might look like:
+
+```ruby
+{
+  components: {
+    foundation_layer: ["AgentRegistry", "CapabilityManager", "MetaLearningSystem"],
+    runtime_layer: ["TaskFactory", "PlanOrchestrator", "ExecutionContext", "Task"],
+    verification_layer: ["VerificationHub", "CriticFramework", "AdaptationEngine"],
+    extension_system: ["PluginManager", "DomainAdapter", "ProtocolHandler"],
+    learning_system: ["ExecutionHistoryStore", "PatternRecognizer", "StrategyOptimizer"],
+    human_interface: ["InterventionPortal", "ExplanationEngine", "ConfigurationInterface"]
+  },
+  dependencies: [
+    {from: "Task", to: "PromptBuilder", relationship: "uses"},
+    {from: "PlanOrchestrator", to: "Task", relationship: "manages"},
+    # Additional dependencies...
+  ],
+  interfaces: [
+    {name: "Agent", methods: ["execute", "get_capabilities"]},
+    {name: "Task", methods: ["perform", "verify", "to_h"]},
+    # Additional interfaces...
+  ]
+}
+```
+
+### Task 2: Design Class Hierarchy
+
+```ruby
+# Creating the Software Architect agent
+software_architect = Agentic::Agent.build do |agent|
+  agent.role = "Software Architect"
+  agent.purpose = "To design robust and extensible software architectures"
+  agent.backstory = "You are an experienced software architect specializing in object-oriented design and scalable systems."
+  agent.tools = [:uml_generation, :dependency_analysis]
+end
+
+# Define the task with input from previous task
+design_task = Agentic::Task.new(
+  description: "Create a hierarchical representation of all classes, modules, and their relationships",
+  agent_spec: {
+    name: "SoftwareArchitect",
+    description: "Designs software architectures with a focus on extensibility",
+    instructions: "Based on the provided component analysis, create a class hierarchy diagram. Include inheritance relationships, module inclusions, and key methods."
+  },
+  input: architecture_analysis
+)
+
+# Execute the task
+class_hierarchy = design_task.perform(software_architect)
+```
+
+The output might include a structured representation of the class hierarchy:
+
+```ruby
+{
+  modules: {
+    "Agentic": {
+      classes: ["Agent", "Task", "TaskPlanner", "LlmClient", "LlmConfig"],
+      modules: ["FactoryMethods", "StructuredOutputs"]
+    },
+    "Agentic::Foundation": {
+      classes: ["AgentRegistry", "CapabilityManager", "MetaLearningSystem"]
+    },
+    # Additional modules...
+  },
+  inheritance: [
+    {parent: "Object", child: "Agentic::Agent"},
+    {parent: "Object", child: "Agentic::Task"},
+    # Additional relationships...
+  ],
+  includes: [
+    {module: "FactoryMethods", class: "Agent"},
+    {module: "Singleton", class: "AgentRegistry"},
+    # Additional inclusions...
+  ]
+}
+```
+
+### Task 3: Implement Foundation Layer
+
+```ruby
+# Creating the Core Developer agent
+core_developer = Agentic::Agent.build do |agent|
+  agent.role = "Core Developer"
+  agent.purpose = "To implement foundational software components"
+  agent.backstory = "You are a systems programmer with expertise in building core libraries and frameworks."
+  agent.tools = [:code_generation, :testing]
+end
+
+# Define the task with input from previous tasks
+implement_foundation_task = Agentic::Task.new(
+  description: "Implement the foundation layer components",
+  agent_spec: {
+    name: "CoreDeveloper",
+    description: "Implements core system components",
+    instructions: "Based on the architectural analysis and class hierarchy, implement the foundation layer components: AgentRegistry, CapabilityManager, and MetaLearningSystem. Include all necessary methods, properties, and documentation."
+  },
+  input: {
+    architecture_analysis: architecture_analysis,
+    class_hierarchy: class_hierarchy
+  }
+)
+
+# Execute the task
+foundation_implementation = implement_foundation_task.perform(core_developer)
+```
+
+The output would contain the actual code for the foundation layer:
+
+```ruby
+{
+  files: {
+    "lib/agentic/foundation/agent_registry.rb": "# frozen_string_literal: true\n\nmodule Agentic\n  module Foundation\n    class AgentRegistry\n      include Singleton\n      \n      def initialize\n        @agents = {}\n      end\n      \n      def register(agent_type, agent_class)\n        @agents[agent_type] = agent_class\n      end\n      \n      def get(agent_type)\n        @agents[agent_type]\n      end\n      \n      def create(agent_type, **args)\n        agent_class = get(agent_type)\n        raise \"Unknown agent type: #{agent_type}\" unless agent_class\n        \n        agent_class.build(**args)\n      end\n    end\n  end\nend",
+    # Additional files...
+  },
+  tests: {
+    "spec/agentic/foundation/agent_registry_spec.rb": "# frozen_string_literal: true\n\nRSpec.describe Agentic::Foundation::AgentRegistry do\n  let(:registry) { described_class.instance }\n  \n  describe \"#register and #get\" do\n    it \"stores and retrieves agent classes\" do\n      test_agent_class = Class.new\n      registry.register(:test_agent, test_agent_class)\n      \n      expect(registry.get(:test_agent)).to eq(test_agent_class)\n    end\n  end\n  \n  # Additional tests...\nend",
+    # Additional test files...
+  }
+}
+```
+
+## Continuing the Implementation
+
+This process would continue through all subsequent tasks, with each agent focusing on its area of expertise:
+
+- The **SystemDeveloper** would implement the Runtime Layer
+- The **QualityEngineer** would implement the Verification Layer  
+- The **IntegrationSpecialist** would implement the Extension System
+- The **MLEngineer** would implement the Learning System
+- The **TestEngineer** would create comprehensive tests
+- The **TechnicalWriter** would document the API
+- The **ApplicationDeveloper** would create example applications
+
+## Feedback Loop and Verification
+
+As each component is implemented, it would undergo verification:
+
+```ruby
+# Creating a Critic agent for verification
+critic = Agentic::Agent.build do |agent|
+  agent.role = "Code Critic"
+  agent.purpose = "To evaluate code quality and adherence to specifications"
+  agent.backstory = "You are an expert code reviewer with deep knowledge of software design principles and best practices."
+  agent.tools = [:code_analysis, :architectural_validation]
+end
+
+# Verify each implementation
+foundation_verification = VerificationHub.instance.verify(
+  foundation_implementation,
+  critic
+)
+
+# Process verification results
+if foundation_verification.confidence_score < 0.8
+  # Request human intervention or automated improvements
+  adaptation_engine.adapt(foundation_implementation, foundation_verification)
+end
+```
+
+## Self-Improvement Cycle
+
+Once the initial implementation is complete, the system could begin improving itself:
+
+```ruby
+# Analyze the implementation for potential improvements
+improvement_task = Agentic::Task.new(
+  description: "Identify opportunities for improving the Agentic framework",
+  agent_spec: {
+    name: "SystemAnalyst",
+    description: "Identifies system improvements and optimizations",
+    instructions: "Analyze the current implementation of the Agentic framework. Identify areas where performance could be improved, code could be simplified, or architecture could be enhanced."
+  },
+  input: {
+    current_implementation: {
+      # Implementation details...
+    },
+    usage_metrics: {
+      # Performance metrics...
+    }
+  }
+)
+
+# Execute the task using an analyst agent
+analyst = Agentic::Agent.build do |agent|
+  agent.role = "System Analyst"
+  agent.purpose = "To identify system improvements"
+  agent.backstory = "You are an expert at analyzing systems and identifying optimization opportunities."
+  agent.tools = [:performance_analysis, :code_metrics]
+end
+
+improvement_recommendations = improvement_task.perform(analyst)
+
+# Implement the recommended improvements
+# ...
+```
+
+## Benefits of the Meta-Approach
+
+This exercise demonstrates several key benefits:
+
+1. **Dogfooding**: Using the system to build itself provides deep insights into usability
+2. **Completeness Testing**: Ensures all components work together as intended
+3. **Example Implementation**: Creates a concrete example of the framework in action
+4. **Learning Opportunity**: Demonstrates how different agents can collaborate 
+5. **System Evolution**: Shows how the system can improve itself over time
+
+## Conclusion
+
+This meta-exercise of building Agentic with Agentic demonstrates the framework's potential power and flexibility. It provides a concrete example of how different specialized agents can collaborate on a complex software project, how tasks can be broken down and chained together, and how verification ensures quality at each step.
+
+While initially this would be a theoretical exercise, as the framework matures, it could become increasingly practical to use the framework to enhance and extend itself, creating a virtuous cycle of improvement.

data/.architecture/planning/session_compaction_rule.md

@@ -0,0 +1,43 @@
+# Session Compaction Rule
+
+## Purpose
+Store session history summaries in a structured format to maintain an accessible record of design decisions, implementation activities, and architectural evolution across the project.
+
+## Rules
+
+1. **Directory Structure**
+   - All session summaries are stored in the `.claude-sessions/` directory
+   - Directory is created if it doesn't exist
+
+2. **Naming Convention**
+   - Files follow the format: `NNN-descriptive-session-name.md`
+   - Where NNN is a sequential number (e.g., 001, 002)
+   - The descriptive name should concisely capture the session's main focus
+
+3. **Content Structure**
+   - Each file includes:
+     - Primary request and intent
+     - Key technical concepts
+     - Files and code sections impacted
+     - Problem-solving approach
+     - Pending tasks and next steps
+
+4. **When to Compact**
+   - After completing significant architectural or implementation work
+   - When switching to a new major feature or component
+   - Periodically during long sessions to maintain continuity
+
+5. **Process**
+   - Generate a comprehensive summary of the conversation
+   - Save to a new file in the `.claude-sessions/` directory
+   - Ensure all key decisions and implementation details are captured
+
+## Integration with Architecture Documentation
+
+Session compaction summaries should cross-reference with other architectural documents to maintain a coherent history of the project's evolution. Key architectural decisions identified during sessions should be properly documented in:
+
+- Architecture Decision Records (ADRs) in the `.architecture-review/` directory
+- Updates to `ArchitectureConsiderations.md` for high-level changes
+- Component-specific documentation files
+
+This ensures that the project maintains both a record of what was done (session summaries) and the rationale behind architectural decisions (formal documentation).

data/.architecture/planning/streaming_observability_feature.md

@@ -0,0 +1,223 @@
+# Streaming Observability Feature Planning
+
+## Feature Overview
+
+**Feature Name**: Real-Time Streaming Observability  
+**Feature Type**: Infrastructure Enhancement  
+**Priority**: High  
+**Complexity**: Medium-High  
+**Estimated Effort**: 8-12 weeks  
+
+## Problem Statement
+
+The current Agentic framework lacks real-time observability into the internal workings of:
+- Task execution intermediate steps
+- Agent assembly decision-making process  
+- Plan building and goal analysis phases
+- Orchestrator scheduling and dependency resolution
+- LLM interactions and token usage
+
+This limits debugging capability, reduces transparency, and makes it difficult to optimize performance or understand system behavior in production.
+
+## Business Value
+
+### For Developers
+- **Enhanced Debugging**: Real-time visibility into task execution steps
+- **Performance Optimization**: Live metrics for LLM usage and execution times
+- **Better Understanding**: Insight into agent assembly and capability selection
+
+### For Operations
+- **Production Monitoring**: Real-time system health and performance metrics
+- **Issue Diagnosis**: Detailed execution traces for troubleshooting
+- **Capacity Planning**: Usage patterns and resource consumption data
+
+### For Framework Evolution
+- **Data-Driven Improvements**: Execution patterns inform architectural decisions
+- **Community Adoption**: Better observability increases framework usability
+- **Integration Readiness**: Stream data enables external monitoring systems
+
+## Architectural Impact Assessment
+
+### System Layers Affected
+
+#### 1. Foundation Layer
+- **Observable Pattern Extension**: StreamingObservable module
+- **Central Coordination**: StreamingObservabilityHub singleton
+- **Configuration Management**: Integration with existing LlmConfig
+
+#### 2. Runtime Layer  
+- **Task Enhancement**: Stream task lifecycle and execution steps
+- **Orchestrator Enhancement**: Stream scheduling and dependency resolution
+- **Agent Enhancement**: Stream assembly process and capability selection
+
+#### 3. Verification Layer
+- **Metrics Integration**: Real-time verification and quality metrics
+- **Performance Tracking**: Stream verification execution times
+
+#### 4. Extension System
+- **Plugin Support**: Pluggable stream processors and formatters
+- **Domain Adapters**: Domain-specific observability extensions
+
+### Interface Changes
+
+**New Interfaces:**
+- `ObservabilityStream` - Generic streaming interface
+- `StreamProcessor` - Event processing and filtering
+- `MetricsAggregator` - Real-time metrics calculation
+
+**Enhanced Interfaces:**
+- `Observable` → `StreamingObservable` - Backwards compatible extension
+- `Task#perform` - Optional streaming callback support
+- `PlanOrchestrator` - Enhanced lifecycle hooks
+
+### Dependency Impact
+
+**New Dependencies (Minimal):**
+- `concurrent-ruby` - Already present, leverage for thread safety
+- `websocket-driver` - Optional, for WebSocket support only
+
+**No Breaking Changes:**
+- All existing APIs remain unchanged
+- Streaming is opt-in through configuration
+- Graceful degradation when streams fail
+
+## Technical Design Summary
+
+### Core Components
+
+1. **StreamingObservabilityHub**
+   - Singleton coordinator for all streaming events
+   - Stream registration and management
+   - Event routing and distribution
+
+2. **ObservabilityStream Interface**
+   - ConsoleStream, FileStream, MemoryStream implementations
+   - WebSocketStream for dashboard integration
+   - Pluggable backend support
+
+3. **Enhanced Observable Pattern**
+   - StreamingObservable mixin extending existing Observable
+   - Progress events, batch notifications
+   - Thread-safe streaming with buffering
+
+### Integration Strategy
+
+**Phase 1: Infrastructure** (3 weeks)
+- Core streaming classes and interfaces
+- Basic console and file stream implementations
+- Enhanced Observable pattern
+
+**Phase 2: Task & Agent Streaming** (4 weeks)  
+- Task execution step streaming
+- Agent assembly process streaming
+- Enhanced CLI real-time feedback
+
+**Phase 3: Orchestration Streaming** (3 weeks)
+- Plan building and goal analysis streaming
+- Orchestrator scheduling and metrics
+- Real-time aggregation and reporting
+
+**Phase 4: Advanced Features** (2 weeks)
+- WebSocket support for dashboards
+- Advanced filtering and processing
+- Configuration and CLI integration
+
+## Risk Assessment
+
+### Technical Risks
+
+**Performance Impact**  
+- *Risk*: Streaming overhead slows execution
+- *Mitigation*: Non-blocking design, comprehensive benchmarking
+- *Monitoring*: Performance metrics and circuit breakers
+
+**Complexity Risk**
+- *Risk*: Added complexity affects maintainability  
+- *Mitigation*: Clean interfaces, extensive documentation
+- *Testing*: Comprehensive unit and integration test coverage
+
+**Backwards Compatibility**
+- *Risk*: Changes break existing functionality
+- *Mitigation*: Maintain API compatibility, opt-in streaming
+- *Validation*: Regression testing for all existing features
+
+### Implementation Risks
+
+**Resource Allocation**
+- *Risk*: Extended timeline affects other priorities
+- *Mitigation*: Phased implementation with incremental value
+- *Flexibility*: Can pause between phases if needed
+
+**Community Impact**
+- *Risk*: Complex feature reduces framework accessibility
+- *Mitigation*: Comprehensive documentation and examples
+- *Support*: Clear migration guides and troubleshooting
+
+## Success Criteria
+
+### Phase 1 Success
+- [ ] Streaming infrastructure implemented without performance regression
+- [ ] Clean integration with existing Observable pattern
+- [ ] Basic console streaming working for CLI
+
+### Phase 2 Success  
+- [ ] Real-time task execution visibility
+- [ ] Agent assembly insights available
+- [ ] Enhanced debugging capability demonstrated
+
+### Phase 3 Success
+- [ ] Complete orchestration observability
+- [ ] Real-time performance metrics
+- [ ] Production-ready monitoring capability
+
+### Final Success
+- [ ] Framework adoption improved through better observability
+- [ ] Production deployments using streaming for monitoring
+- [ ] Community feedback validates approach
+
+## Resource Requirements
+
+### Development Time
+- **Total Estimate**: 8-12 weeks
+- **Core Infrastructure**: 3 weeks (1 developer)
+- **Feature Implementation**: 6 weeks (1 developer)  
+- **Testing & Documentation**: 2-3 weeks (distributed)
+
+### Skills Required
+- Ruby concurrency and threading expertise
+- Streaming architecture experience
+- Performance optimization knowledge
+- Integration testing capabilities
+
+## Next Steps
+
+### Immediate Actions
+1. **Architectural Review**: Present design to architectural review board
+2. **ADR Creation**: Document key architectural decisions
+3. **Prototype Development**: Build minimal viable implementation
+4. **Performance Baseline**: Establish current performance benchmarks
+
+### Planning Phase
+1. **Detailed Task Breakdown**: Decompose phases into specific tasks
+2. **Timeline Refinement**: Adjust estimates based on team capacity
+3. **Dependency Coordination**: Align with other framework initiatives
+4. **Community Communication**: Share plans with framework users
+
+## Related Documentation
+
+- `ArchitectureConsiderations.md` - System layer documentation
+- `ArchitecturalFeatureBuilder.md` - Implementation guidelines
+- `.architecture/principles.md` - Architectural principles compliance
+- `docs/streaming_observability_architecture.md` - Detailed technical design
+- Future ADR: Streaming architecture decisions
+
+## Approval Status
+
+**Planning Phase**: ✅ Complete  
+**Architectural Review**: ⏳ Pending  
+**Implementation Approval**: ⏳ Pending  
+**Resource Allocation**: ⏳ Pending  
+
+---
+
+*This feature planning document follows the architectural planning framework established in `.architecture/planning/` and aligns with the principles defined in `.architecture/principles.md`.*