agentic 0.1.0 → 0.2.0

data/.architecture/decisions/adrs/ADR-016-agent-assembly-engine.md

@@ -0,0 +1,117 @@
+# ADR-016: Agent Assembly Engine
+
+## Status
+
+Accepted
+
+## Context
+
+A key requirement for the Agentic framework is the ability to dynamically assemble agents based on task requirements. Currently, agents are constructed manually or through predefined configurations, which:
+
+1. Requires foreknowledge of task requirements
+2. Results in either overly generic or overly specialized agents
+3. Misses opportunities for capability reuse and optimization
+4. Does not leverage knowledge from previous similar tasks
+
+We need a system that can analyze task requirements, select appropriate capabilities, and assemble agents optimized for specific tasks. This system should also integrate with our persistent storage to reuse existing agents when appropriate.
+
+## Decision
+
+We will implement an `AgentAssemblyEngine` that handles the dynamic construction of agents based on task analysis. The engine will:
+
+1. Analyze task requirements to determine needed capabilities
+2. Select appropriate capabilities using pluggable selection strategies
+3. Build agents with the selected capabilities
+4. Store assembled agents for future reuse
+5. Find and reuse suitable existing agents when possible
+
+Key components:
+
+```ruby
+# Main engine
+engine = AgentAssemblyEngine.new(registry, agent_store)
+
+# Assemble an agent for a task
+agent = engine.assemble_agent(task, strategy: strategy, store: true)
+
+# Analyze task requirements
+requirements = engine.analyze_requirements(task)
+
+# Select capabilities based on requirements
+capabilities = engine.select_capabilities(requirements, strategy)
+
+# Build an agent with selected capabilities
+agent = engine.build_agent(task, capabilities)
+```
+
+The engine will support multiple composition strategies through a Strategy pattern:
+
+1. `DefaultCompositionStrategy`: Basic capability selection based on requirements
+2. `LlmAssistedCompositionStrategy`: Uses an LLM to suggest optimal capabilities
+3. Custom strategies can be implemented for specialized domains
+
+## Consequences
+
+### Positive
+
+1. Enables task-optimized agent assembly
+2. Supports agent reuse through integration with persistent store
+3. Pluggable strategies allow domain-specific optimization
+4. Reduces manual configuration requirements
+5. Promotes capability-based agent design
+
+### Negative
+
+1. Increases system complexity
+2. Requires accurate task requirement analysis
+3. May lead to capability explosion without governance
+4. LLM-assisted strategies add external dependencies
+
+### Neutral
+
+1. Changes agent construction paradigm from static to dynamic
+2. Requires integration with the capability registry and agent store
+3. May require adjustment of existing agent usage patterns
+
+## Implementation Notes
+
+### Task Requirement Analysis
+
+The engine will analyze tasks using multiple approaches:
+
+1. Task description analysis using pattern matching or NLP
+2. Agent specification analysis for explicit capability requirements
+3. Task input analysis for capability hints
+4. LLM-assisted analysis for complex requirements
+
+### Capability Selection
+
+Selection strategies will consider:
+- Capability relevance to the task
+- Capability performance metrics from learning system
+- Capability dependencies and compatibility
+- Agent resource constraints
+
+### Agent Matching
+
+When finding existing agents for a task, the engine will:
+1. Identify primary capabilities required for the task
+2. Find agents with these capabilities
+3. Score agents based on capability match
+4. Return the best matching agent if score exceeds threshold
+
+### Integration with Learning System
+
+The engine will integrate with the learning system to:
+1. Record capability selection decisions
+2. Track assembled agent performance
+3. Improve selection strategies based on outcomes
+4. Optimize capability compositions
+
+## Future Considerations
+
+1. Support for agent composition constraints (e.g., maximum capabilities)
+2. Advanced capability compatibility checking
+3. Agent performance benchmarking for selection decisions
+4. Multi-agent assembly for complex tasks
+5. Capability governance and approval workflows

data/.architecture/decisions/adrs/ADR-017-streaming-observability.md

@@ -0,0 +1,171 @@
+# ADR-017: Streaming Observability Architecture
+
+## Status
+
+Proposed
+
+## Context
+
+The Agentic framework currently provides basic observability through the Observable pattern and ExecutionObserver, but lacks real-time insights into intermediate execution steps. Users need visibility into:
+
+1. **Task Execution Steps**: What happens during agent.execute() beyond start/end events
+2. **Agent Assembly Process**: How capabilities are analyzed, selected, and assembled
+3. **Plan Building**: How goals are analyzed and broken down into tasks
+4. **Orchestration Decisions**: Task scheduling, dependency resolution, and resource allocation
+5. **LLM Interactions**: Token usage, response times, and content flow
+
+This visibility is crucial for debugging, performance optimization, and production monitoring. The current system provides only discrete lifecycle events without streaming intermediate states or progress updates.
+
+## Decision Drivers
+
+* **Debugging Complexity**: Complex multi-agent workflows are difficult to debug without intermediate visibility
+* **Performance Optimization**: Real-time metrics needed to identify bottlenecks and optimize resource usage
+* **Production Monitoring**: Operations teams need live insights into system behavior and health
+* **Framework Adoption**: Better observability increases developer confidence and framework usability
+* **Non-Breaking Requirement**: Must maintain backwards compatibility with existing Observable pattern
+* **Performance Requirement**: Streaming must not significantly impact execution performance
+
+## Decision
+
+We will implement a **Streaming Observability Architecture** that extends the existing Observable pattern with real-time event streaming capabilities while maintaining full backwards compatibility.
+
+**Architectural Components Affected:**
+* Foundation Layer: StreamingObservabilityHub, Enhanced Observable pattern
+* Runtime Layer: Task, PlanOrchestrator, AgentAssemblyEngine streaming integration
+* Verification Layer: Real-time metrics aggregation and reporting
+* Extension System: Pluggable stream processors and formatters
+
+**Interface Changes:**
+* New `StreamingObservable` module extending existing `Observable`
+* New `ObservabilityStream` interface with multiple implementations
+* New `StreamProcessor` interface for event filtering and transformation
+* Enhanced lifecycle hooks in `PlanOrchestrator` with streaming support
+* Optional streaming callbacks in task execution methods
+
+## Consequences
+
+### Positive
+
+* **Enhanced Debugging**: Real-time visibility into all execution phases enables rapid issue identification
+* **Performance Insights**: Live metrics help identify optimization opportunities and resource bottlenecks
+* **Production Readiness**: Stream data enables external monitoring system integration
+* **Framework Usability**: Better observability reduces learning curve and increases developer confidence
+* **Future Extensibility**: Streaming infrastructure supports advanced features like dashboards and analytics
+* **Community Value**: Open observability data enables community-driven improvements and integrations
+
+### Negative
+
+* **Implementation Complexity**: Adds significant code complexity across multiple system layers
+* **Performance Overhead**: Event emission and processing may impact execution performance
+* **Memory Usage**: Event buffering and stream management increases memory footprint
+* **Testing Complexity**: Streaming behavior requires additional test scenarios and validation
+* **Documentation Burden**: New observability features require comprehensive documentation and examples
+* **Maintenance Cost**: Additional code surface area increases long-term maintenance requirements
+
+### Neutral
+
+* **Optional Feature**: Streaming is opt-in, so existing users unaffected until they choose to enable it
+* **Backwards Compatibility**: All existing Observable behavior preserved without changes
+* **Incremental Rollout**: Phased implementation allows gradual adoption and validation
+* **Configuration Flexibility**: Multiple stream types and filtering options accommodate different use cases
+
+## Implementation
+
+**Phase 1: Core Infrastructure (3 weeks)**
+* Implement `StreamingObservabilityHub` singleton for event coordination
+* Create `ObservabilityStream` interface with `ConsoleStream`, `FileStream`, `MemoryStream`
+* Extend `Observable` pattern with `StreamingObservable` mixin
+* Integrate basic streaming with existing `ExecutionObserver`
+
+**Phase 2: Task and Agent Streaming (4 weeks)**
+* Enhance `Task#perform` with optional streaming callbacks
+* Add streaming to `AgentAssemblyEngine` capability analysis and selection
+* Implement LLM interaction streaming in agent execution
+* Create enhanced real-time CLI feedback
+
+**Phase 3: Orchestration and Planning Streaming (3 weeks)**
+* Add streaming to `TaskPlanner` goal analysis and task generation
+* Enhance `PlanOrchestrator` with dependency analysis and scheduling streams
+* Implement `MetricsAggregator` for real-time performance metrics
+* Create orchestration visualization and reporting
+
+**Phase 4: Advanced Features (2 weeks)**
+* Implement `WebSocketStream` for dashboard integration
+* Add `StreamProcessor` framework for filtering and transformation
+* Create configuration management and CLI integration
+* Develop example dashboard and monitoring integrations
+
+## Alternatives Considered
+
+### Alternative 1: External Instrumentation
+
+Use external instrumentation libraries (OpenTelemetry, StatsD) for observability.
+
+**Pros:**
+* Leverages existing infrastructure and tooling
+* Standard metrics format for integration
+* Proven performance characteristics
+
+**Cons:**
+* Less control over event structure and timing
+* Limited ability to provide domain-specific insights
+* Additional dependency on external libraries
+* Doesn't provide real-time streaming within the framework
+
+### Alternative 2: Callback-Based Hooks
+
+Extend existing lifecycle hooks with detailed callback parameters.
+
+**Pros:**
+* Simpler implementation using existing patterns
+* No new infrastructure required
+* Maintains current performance characteristics
+
+**Cons:**
+* Limited to predefined hook points
+* No support for intermediate streaming
+* Difficult to add filtering and processing
+* Poor scalability for complex observability needs
+
+### Alternative 3: Event Sourcing Pattern
+
+Implement full event sourcing with persistent event store.
+
+**Pros:**
+* Complete audit trail of all system events
+* Enables replay and analysis capabilities
+* Strong consistency guarantees
+
+**Cons:**
+* Significant implementation complexity
+* Performance impact from persistent storage
+* Overkill for real-time observability needs
+* Changes fundamental system architecture
+
+## Validation
+
+**Acceptance Criteria:**
+- [ ] Streaming adds <5% overhead to task execution performance
+- [ ] All existing Observable behavior preserved without changes
+- [ ] Real-time task execution steps visible in CLI
+- [ ] Agent assembly process insights available
+- [ ] Plan building and orchestration streaming functional
+- [ ] WebSocket streaming supports external dashboards
+- [ ] Comprehensive test coverage for all streaming scenarios
+- [ ] Production-ready configuration and monitoring
+
+**Testing Approach:**
+* Performance benchmarks comparing execution with/without streaming
+* Compatibility tests ensuring existing Observable behavior unchanged
+* Integration tests validating end-to-end streaming flows
+* Load testing for stream performance under high event volume
+* Memory usage testing for event buffering and cleanup
+* Thread safety testing for concurrent streaming operations
+
+## References
+
+* [Architectural Review: Streaming Observability](../reviews/streaming_observability_review.md)
+* [Feature Planning: Streaming Observability](../planning/streaming_observability_feature.md)
+* [System Architecture](../../ArchitectureConsiderations.md)
+* [Observable Pattern Implementation](./ADR-011-task-observable-pattern.md)
+* [Architectural Principles](../principles.md)

data/.architecture/decisions/capability_tools_distinction.md

@@ -0,0 +1,150 @@
+# Capability and Tools Distinction
+
+## Overview
+
+This document clarifies the distinction between "capabilities" and "tools" in the Agentic framework. While both concepts relate to agent functionality, they serve different purposes and have different characteristics.
+
+## Capabilities
+
+Capabilities represent rich, composable components that provide significant functionality to agents. They are formal, versioned interfaces with well-defined inputs and outputs.
+
+### Key Characteristics
+
+1. **Rich Interface Definition**
+   - Formal specification of inputs and outputs
+   - Type definitions and validation
+   - Required vs. optional parameters
+
+2. **Semantic Versioning**
+   - Major.Minor.Patch versioning scheme
+   - Backward compatibility guarantees
+   - Version constraints for dependencies
+
+3. **Dependency Management**
+   - Explicit declaration of dependencies
+   - Version constraints for dependencies
+   - Automatic resolution of dependency graphs
+
+4. **Composition Support**
+   - Capabilities can be composed from other capabilities
+   - Higher-order capabilities encapsulate multiple capabilities
+   - Composition strategies for complex capability combinations
+
+5. **Provider Implementation**
+   - Separation of interface (specification) from implementation
+   - Multiple providers can implement the same capability
+   - Providers can be swapped without affecting consumers
+
+6. **Performance Tracking**
+   - Execution metrics collection
+   - Success/failure tracking
+   - Performance optimization based on historical data
+
+### Example
+
+```ruby
+# Define a capability specification
+text_gen_capability = Agentic::CapabilitySpecification.new(
+  name: "text_generation",
+  description: "Generates text based on a prompt",
+  version: "1.0.0",
+  inputs: {
+    prompt: { type: "string", required: true, description: "The prompt to generate text from" },
+    max_tokens: { type: "integer", default: 100, description: "Maximum tokens to generate" }
+  },
+  outputs: {
+    response: { type: "string", description: "The generated text" }
+  }
+)
+
+# Create a provider for the capability
+text_gen_provider = Agentic::CapabilityProvider.new(
+  capability: text_gen_capability,
+  implementation: ->(inputs) { { response: "Generated text for: #{inputs[:prompt]}" } }
+)
+
+# Register with the system
+registry.register(text_gen_capability, text_gen_provider)
+```
+
+## Tools
+
+Tools are simpler function-like utilities that agents can use for specific, focused tasks. They provide a lightweight mechanism for extending agent functionality without the formality of capabilities.
+
+### Key Characteristics
+
+1. **Simple Interface**
+   - Function-like calling convention
+   - Minimal parameter validation
+   - Limited formal documentation
+
+2. **No Versioning**
+   - Single implementation at a time
+   - No formal backward compatibility guarantees
+   - No dependency management
+
+3. **Direct Implementation**
+   - No separation between interface and implementation
+   - Direct execution without provider indirection
+   - Typically implemented as lambda functions or simple methods
+
+4. **Limited Composition**
+   - No formal composition support
+   - Manual composition through function calls
+   - No dependency resolution
+
+5. **Minimal Tracking**
+   - Basic usage logging only
+   - No formal performance metrics
+   - Limited optimization opportunities
+
+### Example
+
+```ruby
+# Define a simple tool
+agent.add_tool("calculate_sum") do |numbers|
+  numbers.sum
+end
+
+# Use the tool
+result = agent.use_tool("calculate_sum", [1, 2, 3, 4, 5])
+# => 15
+```
+
+## When to Use Each
+
+### Use Capabilities When:
+
+- The functionality is complex and reusable across multiple agent types
+- Formal versioning and backward compatibility are important
+- The functionality has dependencies on other capabilities
+- Performance tracking and optimization are needed
+- Multiple implementations might be required
+- The functionality will evolve over time
+
+### Use Tools When:
+
+- The functionality is simple and focused
+- Formal versioning isn't necessary
+- The tool has no dependencies on other components
+- Quick implementation is more important than formality
+- The tool is specific to a particular agent or use case
+- Performance tracking isn't critical
+
+## Compatibility Layer
+
+For backward compatibility, the framework provides a compatibility layer that allows tools to be used as capabilities and vice versa:
+
+```ruby
+# Convert a tool to a capability
+capability = registry.tool_to_capability("calculate_sum", agent.tools["calculate_sum"])
+
+# Convert a capability to a tool
+agent.add_tool_from_capability("text_generation", registry.get("text_generation"))
+```
+
+## Future Direction
+
+The framework is gradually moving toward a capability-centric approach, as capabilities provide richer functionality, better composition, and improved maintainability. However, tools will continue to be supported for simpler use cases and backward compatibility.
+
+Over time, we expect more functionality to migrate from tools to capabilities, especially as the agent assembly system evolves to leverage capability-based composition more effectively.

data/.architecture/decisions/cli_command_structure.md

@@ -0,0 +1,61 @@
+# Architectural Decision Record: CLI Command Structure
+
+## Context
+
+The Agentic CLI currently has duplicate command definitions appearing in the help output. Commands related to agent management and configuration appear both as top-level commands and as subcommands. This is causing confusion for users trying to understand the proper command structure.
+
+Investigation revealed that two implementations of the same functionality exist:
+1. Nested classes within the main CLI class (`AgentCommands` and `ConfigCommands` in cli.rb) with enhanced UI
+2. Standalone files in the cli/ directory (`agent.rb` and `config.rb`) with simpler implementations
+
+Both implementations are being loaded and registered, resulting in duplicate commands in the help output.
+
+## Decision
+
+We will standardize on the nested class implementation and remove the standalone file implementations. This approach:
+
+1. Provides enhanced UI with colorization and box output for a better user experience
+2. Maintains a clear hierarchy of commands
+3. Eliminates duplicate commands in the help output
+
+We will:
+1. Remove or disable the standalone CLI command files (agent.rb, config.rb)
+2. Update requires in agentic.rb to reflect this change
+3. Ensure proper Thor subcommand registration
+
+## Rationale
+
+The nested class implementation provides a superior user experience with colorized output and formatted boxes. While moving to standalone files might provide better modularity in the long term, the current priority is to fix the command duplication and maintain the enhanced UI.
+
+## Consequences
+
+### Positive
+- Elimination of duplicate commands in CLI help output
+- Consistent, enhanced UI for all commands
+- Clear command hierarchy
+
+### Negative
+- The CLI implementation remains in a single file, which may be less modular
+- Future changes to the CLI may require more coordination
+- Long-term maintenance might be more challenging
+
+### Neutral
+- This approach prioritizes user experience over code modularity
+- A future refactoring to a hybrid approach may still be desirable
+
+## Future Considerations
+
+In future versions, we may want to consider:
+1. Refactoring to a hybrid approach that maintains the enhanced UI while improving modularity
+2. Separating UI presentation from command logic
+3. Creating a more comprehensive testing framework for CLI commands
+
+## Implementation Notes
+
+The implementation will:
+1. Remove or comment out requires for standalone CLI files in agentic.rb
+2. Ensure only nested class implementations are being registered
+3. Test all CLI commands to ensure they work as expected
+4. Update documentation to reflect the command structure
+
+Date: May 22, 2024