@diottodev/runic 1.0.0 → 1.1.0

package/engineering/skills/agent-designer/README.md

@@ -0,0 +1,430 @@
+# Agent Designer - Multi-Agent System Architecture Toolkit
+
+**Tier:** POWERFUL  
+**Category:** Engineering  
+**Tags:** AI agents, architecture, system design, orchestration, multi-agent systems
+
+A comprehensive toolkit for designing, architecting, and evaluating multi-agent systems. Provides structured approaches to agent architecture patterns, tool design principles, communication strategies, and performance evaluation frameworks.
+
+## Overview
+
+The Agent Designer skill includes three core components:
+
+1. **Agent Planner** (`agent_planner.py`) - Designs multi-agent system architectures
+2. **Tool Schema Generator** (`tool_schema_generator.py`) - Creates structured tool schemas
+3. **Agent Evaluator** (`agent_evaluator.py`) - Evaluates system performance and identifies optimizations
+
+## Quick Start
+
+### 1. Design a Multi-Agent Architecture
+
+```bash
+# Use sample requirements or create your own
+python agent_planner.py assets/sample_system_requirements.json -o my_architecture
+
+# This generates:
+# - my_architecture.json (complete architecture)
+# - my_architecture_diagram.mmd (Mermaid diagram)
+# - my_architecture_roadmap.json (implementation plan)
+```
+
+### 2. Generate Tool Schemas
+
+```bash
+# Use sample tool descriptions or create your own
+python tool_schema_generator.py assets/sample_tool_descriptions.json -o my_tools
+
+# This generates:
+# - my_tools.json (complete schemas)
+# - my_tools_openai.json (OpenAI format)
+# - my_tools_anthropic.json (Anthropic format)
+# - my_tools_validation.json (validation rules)
+# - my_tools_examples.json (usage examples)
+```
+
+### 3. Evaluate System Performance
+
+```bash
+# Use sample execution logs or your own
+python agent_evaluator.py assets/sample_execution_logs.json -o evaluation
+
+# This generates:
+# - evaluation.json (complete report)
+# - evaluation_summary.json (executive summary)
+# - evaluation_recommendations.json (optimization suggestions)
+# - evaluation_errors.json (error analysis)
+```
+
+## Detailed Usage
+
+### Agent Planner
+
+The Agent Planner designs multi-agent architectures based on system requirements.
+
+#### Input Format
+
+Create a JSON file with system requirements:
+
+```json
+{
+  "goal": "Your system's primary objective",
+  "description": "Detailed system description",
+  "tasks": ["List", "of", "required", "tasks"],
+  "constraints": {
+    "max_response_time": 30000,
+    "budget_per_task": 1.0,
+    "quality_threshold": 0.9
+  },
+  "team_size": 6,
+  "performance_requirements": {
+    "high_throughput": true,
+    "fault_tolerance": true,
+    "low_latency": false
+  },
+  "safety_requirements": [
+    "Input validation and sanitization",
+    "Output content filtering"
+  ]
+}
+```
+
+#### Command Line Options
+
+```bash
+python agent_planner.py <input_file> [OPTIONS]
+
+Options:
+  -o, --output PREFIX    Output file prefix (default: agent_architecture)
+  --format FORMAT        Output format: json, both (default: both)
+```
+
+#### Output Files
+
+- **Architecture JSON**: Complete system design with agents, communication topology, and scaling strategy
+- **Mermaid Diagram**: Visual representation of the agent architecture
+- **Implementation Roadmap**: Phased implementation plan with timelines and risks
+
+#### Architecture Patterns
+
+The planner automatically selects from these patterns based on requirements:
+
+- **Single Agent**: Simple, focused tasks (1 agent)
+- **Supervisor**: Hierarchical delegation (2-8 agents)
+- **Swarm**: Peer-to-peer collaboration (3-20 agents)
+- **Hierarchical**: Multi-level management (5-50 agents)
+- **Pipeline**: Sequential processing (3-15 agents)
+
+### Tool Schema Generator
+
+Generates structured tool schemas compatible with OpenAI and Anthropic formats.
+
+#### Input Format
+
+Create a JSON file with tool descriptions:
+
+```json
+{
+  "tools": [
+    {
+      "name": "tool_name",
+      "purpose": "What the tool does",
+      "category": "Tool category (search, data, api, etc.)",
+      "inputs": [
+        {
+          "name": "parameter_name",
+          "type": "string",
+          "description": "Parameter description",
+          "required": true,
+          "examples": ["example1", "example2"]
+        }
+      ],
+      "outputs": [
+        {
+          "name": "result_field",
+          "type": "object",
+          "description": "Output description"
+        }
+      ],
+      "error_conditions": ["List of possible errors"],
+      "side_effects": ["List of side effects"],
+      "idempotent": true,
+      "rate_limits": {
+        "requests_per_minute": 60
+      }
+    }
+  ]
+}
+```
+
+#### Command Line Options
+
+```bash
+python tool_schema_generator.py <input_file> [OPTIONS]
+
+Options:
+  -o, --output PREFIX    Output file prefix (default: tool_schemas)
+  --format FORMAT        Output format: json, both (default: both)
+  --validate             Validate generated schemas
+```
+
+#### Output Files
+
+- **Complete Schemas**: All schemas with validation and examples
+- **OpenAI Format**: Schemas compatible with OpenAI function calling
+- **Anthropic Format**: Schemas compatible with Anthropic tool use
+- **Validation Rules**: Input validation specifications
+- **Usage Examples**: Example calls and responses
+
+#### Schema Features
+
+- **Input Validation**: Comprehensive parameter validation rules
+- **Error Handling**: Structured error response formats
+- **Rate Limiting**: Configurable rate limit specifications
+- **Documentation**: Auto-generated usage examples
+- **Security**: Built-in security considerations
+
+### Agent Evaluator
+
+Analyzes agent execution logs to identify performance issues and optimization opportunities.
+
+#### Input Format
+
+Create a JSON file with execution logs:
+
+```json
+{
+  "execution_logs": [
+    {
+      "task_id": "unique_task_identifier",
+      "agent_id": "agent_identifier", 
+      "task_type": "task_category",
+      "start_time": "2024-01-15T09:00:00Z",
+      "end_time": "2024-01-15T09:02:34Z",
+      "duration_ms": 154000,
+      "status": "success",
+      "actions": [
+        {
+          "type": "tool_call",
+          "tool_name": "web_search",
+          "duration_ms": 2300,
+          "success": true
+        }
+      ],
+      "results": {
+        "summary": "Task results",
+        "quality_score": 0.92
+      },
+      "tokens_used": {
+        "input_tokens": 1250,
+        "output_tokens": 2800,
+        "total_tokens": 4050
+      },
+      "cost_usd": 0.081,
+      "error_details": null,
+      "tools_used": ["web_search"],
+      "retry_count": 0
+    }
+  ]
+}
+```
+
+#### Command Line Options
+
+```bash
+python agent_evaluator.py <input_file> [OPTIONS]
+
+Options:
+  -o, --output PREFIX    Output file prefix (default: evaluation_report)
+  --format FORMAT        Output format: json, both (default: both)
+  --detailed             Include detailed analysis in output
+```
+
+#### Output Files
+
+- **Complete Report**: Comprehensive performance analysis
+- **Executive Summary**: High-level metrics and health assessment
+- **Optimization Recommendations**: Prioritized improvement suggestions
+- **Error Analysis**: Detailed error patterns and solutions
+
+#### Evaluation Metrics
+
+**Performance Metrics**:
+- Task success rate and completion times
+- Token usage and cost efficiency
+- Error rates and retry patterns
+- Throughput and latency distributions
+
+**System Health**:
+- Overall health score (poor/fair/good/excellent)
+- SLA compliance tracking
+- Resource utilization analysis
+- Trend identification
+
+**Bottleneck Analysis**:
+- Agent performance bottlenecks
+- Tool usage inefficiencies  
+- Communication overhead
+- Resource constraints
+
+## Architecture Patterns Guide
+
+### When to Use Each Pattern
+
+#### Single Agent
+- **Best for**: Simple, focused tasks with clear boundaries
+- **Team size**: 1 agent
+- **Complexity**: Low
+- **Examples**: Personal assistant, document summarizer, simple automation
+
+#### Supervisor
+- **Best for**: Hierarchical task decomposition with quality control
+- **Team size**: 2-8 agents  
+- **Complexity**: Medium
+- **Examples**: Research coordinator with specialists, content review workflow
+
+#### Swarm
+- **Best for**: Distributed problem solving with high fault tolerance
+- **Team size**: 3-20 agents
+- **Complexity**: High
+- **Examples**: Parallel data processing, distributed research, competitive analysis
+
+#### Hierarchical  
+- **Best for**: Large-scale operations with organizational structure
+- **Team size**: 5-50 agents
+- **Complexity**: Very High
+- **Examples**: Enterprise workflows, complex business processes
+
+#### Pipeline
+- **Best for**: Sequential processing with specialized stages
+- **Team size**: 3-15 agents
+- **Complexity**: Medium
+- **Examples**: Data ETL pipelines, content processing workflows
+
+## Best Practices
+
+### System Design
+
+1. **Start Simple**: Begin with simpler patterns and evolve
+2. **Clear Responsibilities**: Define distinct roles for each agent
+3. **Robust Communication**: Design reliable message passing
+4. **Error Handling**: Plan for failures and recovery
+5. **Monitor Everything**: Implement comprehensive observability
+
+### Tool Design
+
+1. **Single Responsibility**: Each tool should have one clear purpose
+2. **Input Validation**: Validate all inputs thoroughly
+3. **Idempotency**: Design operations to be safely repeatable
+4. **Error Recovery**: Provide clear error messages and recovery paths
+5. **Documentation**: Include comprehensive usage examples
+
+### Performance Optimization
+
+1. **Measure First**: Use the evaluator to identify actual bottlenecks
+2. **Optimize Bottlenecks**: Focus on highest-impact improvements
+3. **Cache Strategically**: Cache expensive operations and results
+4. **Parallel Processing**: Identify opportunities for parallelization
+5. **Resource Management**: Monitor and optimize resource usage
+
+## Sample Files
+
+The `assets/` directory contains sample files to help you get started:
+
+- **`sample_system_requirements.json`**: Example system requirements for a research platform
+- **`sample_tool_descriptions.json`**: Example tool descriptions for common operations
+- **`sample_execution_logs.json`**: Example execution logs from a running system
+
+The `expected_outputs/` directory shows expected results from processing these samples.
+
+## References
+
+See the `references/` directory for detailed documentation:
+
+- **`agent_architecture_patterns.md`**: Comprehensive catalog of architecture patterns
+- **`tool_design_best_practices.md`**: Best practices for tool design and implementation
+- **`evaluation_methodology.md`**: Detailed methodology for system evaluation
+
+## Integration Examples
+
+### With OpenAI
+
+```python
+import json
+import openai
+
+# Load generated OpenAI schemas
+with open('my_tools_openai.json') as f:
+    schemas = json.load(f)
+
+# Use with OpenAI function calling
+response = openai.ChatCompletion.create(
+    model="gpt-4",
+    messages=[{"role": "user", "content": "Search for AI news"}],
+    functions=schemas['functions']
+)
+```
+
+### With Anthropic Claude
+
+```python
+import json
+import anthropic
+
+# Load generated Anthropic schemas
+with open('my_tools_anthropic.json') as f:
+    schemas = json.load(f)
+
+# Use with Anthropic tool use
+client = anthropic.Anthropic()
+response = client.messages.create(
+    model="claude-3-opus-20240229",
+    messages=[{"role": "user", "content": "Search for AI news"}],
+    tools=schemas['tools']
+)
+```
+
+## Troubleshooting
+
+### Common Issues
+
+**"No valid architecture pattern found"**
+- Check that team_size is reasonable (1-50)
+- Ensure tasks list is not empty
+- Verify performance_requirements are valid
+
+**"Tool schema validation failed"**
+- Check that all required fields are present
+- Ensure parameter types are valid
+- Verify enum values are provided as arrays
+
+**"Insufficient execution logs"**
+- Ensure logs contain required fields (task_id, agent_id, status)
+- Check that timestamps are in ISO 8601 format
+- Verify token usage fields are numeric
+
+### Performance Tips
+
+1. **Large Systems**: For systems with >20 agents, consider breaking into subsystems
+2. **Complex Tools**: Tools with >10 parameters may need simplification
+3. **Log Volume**: For >1000 log entries, consider sampling for faster analysis
+
+## Contributing
+
+This skill is part of the claude-skills repository. To contribute:
+
+1. Fork the repository
+2. Create a feature branch
+3. Make your changes
+4. Add tests and documentation
+5. Submit a pull request
+
+## License
+
+This project is licensed under the MIT License - see the main repository for details.
+
+## Support
+
+For issues and questions:
+- Check the troubleshooting section above
+- Review the reference documentation in `references/`
+- Create an issue in the claude-skills repository

package/engineering/skills/agent-designer/SKILL.md

@@ -0,0 +1,279 @@
+---
+name: "agent-designer"
+description: "Use when the user asks to design multi-agent systems, create agent architectures, define agent communication patterns, or build autonomous agent workflows."
+---
+
+# Agent Designer - Multi-Agent System Architecture
+
+**Tier:** POWERFUL  
+**Category:** Engineering  
+**Tags:** AI agents, architecture, system design, orchestration, multi-agent systems
+
+## Overview
+
+Agent Designer is a comprehensive toolkit for designing, architecting, and evaluating multi-agent systems. It provides structured approaches to agent architecture patterns, tool design principles, communication strategies, and performance evaluation frameworks for building robust, scalable AI agent systems.
+
+## Core Capabilities
+
+### 1. Agent Architecture Patterns
+
+#### Single Agent Pattern
+- **Use Case:** Simple, focused tasks with clear boundaries
+- **Pros:** Minimal complexity, easy debugging, predictable behavior
+- **Cons:** Limited scalability, single point of failure
+- **Implementation:** Direct user-agent interaction with comprehensive tool access
+
+#### Supervisor Pattern
+- **Use Case:** Hierarchical task decomposition with centralized control
+- **Architecture:** One supervisor agent coordinating multiple specialist agents
+- **Pros:** Clear command structure, centralized decision making
+- **Cons:** Supervisor bottleneck, complex coordination logic
+- **Implementation:** Supervisor receives tasks, delegates to specialists, aggregates results
+
+#### Swarm Pattern
+- **Use Case:** Distributed problem solving with peer-to-peer collaboration
+- **Architecture:** Multiple autonomous agents with shared objectives
+- **Pros:** High parallelism, fault tolerance, emergent intelligence
+- **Cons:** Complex coordination, potential conflicts, harder to predict
+- **Implementation:** Agent discovery, consensus mechanisms, distributed task allocation
+
+#### Hierarchical Pattern
+- **Use Case:** Complex systems with multiple organizational layers
+- **Architecture:** Tree structure with managers and workers at different levels
+- **Pros:** Natural organizational mapping, clear responsibilities
+- **Cons:** Communication overhead, potential bottlenecks at each level
+- **Implementation:** Multi-level delegation with feedback loops
+
+#### Pipeline Pattern
+- **Use Case:** Sequential processing with specialized stages
+- **Architecture:** Agents arranged in processing pipeline
+- **Pros:** Clear data flow, specialized optimization per stage
+- **Cons:** Sequential bottlenecks, rigid processing order
+- **Implementation:** Message queues between stages, state handoffs
+
+### 2. Agent Role Definition
+
+#### Role Specification Framework
+- **Identity:** Name, purpose statement, core competencies
+- **Responsibilities:** Primary tasks, decision boundaries, success criteria
+- **Capabilities:** Required tools, knowledge domains, processing limits
+- **Interfaces:** Input/output formats, communication protocols
+- **Constraints:** Security boundaries, resource limits, operational guidelines
+
+#### Common Agent Archetypes
+
+**Coordinator Agent**
+- Orchestrates multi-agent workflows
+- Makes high-level decisions and resource allocation
+- Monitors system health and performance
+- Handles escalations and conflict resolution
+
+**Specialist Agent**
+- Deep expertise in specific domain (code, data, research)
+- Optimized tools and knowledge for specialized tasks
+- High-quality output within narrow scope
+- Clear handoff protocols for out-of-scope requests
+
+**Interface Agent**
+- Handles external interactions (users, APIs, systems)
+- Protocol translation and format conversion
+- Authentication and authorization management
+- User experience optimization
+
+**Monitor Agent**
+- System health monitoring and alerting
+- Performance metrics collection and analysis
+- Anomaly detection and reporting
+- Compliance and audit trail maintenance
+
+### 3. Tool Design Principles
+
+#### Schema Design
+- **Input Validation:** Strong typing, required vs optional parameters
+- **Output Consistency:** Standardized response formats, error handling
+- **Documentation:** Clear descriptions, usage examples, edge cases
+- **Versioning:** Backward compatibility, migration paths
+
+#### Error Handling Patterns
+- **Graceful Degradation:** Partial functionality when dependencies fail
+- **Retry Logic:** Exponential backoff, circuit breakers, max attempts
+- **Error Propagation:** Structured error responses, error classification
+- **Recovery Strategies:** Fallback methods, alternative approaches
+
+#### Idempotency Requirements
+- **Safe Operations:** Read operations with no side effects
+- **Idempotent Writes:** Same operation can be safely repeated
+- **State Management:** Version tracking, conflict resolution
+- **Atomicity:** All-or-nothing operation completion
+
+### 4. Communication Patterns
+
+#### Message Passing
+- **Asynchronous Messaging:** Decoupled agents, message queues
+- **Message Format:** Structured payloads with metadata
+- **Delivery Guarantees:** At-least-once, exactly-once semantics
+- **Routing:** Direct messaging, publish-subscribe, broadcast
+
+#### Shared State
+- **State Stores:** Centralized data repositories
+- **Consistency Models:** Strong, eventual, weak consistency
+- **Access Patterns:** Read-heavy, write-heavy, mixed workloads
+- **Conflict Resolution:** Last-writer-wins, merge strategies
+
+#### Event-Driven Architecture
+- **Event Sourcing:** Immutable event logs, state reconstruction
+- **Event Types:** Domain events, system events, integration events
+- **Event Processing:** Real-time, batch, stream processing
+- **Event Schema:** Versioned event formats, backward compatibility
+
+### 5. Guardrails and Safety
+
+#### Input Validation
+- **Schema Enforcement:** Required fields, type checking, format validation
+- **Content Filtering:** Harmful content detection, PII scrubbing
+- **Rate Limiting:** Request throttling, resource quotas
+- **Authentication:** Identity verification, authorization checks
+
+#### Output Filtering
+- **Content Moderation:** Harmful content removal, quality checks
+- **Consistency Validation:** Logic checks, constraint verification
+- **Formatting:** Standardized output formats, clean presentation
+- **Audit Logging:** Decision trails, compliance records
+
+#### Human-in-the-Loop
+- **Approval Workflows:** Critical decision checkpoints
+- **Escalation Triggers:** Confidence thresholds, risk assessment
+- **Override Mechanisms:** Human judgment precedence
+- **Feedback Loops:** Human corrections improve system behavior
+
+### 6. Evaluation Frameworks
+
+#### Task Completion Metrics
+- **Success Rate:** Percentage of tasks completed successfully
+- **Partial Completion:** Progress measurement for complex tasks
+- **Task Classification:** Success criteria by task type
+- **Failure Analysis:** Root cause identification and categorization
+
+#### Quality Assessment
+- **Output Quality:** Accuracy, relevance, completeness measures
+- **Consistency:** Response variability across similar inputs
+- **Coherence:** Logical flow and internal consistency
+- **User Satisfaction:** Feedback scores, usage patterns
+
+#### Cost Analysis
+- **Token Usage:** Input/output token consumption per task
+- **API Costs:** External service usage and charges
+- **Compute Resources:** CPU, memory, storage utilization
+- **Time-to-Value:** Cost per successful task completion
+
+#### Latency Distribution
+- **Response Time:** End-to-end task completion time
+- **Processing Stages:** Bottleneck identification per stage
+- **Queue Times:** Wait times in processing pipelines
+- **Resource Contention:** Impact of concurrent operations
+
+### 7. Orchestration Strategies
+
+#### Centralized Orchestration
+- **Workflow Engine:** Central coordinator manages all agents
+- **State Management:** Centralized workflow state tracking
+- **Decision Logic:** Complex routing and branching rules
+- **Monitoring:** Comprehensive visibility into all operations
+
+#### Decentralized Orchestration
+- **Peer-to-Peer:** Agents coordinate directly with each other
+- **Service Discovery:** Dynamic agent registration and lookup
+- **Consensus Protocols:** Distributed decision making
+- **Fault Tolerance:** No single point of failure
+
+#### Hybrid Approaches
+- **Domain Boundaries:** Centralized within domains, federated across
+- **Hierarchical Coordination:** Multiple orchestration levels
+- **Context-Dependent:** Strategy selection based on task type
+- **Load Balancing:** Distribute coordination responsibility
+
+### 8. Memory Patterns
+
+#### Short-Term Memory
+- **Context Windows:** Working memory for current tasks
+- **Session State:** Temporary data for ongoing interactions
+- **Cache Management:** Performance optimization strategies
+- **Memory Pressure:** Handling capacity constraints
+
+#### Long-Term Memory
+- **Persistent Storage:** Durable data across sessions
+- **Knowledge Base:** Accumulated domain knowledge
+- **Experience Replay:** Learning from past interactions
+- **Memory Consolidation:** Transferring from short to long-term
+
+#### Shared Memory
+- **Collaborative Knowledge:** Shared learning across agents
+- **Synchronization:** Consistency maintenance strategies
+- **Access Control:** Permission-based memory access
+- **Memory Partitioning:** Isolation between agent groups
+
+### 9. Scaling Considerations
+
+#### Horizontal Scaling
+- **Agent Replication:** Multiple instances of same agent type
+- **Load Distribution:** Request routing across agent instances
+- **Resource Pooling:** Shared compute and storage resources
+- **Geographic Distribution:** Multi-region deployments
+
+#### Vertical Scaling
+- **Capability Enhancement:** More powerful individual agents
+- **Tool Expansion:** Broader tool access per agent
+- **Context Expansion:** Larger working memory capacity
+- **Processing Power:** Higher throughput per agent
+
+#### Performance Optimization
+- **Caching Strategies:** Response caching, tool result caching
+- **Parallel Processing:** Concurrent task execution
+- **Resource Optimization:** Efficient resource utilization
+- **Bottleneck Elimination:** Systematic performance tuning
+
+### 10. Failure Handling
+
+#### Retry Mechanisms
+- **Exponential Backoff:** Increasing delays between retries
+- **Jitter:** Random delay variation to prevent thundering herd
+- **Maximum Attempts:** Bounded retry behavior
+- **Retry Conditions:** Transient vs permanent failure classification
+
+#### Fallback Strategies
+- **Graceful Degradation:** Reduced functionality when systems fail
+- **Alternative Approaches:** Different methods for same goals
+- **Default Responses:** Safe fallback behaviors
+- **User Communication:** Clear failure messaging
+
+#### Circuit Breakers
+- **Failure Detection:** Monitoring failure rates and response times
+- **State Management:** Open, closed, half-open circuit states
+- **Recovery Testing:** Gradual return to normal operation
+- **Cascading Failure Prevention:** Protecting upstream systems
+
+## Implementation Guidelines
+
+### Architecture Decision Process
+1. **Requirements Analysis:** Understand system goals, constraints, scale
+2. **Pattern Selection:** Choose appropriate architecture pattern
+3. **Agent Design:** Define roles, responsibilities, interfaces
+4. **Tool Architecture:** Design tool schemas and error handling
+5. **Communication Design:** Select message patterns and protocols
+6. **Safety Implementation:** Build guardrails and validation
+7. **Evaluation Planning:** Define success metrics and monitoring
+8. **Deployment Strategy:** Plan scaling and failure handling
+
+### Quality Assurance
+- **Testing Strategy:** Unit, integration, and system testing approaches
+- **Monitoring:** Real-time system health and performance tracking
+- **Documentation:** Architecture documentation and runbooks
+- **Security Review:** Threat modeling and security assessments
+
+### Continuous Improvement
+- **Performance Monitoring:** Ongoing system performance analysis
+- **User Feedback:** Incorporating user experience improvements
+- **A/B Testing:** Controlled experiments for system improvements
+- **Knowledge Base Updates:** Continuous learning and adaptation
+
+This skill provides the foundation for designing robust, scalable multi-agent systems that can handle complex tasks while maintaining safety, reliability, and performance at scale.