wicked-garden 12.27.0

package/skills/agentic/context-engineering/refs/cost-optimization-reporting.md

@@ -0,0 +1,300 @@
+---
+phase_relevance: ["design", "review"]
+archetype_relevance: ["*"]
+---
+# Cost Optimization & Reporting
+
+Strategies for estimating, optimizing, and reporting on token costs in agentic systems.
+
+## Cost Estimation
+
+### Estimating Token Count
+
+```python
+class TokenEstimator:
+    """Estimate tokens before making API call."""
+
+    @staticmethod
+    def estimate_tokens(text):
+        """Rough estimate: ~4 characters per token for English."""
+        return len(text) // 4
+
+    @staticmethod
+    def estimate_cost(model, prompt, estimated_output_tokens=500):
+        """Estimate cost before API call."""
+        input_tokens = TokenEstimator.estimate_tokens(prompt)
+        output_tokens = estimated_output_tokens
+
+        calc = CostCalculator()
+        return calc.calculate(model, input_tokens, output_tokens)
+
+# More accurate estimation using tiktoken (for OpenAI models)
+import tiktoken
+
+class AccurateTokenEstimator:
+    def __init__(self, model):
+        self.encoding = tiktoken.encoding_for_model(model)
+
+    def count_tokens(self, text):
+        return len(self.encoding.encode(text))
+
+    def estimate_cost(self, model, prompt, estimated_output_tokens=500):
+        input_tokens = self.count_tokens(prompt)
+        calc = CostCalculator()
+        return calc.calculate(model, input_tokens, estimated_output_tokens)
+```
+
+### Predictive Cost Modeling
+
+```python
+class CostPredictor:
+    """Predict costs based on historical data."""
+
+    def __init__(self):
+        self.history = []
+
+    def record_actual(self, task_type, model, actual_tokens, actual_cost):
+        self.history.append({
+            'task_type': task_type,
+            'model': model,
+            'tokens': actual_tokens,
+            'cost': actual_cost
+        })
+
+    def predict_cost(self, task_type, model):
+        """Predict cost for task based on historical similar tasks."""
+        similar_tasks = [
+            h for h in self.history
+            if h['task_type'] == task_type and h['model'] == model
+        ]
+
+        if not similar_tasks:
+            # No history - use conservative estimate
+            return {
+                'estimated_tokens': 5000,
+                'estimated_cost': CostCalculator().calculate(model, 5000, 1000)['total_cost']
+            }
+
+        # Use average of similar tasks
+        avg_tokens = np.mean([t['tokens'] for t in similar_tasks])
+        avg_cost = np.mean([t['cost'] for t in similar_tasks])
+
+        # Add 20% buffer for safety
+        return {
+            'estimated_tokens': int(avg_tokens * 1.2),
+            'estimated_cost': avg_cost * 1.2
+        }
+```
+
+## Cost Optimization Strategies
+
+### Model Selection for Cost
+
+```python
+class CostOptimizer:
+    def select_model_for_task(self, task, quality_threshold=0.8, max_cost=0.10):
+        """Select cheapest model that meets quality threshold."""
+
+        # Models ordered by cost (cheapest first)
+        models_by_cost = [
+            'claude-haiku-4',
+            'gpt-3.5-turbo',
+            'claude-sonnet-4-5',
+            'gpt-4-turbo',
+            'claude-opus-4-6',
+            'gpt-4'
+        ]
+
+        for model in models_by_cost:
+            # Estimate cost for this model
+            est_cost = self.estimate_task_cost(task, model)
+
+            if est_cost > max_cost:
+                continue
+
+            # Check if quality is sufficient (would need ML model for real impl)
+            expected_quality = self.predict_quality(task, model)
+
+            if expected_quality >= quality_threshold:
+                return {
+                    'model': model,
+                    'estimated_cost': est_cost,
+                    'expected_quality': expected_quality
+                }
+
+        raise NoSuitableModelError("No model meets quality/cost requirements")
+```
+
+### Batch Processing for Cost Savings
+
+```python
+class BatchCostOptimizer:
+    """Process multiple items in batches to save on API overhead."""
+
+    async def process_items(self, items, model):
+        # Processing individually
+        individual_cost = 0
+        for item in items:
+            cost = await self.process_one(item, model)
+            individual_cost += cost
+
+        # Processing as batch
+        batch_cost = await self.process_batch(items, model)
+
+        savings = individual_cost - batch_cost
+        savings_pct = (savings / individual_cost) * 100
+
+        print(f"Batch processing saved ${savings:.4f} ({savings_pct:.1f}%)")
+
+        return batch_cost
+
+    async def process_batch(self, items, model):
+        """Process all items in one API call."""
+        batch_prompt = "Process each item:\n\n"
+        for i, item in enumerate(items):
+            batch_prompt += f"Item {i+1}: {item}\n"
+
+        input_tokens = self.count_tokens(batch_prompt)
+        # Single output with all results
+        output_tokens = len(items) * 100  # Estimate
+
+        calc = CostCalculator()
+        return calc.calculate(model, input_tokens, output_tokens)['total_cost']
+```
+
+### Caching for Cost Reduction
+
+```python
+class CostAwareCache:
+    """Cache expensive LLM calls."""
+
+    def __init__(self):
+        self.cache = {}
+        self.hits = 0
+        self.misses = 0
+        self.savings = 0.0
+
+    async def get_or_generate(self, prompt, model, cost_calculator):
+        cache_key = hashlib.md5(f"{prompt}{model}".encode()).hexdigest()
+
+        if cache_key in self.cache:
+            # Cache hit
+            self.hits += 1
+
+            # Calculate cost saved
+            estimated_cost = cost_calculator.estimate(prompt, model)
+            self.savings += estimated_cost
+
+            return self.cache[cache_key]['response']
+
+        # Cache miss
+        self.misses += 1
+        response, actual_cost = await self.generate_with_cost(prompt, model)
+
+        # Store in cache
+        self.cache[cache_key] = {
+            'response': response,
+            'cost': actual_cost,
+            'timestamp': datetime.now()
+        }
+
+        return response
+
+    def get_cache_stats(self):
+        total = self.hits + self.misses
+        hit_rate = (self.hits / total * 100) if total > 0 else 0
+
+        return {
+            'hits': self.hits,
+            'misses': self.misses,
+            'hit_rate': hit_rate,
+            'savings': self.savings
+        }
+```
+
+## Cost Reporting
+
+### Comprehensive Cost Report
+
+```python
+class CostReporter:
+    def __init__(self, tracker):
+        self.tracker = tracker
+
+    def generate_report(self, period='daily'):
+        stats = self.tracker.get_stats()
+
+        report = f"""
+Cost Report - {period}
+{'='*50}
+
+Summary:
+- Total Cost: ${stats['total_cost']:.4f}
+- Total Calls: {stats['total_calls']}
+- Total Tokens: {stats['total_tokens']:,}
+- Avg Cost/Call: ${stats['avg_cost_per_call']:.4f}
+- Avg Tokens/Call: {stats['avg_tokens_per_call']:.0f}
+
+By Agent:
+"""
+        for agent_id, cost in stats['by_agent'].items():
+            pct = (cost / stats['total_cost']) * 100
+            report += f"  {agent_id}: ${cost:.4f} ({pct:.1f}%)\n"
+
+        report += "\nBy Model:\n"
+        for model, cost in stats['by_model'].items():
+            pct = (cost / stats['total_cost']) * 100
+            report += f"  {model}: ${cost:.4f} ({pct:.1f}%)\n"
+
+        return report
+
+    def identify_optimization_opportunities(self):
+        """Find cost optimization opportunities."""
+        stats = self.tracker.get_stats()
+        opportunities = []
+
+        # Check for expensive agents
+        for agent_id, cost in stats['by_agent'].items():
+            if cost > stats['total_cost'] * 0.3:  # Agent using >30% of budget
+                opportunities.append({
+                    'type': 'expensive_agent',
+                    'agent': agent_id,
+                    'cost': cost,
+                    'recommendation': f"Optimize {agent_id} - using {(cost/stats['total_cost']*100):.1f}% of budget"
+                })
+
+        # Check for expensive models
+        for model, cost in stats['by_model'].items():
+            if model in ['claude-opus-4-6', 'gpt-4']:
+                # Check if cheaper model could work
+                opportunities.append({
+                    'type': 'expensive_model',
+                    'model': model,
+                    'cost': cost,
+                    'recommendation': f"Consider using cheaper model instead of {model}"
+                })
+
+        return opportunities
+```
+
+## Real-World Cost Examples
+
+**Code Review System** (100 files/day, ~10K tokens/file):
+- Claude Opus: ~$18.75/day → ~$562/month
+- Claude Sonnet: ~$3.75/day → ~$112/month — **~$450/month savings**
+
+**Customer Support Bot** (1000 conversations/day × 10 messages × 100 tokens):
+- Claude Haiku: ~$1.60/day → ~$48/month → ~$584/year
+
+## Best Practices
+
+1. **Track Everything**: Record all LLM calls with model, tokens, cost
+2. **Set Budgets**: Per session, daily, monthly limits
+3. **Estimate First**: Predict cost before expensive calls
+4. **Cache Aggressively**: Avoid redundant API calls
+5. **Batch When Possible**: Combine multiple items per call
+6. **Right-Size Models**: Use cheapest model that meets quality bar
+7. **Monitor Trends**: Watch for cost increases over time
+8. **Optimize Hot Paths**: Focus on highest-cost agents/operations
+9. **User Limits**: Prevent abuse with per-user budgets
+10. **Alert on Spikes**: Get notified of unusual cost patterns

package/skills/agentic/context-engineering/refs/selective-loading.md

@@ -0,0 +1,179 @@
+---
+phase_relevance: ["design", "review"]
+archetype_relevance: ["*"]
+---
+# Selective Loading & Token-Aware Prompting
+
+Strategies for loading only relevant context and adapting prompts to available token budgets.
+
+## Selective Loading Strategies
+
+### 5. Relevance-Based Filtering
+
+Load only context relevant to current task.
+
+```python
+class RelevanceFilter:
+    def __init__(self, vector_db):
+        self.vector_db = vector_db
+
+    async def filter_context(self, all_context, current_task, top_k=5):
+        # Embed current task
+        task_embedding = await self.embed(current_task)
+
+        # Score all context items by relevance
+        scored_items = []
+        for item in all_context:
+            item_embedding = await self.embed(item)
+            similarity = cosine_similarity([task_embedding], [item_embedding])[0][0]
+            scored_items.append((item, similarity))
+
+        # Sort by relevance and take top K
+        scored_items.sort(key=lambda x: x[1], reverse=True)
+        relevant_items = [item for item, score in scored_items[:top_k]]
+
+        return relevant_items
+
+# Usage
+filter = RelevanceFilter(vector_db)
+relevant = await filter.filter_context(
+    all_context=user.full_history,
+    current_task="Write a Python function",
+    top_k=3
+)
+```
+
+### 6. Time-Based Decay
+
+Weight recent context more heavily.
+
+```python
+from datetime import datetime, timedelta
+
+class TimeDecayFilter:
+    def score_by_recency(self, items, decay_factor=0.5):
+        """Score items with exponential time decay."""
+        now = datetime.now()
+        scored_items = []
+
+        for item in items:
+            age = (now - item.timestamp).total_seconds() / 3600  # hours
+            decay = decay_factor ** (age / 24)  # Half-life of 24 hours
+            scored_items.append((item, decay))
+
+        return sorted(scored_items, key=lambda x: x[1], reverse=True)
+
+# Combine with relevance
+class HybridFilter:
+    def score(self, item, task_embedding):
+        # Relevance score
+        relevance = cosine_similarity(
+            [task_embedding],
+            [item.embedding]
+        )[0][0]
+
+        # Recency score
+        age_hours = (datetime.now() - item.timestamp).total_seconds() / 3600
+        recency = 0.5 ** (age_hours / 24)
+
+        # Weighted combination
+        return 0.7 * relevance + 0.3 * recency
+```
+
+### 7. Query-Specific Pruning
+
+Remove context not needed for specific query type.
+
+```python
+class QuerySpecificPruner:
+    def prune(self, context, query_type):
+        if query_type == 'factual':
+            # Keep facts, remove opinions/summaries
+            return [c for c in context if c.type == 'fact']
+
+        elif query_type == 'code_generation':
+            # Keep code examples, remove general discussion
+            return [c for c in context if c.type in ['code', 'api_doc']]
+
+        elif query_type == 'decision':
+            # Keep past decisions and outcomes
+            return [c for c in context if c.type in ['decision', 'outcome']]
+
+        else:
+            # Default: keep all
+            return context
+```
+
+## Token-Aware Prompting
+
+### 8. Dynamic Prompt Sizing
+
+Adjust prompt detail based on available tokens.
+
+```python
+class DynamicPrompt:
+    def __init__(self, max_tokens):
+        self.max_tokens = max_tokens
+
+    def build_prompt(self, task, available_tokens):
+        # Base prompt (always include)
+        base = f"Task: {task}\n\n"
+        base_tokens = self.count_tokens(base)
+
+        remaining = available_tokens - base_tokens - 1000  # Reserve for output
+
+        if remaining > 5000:
+            # Verbose mode
+            return base + self.get_detailed_instructions()
+
+        elif remaining > 2000:
+            # Standard mode
+            return base + self.get_standard_instructions()
+
+        else:
+            # Minimal mode
+            return base + self.get_minimal_instructions()
+
+    def get_detailed_instructions(self):
+        return """Detailed instructions with examples:
+        1. First step with example...
+        2. Second step with example...
+        ...
+        """
+
+    def get_standard_instructions(self):
+        return """Standard instructions:
+        1. First step
+        2. Second step
+        ...
+        """
+
+    def get_minimal_instructions(self):
+        return "Follow standard procedure."
+```
+
+### 9. Token-Budgeted Retrieval
+
+Retrieve as much as token budget allows.
+
+```python
+class BudgetedRetriever:
+    async def retrieve(self, query, token_budget):
+        # Get candidates
+        candidates = await self.vector_db.search(query, limit=100)
+
+        # Add candidates until budget exhausted
+        selected = []
+        tokens_used = 0
+
+        for candidate in candidates:
+            candidate_tokens = self.count_tokens(candidate.content)
+
+            if tokens_used + candidate_tokens <= token_budget:
+                selected.append(candidate)
+                tokens_used += candidate_tokens
+            else:
+                break
+
+        return selected, tokens_used
+```

package/skills/agentic/frameworks/SKILL.md

@@ -0,0 +1,191 @@
+---
+name: frameworks
+description: |
+  Use when selecting or comparing agentic frameworks (LangChain, LangGraph, CrewAI, AutoGen, etc.) —
+  curated comparison by use case, language, and maturity. Gets latest context via Context7 when available.
+  NOT for reviewing existing agentic code (use review-methodology) or architecture patterns (use agentic-patterns).
+portability: portable
+phase_relevance: ["design", "review"]
+archetype_relevance: ["*"]
+---
+
+# Agentic Frameworks
+
+Comprehensive guide to agentic frameworks, their strengths, and how to choose the right one.
+
+## Quick Comparison Table
+
+| Framework | Language | Best For | Maturity | Learning Curve |
+|-----------|----------|----------|----------|----------------|
+| Anthropic ADK | TypeScript | Claude-specific, production | High | Low |
+| LangGraph | Python | Complex workflows, state | High | Medium |
+| CrewAI | Python | Role-based teams | Medium | Low |
+| AutoGen | Python | Multi-agent conversations | Medium | Medium |
+| Pydantic AI | Python | Type-safe agents | Medium | Low |
+| OpenAI Agents SDK | Python | OpenAI-specific | Low | Low |
+| LlamaIndex Agents | Python | RAG-heavy applications | High | Medium |
+| Haystack | Python | Production pipelines | High | Medium |
+| Semantic Kernel | C#/Python | Microsoft ecosystem | Medium | Medium |
+| LangChain | Python | Rapid prototyping | High | Medium-High |
+| Agency Swarm | Python | OpenAI Assistants API | Low | Low |
+| Dify | Low-code | No-code workflows | Medium | Very Low |
+
+## Selection Criteria
+
+### 1. Orchestration Capabilities
+
+**Simple Sequential:**
+- LangChain (chains)
+- Haystack (pipelines)
+- Pydantic AI
+
+**Complex Workflows:**
+- LangGraph (state machines)
+- ADK (delegated workflows)
+- AutoGen (conversation patterns)
+
+**Team-Based:**
+- CrewAI (role-based)
+- Agency Swarm (org structure)
+
+### 2. State Management
+
+**No Built-in State:** LangChain, Pydantic AI
+
+**Checkpointed State:** LangGraph (built-in), ADK (context preservation)
+
+**Distributed State:** Custom implementation needed for all
+
+### 3. Tool Integration
+
+**Extensive Tool Libraries:** LangChain (largest ecosystem), LlamaIndex (RAG-focused), Haystack (production tools)
+
+**Easy Tool Definition:** Pydantic AI (type-safe), ADK (TypeScript decorators), OpenAI Agents SDK (function calling)
+
+**Custom Tools:** All frameworks support custom tools
+
+### 4. Error Handling
+
+**Built-in Retry/Fallback:** ADK (comprehensive), LangGraph (error handling nodes), Haystack (pipeline error handling)
+
+**Manual Error Handling:** CrewAI, AutoGen, Pydantic AI
+
+### 5. Observability
+
+**Native Tracing:** LangSmith (for LangChain/LangGraph), Braintrust (for ADK)
+
+**Third-Party Integration:** All support OpenTelemetry, most support LangFuse, Arize
+
+## Framework Profiles
+
+See `refs/framework-profiles-1.md` (ADK, LangGraph, CrewAI) and `refs/framework-profiles-2.md` (AutoGen, Pydantic AI, LlamaIndex) for detailed profiles.
+
+### Anthropic Agent Developer Kit (ADK)
+**Best for:** Production Claude applications
+**Strengths:** TypeScript with type safety, built-in context management, comprehensive error handling, delegated workflows
+**Weaknesses:** Claude-only, TypeScript/Node only, smaller community
+**When to choose:** Building on Claude exclusively, TypeScript/Node stack, need production-ready patterns
+
+### LangGraph
+**Best for:** Complex stateful workflows
+**Strengths:** State machine abstraction, built-in checkpointing, human-in-the-loop support, time-travel debugging
+**Weaknesses:** Steeper learning curve, can be overkill, more boilerplate
+**When to choose:** Complex workflows with branches/loops, need state persistence, want human approval gates
+
+### CrewAI
+**Best for:** Role-based agent teams
+**Strengths:** Intuitive role/task abstraction, simple API, good for hierarchical teams
+**Weaknesses:** Less mature, limited state management, fewer production features
+**When to choose:** Team-based workflows, quick prototyping, straightforward delegation
+
+### AutoGen
+**Best for:** Multi-agent conversations
+**Strengths:** Flexible conversation patterns, group chat capabilities, human-in-the-loop
+**Weaknesses:** Can be verbose, conversation management complexity
+**When to choose:** Agents need to debate/collaborate, conversational workflows
+
+### Pydantic AI
+**Best for:** Type-safe Python agents
+**Strengths:** Type safety via Pydantic, simple clean API, dependency injection, multi-provider
+**Weaknesses:** New/less mature, smaller ecosystem, limited orchestration patterns
+**When to choose:** Want type safety, simple agent use cases, already using Pydantic
+
+### LlamaIndex Agents
+**Best for:** RAG-heavy applications
+**Strengths:** Excellent retrieval capabilities, query planning, tool use with data
+**Weaknesses:** Best for RAG use cases, heavier framework
+**When to choose:** Heavy RAG requirements, complex data retrieval, query planning needs
+
+## Decision Tree
+
+```
+Start: What's your primary use case?
+
+├─ Complex stateful workflow with branches/loops
+│  └─ Use: LangGraph
+
+├─ Role-based team of agents
+│  └─ Use: CrewAI or ADK
+
+├─ RAG-heavy application
+│  └─ Use: LlamaIndex Agents
+
+├─ Multi-agent conversations/debates
+│  └─ Use: AutoGen
+
+├─ Simple sequential workflow
+│  ├─ TypeScript?
+│  │  └─ Use: ADK
+│  └─ Python?
+│     └─ Use: Pydantic AI or LangChain
+
+├─ Production pipeline
+│  └─ Use: Haystack or ADK
+
+└─ Maximum flexibility
+   └─ Build from scratch or use LangGraph
+```
+
+## Language Considerations
+
+**Python Frameworks:** LangChain, LangGraph, CrewAI, AutoGen, Pydantic AI, LlamaIndex
+- Largest ecosystem, most tutorials/examples, best for data science/ML integration
+
+**TypeScript Frameworks:** Anthropic ADK
+- Better type safety, Node.js ecosystem, good for web applications
+
+**C# Frameworks:** Semantic Kernel
+- Microsoft ecosystem, .NET integration
+
+## Multi-Provider vs Single-Provider
+
+**Multi-Provider (LLM-agnostic):** LangChain, LangGraph, CrewAI, AutoGen, Pydantic AI
+- Can switch between OpenAI, Anthropic, etc.
+- More flexibility but may not leverage provider-specific features
+
+**Single-Provider (Optimized):** ADK (Claude), OpenAI Agents SDK (OpenAI)
+- Better integration with specific provider
+- Access to provider-specific features but less flexibility
+
+## Production Readiness
+
+**Most Production-Ready:** Anthropic ADK, LangGraph, Haystack, LlamaIndex
+
+**Good for Production:** CrewAI, LangChain, AutoGen
+
+**Early/Experimental:** Pydantic AI, OpenAI Agents SDK, Agency Swarm
+
+## When NOT to Use a Framework
+
+Build from scratch if: very simple use case, specific requirements unmet, want maximum control, or learning exercise. Framework overhead not worth it for single LLM calls, static prompts, or no agent behavior.
+
+## Quick Recommendations
+
+**Just getting started:** CrewAI or Pydantic AI | **State management:** LangGraph | **TypeScript:** ADK | **RAG:** LlamaIndex | **Team-based:** CrewAI or ADK | **Max flexibility:** LangGraph | **Production Claude:** ADK
+
+## References
+
+- `refs/framework-profiles-1.md` - ADK, LangGraph, CrewAI detailed profiles
+- `refs/framework-profiles-2.md` - AutoGen, Pydantic AI, LlamaIndex profiles + comparison matrix
+- `refs/migration-patterns-paths.md` - Common migration paths between frameworks
+- `refs/migration-patterns-testing.md` - Migration testing, rollback, and effort estimates