cc-mirror 1.0.3 → 1.1.0

package/dist/skills/multi-agent-orchestrator/references/domains/data-analysis.md

@@ -0,0 +1,347 @@
+# Data Analysis Orchestration Patterns
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                                                             │
+│   Data yields insights faster when explored in parallel.    │
+│   Multiple dimensions, simultaneous analysis, clear story.  │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
+> **Load when**: Exploratory data analysis, data quality, report generation, ETL pipelines, statistical analysis
+> **Common patterns**: Multi-Dimensional Exploration, Comprehensive Quality Audit, Hypothesis Testing
+
+## Table of Contents
+
+1. [Exploratory Data Analysis](#exploratory-data-analysis)
+2. [Data Quality](#data-quality)
+3. [Report Generation](#report-generation)
+4. [ETL Pipelines](#etl-pipelines)
+5. [Statistical Analysis](#statistical-analysis)
+
+---
+
+## Exploratory Data Analysis
+
+### Pattern: Multi-Dimensional Exploration
+
+```
+User Request: "Analyze this dataset"
+
+Phase 1: FAN-OUT (Parallel initial exploration)
+├─ Agent A: Schema analysis (columns, types, constraints)
+├─ Agent B: Statistical summary (distributions, outliers)
+├─ Agent C: Missing data analysis
+├─ Agent D: Cardinality and uniqueness analysis
+└─ Agent E: Sample data examination
+
+Phase 2: REDUCE
+└─ General-purpose agent: Synthesize initial findings
+
+Phase 3: FAN-OUT (Deep dive based on findings)
+├─ Agent A: Correlation analysis
+├─ Agent B: Time series patterns (if applicable)
+└─ Agent C: Categorical relationship analysis
+
+Phase 4: REDUCE
+└─ General-purpose agent: Complete EDA report
+```
+
+### Pattern: Question-Driven Analysis
+
+```
+User Request: "Why are sales declining?"
+
+Phase 1: EXPLORE
+└─ Explore agent: Understand available data sources
+
+Phase 2: FAN-OUT (Parallel hypothesis investigation)
+├─ Agent A: Analyze sales by region
+├─ Agent B: Analyze sales by product
+├─ Agent C: Analyze sales by customer segment
+├─ Agent D: Analyze external factors (seasonality, competition)
+└─ Agent E: Analyze marketing/promotion effectiveness
+
+Phase 3: REDUCE
+└─ General-purpose agent: Identify key drivers, recommendations
+```
+
+### Pattern: Comparative Analysis
+
+```
+Phase 1: FAN-OUT
+├─ Agent A: Analyze dataset A characteristics
+├─ Agent B: Analyze dataset B characteristics
+└─ Agent C: Analyze overlap/differences
+
+Phase 2: REDUCE
+└─ General-purpose agent: Comparison report with insights
+```
+
+---
+
+## Data Quality
+
+### Pattern: Comprehensive Quality Audit
+
+```
+User Request: "Check data quality for the customer table"
+
+Phase 1: FAN-OUT (Parallel quality dimensions)
+├─ Agent A: Completeness (null rates, missing values)
+├─ Agent B: Accuracy (format validation, range checks)
+├─ Agent C: Consistency (cross-field validation)
+├─ Agent D: Timeliness (freshness, update patterns)
+├─ Agent E: Uniqueness (duplicates, key integrity)
+└─ Agent F: Validity (business rule compliance)
+
+Phase 2: REDUCE
+└─ General-purpose agent: Quality scorecard with issues
+
+Phase 3: FAN-OUT (Remediation)
+├─ Agent A: Fix completeness issues
+├─ Agent B: Fix accuracy issues
+└─ Agent C: Fix consistency issues
+```
+
+### Pattern: Anomaly Detection
+
+```
+Phase 1: FAN-OUT
+├─ Agent A: Statistical outlier detection
+├─ Agent B: Business rule violations
+├─ Agent C: Pattern anomalies (sudden changes)
+└─ Agent D: Referential integrity issues
+
+Phase 2: REDUCE
+└─ General-purpose agent: Anomaly report with severity
+```
+
+### Pattern: Data Profiling Pipeline
+
+```
+Phase 1: PIPELINE
+├─ General-purpose agent: Extract profiling metrics
+├─ General-purpose agent: Compare against historical baseline
+└─ General-purpose agent: Flag deviations
+
+Phase 2: BACKGROUND
+└─ Background agent: Generate profile report
+```
+
+---
+
+## Report Generation
+
+### Pattern: Multi-Section Report
+
+```
+User Request: "Generate monthly business report"
+
+Phase 1: FAN-OUT (Parallel section generation)
+├─ Agent A: Executive summary section
+├─ Agent B: Sales performance section
+├─ Agent C: Customer metrics section
+├─ Agent D: Product analytics section
+├─ Agent E: Financial summary section
+└─ Agent F: Operational metrics section
+
+Phase 2: REDUCE
+└─ General-purpose agent: Compile sections, add insights
+
+Phase 3: PIPELINE
+└─ General-purpose agent: Format, add visualizations
+```
+
+### Pattern: Automated Dashboard Refresh
+
+```
+Phase 1: FAN-OUT (Parallel data refresh)
+├─ Background agent: Refresh data source 1
+├─ Background agent: Refresh data source 2
+└─ Background agent: Refresh data source 3
+
+Phase 2: PIPELINE
+├─ General-purpose agent: Aggregate refreshed data
+└─ General-purpose agent: Update dashboard calculations
+
+Phase 3: BACKGROUND
+└─ Background agent: Generate and distribute report
+```
+
+### Pattern: Ad-Hoc Query Report
+
+```
+User Request: "Get me sales by region for Q4"
+
+Phase 1: EXPLORE
+└─ Explore agent: Find relevant tables and joins
+
+Phase 2: PIPELINE
+├─ General-purpose agent: Build and execute query
+├─ General-purpose agent: Format results
+└─ General-purpose agent: Add context and insights
+```
+
+---
+
+## ETL Pipelines
+
+### Pattern: ETL Development
+
+```
+User Request: "Create ETL pipeline for user events"
+
+Phase 1: EXPLORE
+└─ Explore agent: Understand source schema, target requirements
+
+Phase 2: PLAN
+└─ Plan agent: Design ETL architecture
+
+Phase 3: FAN-OUT (Parallel component development)
+├─ Agent A: Extract logic (source connectors)
+├─ Agent B: Transform logic (cleaning, mapping)
+├─ Agent C: Load logic (target insertion)
+└─ Agent D: Error handling and logging
+
+Phase 4: PIPELINE
+├─ General-purpose agent: Wire components
+└─ Background agent: Test with sample data
+```
+
+### Pattern: ETL Debugging
+
+```
+User Request: "ETL job is failing"
+
+Phase 1: FAN-OUT (Parallel diagnosis)
+├─ Explore agent: Check job logs
+├─ Explore agent: Check source data quality
+├─ Explore agent: Check target schema compatibility
+└─ Explore agent: Check resource utilization
+
+Phase 2: REDUCE
+└─ General-purpose agent: Root cause identification
+
+Phase 3: PIPELINE
+├─ General-purpose agent: Implement fix
+└─ Background agent: Verify fix with test run
+```
+
+### Pattern: Schema Evolution
+
+```
+Phase 1: EXPLORE
+└─ Explore agent: Identify schema changes
+
+Phase 2: FAN-OUT
+├─ Agent A: Update extract logic
+├─ Agent B: Update transform mappings
+└─ Agent C: Update load targets
+
+Phase 3: PIPELINE
+├─ General-purpose agent: Migration script
+└─ Background agent: Backfill historical data
+```
+
+---
+
+## Statistical Analysis
+
+### Pattern: Hypothesis Testing
+
+```
+User Request: "Did the new feature improve conversion?"
+
+Phase 1: EXPLORE
+└─ Explore agent: Gather pre and post data
+
+Phase 2: FAN-OUT
+├─ Agent A: Descriptive statistics (both groups)
+├─ Agent B: Distribution analysis
+└─ Agent C: Confounding variable check
+
+Phase 3: PIPELINE
+├─ General-purpose agent: Select appropriate test
+├─ General-purpose agent: Run statistical test
+└─ General-purpose agent: Interpret results
+
+Phase 4: REDUCE
+└─ General-purpose agent: Conclusion with confidence
+```
+
+### Pattern: Predictive Modeling
+
+```
+Phase 1: FAN-OUT (Data preparation)
+├─ Agent A: Feature engineering
+├─ Agent B: Data cleaning
+└─ Agent C: Train/test split
+
+Phase 2: SPECULATIVE (Model selection)
+├─ Agent A: Train model type 1
+├─ Agent B: Train model type 2
+└─ Agent C: Train model type 3
+
+Phase 3: REDUCE
+└─ General-purpose agent: Compare models, select best
+
+Phase 4: PIPELINE
+└─ General-purpose agent: Final evaluation, documentation
+```
+
+### Pattern: Trend Analysis
+
+```
+Phase 1: FAN-OUT
+├─ Agent A: Decompose time series
+├─ Agent B: Identify seasonality patterns
+├─ Agent C: Detect change points
+└─ Agent D: Forecast future values
+
+Phase 2: REDUCE
+└─ General-purpose agent: Trend report with insights
+```
+
+---
+
+## Task Management for Data Analysis
+
+Structure data analysis with parallel exploration:
+
+```python
+# Create analysis tasks
+TaskCreate(subject="Understand data sources", description="Schema, types, relationships...")
+TaskCreate(subject="Explore distributions", description="Statistical summaries, outliers...")
+TaskCreate(subject="Analyze missing data", description="Null patterns, imputation needs...")
+TaskCreate(subject="Check data quality", description="Validation, consistency...")
+TaskCreate(subject="Synthesize findings", description="Aggregate insights, recommendations...")
+TaskCreate(subject="Generate report", description="Visualizations, documentation...")
+
+# Parallel exploration after understanding
+TaskUpdate(taskId="2", addBlockedBy=["1"])
+TaskUpdate(taskId="3", addBlockedBy=["1"])
+TaskUpdate(taskId="4", addBlockedBy=["1"])
+TaskUpdate(taskId="5", addBlockedBy=["2", "3", "4"])
+TaskUpdate(taskId="6", addBlockedBy=["5"])
+
+# Spawn parallel analysis agents
+Task(subagent_type="general-purpose", prompt="TaskId 2: Explore distributions...")
+Task(subagent_type="general-purpose", prompt="TaskId 3: Analyze missing data...")
+Task(subagent_type="general-purpose", prompt="TaskId 4: Check data quality...")
+```
+
+## Best Practices
+
+1. **Parallelize exploration** across dimensions
+2. **Validate data quality** before analysis
+3. **Background long queries** to maintain responsiveness
+4. **Document assumptions** in reports
+5. **Include confidence levels** in statistical conclusions
+
+---
+
+```
+─── ◈ Data Analysis ─────────────────────
+```

package/dist/skills/multi-agent-orchestrator/references/domains/devops.md

@@ -0,0 +1,340 @@
+# DevOps Orchestration Patterns
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                                                             │
+│   Infrastructure as carefully orchestrated as code.         │
+│   Safe deployments. Fast pipelines. Reliable operations.    │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
+> **Load when**: CI/CD pipeline, deployment, infrastructure as code, monitoring, incident response
+> **Common patterns**: Pipeline Setup, Zero-Downtime Deployment, Incident Triage
+
+## Table of Contents
+
+1. [CI/CD Pipeline](#cicd-pipeline)
+2. [Deployment](#deployment)
+3. [Infrastructure](#infrastructure)
+4. [Monitoring and Alerting](#monitoring-and-alerting)
+5. [Incident Response](#incident-response)
+
+---
+
+## CI/CD Pipeline
+
+### Pattern: Pipeline Setup
+
+```
+User Request: "Set up CI/CD for this project"
+
+Phase 1: EXPLORE
+└─ Explore agent: Analyze project structure, build system, test setup
+
+Phase 2: PLAN
+└─ Plan agent: Design pipeline stages and requirements
+
+Phase 3: FAN-OUT (Parallel stage implementation)
+├─ Agent A: Build stage (compile, dependencies)
+├─ Agent B: Test stage (unit, integration)
+├─ Agent C: Security scan stage (SAST, dependencies)
+└─ Agent D: Deploy stage (environments, rollback)
+
+Phase 4: PIPELINE
+└─ General-purpose agent: Wire stages, add notifications
+```
+
+### Pattern: Pipeline Optimization
+
+```
+User Request: "Speed up our CI pipeline"
+
+Phase 1: FAN-OUT (Parallel analysis)
+├─ Explore agent: Analyze current pipeline duration by stage
+├─ Explore agent: Find parallelization opportunities
+├─ Explore agent: Check caching effectiveness
+└─ Explore agent: Review resource allocation
+
+Phase 2: REDUCE
+└─ Plan agent: Prioritized optimization plan
+
+Phase 3: FAN-OUT (Implement optimizations)
+├─ Agent A: Implement caching improvements
+├─ Agent B: Parallelize independent jobs
+└─ Agent C: Optimize resource-heavy stages
+```
+
+### Pattern: Multi-Environment Pipeline
+
+```
+Phase 1: EXPLORE
+└─ Explore agent: Map environments (dev, staging, prod)
+
+Phase 2: FAN-OUT
+├─ Agent A: Configure dev deployment
+├─ Agent B: Configure staging deployment
+├─ Agent C: Configure prod deployment (with gates)
+└─ Agent D: Configure environment promotions
+
+Phase 3: PIPELINE
+└─ General-purpose agent: Add approval workflows, rollback triggers
+```
+
+---
+
+## Deployment
+
+### Pattern: Zero-Downtime Deployment
+
+```
+User Request: "Deploy the new version without downtime"
+
+Phase 1: EXPLORE
+└─ Explore agent: Current deployment strategy, infrastructure
+
+Phase 2: PIPELINE
+├─ General-purpose agent: Build and tag new image
+├─ General-purpose agent: Deploy to canary/blue environment
+├─ Background agent: Run smoke tests
+└─ General-purpose agent: Shift traffic, drain old instances
+
+Phase 3: BACKGROUND
+└─ Background agent: Monitor error rates post-deploy
+```
+
+### Pattern: Rollback Preparation
+
+```
+Phase 1: FAN-OUT (Pre-deployment)
+├─ Agent A: Tag current stable version
+├─ Agent B: Document database state
+├─ Agent C: Prepare rollback scripts
+└─ Agent D: Verify rollback path
+
+Phase 2: PIPELINE (Deployment with safety)
+├─ General-purpose agent: Deploy new version
+├─ Background agent: Health check monitoring
+└─ General-purpose agent: (If needed) Execute rollback
+```
+
+### Pattern: Database Migration Deployment
+
+```
+Phase 1: PIPELINE (Safe migration)
+├─ General-purpose agent: Backup current database
+├─ General-purpose agent: Run forward migration
+├─ General-purpose agent: Deploy compatible app version
+└─ Background agent: Verify data integrity
+
+Phase 2: BACKGROUND
+└─ Background agent: Monitor for migration issues
+```
+
+---
+
+## Infrastructure
+
+### Pattern: Infrastructure as Code
+
+```
+User Request: "Create Terraform for our AWS setup"
+
+Phase 1: EXPLORE
+└─ Explore agent: Map current infrastructure, requirements
+
+Phase 2: FAN-OUT (Parallel module creation)
+├─ Agent A: Network module (VPC, subnets, security groups)
+├─ Agent B: Compute module (ECS/EKS, EC2)
+├─ Agent C: Database module (RDS, ElastiCache)
+├─ Agent D: Storage module (S3, EFS)
+└─ Agent E: CDN and DNS module
+
+Phase 3: PIPELINE
+├─ General-purpose agent: Wire modules, configure state backend
+└─ Background agent: Validate and plan
+```
+
+### Pattern: Kubernetes Manifest Generation
+
+```
+Phase 1: EXPLORE
+└─ Explore agent: Analyze application requirements
+
+Phase 2: FAN-OUT
+├─ Agent A: Deployment manifests
+├─ Agent B: Service and ingress manifests
+├─ Agent C: ConfigMaps and Secrets
+├─ Agent D: HPA and PDB configurations
+└─ Agent E: NetworkPolicies
+
+Phase 3: PIPELINE
+├─ General-purpose agent: Kustomize or Helm setup
+└─ Background agent: Validate against cluster
+```
+
+### Pattern: Security Hardening
+
+```
+Phase 1: FAN-OUT (Parallel security checks)
+├─ Agent A: Network security (firewall, security groups)
+├─ Agent B: IAM and access control
+├─ Agent C: Encryption (at rest, in transit)
+├─ Agent D: Secrets management
+└─ Agent E: Compliance checks
+
+Phase 2: REDUCE
+└─ General-purpose agent: Security report with remediations
+
+Phase 3: FAN-OUT (Implement fixes)
+├─ Agents implement security improvements
+```
+
+---
+
+## Monitoring and Alerting
+
+### Pattern: Observability Setup
+
+```
+User Request: "Set up monitoring for the application"
+
+Phase 1: FAN-OUT (Parallel pillar implementation)
+├─ Agent A: Metrics (Prometheus, CloudWatch)
+├─ Agent B: Logging (ELK, CloudWatch Logs)
+├─ Agent C: Tracing (Jaeger, X-Ray)
+└─ Agent D: Dashboards (Grafana, custom)
+
+Phase 2: PIPELINE
+└─ General-purpose agent: Configure alerting rules, runbooks
+```
+
+### Pattern: Alert Tuning
+
+```
+Phase 1: EXPLORE
+└─ Explore agent: Analyze current alerts, noise ratio
+
+Phase 2: FAN-OUT
+├─ Agent A: Tune threshold-based alerts
+├─ Agent B: Implement anomaly detection
+└─ Agent C: Configure alert routing
+
+Phase 3: PIPELINE
+└─ General-purpose agent: Document runbooks for each alert
+```
+
+### Pattern: SLO Definition
+
+```
+Phase 1: EXPLORE
+└─ Explore agent: Identify critical user journeys
+
+Phase 2: FAN-OUT
+├─ Agent A: Define availability SLIs/SLOs
+├─ Agent B: Define latency SLIs/SLOs
+├─ Agent C: Define error rate SLIs/SLOs
+└─ Agent D: Configure error budget alerts
+
+Phase 3: PIPELINE
+└─ General-purpose agent: Create SLO dashboard
+```
+
+---
+
+## Incident Response
+
+### Pattern: Incident Triage
+
+```
+User Request: "Production is down!"
+
+Phase 1: FAN-OUT (Rapid parallel diagnosis)
+├─ Agent A: Check application logs for errors
+├─ Agent B: Check infrastructure metrics
+├─ Agent C: Check recent deployments
+├─ Agent D: Check external dependencies
+└─ Agent E: Check database health
+
+Phase 2: REDUCE (Fast)
+└─ General-purpose agent: Identify most likely cause
+
+Phase 3: PIPELINE
+├─ General-purpose agent: Implement fix or rollback
+└─ Background agent: Verify recovery
+```
+
+### Pattern: Post-Incident Review
+
+```
+Phase 1: FAN-OUT (Evidence gathering)
+├─ Explore agent: Timeline of events
+├─ Explore agent: Relevant logs and metrics
+├─ Explore agent: Changes before incident
+└─ Explore agent: Response actions taken
+
+Phase 2: PIPELINE
+├─ General-purpose agent: Root cause analysis
+├─ General-purpose agent: Impact assessment
+└─ General-purpose agent: Action items and preventions
+
+Phase 3: REDUCE
+└─ General-purpose agent: Post-mortem document
+```
+
+### Pattern: Runbook Execution
+
+```
+Phase 1: EXPLORE
+└─ Explore agent: Find relevant runbook
+
+Phase 2: PIPELINE (Step-by-step)
+├─ General-purpose agent: Execute step 1
+├─ General-purpose agent: Verify step 1
+├─ General-purpose agent: Execute step 2
+└─ ... continue until resolved
+
+Phase 3: BACKGROUND
+└─ Background agent: Continued monitoring
+```
+
+---
+
+## Task Management for DevOps
+
+Structure infrastructure work with safety checkpoints:
+
+```python
+# Create DevOps tasks
+TaskCreate(subject="Assess infrastructure", description="Current state, requirements...")
+TaskCreate(subject="Plan changes", description="Design with minimal disruption...")
+TaskCreate(subject="Implement network changes", description="VPC, security groups...")
+TaskCreate(subject="Implement compute changes", description="ECS, scaling...")
+TaskCreate(subject="Validate deployment", description="Health checks, smoke tests...")
+TaskCreate(subject="Configure monitoring", description="Alerts, dashboards...")
+
+# Sequential safety gates
+TaskUpdate(taskId="2", addBlockedBy=["1"])
+TaskUpdate(taskId="3", addBlockedBy=["2"])
+TaskUpdate(taskId="4", addBlockedBy=["2"])  # Can parallel with network
+TaskUpdate(taskId="5", addBlockedBy=["3", "4"])
+TaskUpdate(taskId="6", addBlockedBy=["5"])
+
+# Parallel infrastructure implementation
+Task(subagent_type="general-purpose", prompt="TaskId 3: Implement network changes...")
+Task(subagent_type="general-purpose", prompt="TaskId 4: Implement compute changes...")
+```
+
+## Safety Principles
+
+1. **Always have rollback plan** before deploying
+2. **Background monitor** during and after deployment
+3. **Parallel diagnosis** during incidents for speed
+4. **Document everything** for future incidents
+5. **Test in staging** before production
+
+---
+
+```
+─── ◈ DevOps ────────────────────────────
+```