mumucc 0.4.4 → 0.4.10

package/defaults/plugin-seed/marketplaces/claude-code-workflows/plugins/machine-learning-ops/commands/ml-pipeline.md

@@ -0,0 +1,301 @@
+# Machine Learning Pipeline - Multi-Agent MLOps Orchestration
+
+Design and implement a complete ML pipeline for: $ARGUMENTS
+
+## Thinking
+
+This workflow orchestrates multiple specialized agents to build a production-ready ML pipeline following modern MLOps best practices. The approach emphasizes:
+
+- **Phase-based coordination**: Each phase builds upon previous outputs, with clear handoffs between agents
+- **Modern tooling integration**: MLflow/W&B for experiments, Feast/Tecton for features, KServe/Seldon for serving
+- **Production-first mindset**: Every component designed for scale, monitoring, and reliability
+- **Reproducibility**: Version control for data, models, and infrastructure
+- **Continuous improvement**: Automated retraining, A/B testing, and drift detection
+
+The multi-agent approach ensures each aspect is handled by domain experts:
+
+- Data engineers handle ingestion and quality
+- Data scientists design features and experiments
+- ML engineers implement training pipelines
+- MLOps engineers handle production deployment
+- Observability engineers ensure monitoring
+
+## Phase 1: Data & Requirements Analysis
+
+<Task>
+subagent_type: data-engineer
+prompt: |
+  Analyze and design data pipeline for ML system with requirements: $ARGUMENTS
+
+Deliverables:
+
+1. Data source audit and ingestion strategy:
+   - Source systems and connection patterns
+   - Schema validation using Pydantic/Great Expectations
+   - Data versioning with DVC or lakeFS
+   - Incremental loading and CDC strategies
+
+2. Data quality framework:
+   - Profiling and statistics generation
+   - Anomaly detection rules
+   - Data lineage tracking
+   - Quality gates and SLAs
+
+3. Storage architecture:
+   - Raw/processed/feature layers
+   - Partitioning strategy
+   - Retention policies
+   - Cost optimization
+
+Provide implementation code for critical components and integration patterns.
+</Task>
+
+<Task>
+subagent_type: data-scientist
+prompt: |
+  Design feature engineering and model requirements for: $ARGUMENTS
+  Using data architecture from: {phase1.data-engineer.output}
+
+Deliverables:
+
+1. Feature engineering pipeline:
+   - Transformation specifications
+   - Feature store schema (Feast/Tecton)
+   - Statistical validation rules
+   - Handling strategies for missing data/outliers
+
+2. Model requirements:
+   - Algorithm selection rationale
+   - Performance metrics and baselines
+   - Training data requirements
+   - Evaluation criteria and thresholds
+
+3. Experiment design:
+   - Hypothesis and success metrics
+   - A/B testing methodology
+   - Sample size calculations
+   - Bias detection approach
+
+Include feature transformation code and statistical validation logic.
+</Task>
+
+## Phase 2: Model Development & Training
+
+<Task>
+subagent_type: ml-engineer
+prompt: |
+  Implement training pipeline based on requirements: {phase1.data-scientist.output}
+  Using data pipeline: {phase1.data-engineer.output}
+
+Build comprehensive training system:
+
+1. Training pipeline implementation:
+   - Modular training code with clear interfaces
+   - Hyperparameter optimization (Optuna/Ray Tune)
+   - Distributed training support (Horovod/PyTorch DDP)
+   - Cross-validation and ensemble strategies
+
+2. Experiment tracking setup:
+   - MLflow/Weights & Biases integration
+   - Metric logging and visualization
+   - Artifact management (models, plots, data samples)
+   - Experiment comparison and analysis tools
+
+3. Model registry integration:
+   - Version control and tagging strategy
+   - Model metadata and lineage
+   - Promotion workflows (dev -> staging -> prod)
+   - Rollback procedures
+
+Provide complete training code with configuration management.
+</Task>
+
+<Task>
+subagent_type: python-pro
+prompt: |
+  Optimize and productionize ML code from: {phase2.ml-engineer.output}
+
+Focus areas:
+
+1. Code quality and structure:
+   - Refactor for production standards
+   - Add comprehensive error handling
+   - Implement proper logging with structured formats
+   - Create reusable components and utilities
+
+2. Performance optimization:
+   - Profile and optimize bottlenecks
+   - Implement caching strategies
+   - Optimize data loading and preprocessing
+   - Memory management for large-scale training
+
+3. Testing framework:
+   - Unit tests for data transformations
+   - Integration tests for pipeline components
+   - Model quality tests (invariance, directional)
+   - Performance regression tests
+
+Deliver production-ready, maintainable code with full test coverage.
+</Task>
+
+## Phase 3: Production Deployment & Serving
+
+<Task>
+subagent_type: mlops-engineer
+prompt: |
+  Design production deployment for models from: {phase2.ml-engineer.output}
+  With optimized code from: {phase2.python-pro.output}
+
+Implementation requirements:
+
+1. Model serving infrastructure:
+   - REST/gRPC APIs with FastAPI/TorchServe
+   - Batch prediction pipelines (Airflow/Kubeflow)
+   - Stream processing (Kafka/Kinesis integration)
+   - Model serving platforms (KServe/Seldon Core)
+
+2. Deployment strategies:
+   - Blue-green deployments for zero downtime
+   - Canary releases with traffic splitting
+   - Shadow deployments for validation
+   - A/B testing infrastructure
+
+3. CI/CD pipeline:
+   - GitHub Actions/GitLab CI workflows
+   - Automated testing gates
+   - Model validation before deployment
+   - ArgoCD for GitOps deployment
+
+4. Infrastructure as Code:
+   - Terraform modules for cloud resources
+   - Helm charts for Kubernetes deployments
+   - Docker multi-stage builds for optimization
+   - Secret management with Vault/Secrets Manager
+
+Provide complete deployment configuration and automation scripts.
+</Task>
+
+<Task>
+subagent_type: kubernetes-architect
+prompt: |
+  Design Kubernetes infrastructure for ML workloads from: {phase3.mlops-engineer.output}
+
+Kubernetes-specific requirements:
+
+1. Workload orchestration:
+   - Training job scheduling with Kubeflow
+   - GPU resource allocation and sharing
+   - Spot/preemptible instance integration
+   - Priority classes and resource quotas
+
+2. Serving infrastructure:
+   - HPA/VPA for autoscaling
+   - KEDA for event-driven scaling
+   - Istio service mesh for traffic management
+   - Model caching and warm-up strategies
+
+3. Storage and data access:
+   - PVC strategies for training data
+   - Model artifact storage with CSI drivers
+   - Distributed storage for feature stores
+   - Cache layers for inference optimization
+
+Provide Kubernetes manifests and Helm charts for entire ML platform.
+</Task>
+
+## Phase 4: Monitoring & Continuous Improvement
+
+<Task>
+subagent_type: observability-engineer
+prompt: |
+  Implement comprehensive monitoring for ML system deployed in: {phase3.mlops-engineer.output}
+  Using Kubernetes infrastructure: {phase3.kubernetes-architect.output}
+
+Monitoring framework:
+
+1. Model performance monitoring:
+   - Prediction accuracy tracking
+   - Latency and throughput metrics
+   - Feature importance shifts
+   - Business KPI correlation
+
+2. Data and model drift detection:
+   - Statistical drift detection (KS test, PSI)
+   - Concept drift monitoring
+   - Feature distribution tracking
+   - Automated drift alerts and reports
+
+3. System observability:
+   - Prometheus metrics for all components
+   - Grafana dashboards for visualization
+   - Distributed tracing with Jaeger/Zipkin
+   - Log aggregation with ELK/Loki
+
+4. Alerting and automation:
+   - PagerDuty/Opsgenie integration
+   - Automated retraining triggers
+   - Performance degradation workflows
+   - Incident response runbooks
+
+5. Cost tracking:
+   - Resource utilization metrics
+   - Cost allocation by model/experiment
+   - Optimization recommendations
+   - Budget alerts and controls
+
+Deliver monitoring configuration, dashboards, and alert rules.
+</Task>
+
+## Configuration Options
+
+- **experiment_tracking**: mlflow | wandb | neptune | clearml
+- **feature_store**: feast | tecton | databricks | custom
+- **serving_platform**: kserve | seldon | torchserve | triton
+- **orchestration**: kubeflow | airflow | prefect | dagster
+- **cloud_provider**: aws | azure | gcp | oci | multi-cloud
+- **deployment_mode**: realtime | batch | streaming | hybrid
+- **monitoring_stack**: prometheus | datadog | newrelic | custom
+
+## Success Criteria
+
+1. **Data Pipeline Success**:
+   - < 0.1% data quality issues in production
+   - Automated data validation passing 99.9% of time
+   - Complete data lineage tracking
+   - Sub-second feature serving latency
+
+2. **Model Performance**:
+   - Meeting or exceeding baseline metrics
+   - < 5% performance degradation before retraining
+   - Successful A/B tests with statistical significance
+   - No undetected model drift > 24 hours
+
+3. **Operational Excellence**:
+   - 99.9% uptime for model serving
+   - < 200ms p99 inference latency
+   - Automated rollback within 5 minutes
+   - Complete observability with < 1 minute alert time
+
+4. **Development Velocity**:
+   - < 1 hour from commit to production
+   - Parallel experiment execution
+   - Reproducible training runs
+   - Self-service model deployment
+
+5. **Cost Efficiency**:
+   - < 20% infrastructure waste
+   - Optimized resource allocation
+   - Automatic scaling based on load
+   - Spot instance utilization > 60%
+
+## Final Deliverables
+
+Upon completion, the orchestrated pipeline will provide:
+
+- End-to-end ML pipeline with full automation
+- Comprehensive documentation and runbooks
+- Production-ready infrastructure as code
+- Complete monitoring and alerting system
+- CI/CD pipelines for continuous improvement
+- Cost optimization and scaling strategies
+- Disaster recovery and rollback procedures

package/defaults/plugin-seed/marketplaces/claude-code-workflows/plugins/machine-learning-ops/skills/ml-pipeline-workflow/SKILL.md

@@ -0,0 +1,248 @@
+---
+name: ml-pipeline-workflow
+description: Build end-to-end MLOps pipelines from data preparation through model training, validation, and production deployment. Use when creating ML pipelines, implementing MLOps practices, or automating model training and deployment workflows.
+---
+
+# ML Pipeline Workflow
+
+Complete end-to-end MLOps pipeline orchestration from data preparation through model deployment.
+
+## Overview
+
+This skill provides comprehensive guidance for building production ML pipelines that handle the full lifecycle: data ingestion → preparation → training → validation → deployment → monitoring.
+
+## When to Use This Skill
+
+- Building new ML pipelines from scratch
+- Designing workflow orchestration for ML systems
+- Implementing data → model → deployment automation
+- Setting up reproducible training workflows
+- Creating DAG-based ML orchestration
+- Integrating ML components into production systems
+
+## What This Skill Provides
+
+### Core Capabilities
+
+1. **Pipeline Architecture**
+   - End-to-end workflow design
+   - DAG orchestration patterns (Airflow, Dagster, Kubeflow)
+   - Component dependencies and data flow
+   - Error handling and retry strategies
+
+2. **Data Preparation**
+   - Data validation and quality checks
+   - Feature engineering pipelines
+   - Data versioning and lineage
+   - Train/validation/test splitting strategies
+
+3. **Model Training**
+   - Training job orchestration
+   - Hyperparameter management
+   - Experiment tracking integration
+   - Distributed training patterns
+
+4. **Model Validation**
+   - Validation frameworks and metrics
+   - A/B testing infrastructure
+   - Performance regression detection
+   - Model comparison workflows
+
+5. **Deployment Automation**
+   - Model serving patterns
+   - Canary deployments
+   - Blue-green deployment strategies
+   - Rollback mechanisms
+
+### Reference Documentation
+
+See the `references/` directory for detailed guides:
+
+- **data-preparation.md** - Data cleaning, validation, and feature engineering
+- **model-training.md** - Training workflows and best practices
+- **model-validation.md** - Validation strategies and metrics
+- **model-deployment.md** - Deployment patterns and serving architectures
+
+### Assets and Templates
+
+The `assets/` directory contains:
+
+- **pipeline-dag.yaml.template** - DAG template for workflow orchestration
+- **training-config.yaml** - Training configuration template
+- **validation-checklist.md** - Pre-deployment validation checklist
+
+## Usage Patterns
+
+### Basic Pipeline Setup
+
+```python
+# 1. Define pipeline stages
+stages = [
+    "data_ingestion",
+    "data_validation",
+    "feature_engineering",
+    "model_training",
+    "model_validation",
+    "model_deployment"
+]
+
+# 2. Configure dependencies
+# See assets/pipeline-dag.yaml.template for full example
+```
+
+### Production Workflow
+
+1. **Data Preparation Phase**
+   - Ingest raw data from sources
+   - Run data quality checks
+   - Apply feature transformations
+   - Version processed datasets
+
+2. **Training Phase**
+   - Load versioned training data
+   - Execute training jobs
+   - Track experiments and metrics
+   - Save trained models
+
+3. **Validation Phase**
+   - Run validation test suite
+   - Compare against baseline
+   - Generate performance reports
+   - Approve for deployment
+
+4. **Deployment Phase**
+   - Package model artifacts
+   - Deploy to serving infrastructure
+   - Configure monitoring
+   - Validate production traffic
+
+## Best Practices
+
+### Pipeline Design
+
+- **Modularity**: Each stage should be independently testable
+- **Idempotency**: Re-running stages should be safe
+- **Observability**: Log metrics at every stage
+- **Versioning**: Track data, code, and model versions
+- **Failure Handling**: Implement retry logic and alerting
+
+### Data Management
+
+- Use data validation libraries (Great Expectations, TFX)
+- Version datasets with DVC or similar tools
+- Document feature engineering transformations
+- Maintain data lineage tracking
+
+### Model Operations
+
+- Separate training and serving infrastructure
+- Use model registries (MLflow, Weights & Biases)
+- Implement gradual rollouts for new models
+- Monitor model performance drift
+- Maintain rollback capabilities
+
+### Deployment Strategies
+
+- Start with shadow deployments
+- Use canary releases for validation
+- Implement A/B testing infrastructure
+- Set up automated rollback triggers
+- Monitor latency and throughput
+
+## Integration Points
+
+### Orchestration Tools
+
+- **Apache Airflow**: DAG-based workflow orchestration
+- **Dagster**: Asset-based pipeline orchestration
+- **Kubeflow Pipelines**: Kubernetes-native ML workflows
+- **Prefect**: Modern dataflow automation
+
+### Experiment Tracking
+
+- MLflow for experiment tracking and model registry
+- Weights & Biases for visualization and collaboration
+- TensorBoard for training metrics
+
+### Deployment Platforms
+
+- AWS SageMaker for managed ML infrastructure
+- Google Vertex AI for GCP deployments
+- Azure ML for Azure cloud
+- OCI Data Science for Oracle Cloud Infrastructure deployments
+- Kubernetes + KServe for cloud-agnostic serving
+
+## Progressive Disclosure
+
+Start with the basics and gradually add complexity:
+
+1. **Level 1**: Simple linear pipeline (data → train → deploy)
+2. **Level 2**: Add validation and monitoring stages
+3. **Level 3**: Implement hyperparameter tuning
+4. **Level 4**: Add A/B testing and gradual rollouts
+5. **Level 5**: Multi-model pipelines with ensemble strategies
+
+## Common Patterns
+
+### Batch Training Pipeline
+
+```yaml
+# See assets/pipeline-dag.yaml.template
+stages:
+  - name: data_preparation
+    dependencies: []
+  - name: model_training
+    dependencies: [data_preparation]
+  - name: model_evaluation
+    dependencies: [model_training]
+  - name: model_deployment
+    dependencies: [model_evaluation]
+```
+
+### Real-time Feature Pipeline
+
+```python
+# Stream processing for real-time features
+# Combined with batch training
+# See references/data-preparation.md
+```
+
+### Continuous Training
+
+```python
+# Automated retraining on schedule
+# Triggered by data drift detection
+# See references/model-training.md
+```
+
+## Troubleshooting
+
+### Common Issues
+
+- **Pipeline failures**: Check dependencies and data availability
+- **Training instability**: Review hyperparameters and data quality
+- **Deployment issues**: Validate model artifacts and serving config
+- **Performance degradation**: Monitor data drift and model metrics
+
+### Debugging Steps
+
+1. Check pipeline logs for each stage
+2. Validate input/output data at boundaries
+3. Test components in isolation
+4. Review experiment tracking metrics
+5. Inspect model artifacts and metadata
+
+## Next Steps
+
+After setting up your pipeline:
+
+1. Explore **hyperparameter-tuning** skill for optimization
+2. Learn **experiment-tracking-setup** for MLflow/W&B
+3. Review **model-deployment-patterns** for serving strategies
+4. Implement monitoring with observability tools
+
+## Related Skills
+
+- **experiment-tracking-setup**: MLflow and Weights & Biases integration
+- **hyperparameter-tuning**: Automated hyperparameter optimization
+- **model-deployment-patterns**: Advanced deployment strategies

package/defaults/plugin-seed/marketplaces/claude-code-workflows/plugins/meigen-ai-design/.claude-plugin/plugin.json

@@ -0,0 +1,10 @@
+{
+  "name": "meigen-ai-design",
+  "version": "1.0.0",
+  "description": "AI image generation with creative workflow orchestration, parallel multi-direction output, prompt engineering, and a 1,300+ curated inspiration library. Requires MeiGen MCP server (supports MeiGen Cloud, local ComfyUI, and OpenAI-compatible APIs).",
+  "author": {
+    "name": "MeiGen",
+    "url": "https://www.meigen.ai"
+  },
+  "license": "MIT"
+}

package/defaults/plugin-seed/marketplaces/claude-code-workflows/plugins/meigen-ai-design/README.md

@@ -0,0 +1,112 @@
+# MeiGen AI Design
+
+AI image generation plugin with creative workflow orchestration, parallel multi-direction output, prompt engineering, and a 1,300+ curated inspiration library.
+
+## Prerequisites
+
+This plugin requires the **meigen** MCP server. Install it by adding to your project's `.mcp.json`:
+
+```json
+{
+  "mcpServers": {
+    "meigen": {
+      "command": "npx",
+      "args": ["-y", "meigen@1.2.5"]
+    }
+  }
+}
+```
+
+**Restart Claude Code** after adding the configuration.
+
+Requires Node.js 18+.
+
+## Provider Setup
+
+Configure at least one provider to enable image generation. **Free features** (gallery search, prompt enhancement, model listing, preference management) work without any provider configured.
+
+### Option A: MeiGen Cloud (Recommended)
+
+1. Sign up at [meigen.ai](https://www.meigen.ai)
+2. Click avatar → Settings → API Keys → create a new key (starts with `meigen_sk_`)
+3. Set environment variable:
+
+```bash
+export MEIGEN_API_TOKEN=meigen_sk_your_token_here
+```
+
+### Option B: Local ComfyUI (Free, Private)
+
+1. Install and run [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
+2. Set environment variable:
+
+```bash
+export COMFYUI_URL=http://localhost:8188
+```
+
+3. Export workflows from ComfyUI (API format JSON) and import via the plugin
+
+### Option C: OpenAI-compatible API (Bring Your Own Key)
+
+Works with any OpenAI-compatible image generation API (OpenAI, Together AI, Fireworks AI, etc.):
+
+```bash
+export OPENAI_API_KEY=your_key_here
+export OPENAI_BASE_URL=https://api.openai.com/v1
+export OPENAI_MODEL=gpt-image-1
+```
+
+Alternatively, store configuration in `~/.config/meigen/config.json`.
+
+## Agents
+
+| Agent | Model | Purpose |
+|-------|-------|---------|
+| `gallery-researcher` | haiku | Search 1,300+ curated gallery, find references, build mood boards |
+| `prompt-crafter` | haiku | Write multiple distinct prompts for parallel image generation |
+| `image-generator` | inherit | Execute `generate_image` calls and relay results |
+
+## Commands
+
+| Command | Description |
+|---------|-------------|
+| `/meigen-ai-design:gen <prompt>` | Quick image generation — skips intent assessment |
+| `/meigen-ai-design:find <keywords>` | Quick gallery search — browse inspiration |
+
+## Free Features (No API Key)
+
+These tools work immediately without any provider setup:
+- **search_gallery** — Search 1,300+ curated AI image prompts
+- **enhance_prompt** — Expand brief descriptions into detailed generation prompts
+- **list_models** — List available models across all configured providers
+- **manage_preferences** — Save default style, aspect ratio, and favorite prompts
+
+## Verification
+
+After setup, test with a free feature:
+> "Search the gallery for product photography inspiration"
+
+Then test generation:
+> "Generate a product photo for a glass perfume bottle on marble"
+
+## Troubleshooting
+
+| Problem | Solution |
+|---------|----------|
+| MCP tools not available | Restart Claude Code after adding `.mcp.json` configuration |
+| "No image generation providers configured" | Configure at least one provider (see Provider Setup above) |
+| API key / authentication errors | Verify env vars or check `~/.config/meigen/config.json` |
+| ComfyUI connection refused | Ensure ComfyUI is running at the configured URL |
+| Generation timeout | Server may be under high load — try again in a moment |
+| Empty gallery search results | Try different keywords or broader terms |
+
+## Links
+
+- [npm package](https://www.npmjs.com/package/meigen)
+- [GitHub](https://github.com/jau123/MeiGen-AI-Design-MCP)
+- [Homepage](https://www.meigen.ai)
+- [Demo Video](https://youtu.be/JQ3DZ1DXqvs)
+
+## License
+
+MIT