@ngxtm/devkit 3.7.0 → 3.8.0

package/merged-commands/senior-data-engineer.md

@@ -0,0 +1,226 @@
+---
+name: senior-data-engineer
+description: World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data architectures, building data pipelines, optimizing data workflows, or implementing data governance.
+---
+
+# Senior Data Engineer
+
+World-class senior data engineer skill for production-grade AI/ML/Data systems.
+
+## Quick Start
+
+### Main Capabilities
+
+```bash
+# Core Tool 1
+python3 ~/.{TOOL}/skills/senior-data-engineer/scripts/pipeline_orchestrator.py --input data/ --output results/
+
+# Core Tool 2  
+python3 ~/.{TOOL}/skills/senior-data-engineer/scripts/data_quality_validator.py --target project/ --analyze
+
+# Core Tool 3
+python3 ~/.{TOOL}/skills/senior-data-engineer/scripts/etl_performance_optimizer.py --config config.yaml --deploy
+```
+
+## Core Expertise
+
+This skill covers world-class capabilities in:
+
+- Advanced production patterns and architectures
+- Scalable system design and implementation
+- Performance optimization at scale
+- MLOps and DataOps best practices
+- Real-time processing and inference
+- Distributed computing frameworks
+- Model deployment and monitoring
+- Security and compliance
+- Cost optimization
+- Team leadership and mentoring
+
+## Tech Stack
+
+**Languages:** Python, SQL, R, Scala, Go
+**ML Frameworks:** PyTorch, TensorFlow, Scikit-learn, XGBoost
+**Data Tools:** Spark, Airflow, dbt, Kafka, Databricks
+**LLM Frameworks:** LangChain, LlamaIndex, DSPy
+**Deployment:** Docker, Kubernetes, AWS/GCP/Azure
+**Monitoring:** MLflow, Weights & Biases, Prometheus
+**Databases:** PostgreSQL, BigQuery, Snowflake, Pinecone
+
+## Reference Documentation
+
+### 1. Data Pipeline Architecture
+
+Comprehensive guide available in `references/data_pipeline_architecture.md` covering:
+
+- Advanced patterns and best practices
+- Production implementation strategies
+- Performance optimization techniques
+- Scalability considerations
+- Security and compliance
+- Real-world case studies
+
+### 2. Data Modeling Patterns
+
+Complete workflow documentation in `references/data_modeling_patterns.md` including:
+
+- Step-by-step processes
+- Architecture design patterns
+- Tool integration guides
+- Performance tuning strategies
+- Troubleshooting procedures
+
+### 3. Dataops Best Practices
+
+Technical reference guide in `references/dataops_best_practices.md` with:
+
+- System design principles
+- Implementation examples
+- Configuration best practices
+- Deployment strategies
+- Monitoring and observability
+
+## Production Patterns
+
+### Pattern 1: Scalable Data Processing
+
+Enterprise-scale data processing with distributed computing:
+
+- Horizontal scaling architecture
+- Fault-tolerant design
+- Real-time and batch processing
+- Data quality validation
+- Performance monitoring
+
+### Pattern 2: ML Model Deployment
+
+Production ML system with high availability:
+
+- Model serving with low latency
+- A/B testing infrastructure
+- Feature store integration
+- Model monitoring and drift detection
+- Automated retraining pipelines
+
+### Pattern 3: Real-Time Inference
+
+High-throughput inference system:
+
+- Batching and caching strategies
+- Load balancing
+- Auto-scaling
+- Latency optimization
+- Cost optimization
+
+## Best Practices
+
+### Development
+
+- Test-driven development
+- Code reviews and pair programming
+- Documentation as code
+- Version control everything
+- Continuous integration
+
+### Production
+
+- Monitor everything critical
+- Automate deployments
+- Feature flags for releases
+- Canary deployments
+- Comprehensive logging
+
+### Team Leadership
+
+- Mentor junior engineers
+- Drive technical decisions
+- Establish coding standards
+- Foster learning culture
+- Cross-functional collaboration
+
+## Performance Targets
+
+**Latency:**
+- P50: < 50ms
+- P95: < 100ms
+- P99: < 200ms
+
+**Throughput:**
+- Requests/second: > 1000
+- Concurrent users: > 10,000
+
+**Availability:**
+- Uptime: 99.9%
+- Error rate: < 0.1%
+
+## Security & Compliance
+
+- Authentication & authorization
+- Data encryption (at rest & in transit)
+- PII handling and anonymization
+- GDPR/CCPA compliance
+- Regular security audits
+- Vulnerability management
+
+## Common Commands
+
+```bash
+# Development
+python -m pytest tests/ -v --cov
+python -m black src/
+python -m pylint src/
+
+# Training
+python3 ~/.{TOOL}/skills/senior-data-engineer/scripts/train.py --config prod.yaml
+python3 ~/.{TOOL}/skills/senior-data-engineer/scripts/evaluate.py --model best.pth
+
+# Deployment
+docker build -t service:v1 .
+kubectl apply -f k8s/
+helm upgrade service ./charts/
+
+# Monitoring
+kubectl logs -f deployment/service
+python3 ~/.{TOOL}/skills/senior-data-engineer/scripts/health_check.py
+```
+
+## Resources
+
+- Advanced Patterns: `references/data_pipeline_architecture.md`
+- Implementation Guide: `references/data_modeling_patterns.md`
+- Technical Reference: `references/dataops_best_practices.md`
+- Automation Scripts: `scripts/` directory
+
+## Senior-Level Responsibilities
+
+As a world-class senior professional:
+
+1. **Technical Leadership**
+   - Drive architectural decisions
+   - Mentor team members
+   - Establish best practices
+   - Ensure code quality
+
+2. **Strategic Thinking**
+   - Align with business goals
+   - Evaluate trade-offs
+   - Plan for scale
+   - Manage technical debt
+
+3. **Collaboration**
+   - Work across teams
+   - Communicate effectively
+   - Build consensus
+   - Share knowledge
+
+4. **Innovation**
+   - Stay current with research
+   - Experiment with new approaches
+   - Contribute to community
+   - Drive continuous improvement
+
+5. **Production Excellence**
+   - Ensure high availability
+   - Monitor proactively
+   - Optimize performance
+   - Respond to incidents

package/merged-commands/senior-data-scientist.md

@@ -0,0 +1,226 @@
+---
+name: senior-data-scientist
+description: World-class data science skill for statistical modeling, experimentation, causal inference, and advanced analytics. Expertise in Python (NumPy, Pandas, Scikit-learn), R, SQL, statistical methods, A/B testing, time series, and business intelligence. Includes experiment design, feature engineering, model evaluation, and stakeholder communication. Use when designing experiments, building predictive models, performing causal analysis, or driving data-driven decisions.
+---
+
+# Senior Data Scientist
+
+World-class senior data scientist skill for production-grade AI/ML/Data systems.
+
+## Quick Start
+
+### Main Capabilities
+
+```bash
+# Core Tool 1
+python3 ~/.{TOOL}/skills/senior-data-scientist/scripts/experiment_designer.py --input data/ --output results/
+
+# Core Tool 2  
+python3 ~/.{TOOL}/skills/senior-data-scientist/scripts/feature_engineering_pipeline.py --target project/ --analyze
+
+# Core Tool 3
+python3 ~/.{TOOL}/skills/senior-data-scientist/scripts/model_evaluation_suite.py --config config.yaml --deploy
+```
+
+## Core Expertise
+
+This skill covers world-class capabilities in:
+
+- Advanced production patterns and architectures
+- Scalable system design and implementation
+- Performance optimization at scale
+- MLOps and DataOps best practices
+- Real-time processing and inference
+- Distributed computing frameworks
+- Model deployment and monitoring
+- Security and compliance
+- Cost optimization
+- Team leadership and mentoring
+
+## Tech Stack
+
+**Languages:** Python, SQL, R, Scala, Go
+**ML Frameworks:** PyTorch, TensorFlow, Scikit-learn, XGBoost
+**Data Tools:** Spark, Airflow, dbt, Kafka, Databricks
+**LLM Frameworks:** LangChain, LlamaIndex, DSPy
+**Deployment:** Docker, Kubernetes, AWS/GCP/Azure
+**Monitoring:** MLflow, Weights & Biases, Prometheus
+**Databases:** PostgreSQL, BigQuery, Snowflake, Pinecone
+
+## Reference Documentation
+
+### 1. Statistical Methods Advanced
+
+Comprehensive guide available in `references/statistical_methods_advanced.md` covering:
+
+- Advanced patterns and best practices
+- Production implementation strategies
+- Performance optimization techniques
+- Scalability considerations
+- Security and compliance
+- Real-world case studies
+
+### 2. Experiment Design Frameworks
+
+Complete workflow documentation in `references/experiment_design_frameworks.md` including:
+
+- Step-by-step processes
+- Architecture design patterns
+- Tool integration guides
+- Performance tuning strategies
+- Troubleshooting procedures
+
+### 3. Feature Engineering Patterns
+
+Technical reference guide in `references/feature_engineering_patterns.md` with:
+
+- System design principles
+- Implementation examples
+- Configuration best practices
+- Deployment strategies
+- Monitoring and observability
+
+## Production Patterns
+
+### Pattern 1: Scalable Data Processing
+
+Enterprise-scale data processing with distributed computing:
+
+- Horizontal scaling architecture
+- Fault-tolerant design
+- Real-time and batch processing
+- Data quality validation
+- Performance monitoring
+
+### Pattern 2: ML Model Deployment
+
+Production ML system with high availability:
+
+- Model serving with low latency
+- A/B testing infrastructure
+- Feature store integration
+- Model monitoring and drift detection
+- Automated retraining pipelines
+
+### Pattern 3: Real-Time Inference
+
+High-throughput inference system:
+
+- Batching and caching strategies
+- Load balancing
+- Auto-scaling
+- Latency optimization
+- Cost optimization
+
+## Best Practices
+
+### Development
+
+- Test-driven development
+- Code reviews and pair programming
+- Documentation as code
+- Version control everything
+- Continuous integration
+
+### Production
+
+- Monitor everything critical
+- Automate deployments
+- Feature flags for releases
+- Canary deployments
+- Comprehensive logging
+
+### Team Leadership
+
+- Mentor junior engineers
+- Drive technical decisions
+- Establish coding standards
+- Foster learning culture
+- Cross-functional collaboration
+
+## Performance Targets
+
+**Latency:**
+- P50: < 50ms
+- P95: < 100ms
+- P99: < 200ms
+
+**Throughput:**
+- Requests/second: > 1000
+- Concurrent users: > 10,000
+
+**Availability:**
+- Uptime: 99.9%
+- Error rate: < 0.1%
+
+## Security & Compliance
+
+- Authentication & authorization
+- Data encryption (at rest & in transit)
+- PII handling and anonymization
+- GDPR/CCPA compliance
+- Regular security audits
+- Vulnerability management
+
+## Common Commands
+
+```bash
+# Development
+python -m pytest tests/ -v --cov
+python -m black src/
+python -m pylint src/
+
+# Training
+python3 ~/.{TOOL}/skills/senior-data-scientist/scripts/train.py --config prod.yaml
+python3 ~/.{TOOL}/skills/senior-data-scientist/scripts/evaluate.py --model best.pth
+
+# Deployment
+docker build -t service:v1 .
+kubectl apply -f k8s/
+helm upgrade service ./charts/
+
+# Monitoring
+kubectl logs -f deployment/service
+python3 ~/.{TOOL}/skills/senior-data-scientist/scripts/health_check.py
+```
+
+## Resources
+
+- Advanced Patterns: `references/statistical_methods_advanced.md`
+- Implementation Guide: `references/experiment_design_frameworks.md`
+- Technical Reference: `references/feature_engineering_patterns.md`
+- Automation Scripts: `scripts/` directory
+
+## Senior-Level Responsibilities
+
+As a world-class senior professional:
+
+1. **Technical Leadership**
+   - Drive architectural decisions
+   - Mentor team members
+   - Establish best practices
+   - Ensure code quality
+
+2. **Strategic Thinking**
+   - Align with business goals
+   - Evaluate trade-offs
+   - Plan for scale
+   - Manage technical debt
+
+3. **Collaboration**
+   - Work across teams
+   - Communicate effectively
+   - Build consensus
+   - Share knowledge
+
+4. **Innovation**
+   - Stay current with research
+   - Experiment with new approaches
+   - Contribute to community
+   - Drive continuous improvement
+
+5. **Production Excellence**
+   - Ensure high availability
+   - Monitor proactively
+   - Optimize performance
+   - Respond to incidents

package/merged-commands/senior-devops.md

@@ -0,0 +1,209 @@
+---
+name: senior-devops
+description: Comprehensive DevOps skill for CI/CD, infrastructure automation, containerization, and cloud platforms (AWS, GCP, Azure). Includes pipeline setup, infrastructure as code, deployment automation, and monitoring. Use when setting up pipelines, deploying applications, managing infrastructure, implementing monitoring, or optimizing deployment processes.
+---
+
+# Senior Devops
+
+Complete toolkit for senior devops with modern tools and best practices.
+
+## Quick Start
+
+### Main Capabilities
+
+This skill provides three core capabilities through automated scripts:
+
+```bash
+# Script 1: Pipeline Generator
+python3 ~/.{TOOL}/skills/senior-devops/scripts/pipeline_generator.py [options]
+
+# Script 2: Terraform Scaffolder
+python3 ~/.{TOOL}/skills/senior-devops/scripts/terraform_scaffolder.py [options]
+
+# Script 3: Deployment Manager
+python3 ~/.{TOOL}/skills/senior-devops/scripts/deployment_manager.py [options]
+```
+
+## Core Capabilities
+
+### 1. Pipeline Generator
+
+Automated tool for pipeline generator tasks.
+
+**Features:**
+- Automated scaffolding
+- Best practices built-in
+- Configurable templates
+- Quality checks
+
+**Usage:**
+```bash
+python3 ~/.{TOOL}/skills/senior-devops/scripts/pipeline_generator.py <project-path> [options]
+```
+
+### 2. Terraform Scaffolder
+
+Comprehensive analysis and optimization tool.
+
+**Features:**
+- Deep analysis
+- Performance metrics
+- Recommendations
+- Automated fixes
+
+**Usage:**
+```bash
+python3 ~/.{TOOL}/skills/senior-devops/scripts/terraform_scaffolder.py <target-path> [--verbose]
+```
+
+### 3. Deployment Manager
+
+Advanced tooling for specialized tasks.
+
+**Features:**
+- Expert-level automation
+- Custom configurations
+- Integration ready
+- Production-grade output
+
+**Usage:**
+```bash
+python3 ~/.{TOOL}/skills/senior-devops/scripts/deployment_manager.py [arguments] [options]
+```
+
+## Reference Documentation
+
+### Cicd Pipeline Guide
+
+Comprehensive guide available in `references/cicd_pipeline_guide.md`:
+
+- Detailed patterns and practices
+- Code examples
+- Best practices
+- Anti-patterns to avoid
+- Real-world scenarios
+
+### Infrastructure As Code
+
+Complete workflow documentation in `references/infrastructure_as_code.md`:
+
+- Step-by-step processes
+- Optimization strategies
+- Tool integrations
+- Performance tuning
+- Troubleshooting guide
+
+### Deployment Strategies
+
+Technical reference guide in `references/deployment_strategies.md`:
+
+- Technology stack details
+- Configuration examples
+- Integration patterns
+- Security considerations
+- Scalability guidelines
+
+## Tech Stack
+
+**Languages:** TypeScript, JavaScript, Python, Go, Swift, Kotlin
+**Frontend:** React, Next.js, React Native, Flutter
+**Backend:** Node.js, Express, GraphQL, REST APIs
+**Database:** PostgreSQL, Prisma, NeonDB, Supabase
+**DevOps:** Docker, Kubernetes, Terraform, GitHub Actions, CircleCI
+**Cloud:** AWS, GCP, Azure
+
+## Development Workflow
+
+### 1. Setup and Configuration
+
+```bash
+# Install dependencies
+npm install
+# or
+pip install -r requirements.txt
+
+# Configure environment
+cp .env.example .env
+```
+
+### 2. Run Quality Checks
+
+```bash
+# Use the analyzer script
+python3 ~/.{TOOL}/skills/senior-devops/scripts/terraform_scaffolder.py .
+
+# Review recommendations
+# Apply fixes
+```
+
+### 3. Implement Best Practices
+
+Follow the patterns and practices documented in:
+- `references/cicd_pipeline_guide.md`
+- `references/infrastructure_as_code.md`
+- `references/deployment_strategies.md`
+
+## Best Practices Summary
+
+### Code Quality
+- Follow established patterns
+- Write comprehensive tests
+- Document decisions
+- Review regularly
+
+### Performance
+- Measure before optimizing
+- Use appropriate caching
+- Optimize critical paths
+- Monitor in production
+
+### Security
+- Validate all inputs
+- Use parameterized queries
+- Implement proper authentication
+- Keep dependencies updated
+
+### Maintainability
+- Write clear code
+- Use consistent naming
+- Add helpful comments
+- Keep it simple
+
+## Common Commands
+
+```bash
+# Development
+npm run dev
+npm run build
+npm run test
+npm run lint
+
+# Analysis
+python3 ~/.{TOOL}/skills/senior-devops/scripts/terraform_scaffolder.py .
+python3 ~/.{TOOL}/skills/senior-devops/scripts/deployment_manager.py --analyze
+
+# Deployment
+docker build -t app:latest .
+docker-compose up -d
+kubectl apply -f k8s/
+```
+
+## Troubleshooting
+
+### Common Issues
+
+Check the comprehensive troubleshooting section in `references/deployment_strategies.md`.
+
+### Getting Help
+
+- Review reference documentation
+- Check script output messages
+- Consult tech stack documentation
+- Review error logs
+
+## Resources
+
+- Pattern Reference: `references/cicd_pipeline_guide.md`
+- Workflow Guide: `references/infrastructure_as_code.md`
+- Technical Guide: `references/deployment_strategies.md`
+- Tool Scripts: `scripts/` directory