agentic-team-templates 0.3.0

package/templates/ml-ai/.cursorrules/overview.md

@@ -0,0 +1,131 @@
+# ML/AI Development Overview
+
+Staff-level guidelines for machine learning and artificial intelligence systems, covering the full ML lifecycle from data engineering to production monitoring.
+
+## Scope
+
+This template applies to:
+
+- Machine learning pipelines (training, evaluation, deployment)
+- Deep learning systems (computer vision, NLP, recommendation systems)
+- MLOps infrastructure (experiment tracking, feature stores, model registries)
+- LLM/GenAI applications (fine-tuning, RAG, prompt engineering)
+- Real-time and batch inference systems
+
+## Core Principles
+
+### 1. Data-Centric Development
+
+Data quality beats algorithm complexity. Most model improvements come from better data, not fancier architectures.
+
+- Validate data at every pipeline boundary
+- Version datasets alongside code
+- Invest in data labeling quality
+- Monitor for data drift continuously
+
+### 2. Reproducibility Is Non-Negotiable
+
+Every experiment must be reproducible. Version everything: data, code, configurations, models, and environments.
+
+- Pin all dependency versions
+- Log random seeds and hyperparameters
+- Use deterministic operations where possible
+- Track data lineage from source to prediction
+
+### 3. Observability Over Uptime
+
+Traditional monitoring (CPU, memory, latency) isn't enough. Models can silently degrade while infrastructure stays green.
+
+- Monitor prediction distributions
+- Detect data and concept drift
+- Track business metrics, not just ML metrics
+- Set up alerts for model performance degradation
+
+### 4. Responsible AI By Default
+
+Fairness, bias detection, and explainability are not afterthoughts—they're requirements.
+
+- Assess fairness across protected groups
+- Document model limitations
+- Provide explanations for predictions
+- Test for adversarial robustness
+
+## Project Structure
+
+```
+ml-project/
+├── data/
+│   ├── raw/                    # Immutable raw data
+│   ├── processed/              # Cleaned, transformed data
+│   └── features/               # Feature store exports
+├── src/
+│   ├── data/                   # Data loading and validation
+│   ├── features/               # Feature engineering
+│   ├── models/                 # Model definitions
+│   ├── training/               # Training logic
+│   ├── evaluation/             # Metrics and analysis
+│   ├── inference/              # Serving code
+│   └── utils/                  # Shared utilities
+├── configs/                    # Experiment configurations
+├── notebooks/                  # Exploration (not production)
+├── tests/
+│   ├── unit/                   # Unit tests
+│   ├── integration/            # Integration tests
+│   └── model/                  # Model behavior tests
+├── pipelines/                  # ML pipeline definitions
+└── deployments/                # Kubernetes/serving configs
+```
+
+## Technology Stack
+
+### Training & Experimentation
+
+- **Frameworks**: PyTorch, TensorFlow, scikit-learn, XGBoost
+- **Experiment Tracking**: MLflow, Weights & Biases, Neptune
+- **Hyperparameter Optimization**: Optuna, Ray Tune
+
+### Data & Features
+
+- **Feature Stores**: Feast, Tecton, Hopsworks
+- **Data Validation**: TensorFlow Data Validation, Great Expectations, Pandera
+- **Data Versioning**: DVC, LakeFS
+
+### Deployment & Serving
+
+- **Model Serving**: KServe, TorchServe, Triton Inference Server, vLLM
+- **Containerization**: Docker, Kubernetes
+- **Orchestration**: Kubeflow, Airflow, Prefect, Dagster
+
+### Monitoring & Observability
+
+- **Drift Detection**: Evidently, WhyLabs, Arize
+- **Logging**: Structured logging with MLflow
+- **Metrics**: Prometheus, custom business metrics
+
+## Definition of Done (ML Model)
+
+A machine learning model is production-ready when:
+
+- [ ] Data validation schemas defined and enforced
+- [ ] Feature engineering code tested and versioned
+- [ ] Experiment tracked with all parameters and metrics
+- [ ] Model evaluated on multiple metrics (not just accuracy)
+- [ ] Fairness assessed across protected groups
+- [ ] Model registered with signature and artifacts
+- [ ] Inference endpoint tested under load
+- [ ] Drift detection configured
+- [ ] Monitoring dashboards created
+- [ ] Rollback procedure documented
+- [ ] All tests passing (unit, model, integration)
+
+## Definition of Done (ML Pipeline)
+
+An ML pipeline is production-ready when:
+
+- [ ] Each stage has clear inputs and outputs
+- [ ] Data validation at pipeline boundaries
+- [ ] Idempotent and retryable stages
+- [ ] Failure alerts configured
+- [ ] Logging captures lineage
+- [ ] Pipeline tested end-to-end
+- [ ] Documentation complete