@zigrivers/scaffold 3.8.0 → 3.9.1

package/content/knowledge/ml/ml-observability.md

@@ -0,0 +1,253 @@
+---
+name: ml-observability
+description: Model monitoring for drift and decay, prediction logging, explainability tools, and alerting on accuracy drops in production ML systems
+topics: [ml, observability, monitoring, drift, model-decay, explainability, alerting, prediction-logging]
+---
+
+A model deployed to production without monitoring is a ticking clock. Models decay silently: the world changes, input distributions shift, and accuracy degrades while dashboards show green. Unlike software bugs that throw exceptions, model degradation has no stack trace — predictions simply become less useful. ML observability is the discipline of detecting these degradations before users notice them, through systematic monitoring of model inputs, outputs, and outcomes.
+
+## Summary
+
+ML observability covers four pillars: input monitoring (feature drift detection), output monitoring (prediction distribution shifts), outcome monitoring (accuracy against labels), and operational monitoring (latency, error rate). Complement monitoring with prediction logging for post-hoc analysis and explainability tools (SHAP, LIME) for understanding individual predictions and debugging systematic failures. Alert thresholds and on-call rotation for model health are as important as for service health.
+
+## Deep Guidance
+
+### The Four Pillars of ML Observability
+
+**Pillar 1 — Input monitoring (data drift)**: Detect when the distribution of model inputs changes from the training distribution. A model trained on winter data receiving summer data will degrade without any software change.
+
+**Pillar 2 — Output monitoring (prediction drift)**: Detect when the model's prediction distribution changes — e.g., a fraud model that suddenly classifies 10% of transactions as fraud (vs. the baseline 0.1%).
+
+**Pillar 3 — Outcome monitoring (accuracy/concept drift)**: Detect when model accuracy changes on labelled outcomes. Requires ground truth labels, which often arrive with delay (e.g., actual fraud confirmed days after prediction).
+
+**Pillar 4 — Operational monitoring**: Latency, throughput, error rate, memory usage. Standard SRE metrics applied to the model serving layer.
+
+### Feature Drift Detection
+
+Measure drift between training and serving feature distributions using statistical tests:
+
+```python
+from scipy import stats
+import numpy as np
+from dataclasses import dataclass
+from typing import Optional
+
+@dataclass
+class DriftReport:
+    feature: str
+    psi: float           # Population Stability Index
+    ks_statistic: float  # Kolmogorov-Smirnov statistic
+    ks_p_value: float
+    is_drifted: bool
+
+def compute_psi(
+    expected: np.ndarray,
+    actual: np.ndarray,
+    buckets: int = 10,
+) -> float:
+    """Population Stability Index. PSI < 0.1: stable, 0.1-0.2: minor drift, >0.2: significant drift."""
+    eps = 1e-6
+    expected_pcts, bins = np.histogram(expected, bins=buckets)
+    actual_pcts, _ = np.histogram(actual, bins=bins)
+
+    expected_pcts = expected_pcts / expected_pcts.sum() + eps
+    actual_pcts = actual_pcts / actual_pcts.sum() + eps
+
+    return float(np.sum((actual_pcts - expected_pcts) * np.log(actual_pcts / expected_pcts)))
+
+def detect_drift(
+    training_values: np.ndarray,
+    serving_values: np.ndarray,
+    feature_name: str,
+    psi_threshold: float = 0.2,
+    ks_alpha: float = 0.05,
+) -> DriftReport:
+    psi = compute_psi(training_values, serving_values)
+    ks_stat, ks_p = stats.ks_2samp(training_values, serving_values)
+    return DriftReport(
+        feature=feature_name,
+        psi=psi,
+        ks_statistic=ks_stat,
+        ks_p_value=ks_p,
+        is_drifted=psi > psi_threshold or ks_p < ks_alpha,
+    )
+```
+
+**Reference distribution maintenance**: Store feature statistics (mean, std, percentiles, histogram) from the training set as a "reference profile." Compare each day's serving data to this profile. Refresh the reference when the model is retrained.
+
+**PSI thresholds** (industry standard):
+- PSI < 0.1: No significant drift — monitor as normal
+- 0.1 ≤ PSI < 0.2: Minor drift — investigate, consider retraining
+- PSI ≥ 0.2: Significant drift — trigger retraining or alert
+
+### Prediction Logging
+
+Every prediction made in production should be logged for monitoring and post-hoc analysis:
+
+```python
+# src/serving/prediction_logger.py
+import json
+import time
+from dataclasses import dataclass, asdict
+from typing import Any
+
+@dataclass
+class PredictionRecord:
+    prediction_id: str       # UUID for correlation
+    model_version: str
+    timestamp: float
+    request_id: str          # Trace ID for distributed tracing
+    input_features: dict     # Logged features (scrub PII before logging)
+    prediction: Any
+    confidence: float
+    latency_ms: float
+
+class PredictionLogger:
+    def __init__(self, sink):  # sink: Kafka producer, Kinesis, or file
+        self.sink = sink
+
+    def log(self, record: PredictionRecord) -> None:
+        payload = json.dumps(asdict(record))
+        self.sink.send(payload)
+```
+
+**What to log** (balance observability with privacy/cost):
+- Always: prediction ID, model version, timestamp, prediction value, confidence, latency
+- Feature logging: Log features used for prediction (important for drift detection and debugging)
+- PII scrubbing: Never log raw PII fields; log derived features or anonymised values only
+- Sampling: For very high-throughput systems (> 10K RPS), log a representative sample (1–10%)
+
+**Label joining**: When ground truth labels arrive (delayed), join them with prediction logs using the prediction ID to compute accuracy metrics:
+```sql
+SELECT
+    p.model_version,
+    COUNT(*) as n_predictions,
+    AVG(CASE WHEN p.prediction = l.actual_label THEN 1 ELSE 0 END) as accuracy,
+    AVG(p.confidence) as mean_confidence
+FROM predictions p
+JOIN labels l ON p.prediction_id = l.prediction_id
+WHERE p.timestamp >= NOW() - INTERVAL '7 days'
+GROUP BY p.model_version
+```
+
+### Explainability
+
+Explainability tools help debug model failures and satisfy regulatory requirements:
+
+**SHAP (SHapley Additive exPlanations)**: Computes feature importance for individual predictions using game-theoretic Shapley values. Works with any model.
+
+```python
+import shap
+
+# Train a background dataset for the explainer
+background = X_train[np.random.choice(len(X_train), 100, replace=False)]
+explainer = shap.TreeExplainer(model)  # For tree models
+# explainer = shap.DeepExplainer(model, background)  # For neural networks
+# explainer = shap.KernelExplainer(model.predict_proba, background)  # Model-agnostic
+
+# Explain a single prediction
+shap_values = explainer.shap_values(X_test[0:1])
+shap.force_plot(explainer.expected_value[1], shap_values[1][0], X_test[0])
+
+# Explain the entire test set (global feature importance)
+shap_values_all = explainer.shap_values(X_test)
+shap.summary_plot(shap_values_all[1], X_test)
+```
+
+**LIME (Local Interpretable Model-agnostic Explanations)**: Fits a simple interpretable model (linear regression) locally around each prediction.
+
+```python
+from lime.lime_tabular import LimeTabularExplainer
+
+explainer = LimeTabularExplainer(
+    X_train,
+    feature_names=feature_names,
+    class_names=["legitimate", "fraud"],
+    mode="classification",
+)
+
+explanation = explainer.explain_instance(
+    X_test[0],
+    model.predict_proba,
+    num_features=10,
+)
+explanation.show_in_notebook()
+```
+
+**Integrated Gradients** (for neural networks): Attribution method that satisfies axiomatic completeness. Available in Captum (PyTorch):
+```python
+from captum.attr import IntegratedGradients
+
+ig = IntegratedGradients(model)
+attributions = ig.attribute(input_tensor, baseline=torch.zeros_like(input_tensor))
+```
+
+### Alerting Strategy
+
+Define alert thresholds before deployment, not after a production incident:
+
+```yaml
+# monitoring/alerts.yaml
+alerts:
+  - name: accuracy_degradation_warning
+    metric: val_accuracy_7d_rolling
+    condition: "< 0.87"  # Warning: 2pp below target
+    severity: warning
+    action: page_on_call
+
+  - name: accuracy_degradation_critical
+    metric: val_accuracy_7d_rolling
+    condition: "< 0.85"  # Critical: at SLA threshold
+    severity: critical
+    action: page_on_call_and_escalate
+
+  - name: feature_drift_significant
+    metric: max_psi_across_features
+    condition: "> 0.2"
+    severity: warning
+    action: notify_ml_team
+
+  - name: prediction_rate_anomaly
+    metric: fraud_prediction_rate_1h
+    condition: "> 0.05"  # 5x normal rate
+    severity: critical
+    action: page_on_call
+
+  - name: serving_latency_breach
+    metric: p99_latency_ms
+    condition: "> 200"
+    severity: warning
+    action: notify_ml_team
+```
+
+**Alerting anti-patterns**:
+- Alert fatigue: Too many low-signal alerts causes teams to ignore them. Start with critical-only, add warnings after establishing baselines.
+- Static thresholds for seasonal data: Use rolling baselines that adapt to weekly/seasonal patterns.
+- No runbook: Every alert must have a runbook link: "When this fires, do X, check Y, escalate to Z."
+
+### Model Monitoring Dashboard
+
+A model health dashboard should show at a glance:
+
+```
+Model: fraud-detector v2.3.1  |  Status: HEALTHY  |  Updated: 5 minutes ago
+
+┌─────────────────┬──────────────────┬──────────────────┐
+│ Accuracy (7d)   │ Prediction Rate  │ P99 Latency      │
+│   87.3%   ✓     │   0.12%    ✓     │   142ms    ✓     │
+│ target: ≥85%   │ baseline: 0.1%  │ SLA: <200ms     │
+└─────────────────┴──────────────────┴──────────────────┘
+
+┌──────────────────────────────────────────────────────┐
+│ Feature Drift (PSI)                                  │
+│ transaction_amount: 0.08  ✓                          │
+│ merchant_category:  0.12  ⚠ (minor drift)            │
+│ user_age_days:      0.04  ✓                          │
+└──────────────────────────────────────────────────────┘
+```
+
+Retraining triggers: Codify when to retrain rather than leaving it to human judgment:
+- Accuracy drops below warning threshold for 48+ consecutive hours
+- PSI > 0.2 on any top-10 feature by SHAP importance
+- Major upstream data source change (schema change, new data source)
+- Scheduled retraining on a fixed cadence (monthly for most models)

package/content/knowledge/ml/ml-project-structure.md

@@ -0,0 +1,216 @@
+---
+name: ml-project-structure
+description: Standard ML project directory layout covering src/data, src/models, src/training, src/serving, notebooks, configs, and model artifact storage
+topics: [ml, project-structure, layout, organization, artifacts, notebooks]
+---
+
+ML projects accumulate files faster than almost any other software domain: datasets, model checkpoints, experiment configs, notebooks, evaluation reports, and serving code. Without a deliberate directory structure, projects become disorganised within weeks and impossible to onboard new team members onto. A well-structured ML project separates concerns clearly: source code from notebooks, training from serving, configs from code, and tracked artifacts from ephemeral outputs.
+
+## Summary
+
+A standard ML project separates source code (`src/`), exploratory notebooks (`notebooks/`), training configurations (`configs/`), and model artifacts (`models/`). Within `src/`, separate data loading (`src/data/`), model architectures (`src/models/`), training logic (`src/training/`), and serving code (`src/serving/`). Keep large artifacts (datasets, checkpoints) out of git using `.gitignore` and DVC or object storage. The structure should be navigable to a new team member within five minutes.
+
+## Deep Guidance
+
+### Top-Level Directory Structure
+
+```
+project-root/
+├── configs/            # All experiment and model configs (YAML/TOML)
+├── data/               # Data directory (gitignored content, DVC-tracked)
+│   ├── raw/            # Immutable raw data as received from source
+│   ├── processed/      # Cleaned, transformed datasets
+│   └── splits/         # Train/val/test split files (CSV/JSON of IDs)
+├── docs/               # Architecture decisions, dataset cards, model cards
+├── models/             # Model artifact storage (gitignored; object storage backed)
+│   ├── checkpoints/    # Training checkpoints (epoch-N.pt)
+│   └── registry/       # Production-promoted model versions
+├── notebooks/          # Jupyter notebooks for exploration (outputs cleared before commit)
+├── reports/            # Evaluation reports, figures, experiment summaries
+├── scripts/            # One-off utility scripts (not part of the pipeline)
+├── src/                # All production source code
+│   ├── data/           # Dataset classes, loaders, preprocessing
+│   ├── models/         # Model architecture definitions
+│   ├── training/       # Training loops, loss functions, callbacks
+│   ├── evaluation/     # Metrics, evaluation runners, result serialisation
+│   └── serving/        # Inference pipelines, API handlers, preprocessing wrappers
+├── tests/              # Unit and integration tests
+├── .dvc/               # DVC metadata (committed to git)
+├── .gitignore          # Excludes data/, models/, __pycache__, .env
+├── pyproject.toml      # Project metadata and dependencies (Poetry)
+├── Makefile            # Task runner: train, evaluate, serve, test
+└── README.md           # Project overview, setup, and usage
+```
+
+### `src/data/` — Data Loading and Preprocessing
+
+This directory contains all code that transforms raw data into model-ready tensors:
+
+```
+src/data/
+├── __init__.py
+├── dataset.py          # PyTorch Dataset or TF Dataset class
+├── datamodule.py       # LightningDataModule or equivalent orchestrator
+├── transforms.py       # Preprocessing transforms (normalize, tokenize, augment)
+├── augmentation.py     # Training-time data augmentation (separated from eval transforms)
+├── collate.py          # Custom batch collation functions
+└── utils.py            # Data utilities (download, checksum, split generation)
+```
+
+Key rules:
+- **Separate training transforms from eval transforms** — augmentation must not be applied at inference
+- Dataset classes must accept a `split` parameter and behave correctly for each split
+- All preprocessing must be reproducible and deterministic at inference time
+- Cache processed data to avoid recomputation on each run
+
+### `src/models/` — Architecture Definitions
+
+Contains model class definitions only — no training logic, no loss functions:
+
+```
+src/models/
+├── __init__.py
+├── backbone.py         # Feature extractor (ResNet, ViT, BERT, etc.)
+├── head.py             # Task-specific head (classification, regression, generation)
+├── model.py            # Composed full model
+└── components/         # Reusable building blocks (attention, MLP, norm layers)
+    ├── attention.py
+    └── ffn.py
+```
+
+Key rules:
+- Models are pure computation graphs — no file I/O, no training state
+- Accept hyperparameters via constructor, not globals
+- Provide a `from_config(cfg)` class method for config-driven instantiation
+- Serialise with `state_dict()` only — never pickle entire model objects
+
+### `src/training/` — Training Logic
+
+```
+src/training/
+├── __init__.py
+├── trainer.py          # Training loop (or LightningModule)
+├── loss.py             # Loss functions
+├── optimizer.py        # Optimizer and scheduler builders
+├── callbacks.py        # Callbacks (early stopping, logging, checkpoint saving)
+└── utils.py            # Gradient clipping, mixed precision helpers
+```
+
+The training loop is separate from the model. A model knows how to compute predictions; the trainer knows how to update weights. This separation enables:
+- Testing model forward passes independently of training
+- Swapping training strategies (single GPU, DDP, FSDP) without changing the model
+- Using the same model class for training and serving
+
+### `src/evaluation/` — Metrics and Evaluation Runners
+
+```
+src/evaluation/
+├── __init__.py
+├── metrics.py          # Metric computation (accuracy, F1, AUC, etc.)
+├── evaluator.py        # Evaluation loop (runs model on eval set, collects predictions)
+├── slice_analysis.py   # Per-slice performance breakdown
+└── reports.py          # Result serialisation and report generation
+```
+
+Evaluation code runs identically offline and online. Do not inline evaluation logic in the training loop — this makes it impossible to re-evaluate a checkpoint independently.
+
+### `src/serving/` — Inference and API
+
+```
+src/serving/
+├── __init__.py
+├── predictor.py        # Prediction class (loads model, runs inference)
+├── preprocessing.py    # Request preprocessing (mirrors training eval transforms)
+├── postprocessing.py   # Response postprocessing (calibration, thresholding)
+├── api.py              # FastAPI/Flask endpoint definitions
+└── handler.py          # TorchServe or Triton handler
+```
+
+The `Predictor` class is the contract between the model and the serving infrastructure. It:
+- Loads a model from a path or registry reference
+- Exposes a `predict(inputs)` method with documented input/output types
+- Uses the exact same preprocessing as training evaluation transforms
+
+### `notebooks/` — Exploratory Analysis
+
+```
+notebooks/
+├── 01-data-exploration.ipynb   # EDA, data quality checks
+├── 02-baseline-model.ipynb     # Baseline experiments
+├── 03-feature-engineering.ipynb
+└── 04-error-analysis.ipynb     # Post-training error analysis
+```
+
+Rules for notebooks:
+- **Clear outputs before committing** — use `nbstripout` as a pre-commit hook
+- Number notebooks in chronological/logical order
+- Notebooks document exploration, not production logic
+- Any reusable code found in notebooks gets refactored into `src/` with tests
+
+### `configs/` — Experiment Configuration
+
+```
+configs/
+├── base.yaml               # Default config merged into all experiments
+├── model/
+│   ├── small.yaml
+│   └── large.yaml
+├── data/
+│   ├── dev.yaml            # Small dataset for fast iteration
+│   └── full.yaml           # Full production dataset
+└── training/
+    ├── debug.yaml          # 1 epoch, no logging, fast feedback
+    └── production.yaml     # Full training run settings
+```
+
+### `models/` — Artifact Storage
+
+Large binary artifacts are not stored in git:
+- Checkpoints and production models live in `models/` but are gitignored
+- Back `models/` with object storage: S3, GCS, Azure Blob Storage
+- Use DVC to track artifact versions alongside the code:
+
+```bash
+dvc add models/registry/v1.2.0/model.pt
+git add models/registry/v1.2.0/model.pt.dvc
+git commit -m "feat: register model v1.2.0"
+dvc push  # Pushes binary to remote storage
+```
+
+Teammates restore the artifact with `dvc pull` — they get the exact binary referenced by the `.dvc` pointer in git.
+
+### `.gitignore` Essentials for ML Projects
+
+```gitignore
+# Data
+data/raw/
+data/processed/
+data/splits/*.csv
+
+# Model artifacts
+models/checkpoints/
+models/registry/
+
+# Notebook outputs
+*.ipynb
+
+# Python
+__pycache__/
+*.pyc
+.venv/
+*.egg-info/
+
+# Experiment tracking
+mlruns/
+wandb/
+
+# Environment
+.env
+*.env
+```
+
+Use `nbstripout` to automatically strip notebook outputs:
+```bash
+pip install nbstripout
+nbstripout --install  # Installs as git filter
+```

package/content/knowledge/ml/ml-requirements.md

@@ -0,0 +1,138 @@
+---
+name: ml-requirements
+description: Model performance metrics (accuracy, latency, throughput), business KPIs, fairness/bias requirements, and SLA definitions for ML systems
+topics: [ml, requirements, metrics, fairness, bias, sla, kpi]
+---
+
+ML requirements differ from traditional software requirements because correctness is probabilistic, not absolute. Before writing a single line of training code, define the target metrics, their measurement methodology, and the business KPIs they serve. Ambiguous requirements — "make the model accurate" — are the root cause of most ML project failures. A requirements document for an ML system must specify numeric thresholds, measurement conditions, and what constitutes an acceptable production deployment.
+
+## Summary
+
+ML requirements must specify concrete numeric thresholds for model performance (accuracy, latency, throughput), tie those metrics to business KPIs, define fairness and bias constraints across protected groups, and establish SLAs for production serving. Requirements without measurement methodology are aspirations, not requirements. Capture them in a Model Requirements Document before training begins and treat them as acceptance criteria for production deployment.
+
+## Deep Guidance
+
+### Performance Metrics by Task Type
+
+Different ML tasks have canonical metrics. Use the right metric for the task — do not default to accuracy for all problems:
+
+**Classification**
+- **Accuracy**: Fraction of correct predictions. Misleading for class-imbalanced datasets (a fraud detector predicting "not fraud" always achieves 99.9% accuracy if fraud is 0.1% of data).
+- **Precision**: Of all positive predictions, how many are actually positive. Optimise when false positives are costly (spam filter flagging legitimate email).
+- **Recall (Sensitivity)**: Of all actual positives, how many were predicted positive. Optimise when false negatives are costly (cancer screening missing a case).
+- **F1 Score**: Harmonic mean of precision and recall. Good single metric when both matter equally.
+- **ROC-AUC**: Area under the Receiver Operating Characteristic curve. Threshold-independent, useful for comparing models. Insensitive to class imbalance.
+- **PR-AUC**: Area under the Precision-Recall curve. Better than ROC-AUC for highly imbalanced datasets.
+
+**Regression**
+- **MAE (Mean Absolute Error)**: Average absolute error. Robust to outliers. Easy to interpret in the target unit.
+- **RMSE (Root Mean Squared Error)**: Penalises large errors more than MAE. Use when large errors are disproportionately harmful.
+- **MAPE (Mean Absolute Percentage Error)**: Scale-independent. Problematic when targets near zero.
+- **R² (Coefficient of Determination)**: Variance explained by the model. Context-dependent — R²=0.9 may be poor for weather forecasting but excellent for pricing.
+
+**Ranking / Recommendation**
+- **NDCG (Normalized Discounted Cumulative Gain)**: Relevance-weighted ranking quality. Standard for search and recommendation.
+- **MRR (Mean Reciprocal Rank)**: Average of 1/rank of first relevant result.
+- **Hit Rate @ K**: Fraction of users for whom a relevant item appears in top-K recommendations.
+
+**Generation (LLM/NLG)**
+- **BLEU / ROUGE**: Reference-based n-gram overlap. Weak proxy for quality — supplement with human evaluation.
+- **Perplexity**: Model confidence on a held-out corpus. Lower is better; useful for comparing language models.
+- **Human evaluation**: Win rate against baseline, Likert scale ratings. Required for production quality gating.
+
+### Business KPI Alignment
+
+Every model metric must map to a business KPI. Without this mapping, teams optimise metrics that do not move the needle:
+
+| Model Metric | Business KPI | Notes |
+|---|---|---|
+| Fraud detection recall | Revenue protected from fraud | 1% recall improvement may not justify infra cost |
+| Recommendation CTR | Gross Merchandise Value | CTR can rise while GMV falls (clicks on cheap items) |
+| Search NDCG | Query success rate, conversion | Offline NDCG and online conversion often diverge |
+| Churn prediction AUC | Customer retention rate | Model accuracy gap vs. treatment effectiveness |
+
+Document this mapping explicitly. When offline metrics improve but the business metric does not, the mapping is wrong.
+
+### Latency Requirements
+
+Latency requirements are determined by the use case, not the model team's preferences:
+
+- **Interactive / real-time**: User-facing features require P95 latency under 100ms. P99 under 500ms. Recommendation, search, and content ranking fall here.
+- **Near-real-time**: Fraud detection at checkout tolerates 200–500ms P95.
+- **Batch / async**: Offline scoring pipelines have no strict latency requirements but throughput requirements (e.g., score 10M records in 4 hours).
+
+Define latency budgets from the user experience backward:
+1. Total page load budget: 2000ms
+2. Backend API budget: 500ms
+3. ML inference budget: 100ms (within the API budget)
+4. Model must fit within that budget at P99 under peak load
+
+**Throughput requirements** are independent of latency: "The model must score 50,000 requests per second at peak." Throughput is met by horizontal scaling; latency is met by model optimisation (quantisation, distillation, hardware selection).
+
+### Fairness and Bias Requirements
+
+Fairness requirements must be defined before training, not audited afterward:
+
+**Protected attributes**: Race, gender, age, disability status, national origin, religion. Model inputs should not include protected attributes directly; proxy features (zip code, name) may encode them.
+
+**Fairness metrics**:
+- **Demographic parity**: Equal positive prediction rate across groups. `P(ŷ=1 | A=0) = P(ŷ=1 | A=1)`
+- **Equalized odds**: Equal TPR and FPR across groups.
+- **Calibration parity**: Predicted probabilities match observed frequencies equally across groups.
+- **Individual fairness**: Similar individuals receive similar predictions.
+
+**Fairness-accuracy tradeoff**: Perfect fairness under multiple definitions simultaneously is mathematically impossible (Impossibility Theorem). Choose the fairness constraint that aligns with the legal and ethical context, then optimise accuracy subject to it.
+
+**Requirement format**: "Model's false positive rate for group A must not exceed the false positive rate for group B by more than 5 percentage points."
+
+### Model Monitoring SLAs
+
+Define monitoring SLAs as part of requirements:
+
+- **Accuracy SLA**: "Model accuracy must remain above 85% on the weekly validation set. Alert if it drops below 87% (warning threshold) or 85% (critical threshold)."
+- **Drift SLA**: "Input feature distribution shift (PSI > 0.2) triggers model retraining within 48 hours."
+- **Prediction latency SLA**: "P99 inference latency must remain under 200ms. Alert at 150ms."
+- **Availability SLA**: "Model serving endpoint must maintain 99.9% uptime (43 minutes downtime/month)."
+
+### Model Requirements Document Template
+
+```markdown
+# Model Requirements: [Model Name]
+
+## Business Context
+- Business problem:
+- KPI being optimised:
+- KPI owner:
+
+## Performance Requirements
+- Primary metric: [metric] >= [threshold] on [evaluation set]
+- Secondary metric: [metric] >= [threshold]
+- Baseline to beat: [current rule-based / previous model performance]
+
+## Latency / Throughput
+- P50 latency: <= [X]ms
+- P99 latency: <= [X]ms
+- Throughput: >= [X] RPS at peak load
+
+## Fairness Requirements
+- Protected groups: [list]
+- Fairness metric: [metric] gap <= [threshold] across groups
+
+## Data Requirements
+- Training data: [source, size, date range, labeling methodology]
+- Minimum training set size: [N]
+- Label quality: [agreement rate, labeling error budget]
+
+## Monitoring SLAs
+- Accuracy degradation alert: < [threshold] on weekly eval
+- Feature drift alert: PSI > [threshold]
+- Retraining trigger: [condition]
+
+## Acceptance Criteria
+- [ ] Primary metric exceeds threshold on holdout set
+- [ ] Fairness constraints satisfied
+- [ ] P99 latency within budget under load test
+- [ ] No critical findings in bias audit
+```
+
+This document is the acceptance test for model deployment. If the model does not satisfy it, it does not go to production.