@zigrivers/scaffold 3.21.0 → 3.23.0

package/content/knowledge/data-science/data-science-data-versioning.md

@@ -0,0 +1,198 @@
+---
+name: data-science-data-versioning
+description: When and how to version data for reproducibility — size-based rule for choosing between git+Parquet, git-lfs, and DVC
+topics: [data-science, data-versioning, dvc, parquet, git-lfs]
+---
+
+If you can't answer "what data produced this result", you can't reproduce it.
+A model trained on 2026-02-14's snapshot will drift from one trained today,
+and without a versioning story you have no way to roll back, diff, or explain
+the difference to a reviewer six months later.
+
+Data versioning answers that question without blowing up the repo — the trick
+is picking a tool proportional to the dataset size. The common failure mode
+is over-engineering (wiring up DVC with a remote for a 40 MB CSV) or
+under-engineering (committing 2 GB of Parquet directly into git and
+discovering three months later that every clone takes twenty minutes).
+
+## Summary
+
+Pick your tool by size.
+
+- Under ~1 GB of text or Parquet, plain git with committed Parquet files is fine.
+- Between 1 and 10 GB, use `git-lfs` if you already use it on the team;
+  otherwise invest in `DVC` (Data Version Control).
+- Above 10 GB — or for any binary artifact (model weights, image corpora,
+  audio) — always use DVC with a remote (s3, gcs, azure).
+- Never version raw third-party data that you can't legally redistribute —
+  store a fetch script and a content hash instead.
+
+## Deep Guidance
+
+### Size-based decision rule
+
+| Dataset | Tool | Why |
+|---------|------|-----|
+| <1 GB text / Parquet | git + Parquet | Columnar compression keeps files small; git history stays sane |
+| 1–10 GB (judgment call) | `git-lfs` if already adopted; DVC if you have the habit | LFS is lower-effort; DVC gives you pipeline stages too |
+| >10 GB or binary artifacts | DVC with remote | Git history will not tolerate binary churn at this scale |
+| Raw third-party data | Don't version — script + hash | Redistribution is often prohibited; raw bytes bloat history |
+
+The sizes above are rules of thumb, not hard thresholds. What actually
+matters is how often the data changes. A 5 GB file that you generate once
+and never touch again can live in `git-lfs` forever without pain. The same
+5 GB file regenerated weekly will accumulate 260 GB of LFS storage in a
+year — that's the point where DVC's content-addressed cache starts to earn
+its complexity.
+
+A second factor is team shape. A solo researcher on a laptop rarely needs a
+remote backing store; a two-person team on different continents almost
+always does. Choose the tool that fits the smallest real collaboration
+pattern you have, not the one that scales to the team you imagine having.
+
+### When git + Parquet is enough
+
+For a solo or small-team project with modest data, commit processed Parquet directly. Keep raw data out of the repo; reserve git for cleaned, analysis-ready files.
+
+```python
+# src/pipelines/clean.py
+import pandas as pd
+
+df = pd.read_csv("data/raw/events.csv")  # data/raw/ is gitignored
+clean = df.dropna(subset=["user_id"]).assign(ts=pd.to_datetime(df["ts"]))
+clean.to_parquet("data/interim/events_clean.parquet", compression="zstd")
+```
+
+```gitignore
+# .gitignore
+data/raw/
+data/external/
+*.csv
+!data/interim/*.parquet   # do commit processed Parquet
+```
+
+Parquet's columnar layout and zstd compression typically shrink tabular data
+5–10x versus CSV. Diffs aren't line-level but file-level content hashes are
+stable, which is enough for "which version produced this model".
+
+Pair the committed Parquet with a short data card — a markdown file in
+`data/interim/events_clean.md` describing row count, schema, source, and the
+commit that generated it — so readers of the repo a year later can tell what
+they're looking at.
+
+### DVC basics
+
+DVC treats large files as pointers tracked in git. The real bytes live on a remote (s3/gcs/azure/ssh), and a small `.dvc` metadata file is committed.
+
+```yaml
+# dvc.yaml — pipeline stages with content-hashed inputs and outputs
+stages:
+  ingest:
+    cmd: python src/ingest.py --out data/raw/events.parquet
+    outs:
+      - data/raw/events.parquet
+  process:
+    cmd: python src/process.py --in data/raw/events.parquet --out data/processed/features.parquet
+    deps:
+      - src/process.py
+      - data/raw/events.parquet
+    outs:
+      - data/processed/features.parquet
+```
+
+Typical flow:
+
+```bash
+dvc init                              # creates .dvc/ directory
+dvc remote add -d storage s3://my-bucket/dvc-store
+dvc add data/raw/big_dataset.csv      # creates data/raw/big_dataset.csv.dvc (commit this)
+dvc repro                             # runs stages whose inputs changed
+dvc push                              # upload tracked files to remote
+git add dvc.yaml dvc.lock data/raw/big_dataset.csv.dvc .gitignore
+git commit -m "track raw events via DVC"
+```
+
+`dvc.lock` records the content hash of every stage input and output, so
+`dvc repro` on a peer's machine rebuilds exactly what you rebuilt. The
+`.dvc/` directory holds local cache and config; the actual bytes never touch
+git.
+
+Mental model: git for code, DVC for data, both pointing at the same commit.
+When you check out an older branch, git restores the source and the `.dvc`
+pointers, and `dvc checkout` pulls matching data from the remote into your
+working tree.
+
+A common starting point: track one or two heavy inputs with `dvc add` (no
+pipeline), and only adopt `dvc.yaml` stages once you have a repeatable
+multi-step workflow. The overhead of stages pays off when you have 3+ steps
+and want `dvc repro` to skip unchanged work; below that, plain `dvc add`
+plus a Makefile is often clearer.
+
+### git-lfs middle ground
+
+If you're already using Git LFS on the team but not ready to adopt DVC, it works acceptably for the 1–10 GB band — especially for a handful of files over the 100 MB GitHub push limit.
+
+```gitattributes
+# .gitattributes
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pkl     filter=lfs diff=lfs merge=lfs -text
+data/models/** filter=lfs diff=lfs merge=lfs -text
+```
+
+```bash
+git lfs install
+git lfs track "*.parquet"
+git add .gitattributes data/features.parquet
+git commit -m "add feature table via LFS"
+```
+
+Reach for `git-lfs` when files are over ~100 MB but you don't need DVC's
+pipeline stages or content-addressed reproducibility. Skip it if you already
+have DVC set up — two tools versioning the same data is a recipe for
+confusion.
+
+LFS has real drawbacks to know about: bandwidth is metered on hosted plans,
+`git clone` pulls every LFS object by default (use `GIT_LFS_SKIP_SMUDGE=1`
+to defer), and you can't selectively prune history without rewriting the
+whole repo. For a working group of 2–5 people on a research project these
+are usually tolerable; for a fleet of CI workers cloning on every build
+they are not.
+
+### What not to version
+
+- **Third-party data with license constraints** — re-commit a fetch script (`scripts/fetch_kaggle.sh`) and record the SHA256 of the pulled file in a README. Re-download on each environment.
+- **Regenerable intermediates** — if `dvc repro` or `make data` can recreate it deterministically from upstream inputs, don't commit the bytes.
+- **Scratch / exploratory outputs** — `notebooks/scratch/`, `data/tmp/`, `*.ipynb_checkpoints/` belong in `.gitignore`.
+- **Anti-pattern: committing 500 MB Parquet files directly to git** — they live forever in history, clone times balloon, and nobody will clean it up later. Move to DVC or LFS *before* the first large commit, not after. Rewriting history to extract large blobs (`git filter-repo`, BFG) is disruptive to every collaborator and should be a last resort.
+- **Anti-pattern: versioning model checkpoints in git** — a single PyTorch checkpoint can be several hundred MB, and training runs produce dozens. Push them to DVC or an artifact store (MLflow, Weights & Biases) keyed by experiment run ID.
+
+### Quick migration path
+
+If you're staring at a repo that has already committed large files to plain
+git, the order of operations is:
+
+1. Decide the target tool (DVC for most cases where you got here).
+2. Run `dvc add` on the file in its current location — this untracks it
+   from git and creates a `.dvc` pointer.
+3. Commit the pointer and the updated `.gitignore`.
+4. Optionally run `git filter-repo` to purge the old blobs from history if
+   clone size has become painful.
+
+Step 4 requires coordination — everyone must re-clone — so defer it until
+the pain justifies the disruption.
+
+### Reproducibility in practice
+
+The goal of all of this is a single concrete question: given a git commit,
+can a teammate rebuild the exact model artifact that the commit describes?
+Answer yes by pinning three things together:
+
+- **Code** — the git commit itself.
+- **Data** — a `.dvc` pointer, an LFS object, or a committed Parquet file,
+  all content-hashed.
+- **Environment** — a pinned `requirements.txt`, `pyproject.toml`, or
+  `conda-lock.yml` committed in the same commit.
+
+If any one of those three is missing, reproducibility is accidental. The
+versioning tool you pick is less important than treating the three as a
+single atomic unit — changed together, reviewed together, reverted together.

package/content/knowledge/data-science/data-science-dev-environment.md

@@ -0,0 +1,159 @@
+---
+name: data-science-dev-environment
+description: Reproducible local Python dev environment for data science using uv, direnv, pre-commit, and pyproject.toml
+topics: [data-science, dev-environment, uv, direnv, pre-commit]
+---
+
+A data-science project that cannot be recreated in minutes is a liability. Notebooks pick up stale package versions, secrets leak into `.bashrc`, and "works on my machine" kills any chance of a collaborator (or future-you) rerunning an experiment. The fix is not complicated: one lockfile, one place for env vars, one pre-commit hook, no bespoke shell scripts. This guide is opinionated toward solo and small-team workflows where local-first beats container-first.
+
+## Summary
+
+Use `uv` as the single Python package manager — it replaces `pip`, `pip-tools`, `venv`, and `virtualenv` with one fast, reproducible tool. Declare every dependency in `pyproject.toml` so there is exactly one source of truth, and commit `uv.lock` so `uv sync` gives any collaborator a byte-identical environment. Layer `direnv` on top for per-repo environment variables (tracking URIs, data paths, secrets pulled from a vault) so nothing leaks into your global shell. Add `pre-commit` with a small set of fast hooks (`ruff-format`, `ruff-check`, end-of-file fixer) so style and obvious bugs never enter a commit. Skip Docker for greenfield solo DS work — reach for it only when you cross an OS boundary (Mac dev, Linux prod) or depend on GPU/CUDA libraries.
+
+## Deep Guidance
+
+### uv for Python environment and dependencies
+
+`uv` is the 2025 default for Python packaging. It is a drop-in replacement for `pip`, `venv`, and `pip-tools`, written in Rust, and roughly 10-100x faster than the tools it replaces. For data science the combination of `uv sync` (reproduces the environment from the lockfile) and `uv run` (executes a script in the managed venv without activation) is the whole workflow.
+
+Bootstrap a new project:
+
+```bash
+uv init --python 3.12 myproject
+cd myproject
+uv add pandas numpy scikit-learn jupyterlab
+uv add --dev ruff pytest pandera
+uv sync            # creates .venv and installs everything
+uv run pytest      # runs in the managed venv, no activation needed
+uv run jupyter lab
+```
+
+A minimal `pyproject.toml`:
+
+```toml
+[project]
+name = "myproject"
+version = "0.1.0"
+description = "Customer churn analysis"
+requires-python = ">=3.12"
+dependencies = [
+    "pandas>=2.2",
+    "numpy>=2.0",
+    "scikit-learn>=1.5",
+    "jupyterlab>=4.2",
+    "pandera>=0.20",
+]
+
+[dependency-groups]
+dev = [
+    "ruff>=0.6",
+    "pytest>=8.0",
+    "pre-commit>=3.8",
+]
+
+[tool.ruff]
+line-length = 100
+target-version = "py312"
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "B", "UP", "PD"]  # PD = pandas-vet
+```
+
+Commit both `pyproject.toml` and `uv.lock`. A collaborator clones the repo and runs `uv sync` — that is the entire setup step. No `pip install -r requirements.txt`, no virtualenv activation, no version drift.
+
+Add a package with `uv add <name>`; remove with `uv remove <name>`. Both edit `pyproject.toml` and update `uv.lock` atomically. To pin a version use `uv add "pandas==2.2.3"`. To upgrade run `uv lock --upgrade-package pandas`.
+
+Two `uv` features worth knowing for data science specifically:
+
+- **`uv run script.py`** executes a file in the project venv with no activation step. Wire this into a `Makefile` or `justfile` so `make train` and `make eval` Just Work for any collaborator.
+- **Inline script metadata (PEP 723).** For one-off analysis scripts that live outside the project, a shebang-style header declares dependencies and `uv run` auto-creates an ephemeral venv:
+
+  ```python
+  # /// script
+  # requires-python = ">=3.12"
+  # dependencies = ["pandas", "duckdb"]
+  # ///
+  import pandas as pd
+  import duckdb
+  ...
+  ```
+
+  Running `uv run oneoff.py` resolves and caches those deps transparently. No more "should I add this to `pyproject.toml`?" for throwaway exploration.
+
+### direnv for env vars
+
+`direnv` loads a per-directory `.envrc` file whenever you `cd` into the project. It keeps secrets and tracking URIs out of your global shell and ensures every terminal session sees the same variables. Skip it if your project has no environment variables; add it the first time you reach for one.
+
+Install once (`brew install direnv`, then hook into your shell per the docs). In the project:
+
+```bash
+# .envrc — commit this file
+use python .venv/bin/python    # pin Python to the uv-managed venv
+source_up                      # inherit vars from parent .envrc if present
+
+# Experiment tracking
+export MLFLOW_TRACKING_URI="http://localhost:5000"
+
+# Data paths (relative to repo root)
+export DATA_DIR="$PWD/data"
+export MODELS_DIR="$PWD/models"
+
+# Make imports work without installing the package
+export PYTHONPATH="$PWD/src:$PYTHONPATH"
+
+# Secrets — source from a local-only file, never commit
+[[ -f .envrc.local ]] && source_env .envrc.local
+```
+
+Add `.envrc.local` to `.gitignore` and put any actual secrets there (API keys, DB passwords). Run `direnv allow` once after creating or editing `.envrc`; `direnv` will refuse to load until you do. The moment you `cd` out of the project, all variables unload — no pollution.
+
+### pre-commit hooks
+
+`pre-commit` runs a configured set of checks every time you `git commit`. Keep the hook list short and fast — anything slower than a second or two trains you to use `--no-verify`, which defeats the point. For data science the right starter set is format, lint, and a couple of sanity hooks.
+
+```yaml
+# .pre-commit-config.yaml
+repos:
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.6.9
+    hooks:
+      - id: ruff-format
+      - id: ruff-check
+        args: [--fix]
+
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v5.0.0
+    hooks:
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+      - id: check-yaml
+      - id: check-added-large-files
+        args: [--maxkb=500]   # block accidental dataset commits
+
+  - repo: https://github.com/kynan/nbstripout
+    rev: 0.7.1
+    hooks:
+      - id: nbstripout          # strips notebook outputs before commit
+```
+
+Install the git hook once per clone:
+
+```bash
+uv run pre-commit install
+uv run pre-commit run --all-files   # bootstrap: fix everything now
+```
+
+Deliberately excluded from this set: `mypy`, `pytest`, and `bandit`. They are all valuable, but they are slow enough that they belong in CI, not in the commit path. Fast local, thorough remote.
+
+### When to add Docker
+
+For greenfield solo data science, Docker is overhead you do not need. `uv sync` already gives you reproducibility on any machine with the same OS, and local iteration is faster without a container layer.
+
+Reach for Docker only when one of these is true:
+
+- **OS mismatch between dev and prod.** You develop on macOS but the model runs on Linux in production, and a native dependency (e.g. a C extension, a specific `libgomp`) behaves differently across platforms.
+- **GPU / CUDA dependencies.** CUDA toolkit versions are tightly coupled to driver versions and OS. A pinned `nvidia/cuda` base image is the only sane way to guarantee training reproducibility across machines.
+- **Handoff to MLOps or serving infra.** Production deployment targets (SageMaker, Vertex, KServe, plain Kubernetes) expect a container. Build one at the handoff boundary, not before.
+- **Onboarding collaborators with hostile local setups.** A Windows colleague who cannot install `uv` natively is a reasonable reason to ship a devcontainer.
+
+When you do add Docker, keep it thin: copy `pyproject.toml` and `uv.lock`, run `uv sync --frozen`, and let the same lockfile drive both local and container builds. That way the container is a packaging detail, not a parallel source of truth.

package/content/knowledge/data-science/data-science-experiment-tracking.md

@@ -0,0 +1,194 @@
+---
+name: data-science-experiment-tracking
+description: Local MLflow setup, run instrumentation, git commit tagging, and run comparison for solo and small-team data science work
+topics: [data-science, experiment-tracking, mlflow, weights-and-biases, reproducibility]
+---
+
+Without experiment tracking, data science becomes archaeology: three weeks after a promising result, a stakeholder asks "which config produced that number?" and answering it turns into a forensic exercise — sifting through notebook history, Slack messages, and commented-out cells. A lightweight experiment tracker fixes this with one discipline: every run logs its hyperparameters, metrics, artifacts, and the git commit SHA that produced it. For a solo DS or small team, you do not need a shared server or a cloud account — a local MLflow instance on SQLite is enough to get the full benefit, and you can graduate to a shared deployment later without changing the instrumentation.
+
+## Summary
+
+Self-host MLflow locally with a SQLite backend and a local artifact directory — it is the minimum setup that still gives you a queryable run history, a browsable UI, and reproducible run IDs. Every run logs the full hyperparameter dict, metrics per epoch (or iteration), the git commit SHA as a tag, dataset version, and any config or report artifacts. Weights & Biases is a reasonable cloud alternative if you value the polished UI and do not mind cloud storage — but for a DS-1 setup it is not the primary recommendation. Never log PII into run metadata or artifacts, and never commit `mlflow.db` or `mlartifacts/` to git.
+
+## Deep Guidance
+
+### What to log per run
+
+Treat every training run, hyperparameter tweak, or evaluation pass as a tracked experiment — even the exploratory ones you think will be throwaway. The cost of logging is trivial; the cost of not logging a run that turns out to matter is measured in hours of re-running and second-guessing. The minimum payload is:
+
+- **Hyperparameters**: the full config dict (learning rate, batch size, seed, feature set, model type, loss weights, regularization). Log it all — future-you does not know which knob will matter and adding knobs retroactively is impossible.
+- **Metrics**: logged with `step=epoch` (or `step=iteration`) so the UI can render a time-series plot. Log train and validation metrics side by side; a single final-value log loses the overfitting story.
+- **Git commit SHA**: a tag pointing to the exact commit that produced the run. Without this, "reproduce run 47" is unanswerable, because the config alone does not capture code changes in the training loop, data loader, or feature engineering.
+- **Dataset version**: a tag or param identifying which dataset snapshot was used — a DVC hash, a filename with a date suffix, or a data commit SHA. Without this, "reproduce run 47" is still unanswerable even if you have the code, because the data moved underneath it.
+- **Run name**: a human-readable name (`baseline-v3-with-dropout`) so the UI list is browsable without clicking every row to read the params.
+- **Artifacts**: the resolved config YAML, the evaluation report JSON, any confusion matrix images, and the final model checkpoint. Small artifacts go inline with the run; large model weights can be stored by reference.
+
+### MLflow self-hosted setup
+
+Run the tracking server locally. SQLite is the right backend for a solo workflow — it gives you the full query API without the ops burden of Postgres, and the `mlflow.db` file is small enough that you can zip and share it with a collaborator if you really need to:
+
+```bash
+mlflow server \
+  --backend-store-uri sqlite:///mlflow.db \
+  --default-artifact-root ./mlartifacts \
+  --host 127.0.0.1 --port 5000
+```
+
+Bind to `127.0.0.1` rather than `0.0.0.0` so you do not accidentally expose an unauthenticated tracking server to your network. Leave it running in a terminal tab, a `tmux` pane, or under `launchd`/`systemd` — whatever keeps it up between sessions.
+
+Point your code at the server via an environment variable. Using `direnv` keeps this per-project and avoids polluting your shell:
+
+```bash
+# .envrc
+export MLFLOW_TRACKING_URI=http://localhost:5000
+export MLFLOW_EXPERIMENT_NAME=churn-baseline
+```
+
+Add the tracking artifacts to `.gitignore` — they are large, local, and not reproducible from source. Committing them bloats the repo and leaks local-path metadata into history:
+
+```gitignore
+# .gitignore
+mlflow.db
+mlflow.db-journal
+mlartifacts/
+mlruns/
+```
+
+When you later graduate to a shared MLflow server (team deployment, S3 artifact store, Postgres backend), the only change is the `MLFLOW_TRACKING_URI` — your instrumentation code stays identical, and historical runs stay on your laptop as a personal archive.
+
+### Instrumenting a training / experiment run
+
+Wrap the training loop in `mlflow.start_run`. The context manager handles start and end timestamps, guarantees the run closes even on exception, and exposes `run.info.run_id` — the stable handle you use later for comparison, export, or model loading:
+
+```python
+import subprocess
+import mlflow
+import yaml
+
+mlflow.set_tracking_uri("http://localhost:5000")
+mlflow.set_experiment("churn-baseline")
+
+def train(cfg: dict) -> dict:
+    with mlflow.start_run(run_name=cfg["experiment"]["name"]) as run:
+        # Log full hyperparameter dict (flatten nested keys to dot-paths)
+        mlflow.log_params(_flatten(cfg))
+
+        # Reproducibility tags — git commit is the single most important one
+        git_sha = subprocess.check_output(
+            ["git", "rev-parse", "HEAD"]
+        ).decode().strip()
+        mlflow.set_tag("git_commit", git_sha)
+        mlflow.set_tag("dataset_version", cfg["data"]["version"])
+        mlflow.set_tag("model_type", cfg["model"]["type"])
+
+        # Per-epoch metrics — step=epoch is what gives you a time-series plot
+        for epoch in range(cfg["training"]["epochs"]):
+            train_metrics = train_epoch(...)
+            val_metrics = evaluate(...)
+            mlflow.log_metrics({
+                "train_loss": train_metrics["loss"],
+                "val_loss": val_metrics["loss"],
+                "val_auc": val_metrics["auc"],
+            }, step=epoch)
+
+        # Artifacts: resolved config + eval report
+        with open("configs/resolved.yaml", "w") as f:
+            yaml.safe_dump(cfg, f)
+        mlflow.log_artifact("configs/resolved.yaml")
+        mlflow.log_artifact("reports/eval_report.json")
+
+        return {"run_id": run.info.run_id, **val_metrics}
+```
+
+A few notes on the shape of this code. `mlflow.log_params` takes a flat dict, so a helper like `_flatten` turns `{"optimizer": {"lr": 1e-3}}` into `{"optimizer.lr": "0.001"}` — values are coerced to strings. Log the **resolved** config after any CLI overrides or hydra composition, not the raw file on disk, so the stored params match what actually ran. If the working tree is dirty at training time, either commit first or log `git status --porcelain` output as a tag so you can tell the logged commit is not the whole story. Keep the returned `run_id` — it is the primary key you will use to find this run in the UI, export its metadata, register its model later, or reference it from a downstream evaluation run via `mlflow.set_tag("parent_run_id", ...)`.
+
+### Run comparison and selection
+
+Open the MLflow UI at `http://localhost:5000`. The three views that earn their keep:
+
+- **Run list** — sort by `metrics.val_auc` or filter by `tags.git_commit = "<sha>"`. Tag filters are the fastest way to find "the runs I launched from this branch." Sort by columns to see the run_id of your best-performing experiment, then click through for the full picture.
+- **Parallel coordinates plot** — select several runs, switch to the parallel coordinates view, and see which hyperparameters correlate with your target metric. This is the view that turns dozens of runs into a readable pattern — hover a line to see the full config, drag axes to filter a band, and the plot re-paints to show only the runs that meet your criterion.
+- **Metric plot** — overlay `val_loss` across selected runs to spot overfitting (train loss drops, val loss rises), bad seeds (wildly different trajectories with the same config), or early-stopping candidates (val metric plateaued ten epochs before training ended).
+
+You can also query programmatically when the UI's filters are not expressive enough:
+
+```python
+from mlflow.tracking import MlflowClient
+import pandas as pd
+
+client = MlflowClient()
+runs = client.search_runs(
+    experiment_ids=[client.get_experiment_by_name("churn-baseline").experiment_id],
+    filter_string="metrics.val_auc > 0.82 and tags.dataset_version = '2026-03'",
+    order_by=["metrics.val_auc DESC"],
+    max_results=20,
+)
+df = pd.DataFrame([{
+    "run_id": r.info.run_id,
+    "name": r.info.run_name,
+    "val_auc": r.data.metrics.get("val_auc"),
+    "lr": r.data.params.get("optimizer.lr"),
+} for r in runs])
+```
+
+When you have a winner, export its config back into the repo for a clean retrain:
+
+```python
+run = client.get_run("<run_id>")
+client.download_artifacts(run.info.run_id, "resolved.yaml", dst_path="configs/")
+```
+
+Commit the exported config so "run 47's exact recipe" becomes a file in git, not a memory and not a database row that lives only on your laptop.
+
+### Weights & Biases as alternative
+
+Weights & Biases is the polished cloud alternative. It has a richer UI, built-in system metric logging (GPU, memory, temperature), gradient histograms via `wandb.watch(model, log="gradients")`, media logging (images, audio, tables, confusion matrices rendered inline), and better collaboration features — named reports, shared dashboards, thread-style comments. For a small team that has already decided it is comfortable with cloud storage, W&B removes the "who runs the server" question entirely, and the onboarding for a new teammate is `pip install wandb && wandb login`.
+
+The instrumentation shape is familiar:
+
+```python
+import wandb
+wandb.init(project="churn-baseline", name=cfg["experiment"]["name"],
+           config=cfg, tags=["baseline", "v2-features"])
+for epoch in range(cfg["training"]["epochs"]):
+    wandb.log({"epoch": epoch, "val/auc": val_auc, "train/loss": loss})
+wandb.finish()
+```
+
+Two things to weigh before picking it over MLflow for a DS-1 setup. First, the free tier has trial limits on private projects, artifact retention, and seats — fine for a solo experimenter, worth pricing out before a team commits. Second, you are shipping your experiment metadata (and potentially artifacts) to a third party, which matters if your dataset values, config parameters, or run names might accidentally encode something sensitive. MLflow self-hosted stays on your laptop; W&B lives in someone else's cloud. If your org has a data-residency or vendor-review process, MLflow skips it entirely.
+
+### Graduating from solo to shared
+
+The path from "SQLite on my laptop" to "shared team tracker" is short and deliberately low-risk, because your instrumentation already speaks the MLflow protocol:
+
+1. **Stand up a shared MLflow server** behind your internal network — Postgres backend, S3 or equivalent object store for artifacts, authentication in front (oauth2-proxy, nginx basic auth, or a cloud load balancer).
+2. **Flip the tracking URI** in each project's `.envrc` to point at the shared server. No code changes.
+3. **Optionally backfill historic runs** using `mlflow artifacts download` plus `search_runs`, then re-log to the shared server — only worth it for runs you want the team to see.
+4. **Keep the local server configured** for offline or air-gapped work — you can still set `MLFLOW_TRACKING_URI=file:./mlruns` for quick local-only iteration when the shared server is down or you are on a plane.
+
+This is why the self-hosted local setup is the right default even if you know you will eventually run a team server: the instrumentation you write today is exactly the instrumentation that will talk to production tomorrow.
+
+### Nested runs for sweeps and evaluations
+
+When you run a small hyperparameter sweep from a laptop — a few learning rates, two or three seeds — use MLflow's nested runs rather than one flat run per trial. A parent run captures the sweep-level config and best-metric summary; each trial becomes a child with its own params and metrics:
+
+```python
+with mlflow.start_run(run_name="lr-sweep") as parent:
+    mlflow.log_param("sweep_type", "grid")
+    best_auc = 0.0
+    for lr in [1e-4, 3e-4, 1e-3, 3e-3]:
+        with mlflow.start_run(run_name=f"lr={lr}", nested=True) as child:
+            mlflow.log_param("lr", lr)
+            val_auc = train_one(lr)
+            mlflow.log_metric("val_auc", val_auc)
+            best_auc = max(best_auc, val_auc)
+    mlflow.log_metric("best_val_auc", best_auc)
+```
+
+In the UI, the parent shows an expandable tree of children, which keeps the run list navigable once you have hundreds of rows. For larger sweeps, pair MLflow with Optuna (`mlflow.start_run(nested=True)` inside an `objective` function) — you get Bayesian search on top of MLflow's persistence.
+
+### Hygiene
+
+**Never log PII.** Experiment metadata and artifacts are easy to share with collaborators, screenshot into a ticket, or accidentally export to a cloud UI when you later migrate to W&B or a shared MLflow. Hyperparameter values, metric names, run names, tags, and artifact contents must all be free of customer identifiers, emails, names, or raw records. If a config carries a data path, keep it at the dataset level (`data/processed/churn_2026_03.parquet`) — never at the row or user level (`data/users/alice@example.com/history.parquet`). Evaluation reports that include example predictions must redact any PII in the input columns before being logged as artifacts. See `data-science-security.md` for the broader no-PII rules that apply across the whole DS workflow.
+
+**Never commit the tracking store.** `mlflow.db`, `mlflow.db-journal`, `mlartifacts/`, and `mlruns/` all belong in `.gitignore`. They are large (easily hundreds of MB once you log a few model checkpoints), machine-local (paths and timestamps are specific to your laptop), and not reproducible from git. If you need a teammate to see a run, export the specific artifacts you want to share via `mlflow artifacts download --run-id <id>` or by running a shared tracking server — do not push the whole store into the repo and do not email `mlflow.db` as an attachment.