@zigrivers/scaffold 3.22.0 → 3.24.0

package/content/knowledge/data-science/data-science-conventions.md

@@ -0,0 +1,233 @@
+---
+name: data-science-conventions
+description: Python coding conventions for solo data-science work — ruff for lint+format, pragmatic type hints, pyproject.toml as single config source, import ordering, module layout, naming, and docstrings
+topics: [data-science, conventions, python, ruff, type-hints]
+---
+
+Solo data-science code drifts faster than any other kind of Python: half of it lives in notebooks, the other half migrates into scripts, and nothing stays stable long enough to earn a style review. Consistent conventions are the only thing that keeps cognitive load bounded when you come back to a project after two months. Encode them in tooling (`ruff`, `pyproject.toml`) so they run on save — not on willpower — and the notebook→script promotion path stays smooth instead of becoming a cleanup tax.
+
+## Summary
+
+Use `ruff` as the single lint + format tool — `ruff format` is Black-compatible and replaces Black, so do not install both. Apply `type hints` pragmatically: typed on any function another module imports, omitted on throwaway notebook helpers. Centralize all project and tool configuration in `pyproject.toml` — one file for build metadata, dependencies, ruff, and pytest. Use `ruff`/`isort`-style import sections (stdlib → third-party → local), a flat `src/` layout with a clear module split, and docstrings sized to the consumer: one-liners for internal helpers, full Google/NumPy style for anything a teammate will call without reading the source.
+
+## Deep Guidance
+
+### Linter + formatter (ruff)
+
+`ruff` is the only Python linter/formatter a solo DS project needs. It replaces `flake8`, `isort`, `pyupgrade`, `pydocstyle`, `pylint` (mostly), and — via `ruff format` — Black. It is an order of magnitude faster than the tools it replaces, configured in one `[tool.ruff]` block, and has no plugin-management overhead. Do not layer Black on top: `ruff format` implements the same formatting contract, and running both just causes churn.
+
+```toml
+# pyproject.toml
+[tool.ruff]
+line-length = 100
+target-version = "py311"
+extend-exclude = ["notebooks/_scratch", "data", "models"]
+
+[tool.ruff.lint]
+select = [
+  "E",   # pycodestyle errors
+  "W",   # pycodestyle warnings
+  "F",   # pyflakes
+  "I",   # isort (import sorting)
+  "N",   # pep8-naming
+  "UP",  # pyupgrade
+  "B",   # flake8-bugbear
+  "C90", # mccabe complexity
+  "D",   # pydocstyle
+]
+ignore = [
+  "D100",  # missing docstring in public module — noisy for scripts
+  "D104",  # missing docstring in public package
+  "E501",  # line-too-long — formatter handles it
+]
+
+[tool.ruff.lint.per-file-ignores]
+# Notebooks and experiment scripts get a lighter hand
+"notebooks/**/*.py" = ["D", "N806", "E402"]
+"scripts/**/*.py"  = ["D"]
+"tests/**/*.py"    = ["D"]
+
+[tool.ruff.lint.pydocstyle]
+convention = "google"
+
+[tool.ruff.lint.mccabe]
+max-complexity = 12
+
+[tool.ruff.format]
+quote-style = "double"
+indent-style = "space"
+```
+
+**Tradeoff**: notebook and exploration code legitimately breaks rules that production code should not — uppercase variable names (`X_train`), imports after executable code, no docstrings. The `per-file-ignores` block disables the rules that fight notebook workflows without weakening the defaults for `src/`. Do not globally ignore `D` or `N` just to silence notebook noise.
+
+Run on save (editor integration) and as a pre-commit hook. In CI, run `ruff check .` and `ruff format --check .` — the `--check` flag fails instead of rewriting.
+
+### Type hints
+
+Python is not a typed language, and pretending it is in exploratory code wastes time. The rule is **import boundary = type boundary**: if another module imports the function, type it. Notebook-local helpers and inline lambdas do not need annotations.
+
+```python
+# src/features/encoders.py — imported by training and serving, fully typed
+from __future__ import annotations
+
+import numpy as np
+import pandas as pd
+
+
+def target_encode(
+    series: pd.Series,
+    target: pd.Series,
+    smoothing: float = 10.0,
+) -> pd.Series:
+    """Smoothed target encoding for a categorical feature.
+
+    Args:
+        series: Categorical feature values (any hashable dtype).
+        target: Numeric target aligned to `series` by index.
+        smoothing: Prior weight; higher values pull rare categories
+            toward the global mean.
+
+    Returns:
+        Series of encoded floats aligned to `series.index`.
+    """
+    global_mean = target.mean()
+    agg = target.groupby(series).agg(["mean", "count"])
+    weight = agg["count"] / (agg["count"] + smoothing)
+    encoding = weight * agg["mean"] + (1 - weight) * global_mean
+    return series.map(encoding).astype(np.float64)
+```
+
+```python
+# notebooks/03_eda.py — throwaway scratch, no annotations needed
+def quick_hist(col):
+    return df[col].value_counts().head(20)
+
+for c in cat_cols:
+    print(c, quick_hist(c).to_dict())
+```
+
+Practical rules:
+- Type every function exported from `src/` — parameters and return.
+- Type dataclasses and `TypedDict` schemas that describe data contracts (row shapes, config dicts).
+- Skip annotations on notebook cells, inline closures, and private helpers inside a single script.
+- Use `from __future__ import annotations` at the top of every `src/` file — it makes all annotations lazy strings, so forward references and expensive-to-import types (`torch.Tensor`, `pd.DataFrame`) cost nothing at import time.
+- Do not run `mypy --strict` on a solo DS project. Run it on `src/` with `--ignore-missing-imports` if you want a safety net, and do not bother with notebooks.
+
+### Project layout and pyproject.toml
+
+One `pyproject.toml` at the repo root configures the build, dependencies, lint, format, and tests. Do not scatter config across `setup.cfg`, `.flake8`, `.isort.cfg`, and `pytest.ini` — everything lives in `pyproject.toml`.
+
+```toml
+# pyproject.toml
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "churn-model"
+version = "0.1.0"
+description = "Customer churn prediction — feature pipeline, training, and serving."
+requires-python = ">=3.11"
+dependencies = [
+  "pandas>=2.1",
+  "numpy>=1.26",
+  "scikit-learn>=1.4",
+  "pydantic>=2.5",
+]
+
+[project.optional-dependencies]
+dev = [
+  "ruff>=0.3",
+  "pytest>=8.0",
+  "pytest-cov>=4.1",
+  "ipykernel>=6.29",
+]
+
+[tool.ruff]
+line-length = 100
+target-version = "py311"
+# ... (see ruff section above)
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-ra --strict-markers --cov=src --cov-report=term-missing"
+markers = [
+  "slow: marks tests as slow (deselect with '-m \"not slow\"')",
+]
+```
+
+Repo layout:
+
+```
+churn-model/
+  pyproject.toml
+  README.md
+  src/
+    churn_model/
+      __init__.py
+      data/          # loaders, schemas, splits
+      features/      # transformers, encoders, selection
+      models/        # model definitions and wrappers
+      training/      # train loops, CV runners
+      evaluation/    # metrics, diagnostics
+      serving/       # inference helpers
+  notebooks/
+    01_data_audit.ipynb
+    02_feature_exploration.ipynb
+  tests/
+    test_features.py
+    test_training.py
+  configs/
+    base.yaml
+```
+
+Use a `src/` layout (not flat) so imports always go through the installed package — this prevents the "works in notebook, breaks in test" failure mode where `from my_module import x` resolves from the CWD instead of the package.
+
+### Import ordering
+
+`ruff` with rule `I` enforces `isort`-compatible sections automatically. The contract:
+
+1. Future imports (`from __future__ import annotations`)
+2. Standard library
+3. Third-party
+4. First-party (your package)
+5. Local relative (`from .utils import ...`)
+
+One blank line between sections, alphabetical within each. Do not hand-maintain this — `ruff check --fix` sorts imports in milliseconds.
+
+```python
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+import numpy as np
+import pandas as pd
+from sklearn.model_selection import KFold
+
+from churn_model.data import load_raw
+from churn_model.features import target_encode
+
+from .utils import timed
+```
+
+### Naming and docstrings
+
+Naming rubric (enforced by `ruff` rule `N`):
+
+- **Modules/files**: `snake_case.py` (`feature_store.py`, not `FeatureStore.py`).
+- **Functions/variables**: `snake_case` (`compute_auc`, `n_splits`).
+- **Classes**: `PascalCase` (`ChurnDataset`, `TargetEncoder`).
+- **Constants**: `UPPER_SNAKE_CASE` at module top level (`DEFAULT_SEED = 42`, `FEATURE_COLUMNS: tuple[str, ...] = (...)`).
+- **Private**: single leading underscore (`_internal_helper`). Double underscore only when you specifically want name-mangling inside a class.
+- **Type variables**: `PascalCase` with suffix (`ModelT = TypeVar("ModelT")`).
+- **DataFrame matrices**: `X`, `y`, `X_train`, `y_test` are the one permitted uppercase exception — this is ML convention and `ruff` can be told to allow it via `N806` ignore in model/training modules.
+
+Docstring style sizing — match the cost of writing the docstring to the consumer:
+
+- **Terse one-liner** for private helpers and obvious utilities. `"""Return the 95th percentile of non-null values."""` is enough.
+- **Full Google-style** (Args/Returns/Raises) for any public function in `src/features/`, `src/models/`, or `src/serving/` — anything a teammate or future-you will call without opening the source. See the `target_encode` example above.
+- **Module docstring** on every `src/` module: one sentence describing what lives there. Skip on `scripts/` and `notebooks/`.
+- **Class docstring** covers the class contract; `__init__` args go in the class docstring, not a separate `__init__` docstring. (This is the Google convention and `ruff`'s `pydocstyle` setting enforces it.)
+
+Pick Google **or** NumPy style — not both — and set it in `[tool.ruff.lint.pydocstyle]`. Google is more compact and reads better in IDE hover; NumPy is better when you have long parameter descriptions with math. For solo DS, Google is the default recommendation.

package/content/knowledge/data-science/data-science-data-versioning.md

@@ -0,0 +1,198 @@
+---
+name: data-science-data-versioning
+description: When and how to version data for reproducibility — size-based rule for choosing between git+Parquet, git-lfs, and DVC
+topics: [data-science, data-versioning, dvc, parquet, git-lfs]
+---
+
+If you can't answer "what data produced this result", you can't reproduce it.
+A model trained on 2026-02-14's snapshot will drift from one trained today,
+and without a versioning story you have no way to roll back, diff, or explain
+the difference to a reviewer six months later.
+
+Data versioning answers that question without blowing up the repo — the trick
+is picking a tool proportional to the dataset size. The common failure mode
+is over-engineering (wiring up DVC with a remote for a 40 MB CSV) or
+under-engineering (committing 2 GB of Parquet directly into git and
+discovering three months later that every clone takes twenty minutes).
+
+## Summary
+
+Pick your tool by size.
+
+- Under ~1 GB of text or Parquet, plain git with committed Parquet files is fine.
+- Between 1 and 10 GB, use `git-lfs` if you already use it on the team;
+  otherwise invest in `DVC` (Data Version Control).
+- Above 10 GB — or for any binary artifact (model weights, image corpora,
+  audio) — always use DVC with a remote (s3, gcs, azure).
+- Never version raw third-party data that you can't legally redistribute —
+  store a fetch script and a content hash instead.
+
+## Deep Guidance
+
+### Size-based decision rule
+
+| Dataset | Tool | Why |
+|---------|------|-----|
+| <1 GB text / Parquet | git + Parquet | Columnar compression keeps files small; git history stays sane |
+| 1–10 GB (judgment call) | `git-lfs` if already adopted; DVC if you have the habit | LFS is lower-effort; DVC gives you pipeline stages too |
+| >10 GB or binary artifacts | DVC with remote | Git history will not tolerate binary churn at this scale |
+| Raw third-party data | Don't version — script + hash | Redistribution is often prohibited; raw bytes bloat history |
+
+The sizes above are rules of thumb, not hard thresholds. What actually
+matters is how often the data changes. A 5 GB file that you generate once
+and never touch again can live in `git-lfs` forever without pain. The same
+5 GB file regenerated weekly will accumulate 260 GB of LFS storage in a
+year — that's the point where DVC's content-addressed cache starts to earn
+its complexity.
+
+A second factor is team shape. A solo researcher on a laptop rarely needs a
+remote backing store; a two-person team on different continents almost
+always does. Choose the tool that fits the smallest real collaboration
+pattern you have, not the one that scales to the team you imagine having.
+
+### When git + Parquet is enough
+
+For a solo or small-team project with modest data, commit processed Parquet directly. Keep raw data out of the repo; reserve git for cleaned, analysis-ready files.
+
+```python
+# src/pipelines/clean.py
+import pandas as pd
+
+df = pd.read_csv("data/raw/events.csv")  # data/raw/ is gitignored
+clean = df.dropna(subset=["user_id"]).assign(ts=pd.to_datetime(df["ts"]))
+clean.to_parquet("data/interim/events_clean.parquet", compression="zstd")
+```
+
+```gitignore
+# .gitignore
+data/raw/
+data/external/
+*.csv
+!data/interim/*.parquet   # do commit processed Parquet
+```
+
+Parquet's columnar layout and zstd compression typically shrink tabular data
+5–10x versus CSV. Diffs aren't line-level but file-level content hashes are
+stable, which is enough for "which version produced this model".
+
+Pair the committed Parquet with a short data card — a markdown file in
+`data/interim/events_clean.md` describing row count, schema, source, and the
+commit that generated it — so readers of the repo a year later can tell what
+they're looking at.
+
+### DVC basics
+
+DVC treats large files as pointers tracked in git. The real bytes live on a remote (s3/gcs/azure/ssh), and a small `.dvc` metadata file is committed.
+
+```yaml
+# dvc.yaml — pipeline stages with content-hashed inputs and outputs
+stages:
+  ingest:
+    cmd: python src/ingest.py --out data/raw/events.parquet
+    outs:
+      - data/raw/events.parquet
+  process:
+    cmd: python src/process.py --in data/raw/events.parquet --out data/processed/features.parquet
+    deps:
+      - src/process.py
+      - data/raw/events.parquet
+    outs:
+      - data/processed/features.parquet
+```
+
+Typical flow:
+
+```bash
+dvc init                              # creates .dvc/ directory
+dvc remote add -d storage s3://my-bucket/dvc-store
+dvc add data/raw/big_dataset.csv      # creates data/raw/big_dataset.csv.dvc (commit this)
+dvc repro                             # runs stages whose inputs changed
+dvc push                              # upload tracked files to remote
+git add dvc.yaml dvc.lock data/raw/big_dataset.csv.dvc .gitignore
+git commit -m "track raw events via DVC"
+```
+
+`dvc.lock` records the content hash of every stage input and output, so
+`dvc repro` on a peer's machine rebuilds exactly what you rebuilt. The
+`.dvc/` directory holds local cache and config; the actual bytes never touch
+git.
+
+Mental model: git for code, DVC for data, both pointing at the same commit.
+When you check out an older branch, git restores the source and the `.dvc`
+pointers, and `dvc checkout` pulls matching data from the remote into your
+working tree.
+
+A common starting point: track one or two heavy inputs with `dvc add` (no
+pipeline), and only adopt `dvc.yaml` stages once you have a repeatable
+multi-step workflow. The overhead of stages pays off when you have 3+ steps
+and want `dvc repro` to skip unchanged work; below that, plain `dvc add`
+plus a Makefile is often clearer.
+
+### git-lfs middle ground
+
+If you're already using Git LFS on the team but not ready to adopt DVC, it works acceptably for the 1–10 GB band — especially for a handful of files over the 100 MB GitHub push limit.
+
+```gitattributes
+# .gitattributes
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pkl     filter=lfs diff=lfs merge=lfs -text
+data/models/** filter=lfs diff=lfs merge=lfs -text
+```
+
+```bash
+git lfs install
+git lfs track "*.parquet"
+git add .gitattributes data/features.parquet
+git commit -m "add feature table via LFS"
+```
+
+Reach for `git-lfs` when files are over ~100 MB but you don't need DVC's
+pipeline stages or content-addressed reproducibility. Skip it if you already
+have DVC set up — two tools versioning the same data is a recipe for
+confusion.
+
+LFS has real drawbacks to know about: bandwidth is metered on hosted plans,
+`git clone` pulls every LFS object by default (use `GIT_LFS_SKIP_SMUDGE=1`
+to defer), and you can't selectively prune history without rewriting the
+whole repo. For a working group of 2–5 people on a research project these
+are usually tolerable; for a fleet of CI workers cloning on every build
+they are not.
+
+### What not to version
+
+- **Third-party data with license constraints** — re-commit a fetch script (`scripts/fetch_kaggle.sh`) and record the SHA256 of the pulled file in a README. Re-download on each environment.
+- **Regenerable intermediates** — if `dvc repro` or `make data` can recreate it deterministically from upstream inputs, don't commit the bytes.
+- **Scratch / exploratory outputs** — `notebooks/scratch/`, `data/tmp/`, `*.ipynb_checkpoints/` belong in `.gitignore`.
+- **Anti-pattern: committing 500 MB Parquet files directly to git** — they live forever in history, clone times balloon, and nobody will clean it up later. Move to DVC or LFS *before* the first large commit, not after. Rewriting history to extract large blobs (`git filter-repo`, BFG) is disruptive to every collaborator and should be a last resort.
+- **Anti-pattern: versioning model checkpoints in git** — a single PyTorch checkpoint can be several hundred MB, and training runs produce dozens. Push them to DVC or an artifact store (MLflow, Weights & Biases) keyed by experiment run ID.
+
+### Quick migration path
+
+If you're staring at a repo that has already committed large files to plain
+git, the order of operations is:
+
+1. Decide the target tool (DVC for most cases where you got here).
+2. Run `dvc add` on the file in its current location — this untracks it
+   from git and creates a `.dvc` pointer.
+3. Commit the pointer and the updated `.gitignore`.
+4. Optionally run `git filter-repo` to purge the old blobs from history if
+   clone size has become painful.
+
+Step 4 requires coordination — everyone must re-clone — so defer it until
+the pain justifies the disruption.
+
+### Reproducibility in practice
+
+The goal of all of this is a single concrete question: given a git commit,
+can a teammate rebuild the exact model artifact that the commit describes?
+Answer yes by pinning three things together:
+
+- **Code** — the git commit itself.
+- **Data** — a `.dvc` pointer, an LFS object, or a committed Parquet file,
+  all content-hashed.
+- **Environment** — a pinned `requirements.txt`, `pyproject.toml`, or
+  `conda-lock.yml` committed in the same commit.
+
+If any one of those three is missing, reproducibility is accidental. The
+versioning tool you pick is less important than treating the three as a
+single atomic unit — changed together, reviewed together, reverted together.

package/content/knowledge/data-science/data-science-dev-environment.md

@@ -0,0 +1,159 @@
+---
+name: data-science-dev-environment
+description: Reproducible local Python dev environment for data science using uv, direnv, pre-commit, and pyproject.toml
+topics: [data-science, dev-environment, uv, direnv, pre-commit]
+---
+
+A data-science project that cannot be recreated in minutes is a liability. Notebooks pick up stale package versions, secrets leak into `.bashrc`, and "works on my machine" kills any chance of a collaborator (or future-you) rerunning an experiment. The fix is not complicated: one lockfile, one place for env vars, one pre-commit hook, no bespoke shell scripts. This guide is opinionated toward solo and small-team workflows where local-first beats container-first.
+
+## Summary
+
+Use `uv` as the single Python package manager — it replaces `pip`, `pip-tools`, `venv`, and `virtualenv` with one fast, reproducible tool. Declare every dependency in `pyproject.toml` so there is exactly one source of truth, and commit `uv.lock` so `uv sync` gives any collaborator a byte-identical environment. Layer `direnv` on top for per-repo environment variables (tracking URIs, data paths, secrets pulled from a vault) so nothing leaks into your global shell. Add `pre-commit` with a small set of fast hooks (`ruff-format`, `ruff-check`, end-of-file fixer) so style and obvious bugs never enter a commit. Skip Docker for greenfield solo DS work — reach for it only when you cross an OS boundary (Mac dev, Linux prod) or depend on GPU/CUDA libraries.
+
+## Deep Guidance
+
+### uv for Python environment and dependencies
+
+`uv` is the 2025 default for Python packaging. It is a drop-in replacement for `pip`, `venv`, and `pip-tools`, written in Rust, and roughly 10-100x faster than the tools it replaces. For data science the combination of `uv sync` (reproduces the environment from the lockfile) and `uv run` (executes a script in the managed venv without activation) is the whole workflow.
+
+Bootstrap a new project:
+
+```bash
+uv init --python 3.12 myproject
+cd myproject
+uv add pandas numpy scikit-learn jupyterlab
+uv add --dev ruff pytest pandera
+uv sync            # creates .venv and installs everything
+uv run pytest      # runs in the managed venv, no activation needed
+uv run jupyter lab
+```
+
+A minimal `pyproject.toml`:
+
+```toml
+[project]
+name = "myproject"
+version = "0.1.0"
+description = "Customer churn analysis"
+requires-python = ">=3.12"
+dependencies = [
+    "pandas>=2.2",
+    "numpy>=2.0",
+    "scikit-learn>=1.5",
+    "jupyterlab>=4.2",
+    "pandera>=0.20",
+]
+
+[dependency-groups]
+dev = [
+    "ruff>=0.6",
+    "pytest>=8.0",
+    "pre-commit>=3.8",
+]
+
+[tool.ruff]
+line-length = 100
+target-version = "py312"
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "B", "UP", "PD"]  # PD = pandas-vet
+```
+
+Commit both `pyproject.toml` and `uv.lock`. A collaborator clones the repo and runs `uv sync` — that is the entire setup step. No `pip install -r requirements.txt`, no virtualenv activation, no version drift.
+
+Add a package with `uv add <name>`; remove with `uv remove <name>`. Both edit `pyproject.toml` and update `uv.lock` atomically. To pin a version use `uv add "pandas==2.2.3"`. To upgrade run `uv lock --upgrade-package pandas`.
+
+Two `uv` features worth knowing for data science specifically:
+
+- **`uv run script.py`** executes a file in the project venv with no activation step. Wire this into a `Makefile` or `justfile` so `make train` and `make eval` Just Work for any collaborator.
+- **Inline script metadata (PEP 723).** For one-off analysis scripts that live outside the project, a shebang-style header declares dependencies and `uv run` auto-creates an ephemeral venv:
+
+  ```python
+  # /// script
+  # requires-python = ">=3.12"
+  # dependencies = ["pandas", "duckdb"]
+  # ///
+  import pandas as pd
+  import duckdb
+  ...
+  ```
+
+  Running `uv run oneoff.py` resolves and caches those deps transparently. No more "should I add this to `pyproject.toml`?" for throwaway exploration.
+
+### direnv for env vars
+
+`direnv` loads a per-directory `.envrc` file whenever you `cd` into the project. It keeps secrets and tracking URIs out of your global shell and ensures every terminal session sees the same variables. Skip it if your project has no environment variables; add it the first time you reach for one.
+
+Install once (`brew install direnv`, then hook into your shell per the docs). In the project:
+
+```bash
+# .envrc — commit this file
+use python .venv/bin/python    # pin Python to the uv-managed venv
+source_up                      # inherit vars from parent .envrc if present
+
+# Experiment tracking
+export MLFLOW_TRACKING_URI="http://localhost:5000"
+
+# Data paths (relative to repo root)
+export DATA_DIR="$PWD/data"
+export MODELS_DIR="$PWD/models"
+
+# Make imports work without installing the package
+export PYTHONPATH="$PWD/src:$PYTHONPATH"
+
+# Secrets — source from a local-only file, never commit
+[[ -f .envrc.local ]] && source_env .envrc.local
+```
+
+Add `.envrc.local` to `.gitignore` and put any actual secrets there (API keys, DB passwords). Run `direnv allow` once after creating or editing `.envrc`; `direnv` will refuse to load until you do. The moment you `cd` out of the project, all variables unload — no pollution.
+
+### pre-commit hooks
+
+`pre-commit` runs a configured set of checks every time you `git commit`. Keep the hook list short and fast — anything slower than a second or two trains you to use `--no-verify`, which defeats the point. For data science the right starter set is format, lint, and a couple of sanity hooks.
+
+```yaml
+# .pre-commit-config.yaml
+repos:
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.6.9
+    hooks:
+      - id: ruff-format
+      - id: ruff-check
+        args: [--fix]
+
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v5.0.0
+    hooks:
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+      - id: check-yaml
+      - id: check-added-large-files
+        args: [--maxkb=500]   # block accidental dataset commits
+
+  - repo: https://github.com/kynan/nbstripout
+    rev: 0.7.1
+    hooks:
+      - id: nbstripout          # strips notebook outputs before commit
+```
+
+Install the git hook once per clone:
+
+```bash
+uv run pre-commit install
+uv run pre-commit run --all-files   # bootstrap: fix everything now
+```
+
+Deliberately excluded from this set: `mypy`, `pytest`, and `bandit`. They are all valuable, but they are slow enough that they belong in CI, not in the commit path. Fast local, thorough remote.
+
+### When to add Docker
+
+For greenfield solo data science, Docker is overhead you do not need. `uv sync` already gives you reproducibility on any machine with the same OS, and local iteration is faster without a container layer.
+
+Reach for Docker only when one of these is true:
+
+- **OS mismatch between dev and prod.** You develop on macOS but the model runs on Linux in production, and a native dependency (e.g. a C extension, a specific `libgomp`) behaves differently across platforms.
+- **GPU / CUDA dependencies.** CUDA toolkit versions are tightly coupled to driver versions and OS. A pinned `nvidia/cuda` base image is the only sane way to guarantee training reproducibility across machines.
+- **Handoff to MLOps or serving infra.** Production deployment targets (SageMaker, Vertex, KServe, plain Kubernetes) expect a container. Build one at the handoff boundary, not before.
+- **Onboarding collaborators with hostile local setups.** A Windows colleague who cannot install `uv` natively is a reasonable reason to ship a devcontainer.
+
+When you do add Docker, keep it thin: copy `pyproject.toml` and `uv.lock`, run `uv sync --frozen`, and let the same lockfile drive both local and container builds. That way the container is a packaging detail, not a parallel source of truth.