feather-control-plane 0.1.0__tar.gz

feather_control_plane-0.1.0/.gitignore

@@ -0,0 +1,21 @@
+.worktrees/
+.claude/worktrees/
+__pycache__/
+*.pyc
+*.egg-info/
+dist/
+.venv/
+*.duckdb.wal
+feather_data.duckdb
+feather_state.duckdb
+feather_validation.json
+feather_discover_state.json
+.env
+.DS_Store
+.coverage
+.testmondata
+.testmondata-journal
+.nvmrc
+repomix-output.xml
+.playwright-mcp/
+.superpowers/

feather_control_plane-0.1.0/PKG-INFO

@@ -0,0 +1,107 @@
+Metadata-Version: 2.5
+Name: feather-control-plane
+Version: 0.1.0
+Summary: Durable failure observability for an extractor fleet: WAL + Postgres control plane + liveness lease
+Project-URL: Homepage, https://github.com/siraj-samsudeen/feather-flow
+Project-URL: Repository, https://github.com/siraj-samsudeen/feather-flow
+Project-URL: Issues, https://github.com/siraj-samsudeen/feather-flow/issues
+Author-email: Siraj Samsudeen <mailsiraj@gmail.com>
+License-Expression: MIT
+Keywords: control-plane,etl,observability,postgres,write-ahead-log
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Database
+Classifier: Topic :: System :: Monitoring
+Requires-Python: >=3.12
+Requires-Dist: tzdata>=2026.1
+Provides-Extra: postgres
+Requires-Dist: psycopg[binary]>=3.1; extra == 'postgres'
+Description-Content-Type: text/markdown
+
+# feather-control-plane
+
+Durable failure observability for an extractor fleet.
+
+A long-running extractor that dies mid-load leaves nothing behind: no row, no error, no
+"it got as far as table X". This library gives it three things that survive the death —
+
+- a **write-ahead log** on local disk, fsynced before anything is shipped, so a crash leaves
+  the breadcrumbs of how far the run got;
+- a **Postgres control plane** holding run lifecycle, per-unit events and budget telemetry,
+  shipped as the run goes rather than written at the end;
+- a **liveness lease**, renewed by a background thread, so a process that stops renewing is
+  known to be dead instead of being assumed still running.
+
+The data plane is somewhere else entirely — this records what happened to the load, not the
+load's data.
+
+## Install
+
+```bash
+pip install "feather-control-plane[postgres]"    # the full facade
+pip install feather-control-plane                # pure-stdlib helpers only
+```
+
+**`psycopg` is an optional extra, and that is deliberate.** Two of the modules here —
+`control_plane.business_day` and `control_plane.status` — are pure stdlib, and consumers import
+them on their own, with no database anywhere in the picture. Making a database driver mandatory
+for a date helper is what drove nine services to copy `business_day.py` around by hand instead
+of depending on the library. Anything that actually opens a connection needs the `postgres`
+extra; `__init__.py` resolves those names lazily (PEP 562), so importing the stdlib half never
+pulls the driver in.
+
+## Import name
+
+The distribution is `feather-control-plane`; the **import name is `control_plane`**:
+
+```python
+from control_plane import ControlPlane, status
+from control_plane.business_day import ist_today
+```
+
+They differ on purpose. The nine services adopting this already have ~40 call sites spelled
+`from control_plane import ...`, so keeping the import name turns adoption into a dependency
+substitution with no source edits.
+
+## Usage
+
+```python
+from control_plane import ControlPlane, status
+
+cp = ControlPlane("gofrugal", dsn=PG_DSN, wal_dir="control_wal")
+with cp.run("nightly", run_id=run_id) as run:
+    run.event(entity="sales_item_wise", unit="1001", business_date=d,
+              status=status.EV_OK, http_status=200, rows=42,
+              source_attempts=1, sink_attempts=1, pull_ms=1200, load_ms=800)
+    run.budget(status.BUDGET_WALLTIME, limit=1800, used=1800, remaining=0, hit=True)
+    run.finish(status.RUN_PARTIAL, stop_reason=status.STOP_BUDGET_TIME,
+               n_ok=47, n_err=1, n_skipped=0)
+```
+
+`dsn=None` is the explicit no-control-plane mode for dev and test: the WAL still records
+everything, nothing ships. With a DSN configured, Postgres is a **hard dependency of the run** —
+if it is unreachable at startup the run aborts rather than loading blind, because a load with no
+provenance is worse than no load.
+
+## Tests
+
+```bash
+cd packages/feather-control-plane && uv run --package feather-control-plane pytest
+# or, from the repo root, naming the path:
+uv run --package feather-control-plane pytest packages/feather-control-plane/tests
+```
+
+`--package` picks the *environment*; it does not pick the tests. pytest takes its config from the
+nearest `pyproject.toml` above the working directory, so a bare `pytest` run from the repo root
+collects feather-flow's suite no matter what `--package` says — green, 1050 tests, and not one of
+them this package's. Either `cd` in or name the path.
+
+The pure-logic tests (WAL framing, status vocabulary, business day) run anywhere. The rest need a
+real Postgres and find one in this order: `CONTROL_PG_DSN` if set → a `postgres:16` Docker
+container → a local Postgres binary (`pg_ctl` on `PATH`, else Homebrew's keg-only
+`postgresql@N`), `initdb`'d into a temp directory on a throwaway port and torn down afterwards.
+With none of the three available they skip.

feather_control_plane-0.1.0/README.md

@@ -0,0 +1,83 @@
+# feather-control-plane
+
+Durable failure observability for an extractor fleet.
+
+A long-running extractor that dies mid-load leaves nothing behind: no row, no error, no
+"it got as far as table X". This library gives it three things that survive the death —
+
+- a **write-ahead log** on local disk, fsynced before anything is shipped, so a crash leaves
+  the breadcrumbs of how far the run got;
+- a **Postgres control plane** holding run lifecycle, per-unit events and budget telemetry,
+  shipped as the run goes rather than written at the end;
+- a **liveness lease**, renewed by a background thread, so a process that stops renewing is
+  known to be dead instead of being assumed still running.
+
+The data plane is somewhere else entirely — this records what happened to the load, not the
+load's data.
+
+## Install
+
+```bash
+pip install "feather-control-plane[postgres]"    # the full facade
+pip install feather-control-plane                # pure-stdlib helpers only
+```
+
+**`psycopg` is an optional extra, and that is deliberate.** Two of the modules here —
+`control_plane.business_day` and `control_plane.status` — are pure stdlib, and consumers import
+them on their own, with no database anywhere in the picture. Making a database driver mandatory
+for a date helper is what drove nine services to copy `business_day.py` around by hand instead
+of depending on the library. Anything that actually opens a connection needs the `postgres`
+extra; `__init__.py` resolves those names lazily (PEP 562), so importing the stdlib half never
+pulls the driver in.
+
+## Import name
+
+The distribution is `feather-control-plane`; the **import name is `control_plane`**:
+
+```python
+from control_plane import ControlPlane, status
+from control_plane.business_day import ist_today
+```
+
+They differ on purpose. The nine services adopting this already have ~40 call sites spelled
+`from control_plane import ...`, so keeping the import name turns adoption into a dependency
+substitution with no source edits.
+
+## Usage
+
+```python
+from control_plane import ControlPlane, status
+
+cp = ControlPlane("gofrugal", dsn=PG_DSN, wal_dir="control_wal")
+with cp.run("nightly", run_id=run_id) as run:
+    run.event(entity="sales_item_wise", unit="1001", business_date=d,
+              status=status.EV_OK, http_status=200, rows=42,
+              source_attempts=1, sink_attempts=1, pull_ms=1200, load_ms=800)
+    run.budget(status.BUDGET_WALLTIME, limit=1800, used=1800, remaining=0, hit=True)
+    run.finish(status.RUN_PARTIAL, stop_reason=status.STOP_BUDGET_TIME,
+               n_ok=47, n_err=1, n_skipped=0)
+```
+
+`dsn=None` is the explicit no-control-plane mode for dev and test: the WAL still records
+everything, nothing ships. With a DSN configured, Postgres is a **hard dependency of the run** —
+if it is unreachable at startup the run aborts rather than loading blind, because a load with no
+provenance is worse than no load.
+
+## Tests
+
+```bash
+cd packages/feather-control-plane && uv run --package feather-control-plane pytest
+# or, from the repo root, naming the path:
+uv run --package feather-control-plane pytest packages/feather-control-plane/tests
+```
+
+`--package` picks the *environment*; it does not pick the tests. pytest takes its config from the
+nearest `pyproject.toml` above the working directory, so a bare `pytest` run from the repo root
+collects feather-flow's suite no matter what `--package` says — green, 1050 tests, and not one of
+them this package's. Either `cd` in or name the path.
+
+The pure-logic tests (WAL framing, status vocabulary, business day) run anywhere. The rest need a
+real Postgres and find one in this order: `CONTROL_PG_DSN` if set → a `postgres:16` Docker
+container → a local Postgres binary (`pg_ctl` on `PATH`, else Homebrew's keg-only
+`postgresql@N`), `initdb`'d into a temp directory on a throwaway port and torn down afterwards.
+With none of the three available they skip.

feather_control_plane-0.1.0/pyproject.toml

@@ -0,0 +1,67 @@
+[project]
+name = "feather-control-plane"
+version = "0.1.0"
+description = "Durable failure observability for an extractor fleet: WAL + Postgres control plane + liveness lease"
+readme = "README.md"
+license = "MIT"
+authors = [
+    { name = "Siraj Samsudeen", email = "mailsiraj@gmail.com" }
+]
+# The distribution is `feather-control-plane`; the IMPORT name stays `control_plane`.
+# ~40 call sites across nine services already say `from control_plane import ControlPlane`, so
+# keeping the import name makes adopting this package a dependency substitution with no code edits.
+# `>=3.12` is a CONSUMER constraint, not a language one: this code runs fine on 3.12, 3.13 and 3.14
+# (44 passed on each). Every service in JeyaramaGroup/data-warehouse that will depend on this package
+# declares `>=3.12`, and that repo's root is capped `>=3.12,<3.13` — so a floor of 3.13 here would
+# make the package uninstallable by the nine services it exists to serve. Raise this only in step
+# with them. uv locks a workspace at the INTERSECTION of its members' ranges, so this and
+# feather-flow's own floor must move together (#85).
+requires-python = ">=3.12"
+keywords = ["observability", "etl", "postgres", "write-ahead-log", "control-plane"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Database",
+    "Topic :: System :: Monitoring",
+]
+dependencies = [
+    # business_day.py builds ZoneInfo("Asia/Kolkata") AT IMPORT. psycopg already requires tzdata but
+    # ONLY under `sys_platform == 'win32'`, so on Linux it is absent and `zoneinfo` would be left
+    # depending on the image's /usr/share/zoneinfo — which bookworm-slim does not guarantee.
+    "tzdata>=2026.1",
+]
+
+[project.optional-dependencies]
+# psycopg is NOT a base dependency: `control_plane.business_day` and `control_plane.status` are
+# pure-stdlib and are imported on their own by consumers that have no database. Everything that
+# actually talks to Postgres lives behind this extra, which the lazy `__getattr__` in
+# `__init__.py` makes reachable without eagerly importing a driver.
+postgres = ["psycopg[binary]>=3.1"]
+
+[project.urls]
+Homepage = "https://github.com/siraj-samsudeen/feather-flow"
+Repository = "https://github.com/siraj-samsudeen/feather-flow"
+Issues = "https://github.com/siraj-samsudeen/feather-flow/issues"
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/control_plane"]
+
+[tool.hatch.build.targets.sdist]
+include = ["src/", "tests/", "README.md", "pyproject.toml"]
+
+[dependency-groups]
+dev = [
+    "pytest>=8.0",
+    "psycopg[binary]>=3.1",
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

feather_control_plane-0.1.0/src/control_plane/__init__.py

@@ -0,0 +1,69 @@
+"""control_plane — durable failure observability for an extractor fleet.
+
+A WAL for crash durability, a Postgres control plane for run lifecycle / events / budget
+telemetry, and a lease for deterministic death detection. The data plane lives elsewhere.
+
+Usage:
+    from control_plane import ControlPlane, status
+    cp = ControlPlane("gofrugal", dsn=PG_DSN, wal_dir="control_wal")
+    with cp.run("nightly", run_id=run_id) as run:
+        run.event(entity="sales_item_wise", unit="1001", business_date=d,
+                  status=status.EV_OK, http_status=200, rows=42,
+                  source_attempts=1, sink_attempts=1, pull_ms=1200, load_ms=800)
+        run.budget(status.BUDGET_WALLTIME, limit=1800, used=1800, remaining=0, hit=True)
+        run.finish(status.RUN_PARTIAL, stop_reason=status.STOP_BUDGET_TIME,
+                   n_ok=47, n_err=1, n_skipped=0)
+
+The facade above is resolved LAZILY (PEP 562), and that is load-bearing rather than tidy.
+`control_plane.business_day` and `control_plane.status` are pure stdlib, and consumers want them
+on their own — a cron tick asking what the business day is has no database in the picture. A
+plain `from .run import ControlPlane` at the top of this file runs on `import
+control_plane.business_day` too, because a submodule import executes its package first, so every
+such consumer was forced to have psycopg installed. That tax is why the same `business_day.py`
+ended up hand-copied into scripts that only needed `ist_today()`.
+"""
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+__all__ = ["ControlPlane", "ControlPlaneError", "RunContext", "new_run_id", "status"]
+
+# Name -> the submodule that defines it. `status` is absent deliberately: `from control_plane
+# import status` is a submodule import, which Python's import machinery already resolves on its
+# own without help from here.
+_LAZY = {
+    "ControlPlane": ".run",
+    "ControlPlaneError": ".run",
+    "RunContext": ".run",
+    "new_run_id": ".run",
+}
+
+if TYPE_CHECKING:  # the lazy names, visible to type checkers and IDEs
+    from . import status as status
+    from .run import ControlPlane as ControlPlane
+    from .run import ControlPlaneError as ControlPlaneError
+    from .run import RunContext as RunContext
+    from .run import new_run_id as new_run_id
+
+
+def __getattr__(name: str):
+    module = _LAZY.get(name)
+    if module is None:
+        raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+    import importlib
+
+    try:
+        return getattr(importlib.import_module(module, __name__), name)
+    except ModuleNotFoundError as exc:
+        if exc.name != "psycopg":
+            raise
+        raise ModuleNotFoundError(
+            f"control_plane.{name} talks to Postgres and needs the psycopg driver, which is an "
+            "optional dependency: install `feather-control-plane[postgres]`. The stdlib-only half "
+            "(control_plane.business_day, control_plane.status) needs no extra.",
+            name="psycopg",
+        ) from exc
+
+
+def __dir__() -> list[str]:
+    return sorted(__all__)

feather_control_plane-0.1.0/src/control_plane/business_day.py

@@ -0,0 +1,54 @@
+"""The warehouse's business day, in IST (#3854).
+
+Two clocks, and confusing them is the bug this module exists to prevent:
+
+  * a **business day** — a load window, a grid key, an audit window, a report "as of" —
+    is a partition of the *business*, and the business runs on IST;
+  * an **instant** — `_loaded_at`, a run id, a log stamp, an elapsed/deadline comparison,
+    a cron hour compared against `REPAIR_UTC_HOURS` — is UTC (ADR 0027).
+
+This module is the first half only. Never reach for it to stamp an instant.
+
+Import it rather than writing `ZoneInfo("Asia/Kolkata")` inline: this docstring is the one
+place the reason is recorded, and a literal carries no reason with it.
+"""
+from __future__ import annotations
+
+import datetime as dt
+from zoneinfo import ZoneInfo
+
+IST = ZoneInfo("Asia/Kolkata")
+
+
+def ist_now() -> dt.datetime:
+    """The current instant as an IST-aware datetime — for the callers that need the HOUR.
+
+    A tick that asks "is it past 22:00 IST?" needs a datetime, not a date, and building
+    `now_utc().astimezone(ZoneInfo(...))` inline is how a module ends up with two spellings of IST
+    and no rule about which to use. Use this, then `.hour` / `.date()` off it.
+
+    This is still a *business* clock, not an instant to store: stamp `_loaded_at` with
+    `datetime.now(timezone.utc)` (ADR 0027), never with this.
+    """
+    return dt.datetime.now(IST)
+
+
+def ist_today() -> dt.date:
+    """The calendar date it is *now in India* — the warehouse's business day.
+
+    Never `date.today()`. No service sets `TZ`, so every container runs UTC, and between
+    00:00 and 05:30 IST `date.today()` returns the PREVIOUS business day. Every window
+    derived from it then shifts a day, and the run still reports success: a GoFrugal
+    nightly started at 04:48 IST on 20-Sep-2026 (23:18 UTC on the 19th) pulled 18/17/16-Sep
+    and lost 19-Sep entirely, with a clean `control.run` row and an all-`ok` grid (#3854).
+    """
+    return ist_now().date()
+
+
+def ist_yesterday() -> dt.date:
+    """The business day before `ist_today()` — the usual edge / nightly anchor.
+
+    Plain calendar arithmetic, deliberately: the group's stores trade seven days a week, so
+    there is no trading-day calendar to skip weekends or holidays against.
+    """
+    return ist_today() - dt.timedelta(days=1)

feather_control_plane-0.1.0/src/control_plane/recovery.py

@@ -0,0 +1,33 @@
+"""Startup recovery: replay un-shipped WAL tails of prior runs (issue #204).
+
+ARIES-simplified. Before a run does any work it replays the WAL tails left by *prior* runs
+of this extractor on this host — pushing their last-logged events (incl. a dead run's cause)
+into Postgres. Reconciling those runs to `died` is done by the caller, *after* it acquires
+the lease (holding the lease ⇒ no prior run is alive ⇒ unfinished runs are dead), so death
+detection stays deterministic — never a runtime guess.
+"""
+from __future__ import annotations
+
+import glob
+import os
+
+import psycopg
+
+from . import shipper
+
+
+def replay_prior(conn: psycopg.Connection, extractor: str, wal_dir: str, this_run_id: str) -> int:
+    """Ship the un-shipped tail of every prior run's WAL for this extractor. Returns total
+    records shipped. Best-effort per file: one unreadable WAL never blocks the others."""
+    shipped = 0
+    pattern = os.path.join(wal_dir, extractor, "*.jsonl")
+    for path in sorted(glob.glob(pattern)):
+        run_id = os.path.basename(path)[: -len(".jsonl")]
+        if run_id == this_run_id:  # @spec prior_wal_replayed_before_lease
+            continue
+        try:
+            shipped += shipper.ship(conn, path)
+        except Exception as exc:  # noqa: BLE001 — one bad WAL must not abort recovery
+            print(f"[control_plane] WAL replay failed for {path}: "
+                  f"{type(exc).__name__}: {str(exc)[:160]}", flush=True)
+    return shipped

feather_control_plane-0.1.0/src/control_plane/run.py

@@ -0,0 +1,237 @@
+"""The consumer-facing facade (issue #204).
+
+An extractor wraps its work in `ControlPlane(...).run(mode) as run:` and calls `run.event(...)`
+per unit, `run.budget(...)` for quota telemetry, and `run.finish(...)` with final counts. The
+facade handles WAL durability, ship-as-you-go to Postgres, startup recovery, the lease, and a
+background renewer thread.
+
+Postgres is a **hard dependency** (ADR 0028): if a DSN is configured and unreachable at startup,
+the run **aborts** — no blind runs (a silent load with no provenance is worse than no load). The
+WAL is a local **crash redo-log** *under* the Postgres writes, not an outage buffer: every event
+hits the WAL first (fsync), then ships to PG; if the process dies, the next run replays the tail so
+the death's breadcrumbs survive ("died after table X / chunk Y"), and the lease supplies the fact
+of death. `dsn=None` is the explicit no-control-plane mode for dev/test (WAL-only).
+"""
+from __future__ import annotations
+
+import datetime as _dt
+import os
+import secrets
+import socket
+import threading
+from contextlib import contextmanager
+
+import psycopg
+
+from . import recovery, shipper, status, store
+from .status import ERROR_DETAIL_CAP
+from .wal import Wal
+
+
+def _now() -> _dt.datetime:
+    return _dt.datetime.now(_dt.timezone.utc)
+
+
+def new_run_id(extractor: str) -> str:  # @spec run_id_unique_across_extractors
+    """`<UTC second>-<extractor>-<6 hex>`. `control.run` is keyed on run_id alone, and every Railway
+    container is pid 1, so the old `<second>-<pid>` gave two services starting in the same second
+    one id (#4614). The timestamp stays first: SAP names backups `run_id[:8]`."""
+    return f"{_now().strftime('%Y%m%dT%H%M%SZ')}-{extractor}-{secrets.token_hex(3)}"
+
+
+class ControlPlaneError(RuntimeError):
+    pass
+
+
+class _Renewer:
+    """Renews the lease on its OWN connection, decoupled from the work loop — so a 14h job or
+    a 90-min SAP chunk keeps proving liveness and is never falsely declared dead. Swallows
+    transient PG errors and retries on the next tick (reconnecting if needed)."""
+
+    def __init__(self, dsn: str, extractor: str, run_id: str, ttl_s: int, interval_s: int):
+        self._dsn = dsn
+        self._extractor = extractor
+        self._run_id = run_id
+        self._ttl = ttl_s
+        self._interval = interval_s
+        self._stop = threading.Event()
+        self._thread = threading.Thread(target=self._loop, name="cp-lease-renewer", daemon=True)
+        self._conn: psycopg.Connection | None = None
+
+    def start(self) -> None:
+        self._thread.start()
+
+    def stop(self) -> None:
+        self._stop.set()
+        self._thread.join(timeout=self._interval + 5)
+        if self._conn is not None:
+            try:
+                self._conn.close()
+            except Exception:  # noqa: BLE001
+                pass
+
+    def _loop(self) -> None:
+        while not self._stop.wait(self._interval):  # @spec lease_renewed_in_background
+            try:
+                if self._conn is None or self._conn.closed:
+                    self._conn = psycopg.connect(self._dsn, connect_timeout=10)
+                if not store.lease_renew(self._conn, self._extractor, self._run_id, self._ttl):
+                    # @spec lease_loss_does_not_stop_run
+                    print(f"[control_plane] lease lost for {self._extractor}/{self._run_id} "
+                          "(another run reclaimed it?)", flush=True)
+            except Exception as exc:  # noqa: BLE001 — renew must never crash the run  # @spec false_death_of_live_run
+                print(f"[control_plane] lease renew failed (will retry): "
+                      f"{type(exc).__name__}: {str(exc)[:160]}", flush=True)
+                if self._conn is not None:
+                    try:
+                        self._conn.close()
+                    except Exception:  # noqa: BLE001
+                        pass
+                self._conn = None
+
+
+class RunContext:
+    def __init__(self, cp: "ControlPlane", run_id: str, wal: Wal, conn: psycopg.Connection | None):
+        self.cp = cp
+        self.run_id = run_id
+        self.wal = wal
+        self.conn = conn            # None ⇒ PG was down at start; WAL-only degraded mode
+        self._seq = 0
+        self._finished = False
+
+    # --- internal ---
+    def _ship_now(self) -> None:
+        if self.conn is None:
+            return
+        try:
+            shipper.ship(self.conn, self.wal.path)
+        except Exception as exc:  # noqa: BLE001 — WAL already holds it; recovery ships later  # @spec ship_as_you_go
+            print(f"[control_plane] ship failed (buffered in WAL): "
+                  f"{type(exc).__name__}: {str(exc)[:160]}", flush=True)
+
+    def _emit(self, record: dict) -> None:
+        self.wal.append(record)   # durable first  # @spec wal_before_network
+        self._ship_now()          # then best-effort to PG
+
+    # --- public API ---
+    def event(self, *, entity: str, status: str, unit: str | None = None,
+              business_date=None, http_status: int | None = None, rows: int | None = None,
+              source_attempts: int | None = None, sink_attempts: int | None = None,
+              pull_ms: int | None = None, load_ms: int | None = None,
+              error_detail: str | None = None, extra: dict | None = None) -> None:
+        """Record one unit/cell outcome (one row in control.event). `status` here is the
+        event-status string (status.EV_*), not the module. `extra` is a small JSONB bag for
+        source-specific metrics that don't warrant a shared column (e.g. Zakya's n_requests /
+        gen_ms / rate_remaining); leave it None when there's nothing source-specific to keep."""
+        self._emit({
+            "kind": "event", "extractor": self.cp.extractor, "run_id": self.run_id,
+            "seq": self._seq, "ran_at": _now(), "entity": entity, "unit": unit,
+            "business_date": business_date, "status": status, "http_status": http_status,
+            "rows": rows, "source_attempts": source_attempts, "sink_attempts": sink_attempts,
+            "pull_ms": pull_ms, "load_ms": load_ms,
+            "error_detail": (error_detail[:ERROR_DETAIL_CAP] if error_detail else None),  # @spec error_detail_capped
+            "extra": extra,
+        })
+        self._seq += 1  # @spec event_one_row_per_unit
+
+    def budget(self, name: str, *, limit: int | None, used: int | None,
+               remaining: int | None, hit: bool) -> None:
+        """Record quota utilisation (every run, not only on a hit) → headroom is queryable."""
+        self._emit({
+            "kind": "budget", "extractor": self.cp.extractor, "run_id": self.run_id,
+            "budget_name": name, "limit_val": limit, "used": used, "remaining": remaining,
+            "hit": hit, "ran_at": _now(),
+        })
+
+    def finish(self, run_status: str, *, stop_reason: str, n_ok: int | None = None,
+               n_err: int | None = None, n_skipped: int | None = None) -> None:
+        """Close the run with its final disposition. Idempotent (first call wins)."""
+        if self._finished:  # @spec run_finish_first_call_wins
+            return
+        self._emit({
+            "kind": "run_finish", "extractor": self.cp.extractor, "run_id": self.run_id,
+            "started_at": self.cp._started_at, "finished_at": _now(),
+            "status": run_status, "stop_reason": stop_reason,
+            "n_ok": n_ok, "n_err": n_err, "n_skipped": n_skipped,
+        })
+        self._finished = True
+
+
+class ControlPlane:
+    def __init__(self, extractor: str, dsn: str | None, wal_dir: str, *,
+                 lease_ttl_s: int = 300, renew_interval_s: int = 60, host: str | None = None):
+        self.extractor = extractor
+        self.dsn = dsn
+        self.wal_dir = wal_dir
+        self.lease_ttl_s = lease_ttl_s
+        self.renew_interval_s = renew_interval_s
+        self.host = host or socket.gethostname()
+        self._started_at: str | None = None
+
+    @contextmanager
+    def run(self, mode: str, *, run_id: str | None = None):
+        run_id = run_id or new_run_id(self.extractor)
+        self._started_at = _now().isoformat()
+        wal = Wal(os.path.join(self.wal_dir, self.extractor, run_id + ".jsonl"))
+        conn: psycopg.Connection | None = None
+        renewer: _Renewer | None = None
+
+        # Postgres is a HARD dependency (ADR 0028): if a DSN is configured, the control plane
+        # MUST be reachable or the run aborts — never run blind. A failure here propagates out of
+        # the `with` before any work begins. (dsn=None is the explicit no-control-plane mode for
+        # dev/test → WAL-only.)
+        if self.dsn:  # @spec hard_startup_dependency
+            conn = psycopg.connect(self.dsn, connect_timeout=10)
+            try:
+                store.ensure_schema(conn)
+                recovery.replay_prior(conn, self.extractor, self.wal_dir, run_id)
+                # @spec lease_refuses_second_writer
+                if not store.lease_acquire(conn, self.extractor, run_id, self.lease_ttl_s):
+                    raise ControlPlaneError(
+                        f"another live run holds the lease for {self.extractor!r} "
+                        "(fresh lease present) — refusing to start a second writer")
+                store.reconcile_dead(conn, self.extractor, run_id)  # @spec death_visible_only_at_next_start
+                renewer = _Renewer(self.dsn, self.extractor, run_id,
+                                   self.lease_ttl_s, self.renew_interval_s)
+                renewer.start()
+            except BaseException:
+                # setup failed — clean up and abort; never degrade to a blind run
+                if renewer is not None:
+                    renewer.stop()
+                try:
+                    conn.close()
+                except Exception:  # noqa: BLE001
+                    pass
+                raise
+
+        rc = RunContext(self, run_id, wal, conn)
+        rc._emit({  # @spec run_start_recorded_before_work
+            "kind": "run_start", "run_id": run_id, "extractor": self.extractor,
+            "mode": mode, "host": self.host, "pid": os.getpid(),
+            "started_at": self._started_at, "status": status.RUN_RUNNING,
+            "wal_path": wal.path,
+        })
+
+        try:
+            yield rc
+        except BaseException:
+            if not rc._finished:
+                rc.finish(status.RUN_FAILED, stop_reason=status.STOP_ERROR)  # @spec implicit_finish_on_exit.sys_exit_inside_block
+            raise
+        finally:
+            if not rc._finished:
+                rc.finish(status.RUN_OK, stop_reason=status.STOP_CLEAN)  # @spec implicit_finish_on_exit
+            if renewer is not None:
+                renewer.stop()
+            if conn is not None:
+                try:
+                    store.lease_release(conn, self.extractor, run_id)  # @spec lease_released_on_exit
+                    shipper.ship(conn, wal.path)  # final flush  # @spec final_flush_on_exit
+                except Exception as exc:  # noqa: BLE001
+                    print(f"[control_plane] final flush/release failed (WAL retained): "
+                          f"{type(exc).__name__}: {str(exc)[:160]}", flush=True)
+                finally:
+                    try:
+                        conn.close()
+                    except Exception:  # noqa: BLE001
+                        pass