detectkit 0.60.0__tar.gz → 0.61.0__tar.gz

{detectkit-0.60.0/detectkit.egg-info → detectkit-0.61.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: detectkit
-Version: 0.60.0
+Version: 0.61.0
 Summary: Metric monitoring with automatic anomaly detection
 Author: detectkit team
 License: MIT
@@ -104,6 +104,7 @@ Dynamic: license-file
 - **Alert lifecycle** — consecutive anomalies, fraction-of-window rule (`anomaly_window` + `min_anomaly_share`), cooldown, recovery notifications, no-data alerts
 - **Project-level error alerts** — catch DB outages and pipeline crashes once per run
 - **Database agnostic** — ClickHouse, PostgreSQL, MySQL/MariaDB, DuckDB (no server needed)
+- **Hybrid mode** — read metric SQL from a billed-per-query warehouse while all pipeline state lives in a cheap local database
 - **Idempotent** — resume from interruptions, no duplicate processing
 - **CLI** — `dtk init`, `dtk run --select`, `dtk unlock`, `dtk clean`, tag-based selectors
 - **AI-native onboarding** — `dtk init-claude` sets up Claude Code context (CLAUDE.md + rules + three skills) so an assistant can scaffold metrics, configure databases, and file feedback upstream

{detectkit-0.60.0 → detectkit-0.61.0}/README.md

@@ -25,6 +25,7 @@
 - **Alert lifecycle** — consecutive anomalies, fraction-of-window rule (`anomaly_window` + `min_anomaly_share`), cooldown, recovery notifications, no-data alerts
 - **Project-level error alerts** — catch DB outages and pipeline crashes once per run
 - **Database agnostic** — ClickHouse, PostgreSQL, MySQL/MariaDB, DuckDB (no server needed)
+- **Hybrid mode** — read metric SQL from a billed-per-query warehouse while all pipeline state lives in a cheap local database
 - **Idempotent** — resume from interruptions, no duplicate processing
 - **CLI** — `dtk init`, `dtk run --select`, `dtk unlock`, `dtk clean`, tag-based selectors
 - **AI-native onboarding** — `dtk init-claude` sets up Claude Code context (CLAUDE.md + rules + three skills) so an assistant can scaffold metrics, configure databases, and file feedback upstream

{detectkit-0.60.0 → detectkit-0.61.0}/detectkit/__init__.py

@@ -4,7 +4,7 @@ detectk - Anomaly Detection for Time-Series Metrics
 A Python library for data analysts and engineers to monitor metrics with automatic anomaly detection.
 """
 
-__version__ = "0.60.0"
+__version__ = "0.61.0"
 
 from detectkit.core.interval import Interval
 from detectkit.core.models import ColumnDefinition, TableModel

{detectkit-0.60.0 → detectkit-0.61.0}/detectkit/cli/assets/claude/rules/metrics.md

@@ -11,6 +11,7 @@ keep them in sync. Detectors are covered in `detectors.md`, alerting in
 name: api_response_time        # required, unique across the project
 description: API p95 latency   # optional, shown in alerts
 profile: prod                  # optional, overrides project default_profile
+source_profile: warehouse      # optional — hybrid mode: source-DB profile for THIS metric's SQL (see below)
 enabled: true                  # optional, false → skipped by `dtk run`
 tags: [critical, api]          # optional, used by `--select tag:<t>`
 ai_context:                    # optional — OSI-compatible grounding (descriptive only)
@@ -67,6 +68,19 @@ false_alert_budget: 0.3        # optional — `dtk tune` target false-alert rate
 project-wide default; unset → project, then a built-in `0.5`. Tuning-only — it never
 affects the load/detect/alert pipeline.
 
+`source_profile` is **hybrid mode** for this one metric: the `profiles.yml`
+profile whose database runs *this metric's* SQL query, while `_dtk_*` state
+(datapoints, detections, task locks, alert state) stays in the active
+**state** profile — the one `dtk run` is already connected to for everything
+else. Overrides the project-wide `source_profile` (`project.md`); resolves
+**metric → project → unset** (unset on both = today's behavior — one profile
+for everything). Not the same field as `profile` above (`profile` is dead —
+not applied at runtime by `dtk run`; `source_profile` is live). Only the
+metric's **load** step ever connects to it — detect/alert and every other
+command don't — though every `dtk run` fail-fast validates the resolved name
+up front regardless of `--steps` (a typo exits 1 before any pipeline work).
+See the [Hybrid Mode guide](https://dtk.pipelab.dev/guides/hybrid-mode/).
+
 `ai_context` is **OSI-compatible grounding** you can add to any metric with **no
 OSI model needed** — the metric's business meaning (`instructions`), alternative
 names (`synonyms`) and example values (`examples`), mirroring the

{detectkit-0.60.0 → detectkit-0.61.0}/detectkit/cli/assets/claude/rules/project.md

@@ -33,6 +33,7 @@ timeouts:                      # per-step, seconds
 alert_help_url: null           # optional, see below — "How to read this alert" link
 false_alert_budget: null       # optional — `dtk tune` target FDR (0,1]; per-metric override wins
 loading_delay: null            # optional — default data-maturity delay; per-metric override wins
+source_profile: null           # optional — hybrid mode: default source-DB profile; per-metric override wins
 
 error_alerting:                # optional, see below
   enabled: false
@@ -50,6 +51,26 @@ overrides it (`0` opts that metric out); see `metrics.md` for the full
 load/no-data behavior and the detection-latency trade-off. Only set this
 project-wide when your metrics genuinely share the same upstream schedule.
 
+`source_profile` is **hybrid mode**: a project-wide default `profiles.yml`
+profile whose database runs metric SQL, while *every* `_dtk_*` table
+(datapoints, detections, task locks, alert state) stays in the active
+**state** profile — the one `dtk run` is already connected to for everything
+else. A per-metric `source_profile` overrides it (`metrics.md`); resolves
+**metric → project → unset**, same precedence as `loading_delay`. Unset on
+both = today's behavior (one profile does everything). Only the LOAD step's
+metric-SQL query is affected — detect/alert and every other command
+(`dtk autotune`/`tune`/`ui`/`clean`/`unlock`) only ever touch the state
+profile. A source-side failure raises `SourceDatabaseError` (message leads
+with `source database (profile '<name>'): ...`), distinct from a plain
+exception for a state-side failure — useful for telling a warehouse outage
+apart from a state-DB outage in `error_alerting`'s `{error_type}` /
+`{error_message}`. `dtk run` validates every resolved `source_profile` name
+against `profiles.yml` before opening any connection (unknown name → exit
+`1`, no partial run). Full config example + operational caveats (connecting a
+profile always runs `CREATE DATABASE`/`SCHEMA IF NOT EXISTS` for its own
+`internal_*` location too, even a source-only one) in the [Hybrid Mode
+guide](https://dtk.pipelab.dev/guides/hybrid-mode/).
+
 ### `alert_help_url` — "How to read this alert" link
 
 Every default-rendered alert on every channel carries a `How to read this alert`

{detectkit-0.60.0 → detectkit-0.61.0}/detectkit/cli/commands/run.py

@@ -13,7 +13,7 @@ from typing import Any
 
 import click
 
-from detectkit.config.metric_config import MetricConfig
+from detectkit.config.metric_config import MetricConfig, resolve_source_profile
 from detectkit.config.profile import ProfilesConfig
 from detectkit.config.project_config import ProjectConfig
 from detectkit.config.validator import (
@@ -326,6 +326,21 @@ def _run_impl(
         summary["error"] = f"Error loading profiles.yml: {e}"
         return 1
 
+    # Fail-fast: every selected metric's RESOLVED source_profile (hybrid
+    # mode) must name a real profile. This is a cheap name check — no
+    # connections opened — so a typo'd source_profile fails the whole run
+    # up front instead of surfacing deep inside whichever metric's LOAD step
+    # happens to hit it first.
+    source_profile_error = _validate_source_profiles(metrics, project_config, profiles_config)
+    if source_profile_error:
+        # A config typo, not an outage: exit 1 without paging error_alerting,
+        # consistent with the sibling config-error paths (bad selector,
+        # unparseable profiles.yml). The error alert channel is reserved for
+        # DB-down / DDL / runtime failures.
+        click.echo(click.style(f"Error: {source_profile_error}", fg="red", bold=True))
+        summary["error"] = f"Error: {source_profile_error}"
+        return 1
+
     # Create database manager
     try:
         db_manager = profiles_config.create_manager(profile)
@@ -372,84 +387,136 @@ def _run_impl(
         summary["error"] = f"Error initializing internal tables: {e}"
         return 1
 
+    # The active STATE profile's actual name (mirrors ProfilesConfig.get_profile's
+    # own None -> default_profile resolution) — lets the task manager recognize
+    # a metric's source_profile that happens to name the state profile itself,
+    # and reuse db_manager instead of opening a duplicate connection.
+    state_profile_name: str | None
+    if profile is not None:
+        state_profile_name = profile
+    else:
+        state_profile_name = getattr(profiles_config, "default_profile", None)
+
     # Create task manager
     task_manager = TaskManager(
         internal_manager=internal_manager,
         db_manager=db_manager,
         profiles_config=profiles_config,
         project_config=project_config,
+        state_profile_name=state_profile_name,
     )
 
-    # Process each metric
-    for index, (metric_path, config) in enumerate(metrics):
-        result = process_metric(
-            metric_path=metric_path,
-            config=config,
-            project_root=project_root,
-            task_manager=task_manager,
-            steps=step_list,
-            from_date=from_dt,
-            to_date=to_dt,
-            full_refresh=full_refresh,
-            force=force,
-        )
-        summary["metrics"].append(
-            {
-                "name": config.name,
-                "status": _status_str(result["status"]),
-                "steps_completed": [s.value for s in result["steps_completed"]],
-                "datapoints_loaded": result["datapoints_loaded"],
-                "anomalies_detected": result["anomalies_detected"],
-                "alerts_sent": result["alerts_sent"],
-                "error": result["error"],
-            }
-        )
-
-        # Project-level error alert was dispatched — stop processing the
-        # rest of the metrics. The DB / source is presumed unreachable;
-        # subsequent metrics would all fail with the same error.
-        if result.get("abort_run"):
-            click.echo(
-                click.style(
-                    "✗ Aborting run after project error alert. " "Remaining metrics skipped.",
-                    fg="red",
-                    bold=True,
-                )
+    try:
+        # Process each metric
+        for index, (metric_path, config) in enumerate(metrics):
+            result = process_metric(
+                metric_path=metric_path,
+                config=config,
+                project_root=project_root,
+                task_manager=task_manager,
+                steps=step_list,
+                from_date=from_dt,
+                to_date=to_dt,
+                full_refresh=full_refresh,
+                force=force,
+            )
+            summary["metrics"].append(
+                {
+                    "name": config.name,
+                    "status": _status_str(result["status"]),
+                    "steps_completed": [s.value for s in result["steps_completed"]],
+                    "datapoints_loaded": result["datapoints_loaded"],
+                    "anomalies_detected": result["anomalies_detected"],
+                    "alerts_sent": result["alerts_sent"],
+                    "error": result["error"],
+                }
             )
-            summary["aborted"] = True
-            for _, skipped_config in metrics[index + 1 :]:
-                summary["metrics"].append(
-                    {
-                        "name": skipped_config.name,
-                        "status": "skipped",
-                        "steps_completed": [],
-                        "datapoints_loaded": 0,
-                        "anomalies_detected": 0,
-                        "alerts_sent": 0,
-                        "error": None,
-                    }
-                )
-            break
 
-        # Optional: emit a self-contained HTML report from the freshly-persisted
-        # internal tables (values + bands + anomalies + replayed alerts).
-        if report_path is not None:
-            try:
-                emit_metric_report(
-                    config=config,
-                    project_root=project_root,
-                    internal_manager=internal_manager,
-                    report_path=report_path,
-                    project_name=getattr(project_config, "name", None),
-                    project_loading_delay=getattr(project_config, "loading_delay", None),
-                    from_dt=from_dt,
-                    to_dt=to_dt,
+            # Project-level error alert was dispatched — stop processing the
+            # rest of the metrics. The DB / source is presumed unreachable;
+            # subsequent metrics would all fail with the same error.
+            if result.get("abort_run"):
+                click.echo(
+                    click.style(
+                        "✗ Aborting run after project error alert. " "Remaining metrics skipped.",
+                        fg="red",
+                        bold=True,
+                    )
                 )
-            except Exception as report_error:  # never fail the run on a report
-                click.echo(click.style(f"  │ Report skipped: {report_error}", fg="yellow"))
-
-    failed = any(m["status"] == "failed" for m in summary["metrics"])
-    return 1 if (failed or summary["aborted"]) else 0
+                summary["aborted"] = True
+                for _, skipped_config in metrics[index + 1 :]:
+                    summary["metrics"].append(
+                        {
+                            "name": skipped_config.name,
+                            "status": "skipped",
+                            "steps_completed": [],
+                            "datapoints_loaded": 0,
+                            "anomalies_detected": 0,
+                            "alerts_sent": 0,
+                            "error": None,
+                        }
+                    )
+                break
+
+            # Optional: emit a self-contained HTML report from the freshly-persisted
+            # internal tables (values + bands + anomalies + replayed alerts).
+            if report_path is not None:
+                try:
+                    emit_metric_report(
+                        config=config,
+                        project_root=project_root,
+                        internal_manager=internal_manager,
+                        report_path=report_path,
+                        project_name=getattr(project_config, "name", None),
+                        project_loading_delay=getattr(project_config, "loading_delay", None),
+                        from_dt=from_dt,
+                        to_dt=to_dt,
+                    )
+                except Exception as report_error:  # never fail the run on a report
+                    click.echo(click.style(f"  │ Report skipped: {report_error}", fg="yellow"))
+
+        failed = any(m["status"] == "failed" for m in summary["metrics"])
+        return 1 if (failed or summary["aborted"]) else 0
+    finally:
+        # Close every pooled hybrid-mode SOURCE manager (never db_manager
+        # itself — this function owns that connection's lifecycle, and it
+        # was never explicitly closed here either, before or after hybrid
+        # mode existed). Duck-typed lookup: a test double standing in for
+        # TaskManager need not implement it.
+        close_sources = getattr(task_manager, "close_sources", None)
+        if close_sources is not None:
+            close_sources()
+
+
+def _validate_source_profiles(
+    metrics: list[tuple[Path, MetricConfig]],
+    project_config: ProjectConfig,
+    profiles_config: ProfilesConfig,
+) -> str | None:
+    """Cheap, connection-free check that every metric's RESOLVED
+    ``source_profile`` (hybrid mode) names a real profile.
+
+    Returns an error message naming every offending metric, or ``None`` when
+    every resolved name is either unset (no hybrid override) or a known
+    profile. Runs before any database manager is built.
+    """
+    project_source_profile = getattr(project_config, "source_profile", None)
+    known_profiles = getattr(profiles_config, "profiles", {}) or {}
+    unknown: dict[str, str] = {}
+    for _, config in metrics:
+        name = resolve_source_profile(
+            getattr(config, "source_profile", None), project_source_profile
+        )
+        if name is not None and name not in known_profiles:
+            unknown[config.name] = name
+    if not unknown:
+        return None
+    details = ", ".join(f"{m} -> '{p}'" for m, p in sorted(unknown.items()))
+    available = ", ".join(sorted(known_profiles.keys()))
+    return (
+        f"Unknown source_profile referenced by metric(s): {details}. "
+        f"Available profiles: {available}"
+    )
 
 
 def _resolve_report_path(report_path: str, project_root: Path, metric_name: str) -> Path:

{detectkit-0.60.0 → detectkit-0.61.0}/detectkit/config/metric_config.py

@@ -627,6 +627,8 @@ class MetricConfig(BaseModel):
         description: Optional metric description (supports multi-line text)
         tags: Optional list of tags for metric selection (e.g., ["critical", "api"])
         profile: Profile name to use (overrides default_profile from project config)
+        source_profile: Hybrid mode — profile whose database runs this metric's
+            SQL, while _dtk_* state stays in the active (state) profile
         query: Inline SQL query (mutually exclusive with query_file)
         query_file: Path to SQL file (mutually exclusive with query)
         query_columns: Column name mapping for query results
@@ -684,6 +686,22 @@ class MetricConfig(BaseModel):
     profile: str | None = Field(
         default=None, description="Profile name to use (overrides default_profile)"
     )
+    # Hybrid mode: run this metric's SQL against a different database than the
+    # one holding _dtk_* state. Unlike `profile` above (which is dead — only
+    # round-tripped by autotune's config emitter), this is live: the load step
+    # resolves it to a pooled `ProfilesConfig.create_manager(...)` and executes
+    # the metric's query through it, while every other step (detect/alert) and
+    # save_datapoints itself keep using the active state profile. Falls back to
+    # ProjectConfig.source_profile, then None (= use the state profile).
+    source_profile: str | None = Field(
+        default=None,
+        description=(
+            "profiles.yml profile whose database runs this metric's SQL; "
+            "_dtk_* state stays in the active (state) profile. Falls back to "
+            "the project-level source_profile, then to the state profile "
+            "itself."
+        ),
+    )
     query: str | None = Field(default=None, description="Inline SQL query")
     query_file: Path | None = Field(default=None, description="Path to SQL file")
     query_columns: QueryColumnsConfig | None = Field(
@@ -1054,6 +1072,24 @@ def resolve_loading_delay_seconds(
     return 0
 
 
+def resolve_source_profile(
+    metric_source_profile: str | None, project_source_profile: str | None
+) -> str | None:
+    """Resolve the profiles.yml profile that should run a metric's SQL:
+    metric -> project -> ``None``.
+
+    ``None`` means "run the SQL through the same connection detectkit is
+    already using for ``_dtk_*`` state" — i.e. hybrid mode is off for this
+    metric and the meaning of ``dtk run --profile`` / ``default_profile`` is
+    unchanged. The first configured value wins, mirroring
+    :func:`resolve_loading_delay_seconds`'s metric -> project precedence.
+    """
+    for value in (metric_source_profile, project_source_profile):
+        if value is not None:
+            return value
+    return None
+
+
 def resolve_grid_phase_seconds(loading_start_time: str | None, interval_seconds: int) -> int:
     """Phase of the metric's interval grid on the epoch clock, in ``[0, interval)``.
 

{detectkit-0.60.0 → detectkit-0.61.0}/detectkit/config/project_config.py

@@ -139,6 +139,8 @@ class ProjectConfig(BaseModel):
         tables: Default table names
         timeouts: Operation timeouts
         default_profile: Default database profile to use
+        source_profile: Default hybrid-mode source profile (see
+            MetricConfig.source_profile)
 
     Example YAML:
         ```yaml
@@ -224,6 +226,19 @@ class ProjectConfig(BaseModel):
             "loading_delay overrides it; 0 on a metric opts out."
         ),
     )
+    # Project-wide default source profile for hybrid mode (see
+    # MetricConfig.source_profile). Useful when most metrics read from the
+    # same source database that differs from where _dtk_* state lives. A
+    # per-metric source_profile overrides it; unset on both means every
+    # metric's SQL runs through the active state profile (today's behavior).
+    source_profile: str | None = Field(
+        default=None,
+        description=(
+            "Default profiles.yml profile whose database runs metric SQL "
+            "(hybrid mode); _dtk_* state stays in the active (state) "
+            "profile. Per-metric source_profile overrides it."
+        ),
+    )
 
     @field_validator("loading_delay")
     @classmethod

detectkit-0.61.0/detectkit/loaders/errors.py

@@ -0,0 +1,29 @@
+"""Typed errors for the loading layer.
+
+Kept separate (mirroring ``semantic/errors.py``) so a source-database failure
+in hybrid mode can be told apart from a state-database failure at a glance —
+in logs, in the project-level error alert, and by any caller inspecting
+``type(exc)``.
+"""
+
+from __future__ import annotations
+
+
+class SourceDatabaseError(Exception):
+    """A metric's SOURCE-profile database failed (hybrid mode).
+
+    Wraps the original exception with the source profile's name. Raised only
+    around the two places that actually talk to a source-profile connection:
+    ``MetricLoader.load``'s query execution, and building a pooled
+    source-profile manager in ``TaskManager``. Every other failure (saving to
+    _dtk_* state, detect, alert, or a non-hybrid metric's query — which runs
+    through the same connection as state) stays a plain, unwrapped exception,
+    since there is nothing to disambiguate in that case.
+    """
+
+    def __init__(self, profile_name: str, original: BaseException) -> None:
+        self.profile_name = profile_name
+        self.original = original
+        super().__init__(
+            f"source database (profile '{profile_name}'): " f"{type(original).__name__}: {original}"
+        )

{detectkit-0.60.0 → detectkit-0.61.0}/detectkit/loaders/metric_loader.py

@@ -16,6 +16,7 @@ import numpy as np
 from detectkit.config.metric_config import MetricConfig
 from detectkit.database.internal_tables import InternalTablesManager
 from detectkit.database.manager import BaseDatabaseManager
+from detectkit.loaders.errors import SourceDatabaseError
 from detectkit.loaders.query_template import QueryTemplate
 from detectkit.utils.datetime_utils import now_utc_naive, to_naive_utc
 from detectkit.utils.json_utils import json_dumps_sorted
@@ -50,18 +51,30 @@ class MetricLoader:
         config: MetricConfig,
         db_manager: BaseDatabaseManager,
         internal_manager: InternalTablesManager,
+        source_profile_name: str | None = None,
     ):
         """
         Initialize metric loader.
 
         Args:
             config: Metric configuration
-            db_manager: Database manager for executing queries
+            db_manager: Database manager for executing the metric's SQL. In
+                hybrid mode this is a SOURCE-profile manager, distinct from
+                the manager holding _dtk_* state; ``internal_manager`` below
+                always talks to state.
             internal_manager: Internal tables manager for saving data
+            source_profile_name: Name of the source profile *db_manager*
+                belongs to, when it differs from the active state profile
+                (hybrid mode). ``None`` (the default) means *db_manager* IS
+                the state manager, so a query failure is indistinguishable
+                from a state failure and is left unwrapped. When set, a
+                query failure is wrapped in :class:`SourceDatabaseError` so
+                it can be told apart from a state-database failure.
         """
         self.config = config
         self.db_manager = db_manager
         self.internal_manager = internal_manager
+        self.source_profile_name = source_profile_name
         self.query_template = QueryTemplate()
 
     def load(
@@ -121,8 +134,15 @@ class MetricLoader:
             interval_seconds=interval_seconds,
         )
 
-        # Execute query
-        results = self.db_manager.execute_query(rendered_query)
+        # Execute query. In hybrid mode (source_profile_name set) a failure
+        # here is wrapped so it reads as "source database down", not an
+        # indistinguishable state-database failure.
+        try:
+            results = self.db_manager.execute_query(rendered_query)
+        except Exception as exc:
+            if self.source_profile_name is not None:
+                raise SourceDatabaseError(self.source_profile_name, exc) from exc
+            raise
 
         if not results:
             # No data - return empty arrays

{detectkit-0.60.0 → detectkit-0.61.0}/detectkit/orchestration/task_manager/_base.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 from datetime import datetime
+from typing import Any
 
 from detectkit.alerting.channels.base import BaseAlertChannel
 from detectkit.alerting.channels.factory import AlertChannelFactory
@@ -11,8 +12,10 @@ from detectkit.config.metric_config import (
     MetricConfig,
     resolve_grid_phase_seconds,
     resolve_loading_delay_seconds,
+    resolve_source_profile,
 )
 from detectkit.database.internal_tables import InternalTablesManager
+from detectkit.loaders.errors import SourceDatabaseError
 
 
 class _TaskManagerBase:
@@ -24,11 +27,25 @@ class _TaskManagerBase:
         db_manager,  # BaseDatabaseManager — typed loosely to avoid an import cycle
         profiles_config=None,
         project_config=None,
+        state_profile_name: str | None = None,
     ):
         self.internal = internal_manager
         self.db_manager = db_manager
         self.profiles_config = profiles_config
         self.project_config = project_config
+        # The active STATE profile's name (the one `db_manager` belongs to),
+        # so a metric's resolved `source_profile` can be compared against it —
+        # when they match, hybrid mode is a no-op and `db_manager` is reused
+        # instead of opening a duplicate connection to the same database.
+        # Optional: only needed to short-circuit that comparison; a caller
+        # that never sets it just always pools a source manager by name (fine,
+        # `create_manager` gives an equivalent connection either way).
+        self.state_profile_name = state_profile_name
+        # Lazy pool of SOURCE-profile managers for hybrid mode, keyed by
+        # profile name. A failed construction is cached too (as the raised
+        # SourceDatabaseError) so a bad/unreachable source profile isn't
+        # retried once per metric that references it.
+        self._source_managers: dict[str, Any] = {}
         # In-process flag: dispatch project-level error alert at most once
         # per run. Abort propagation is signalled via result["abort_run"].
         self._error_alert_sent_in_run = False
@@ -57,6 +74,69 @@ class _TaskManagerBase:
             config.get_interval().seconds,
         )
 
+    def _resolve_source_manager(self, config: MetricConfig) -> tuple[Any, str | None]:
+        """Resolve the database manager that should run *config*'s SQL.
+
+        Returns ``(manager, source_profile_name)``:
+
+        - ``source_profile_name`` is ``None`` when the metric has no hybrid
+          override, or its resolved ``source_profile`` names the active
+          STATE profile — in both cases the returned manager IS
+          ``self.db_manager`` (no duplicate connection, and a query failure
+          there is left unwrapped since it's the same connection as state).
+        - Otherwise a manager for the named source profile is built lazily
+          (once per profile — cached in ``self._source_managers``, including
+          a cached failure so a bad profile isn't retried per metric) and
+          returned alongside its name, so the LOAD step can tag it onto
+          :class:`MetricLoader` for :class:`SourceDatabaseError` wrapping.
+
+        Only the LOAD step calls this — detect/alert never touch a source
+        connection.
+        """
+        name = resolve_source_profile(
+            getattr(config, "source_profile", None),
+            getattr(self.project_config, "source_profile", None),
+        )
+        if name is None or name == self.state_profile_name:
+            return self.db_manager, None
+
+        cached = self._source_managers.get(name)
+        if cached is not None:
+            if isinstance(cached, SourceDatabaseError):
+                raise cached
+            return cached, name
+
+        try:
+            if self.profiles_config is None:
+                raise ValueError(
+                    "no profiles configuration available to resolve source " f"profile '{name}'"
+                )
+            manager = self.profiles_config.create_manager(name)
+        except Exception as exc:
+            err = SourceDatabaseError(name, exc)
+            self._source_managers[name] = err
+            raise err from exc
+
+        self._source_managers[name] = manager
+        return manager, name
+
+    def close_sources(self) -> None:
+        """Close every pooled SOURCE-profile manager.
+
+        Never touches ``self.db_manager`` — the caller (``dtk run``) owns
+        that connection's lifecycle. Best-effort: a close failure on one
+        pooled manager is swallowed so it can't mask the run's real outcome
+        or block closing the rest.
+        """
+        for cached in self._source_managers.values():
+            if isinstance(cached, SourceDatabaseError):
+                continue
+            try:
+                cached.close()
+            except Exception:
+                pass
+        self._source_managers.clear()
+
     def _load_recent_detections(
         self,
         metric_name: str,

{detectkit-0.60.0 → detectkit-0.61.0}/detectkit/orchestration/task_manager/_load_step.py

@@ -21,10 +21,15 @@ class _LoadStepMixin(_TaskManagerBase):
         full_refresh: bool,
     ) -> dict[str, int]:
         """Execute the LOAD step end-to-end (resume → batch → save)."""
+        # Hybrid mode: the metric's SQL may run against a different
+        # (SOURCE) profile than the one holding _dtk_* state; save() below
+        # always goes through self.internal (state), untouched either way.
+        source_manager, source_profile_name = self._resolve_source_manager(config)
         loader = MetricLoader(
             config=config,
-            db_manager=self.db_manager,
+            db_manager=source_manager,
             internal_manager=self.internal,
+            source_profile_name=source_profile_name,
         )
 
         if full_refresh:

{detectkit-0.60.0 → detectkit-0.61.0/detectkit.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: detectkit
-Version: 0.60.0
+Version: 0.61.0
 Summary: Metric monitoring with automatic anomaly detection
 Author: detectkit team
 License: MIT
@@ -104,6 +104,7 @@ Dynamic: license-file
 - **Alert lifecycle** — consecutive anomalies, fraction-of-window rule (`anomaly_window` + `min_anomaly_share`), cooldown, recovery notifications, no-data alerts
 - **Project-level error alerts** — catch DB outages and pipeline crashes once per run
 - **Database agnostic** — ClickHouse, PostgreSQL, MySQL/MariaDB, DuckDB (no server needed)
+- **Hybrid mode** — read metric SQL from a billed-per-query warehouse while all pipeline state lives in a cheap local database
 - **Idempotent** — resume from interruptions, no duplicate processing
 - **CLI** — `dtk init`, `dtk run --select`, `dtk unlock`, `dtk clean`, tag-based selectors
 - **AI-native onboarding** — `dtk init-claude` sets up Claude Code context (CLAUDE.md + rules + three skills) so an assistant can scaffold metrics, configure databases, and file feedback upstream