detectkit 0.64.0__tar.gz → 0.65.0__tar.gz

{detectkit-0.64.0/detectkit.egg-info → detectkit-0.65.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: detectkit
-Version: 0.64.0
+Version: 0.65.0
 Summary: Metric monitoring with automatic anomaly detection
 Author: detectkit team
 License: MIT
@@ -42,12 +42,15 @@ Provides-Extra: duckdb
 Requires-Dist: duckdb>=1.1; extra == "duckdb"
 Provides-Extra: snowflake
 Requires-Dist: snowflake-connector-python>=3.12; extra == "snowflake"
+Provides-Extra: bigquery
+Requires-Dist: google-cloud-bigquery>=3.15; extra == "bigquery"
 Provides-Extra: all-db
 Requires-Dist: clickhouse-driver>=0.2.0; extra == "all-db"
 Requires-Dist: psycopg2-binary>=2.9.0; extra == "all-db"
 Requires-Dist: pymysql>=1.0.0; extra == "all-db"
 Requires-Dist: duckdb>=1.1; extra == "all-db"
 Requires-Dist: snowflake-connector-python>=3.12; extra == "all-db"
+Requires-Dist: google-cloud-bigquery>=3.15; extra == "all-db"
 Provides-Extra: prophet
 Requires-Dist: prophet>=1.1.0; extra == "prophet"
 Provides-Extra: timesfm
@@ -65,6 +68,7 @@ Requires-Dist: psycopg2-binary>=2.9.0; extra == "all"
 Requires-Dist: pymysql>=1.0.0; extra == "all"
 Requires-Dist: duckdb>=1.1; extra == "all"
 Requires-Dist: snowflake-connector-python>=3.12; extra == "all"
+Requires-Dist: google-cloud-bigquery>=3.15; extra == "all"
 Requires-Dist: prophet>=1.1.0; extra == "all"
 Requires-Dist: timesfm>=0.1.0; extra == "all"
 Requires-Dist: sqlglot>=20.0; extra == "all"
@@ -110,7 +114,7 @@ Dynamic: license-file
 - **@mentions** — tag users/groups in alerts, each channel formats natively
 - **Alert lifecycle** — consecutive anomalies, fraction-of-window rule (`anomaly_window` + `min_anomaly_share`), cooldown, recovery notifications, no-data alerts
 - **Project-level error alerts** — catch DB outages and pipeline crashes once per run
-- **Database agnostic** — ClickHouse, PostgreSQL, MySQL/MariaDB, DuckDB (no server needed); Snowflake as a hybrid-mode source
+- **Database agnostic** — ClickHouse, PostgreSQL, MySQL/MariaDB, DuckDB (no server needed); Snowflake and BigQuery as hybrid-mode sources
 - **Hybrid mode** — read metric SQL from a billed-per-query warehouse while all pipeline state lives in a cheap local database
 - **Idempotent** — resume from interruptions, no duplicate processing
 - **CLI** — `dtk init`, `dtk run --select`, `dtk unlock`, `dtk clean`, tag-based selectors
@@ -128,6 +132,7 @@ With database drivers:
 pip install detectkit[clickhouse]   # ClickHouse
 pip install detectkit[duckdb]       # DuckDB - single file, no server
 pip install detectkit[snowflake]    # Snowflake - source-only (hybrid mode)
+pip install detectkit[bigquery]     # BigQuery - source-only (hybrid mode)
 pip install detectkit[all-db]       # All databases
 ```
 

{detectkit-0.64.0 → detectkit-0.65.0}/README.md

@@ -24,7 +24,7 @@
 - **@mentions** — tag users/groups in alerts, each channel formats natively
 - **Alert lifecycle** — consecutive anomalies, fraction-of-window rule (`anomaly_window` + `min_anomaly_share`), cooldown, recovery notifications, no-data alerts
 - **Project-level error alerts** — catch DB outages and pipeline crashes once per run
-- **Database agnostic** — ClickHouse, PostgreSQL, MySQL/MariaDB, DuckDB (no server needed); Snowflake as a hybrid-mode source
+- **Database agnostic** — ClickHouse, PostgreSQL, MySQL/MariaDB, DuckDB (no server needed); Snowflake and BigQuery as hybrid-mode sources
 - **Hybrid mode** — read metric SQL from a billed-per-query warehouse while all pipeline state lives in a cheap local database
 - **Idempotent** — resume from interruptions, no duplicate processing
 - **CLI** — `dtk init`, `dtk run --select`, `dtk unlock`, `dtk clean`, tag-based selectors
@@ -42,6 +42,7 @@ With database drivers:
 pip install detectkit[clickhouse]   # ClickHouse
 pip install detectkit[duckdb]       # DuckDB - single file, no server
 pip install detectkit[snowflake]    # Snowflake - source-only (hybrid mode)
+pip install detectkit[bigquery]     # BigQuery - source-only (hybrid mode)
 pip install detectkit[all-db]       # All databases
 ```
 

{detectkit-0.64.0 → detectkit-0.65.0}/detectkit/__init__.py

@@ -4,7 +4,7 @@ detectk - Anomaly Detection for Time-Series Metrics
 A Python library for data analysts and engineers to monitor metrics with automatic anomaly detection.
 """
 
-__version__ = "0.64.0"
+__version__ = "0.65.0"
 
 from detectkit.core.interval import Interval
 from detectkit.core.models import ColumnDefinition, TableModel

{detectkit-0.64.0 → detectkit-0.65.0}/detectkit/cli/assets/claude/CLAUDE.section.md

@@ -14,7 +14,7 @@ conventions.
 **Database access for _you_ (recommended, not required).** detectkit itself
 connects to the database directly via its drivers — it **never** needs an MCP to
 run. But you assist far better with **read access to the same database** (e.g. a
-database MCP for the project's ClickHouse / PostgreSQL / MySQL / MariaDB / DuckDB / Snowflake (source-only, hybrid mode)): you can inspect
+database MCP for the project's ClickHouse / PostgreSQL / MySQL / MariaDB / DuckDB / Snowflake / BigQuery (the last two source-only, hybrid mode)): you can inspect
 a metric's series, find real incidents to label for `dtk autotune`, sanity-check
 a metric query before running it, and confirm detections — instead of asking the
 user to run every query by hand. Without it, fall back to marking incidents

{detectkit-0.64.0 → detectkit-0.65.0}/detectkit/cli/assets/claude/rules/metrics.md

@@ -80,8 +80,8 @@ metric's **load** step ever connects to it — detect/alert and every other
 command don't — though every `dtk run` fail-fast validates the resolved name
 up front regardless of `--steps` (a typo exits 1 before any pipeline work).
 The `source_profile` may point at a **source-only** warehouse profile such as
-`snowflake` (usable *only* as a source, never for `_dtk_*` state — see
-`project.md`), which is exactly the case hybrid mode is built for.
+`snowflake` or `bigquery` (usable *only* as a source, never for `_dtk_*` state
+— see `project.md`), which is exactly the case hybrid mode is built for.
 See the [Hybrid Mode guide](https://dtk.pipelab.dev/guides/hybrid-mode/).
 
 `ai_context` is **OSI-compatible grounding** you can add to any metric with **no

{detectkit-0.64.0 → detectkit-0.65.0}/detectkit/cli/assets/claude/rules/overview.md

@@ -7,8 +7,9 @@ them with one command. Core logic is pure numpy (no pandas). **ClickHouse,
 PostgreSQL, MySQL/MariaDB and DuckDB are all fully supported** — only the
 connection and the SQL dialect of your metric queries differ between them
 (DuckDB additionally needs no server at all — just a local file); **Snowflake**
-is supported as a **source-only** backend (hybrid mode — it runs a metric's
-load SQL, but `_dtk_*` state must live in one of the full backends).
+and **BigQuery** are supported as **source-only** backends (hybrid mode — they
+run a metric's load SQL, but `_dtk_*` state must live in one of the full
+backends).
 
 ## The pipeline: load → detect → alert
 

{detectkit-0.64.0 → detectkit-0.65.0}/detectkit/cli/assets/claude/rules/project.md

@@ -58,8 +58,8 @@ profile whose database runs metric SQL, while *every* `_dtk_*` table
 else. A per-metric `source_profile` overrides it (`metrics.md`); resolves
 **metric → project → unset**, same precedence as `loading_delay`. Unset on
 both = today's behavior (one profile does everything). The source profile can
-be a full backend **or** a **source-only** type like `snowflake` (valid only
-here, never as the state profile). Only the LOAD step's
+be a full backend **or** a **source-only** type (`snowflake`, `bigquery`) —
+valid only here, never as the state profile. Only the LOAD step's
 metric-SQL query is affected — detect/alert and every other command
 (`dtk autotune`/`tune`/`ui`/`clean`/`unlock`) only ever touch the state
 profile. A source-side failure raises `SourceDatabaseError` (message leads
@@ -140,14 +140,14 @@ alert_channels:
 ### Database profiles
 
 > ClickHouse, PostgreSQL, MySQL, MariaDB and DuckDB are all supported as
-> **state** backends (they hold the `_dtk_*` tables); **Snowflake** is
-> **source-only** — see its block below.
+> **state** backends (they hold the `_dtk_*` tables); **Snowflake** and
+> **BigQuery** are **source-only** — see their blocks below.
 > ClickHouse/MySQL/MariaDB use two *databases*; PostgreSQL connects to one
 > `database` and uses two *schemas*; DuckDB is a single *file* (or `:memory:`
 > for tests) holding two *schemas*.
 > `dtk init --db-type {clickhouse,postgres,mysql,mariadb}` scaffolds the right
-> shape — DuckDB and Snowflake profiles are written by hand (not yet a
-> `--db-type` choice).
+> shape — DuckDB, Snowflake and BigQuery profiles are written by hand (not yet
+> a `--db-type` choice).
 
 **ClickHouse**:
 ```yaml
@@ -250,6 +250,38 @@ profiles:
 > hybrid mode (load from Snowflake, keep cheap local state) is the point.
 > `pip install 'detectkit[snowflake]'`.
 
+**BigQuery** (**source-only** — hybrid mode; runs a metric's load SQL, never
+holds `_dtk_*` state):
+```yaml
+profiles:
+  warehouse:
+    type: bigquery
+    project: my-analytics-project    # required — GCP project id billed for the queries
+    credentials_json_path: /etc/detectkit/bq-sa.json   # optional — service-account JSON key file
+    #                                # (unset -> Application Default Credentials)
+    location: EU                     # optional — job location (e.g. EU / US)
+    dataset: analytics               # optional — default dataset so unqualified table names resolve
+    api_endpoint: null               # optional — endpoint override; plain-http (the emulator) -> anonymous
+    #                                # auth when no key file; https endpoints authenticate normally
+    settings:                        # optional — extra QueryJobConfig attributes applied to every query
+      maximum_bytes_billed: 1000000000
+```
+> **Source-only**: a `bigquery` profile is valid **only** as a metric's or the
+> project's `source_profile` (hybrid mode) — `dtk run` refuses it as a state
+> profile (`--profile`/`default_profile`), so pair it with a full backend
+> (DuckDB/Postgres/ClickHouse) that holds the `_dtk_*` tables. See `metrics.md`
+> and the [Hybrid Mode guide](https://dtk.pipelab.dev/guides/hybrid-mode/).
+> Auth: `credentials_json_path` (a service-account key file) when set, else
+> **Application Default Credentials** (gcloud ADC / an attached service account
+> / Workload Identity); host/port/user/password are unused. Timestamps: a
+> BigQuery `TIMESTAMP` column comes back tz-aware UTC (handled by the loader),
+> `DATETIME` comes back naive — prefer `TIMESTAMP` (or a cast) for the metric's
+> timestamp column. Billing note: on-demand queries bill a **10 MiB minimum**
+> of bytes processed per query per referenced table, so frequent small
+> monitoring queries are disproportionately expensive — load from BigQuery,
+> keep cheap local state; `settings: {maximum_bytes_billed: ...}` caps what a
+> single query may scan. `pip install 'detectkit[bigquery]'`.
+
 ### Alert channels
 
 Defined once in `profiles.yml`, referenced by name in each metric's

{detectkit-0.64.0 → detectkit-0.65.0}/detectkit/config/profile.py

@@ -29,22 +29,23 @@ class ProfileConfig(BaseModel):
       duckdb) can hold detectkit state (the ``_dtk_*`` tables) — and, since
       state managers also expose ``execute_query``, double as hybrid-mode
       sources.
-    - **Source-only backends** (``SOURCE_ONLY_TYPES``: snowflake) implement
-      only connect + read query. They are valid **only** as a metric/project
-      ``source_profile``; :meth:`create_manager` refuses them as a state
-      profile with a clear error.
+    - **Source-only backends** (``SOURCE_ONLY_TYPES``: snowflake, bigquery)
+      implement only connect + read query. They are valid **only** as a
+      metric/project ``source_profile``; :meth:`create_manager` refuses them
+      as a state profile with a clear error.
 
     Attributes:
         type: Database type ("clickhouse", "postgres", "mysql", "mariadb",
-            "duckdb"; source-only: "snowflake")
+            "duckdb"; source-only: "snowflake", "bigquery")
         host: Database host (unused for DuckDB — it is an in-process, file-backed
-            database with no network endpoint — and for Snowflake, which
-            connects by `account`)
-        port: Database port (unused for DuckDB/Snowflake; required for every
-            other backend)
-        user: Database user (unused for DuckDB; required explicitly for Snowflake)
-        password: Database password (unused for DuckDB; for Snowflake either
-            `password` or `private_key_path` must be set)
+            database with no network endpoint — and for Snowflake/BigQuery,
+            which connect by `account` / `project`)
+        port: Database port (unused for DuckDB/Snowflake/BigQuery; required
+            for every other backend)
+        user: Database user (unused for DuckDB/BigQuery; required explicitly
+            for Snowflake)
+        password: Database password (unused for DuckDB/BigQuery; for Snowflake
+            either `password` or `private_key_path` must be set)
         database: Connection-target database (PostgreSQL/MySQL/MariaDB; the
             session default database for Snowflake; unused for DuckDB — use
             `path` instead)
@@ -57,6 +58,15 @@ class ProfileConfig(BaseModel):
         private_key_path: Path to a PEM private key for Snowflake key-pair
             auth (recommended for service accounts)
         private_key_passphrase: Passphrase of the private key, if encrypted
+        project: GCP project id billed for the queries (BigQuery only)
+        credentials_json_path: Path to a service-account JSON key file
+            (BigQuery only; unset -> Application Default Credentials)
+        location: Job location, e.g. "EU" (BigQuery only; unset -> inferred
+            from the referenced datasets)
+        dataset: Default dataset for unqualified table names in load queries
+            (BigQuery only)
+        api_endpoint: API endpoint override — the BigQuery emulator or a
+            private endpoint (BigQuery only)
         internal_database: Database/schema for internal tables
         internal_schema: Schema for internal tables (PostgreSQL/DuckDB only)
         data_database: Database for user data tables
@@ -76,10 +86,10 @@ class ProfileConfig(BaseModel):
     )
     # Source-only backends: connect + execute_query only (no DDL/upserts/
     # locks); valid ONLY as a metric/project `source_profile` (hybrid mode).
-    SOURCE_ONLY_TYPES: ClassVar[frozenset[str]] = frozenset({"snowflake"})
-    # Backends with no network port to validate (in-process file DB; an
-    # account-resolved cloud driver).
-    _PORTLESS_TYPES: ClassVar[frozenset[str]] = frozenset({"duckdb", "snowflake"})
+    SOURCE_ONLY_TYPES: ClassVar[frozenset[str]] = frozenset({"snowflake", "bigquery"})
+    # Backends with no network port to validate (in-process file DB;
+    # account/project-resolved cloud drivers).
+    _PORTLESS_TYPES: ClassVar[frozenset[str]] = frozenset({"duckdb", "snowflake", "bigquery"})
 
     type: str = Field(..., description="Database type")
     host: str = Field(default="localhost", description="Database host (unused for DuckDB)")
@@ -141,6 +151,34 @@ class ProfileConfig(BaseModel):
         description="Passphrase of the private key, when it is encrypted (Snowflake only)",
     )
 
+    # BigQuery-only (source-only backend). host/port/user/password are
+    # meaningless for it: the client resolves the endpoint from `project` and
+    # authenticates via a service-account key file or ADC.
+    project: str | None = Field(
+        default=None,
+        description="GCP project id billed for the queries, e.g. 'my-project' (BigQuery only)",
+    )
+    credentials_json_path: str | None = Field(
+        default=None,
+        description="Path to a service-account JSON key file (BigQuery only; "
+        "unset -> Application Default Credentials)",
+    )
+    location: str | None = Field(
+        default=None,
+        description="Job location, e.g. 'EU' (BigQuery only; unset -> inferred "
+        "from the referenced datasets)",
+    )
+    dataset: str | None = Field(
+        default=None,
+        description="Default dataset for unqualified table names in load queries (BigQuery only)",
+    )
+    api_endpoint: str | None = Field(
+        default=None,
+        description="API endpoint override (BigQuery only) — a plain-http endpoint "
+        "(the BigQuery emulator) uses anonymous credentials when no key file is "
+        "set; https endpoints (regional/private) authenticate normally",
+    )
+
     # Internal location for _dtk_* tables
     internal_database: str | None = Field(
         default=None, description="Database for internal tables (ClickHouse/MySQL/MariaDB)"
@@ -215,6 +253,23 @@ class ProfileConfig(BaseModel):
             )
         return self
 
+    @model_validator(mode="after")
+    def _validate_bigquery_fields(self) -> "ProfileConfig":
+        """BigQuery profiles need an explicit GCP project id.
+
+        Auth needs nothing further: `credentials_json_path` when set, else
+        Application Default Credentials (gcloud ADC / attached service
+        account / Workload Identity).
+        """
+        if self.type != "bigquery":
+            return self
+        if not self.project:
+            raise ValueError(
+                "bigquery profiles must set 'project' (the GCP project id billed "
+                "for the queries, e.g. 'my-project')"
+            )
+        return self
+
     def get_internal_location(self) -> str:
         """
         Get internal location (database or schema).
@@ -410,6 +465,19 @@ class ProfileConfig(BaseModel):
                 role=self.role,
                 settings=self.settings,
             )
+        if self.type == "bigquery":
+            from detectkit.database.bigquery_manager import BigQuerySourceManager
+
+            # Guaranteed by `_validate_bigquery_fields` at construction time.
+            assert self.project is not None
+            return BigQuerySourceManager(
+                project=self.project,
+                credentials_json_path=self.credentials_json_path,
+                location=self.location,
+                dataset=self.dataset,
+                api_endpoint=self.api_endpoint,
+                settings=self.settings,
+            )
         return self.create_manager()
 
 

detectkit-0.65.0/detectkit/database/bigquery_manager.py

@@ -0,0 +1,229 @@
+"""BigQuery **source-only** database manager.
+
+BigQuery is supported as a hybrid-mode *source* (a metric's ``load`` SQL runs
+against it) but **not** as a state backend: it implements only the minimal
+:class:`~detectkit.database.source_manager.SourceDatabaseManager` contract
+(connect + :meth:`execute_query` + :meth:`close`), with no DDL, upsert, lock or
+internal-table machinery. :meth:`ProfileConfig.create_manager` refuses a
+``bigquery`` profile as state and routes it here via
+:meth:`ProfileConfig.create_source_manager`; the ``_dtk_*`` state lives in a
+cheaper local backend (DuckDB/PostgreSQL/ClickHouse).
+
+Behaviors that are deliberate and load-bearing:
+
+- **Eager connectivity probe, with bounded retries.** ``bigquery.Client(...)``
+  alone performs no network I/O, so construction runs a ``SELECT 1`` probe — a
+  query with no table references processes 0 bytes and is free on on-demand
+  billing — to fail fast on bad credentials/project at hybrid-pool build
+  (parity with the other backends' eager connect). The probe also applies
+  :meth:`_job_config`, so a typo in ``settings`` or a bad ``dataset`` surfaces
+  immediately. The client library's default retry treats network-level errors
+  (connection refused, DNS) as transient and would retry them for 10+ minutes;
+  the probe caps that at ``_PROBE_TIMEOUT_S`` and load queries at
+  ``_RETRY_TIMEOUT_S`` / ``_JOB_RETRY_TIMEOUT_S``, so an unreachable endpoint
+  fails the run in seconds-to-minutes, not silently stalls it.
+- **Auth resolution.** ``credentials_json_path`` (a service-account key file)
+  when set; otherwise **Application Default Credentials** (gcloud ADC, an
+  attached service account, or Workload Identity). A **plain-``http://``**
+  ``api_endpoint`` — the BigQuery emulator — switches to anonymous credentials
+  so no ADC lookup is attempted; an ``https://`` endpoint override (a regional
+  ``*.rep.googleapis.com`` endpoint, Private Service Connect) is a real,
+  authenticated Google endpoint and resolves auth exactly as above.
+- **Timestamps.** BigQuery ``TIMESTAMP`` columns come back as tz-aware UTC
+  datetimes — the loader converts tz-aware values to naive UTC since v0.62.0.
+  ``DATETIME`` comes back naive and is taken verbatim; use ``TIMESTAMP`` (or
+  cast) for the metric's ``timestamp`` column unless the naive values already
+  are UTC.
+- **No column-name folding.** Unlike Snowflake, BigQuery preserves the case of
+  column aliases, so ``SELECT ts AS timestamp`` reaches the loader unchanged.
+
+Billing note: on-demand queries bill a 10 MiB minimum of bytes processed per
+referenced table, so frequent small monitoring queries are disproportionately
+expensive — the whole point of hybrid mode is to load from BigQuery and keep
+cheap state elsewhere. A ``settings: {maximum_bytes_billed: ...}`` guardrail
+caps what a single load query may scan.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from detectkit.database.source_manager import SourceDatabaseManager
+
+
+class BigQuerySourceManager(SourceDatabaseManager):
+    """Read-only BigQuery connection for hybrid-mode source profiles.
+
+    Constructs the client and runs the free ``SELECT 1`` probe eagerly (see
+    the module docstring for the probe, auth resolution and timestamp rules).
+
+    Args:
+        project: GCP project id billed for the queries (e.g. ``"my-project"``)
+        credentials_json_path: Path to a service-account JSON key file
+            (unset -> Application Default Credentials)
+        location: Job location (e.g. ``"EU"``; unset -> BigQuery infers it
+            from the referenced datasets)
+        dataset: Default dataset for unqualified table names in queries
+        api_endpoint: Override the API endpoint — for the BigQuery emulator
+            (e.g. ``"http://localhost:9050"``) or private endpoints; a
+            plain-``http://`` endpoint without a key file switches auth to
+            anonymous credentials (``https://`` endpoints authenticate
+            normally via key file or ADC)
+        settings: Extra ``QueryJobConfig`` attributes applied to every query
+            (e.g. ``maximum_bytes_billed``, ``labels``, ``use_query_cache``);
+            unknown attribute names are rejected at the probe
+    """
+
+    # The client library's DEFAULT_RETRY/DEFAULT_JOB_RETRY treat network-level
+    # errors as transient and retry them for 600/2400 seconds — an unreachable
+    # endpoint would stall a scheduled run for tens of minutes. These caps keep
+    # the probe genuinely fail-fast and load queries bounded (long-RUNNING
+    # queries are unaffected: waiting on a healthy job is not a retry).
+    _PROBE_TIMEOUT_S = 30.0
+    _RETRY_TIMEOUT_S = 120.0
+    _JOB_RETRY_TIMEOUT_S = 600.0
+
+    def __init__(
+        self,
+        project: str,
+        credentials_json_path: str | None = None,
+        location: str | None = None,
+        dataset: str | None = None,
+        api_endpoint: str | None = None,
+        settings: dict[str, Any] | None = None,
+    ) -> None:
+        try:
+            from google.cloud import bigquery
+        except ImportError as exc:
+            raise ImportError(
+                "google-cloud-bigquery is not installed. "
+                "Install with: pip install detectkit[bigquery]"
+            ) from exc
+
+        self._project = project
+        self._credentials_json_path = credentials_json_path
+        self._location = location
+        self._dataset = dataset
+        self._api_endpoint = api_endpoint
+        self._settings = settings or {}
+
+        self._closed = False
+        self._client = bigquery.Client(**self._client_kwargs())
+        try:
+            self._probe()
+        except Exception:
+            # Don't leak the just-built client (and the HTTP session the probe
+            # request lazily created) — the hybrid pool caches only the error.
+            self.close()
+            raise
+
+    def _client_kwargs(self) -> dict[str, Any]:
+        """Build ``bigquery.Client(...)`` kwargs without any network I/O.
+
+        The testable seam: it resolves auth (service-account key file → ADC →
+        anonymous for the emulator path) and includes ``location`` /
+        ``client_options`` only when set.
+        """
+        kwargs: dict[str, Any] = {"project": self._project}
+
+        if self._credentials_json_path is not None:
+            from google.oauth2 import service_account
+
+            kwargs["credentials"] = service_account.Credentials.from_service_account_file(
+                self._credentials_json_path
+            )
+        elif self._api_endpoint is not None and self._api_endpoint.startswith("http://"):
+            # Plain-HTTP endpoint = the emulator: skip the ADC lookup entirely
+            # (it would fail on a machine with no gcloud auth). An https
+            # endpoint override (regional/private) is a real, authenticated
+            # Google endpoint and falls through to ADC below.
+            from google.auth.credentials import AnonymousCredentials
+
+            kwargs["credentials"] = AnonymousCredentials()
+        # else: Application Default Credentials, resolved by the client itself.
+
+        if self._location is not None:
+            kwargs["location"] = self._location
+        if self._api_endpoint is not None:
+            kwargs["client_options"] = {"api_endpoint": self._api_endpoint}
+
+        return kwargs
+
+    def _job_config(self) -> Any:
+        """Build the per-query ``QueryJobConfig``, or ``None`` when trivial.
+
+        ``dataset`` becomes ``default_dataset`` (unqualified table names in the
+        metric SQL resolve against it); each ``settings`` key must name a real
+        ``QueryJobConfig`` property (typos would otherwise be silently ignored
+        attributes — rejected here instead, surfacing at the connect probe).
+        """
+        if self._dataset is None and not self._settings:
+            return None
+
+        from google.cloud import bigquery
+
+        config = bigquery.QueryJobConfig()
+        if self._dataset is not None:
+            config.default_dataset = bigquery.DatasetReference(self._project, self._dataset)
+        for key, value in self._settings.items():
+            if not isinstance(getattr(type(config), key, None), property):
+                raise ValueError(
+                    f"Unknown BigQuery query setting '{key}': settings keys must be "
+                    f"QueryJobConfig attributes (e.g. maximum_bytes_billed, labels)"
+                )
+            setattr(config, key, value)
+        return config
+
+    def _probe(self) -> None:
+        """Fail fast on bad credentials/project/settings at construction.
+
+        ``SELECT 1`` references no table, processes 0 bytes and is free on
+        on-demand billing; it exercises exactly the permission the loader
+        needs (``bigquery.jobs.create`` on the project). Retries are capped
+        at ``_PROBE_TIMEOUT_S`` (and job-level retry disabled), so an
+        unreachable endpoint fails the pool build in seconds instead of the
+        library default's 10+ minutes of connection-error retries.
+        """
+        from google.cloud.bigquery.retry import DEFAULT_RETRY
+
+        retry = DEFAULT_RETRY.with_timeout(self._PROBE_TIMEOUT_S)
+        self._client.query(
+            "SELECT 1", job_config=self._job_config(), retry=retry, job_retry=None
+        ).result(retry=retry, timeout=self._PROBE_TIMEOUT_S)
+
+    def execute_query(
+        self, query: str, params: dict[str, Any] | None = None
+    ) -> list[dict[str, Any]]:
+        """Execute a read query and return all rows as column-keyed dicts.
+
+        BigQuery takes typed ``ScalarQueryParameter`` objects rather than
+        DB-API params; the loader renders its SQL with Jinja and always calls
+        this param-less, so a non-empty ``params`` is rejected instead of
+        being silently dropped. DDL/DML statements (used by tests to seed the
+        emulator) return an empty row set.
+        """
+        if params:
+            raise ValueError(
+                "BigQuerySourceManager does not support query params; "
+                "render values into the SQL instead (the loader always does)"
+            )
+        from google.cloud.bigquery.retry import DEFAULT_JOB_RETRY, DEFAULT_RETRY
+
+        # Bounded retries (see the class constants): network-level errors stop
+        # being retried after minutes, not tens of minutes. No `timeout=` on
+        # result() — a long-running healthy query may legitimately take longer.
+        retry = DEFAULT_RETRY.with_timeout(self._RETRY_TIMEOUT_S)
+        rows = self._client.query(
+            query,
+            job_config=self._job_config(),
+            retry=retry,
+            job_retry=DEFAULT_JOB_RETRY.with_timeout(self._JOB_RETRY_TIMEOUT_S),
+        ).result(retry=retry)
+        return [dict(row.items()) for row in rows]
+
+    def close(self) -> None:
+        """Close the client's HTTP transport. Safe to call more than once."""
+        if self._closed:
+            return
+        self._closed = True
+        self._client.close()

{detectkit-0.64.0 → detectkit-0.65.0/detectkit.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: detectkit
-Version: 0.64.0
+Version: 0.65.0
 Summary: Metric monitoring with automatic anomaly detection
 Author: detectkit team
 License: MIT
@@ -42,12 +42,15 @@ Provides-Extra: duckdb
 Requires-Dist: duckdb>=1.1; extra == "duckdb"
 Provides-Extra: snowflake
 Requires-Dist: snowflake-connector-python>=3.12; extra == "snowflake"
+Provides-Extra: bigquery
+Requires-Dist: google-cloud-bigquery>=3.15; extra == "bigquery"
 Provides-Extra: all-db
 Requires-Dist: clickhouse-driver>=0.2.0; extra == "all-db"
 Requires-Dist: psycopg2-binary>=2.9.0; extra == "all-db"
 Requires-Dist: pymysql>=1.0.0; extra == "all-db"
 Requires-Dist: duckdb>=1.1; extra == "all-db"
 Requires-Dist: snowflake-connector-python>=3.12; extra == "all-db"
+Requires-Dist: google-cloud-bigquery>=3.15; extra == "all-db"
 Provides-Extra: prophet
 Requires-Dist: prophet>=1.1.0; extra == "prophet"
 Provides-Extra: timesfm
@@ -65,6 +68,7 @@ Requires-Dist: psycopg2-binary>=2.9.0; extra == "all"
 Requires-Dist: pymysql>=1.0.0; extra == "all"
 Requires-Dist: duckdb>=1.1; extra == "all"
 Requires-Dist: snowflake-connector-python>=3.12; extra == "all"
+Requires-Dist: google-cloud-bigquery>=3.15; extra == "all"
 Requires-Dist: prophet>=1.1.0; extra == "all"
 Requires-Dist: timesfm>=0.1.0; extra == "all"
 Requires-Dist: sqlglot>=20.0; extra == "all"
@@ -110,7 +114,7 @@ Dynamic: license-file
 - **@mentions** — tag users/groups in alerts, each channel formats natively
 - **Alert lifecycle** — consecutive anomalies, fraction-of-window rule (`anomaly_window` + `min_anomaly_share`), cooldown, recovery notifications, no-data alerts
 - **Project-level error alerts** — catch DB outages and pipeline crashes once per run
-- **Database agnostic** — ClickHouse, PostgreSQL, MySQL/MariaDB, DuckDB (no server needed); Snowflake as a hybrid-mode source
+- **Database agnostic** — ClickHouse, PostgreSQL, MySQL/MariaDB, DuckDB (no server needed); Snowflake and BigQuery as hybrid-mode sources
 - **Hybrid mode** — read metric SQL from a billed-per-query warehouse while all pipeline state lives in a cheap local database
 - **Idempotent** — resume from interruptions, no duplicate processing
 - **CLI** — `dtk init`, `dtk run --select`, `dtk unlock`, `dtk clean`, tag-based selectors
@@ -128,6 +132,7 @@ With database drivers:
 pip install detectkit[clickhouse]   # ClickHouse
 pip install detectkit[duckdb]       # DuckDB - single file, no server
 pip install detectkit[snowflake]    # Snowflake - source-only (hybrid mode)
+pip install detectkit[bigquery]     # BigQuery - source-only (hybrid mode)
 pip install detectkit[all-db]       # All databases
 ```
 

{detectkit-0.64.0 → detectkit-0.65.0}/detectkit.egg-info/SOURCES.txt

@@ -90,6 +90,7 @@ detectkit/core/interval.py
 detectkit/core/models.py
 detectkit/database/__init__.py
 detectkit/database/_sql_manager.py
+detectkit/database/bigquery_manager.py
 detectkit/database/clickhouse_manager.py
 detectkit/database/duckdb_manager.py
 detectkit/database/manager.py

{detectkit-0.64.0 → detectkit-0.65.0}/detectkit.egg-info/requires.txt

@@ -16,6 +16,7 @@ psycopg2-binary>=2.9.0
 pymysql>=1.0.0
 duckdb>=1.1
 snowflake-connector-python>=3.12
+google-cloud-bigquery>=3.15
 prophet>=1.1.0
 timesfm>=0.1.0
 sqlglot>=20.0
@@ -27,6 +28,10 @@ psycopg2-binary>=2.9.0
 pymysql>=1.0.0
 duckdb>=1.1
 snowflake-connector-python>=3.12
+google-cloud-bigquery>=3.15
+
+[bigquery]
+google-cloud-bigquery>=3.15
 
 [clickhouse]
 clickhouse-driver>=0.2.0