bigquery-data-science 0.7.0__py3-none-any.whl

bigquery_data_science/_skills/bigquery-ml/syntax/mladvanced_weights.md

@@ -0,0 +1,76 @@
+# `ML.ADVANCED_WEIGHTS`
+
+> **Source:** [`bigquery-ml/reference/model-lifecycle-functions.md#mladvanced_weights`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-lifecycle-functions.md#mladvanced_weights)
+
+- **Description:** Extended version of `ML.WEIGHTS` for **linear regression** and **binary logistic regression** models. Returns the same weights plus per-weight **standard errors** and **p-values** (statistical significance). Its output is a superset of `ML.WEIGHTS`.
+- **Use cases:**
+  - Statistical inference on coefficients (significance testing) for econometric / explanatory modeling.
+  - Identify which features have weights statistically distinguishable from zero.
+- **documentation:** https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-advanced-weights
+- **Type:** Table-valued function.
+- **Applies to models:** `LINEAR_REG` and **binary** `LOGISTIC_REG` only. Multiclass logistic regression and matrix factorization are NOT supported. Model must be trained with `CALCULATE_P_VALUES = TRUE`, `CATEGORY_ENCODING_METHOD = 'DUMMY_ENCODING'`, and `L1_REG = 0`.
+
+**Syntax:**
+```sql
+SELECT *
+FROM ML.ADVANCED_WEIGHTS(MODEL `PROJECT_ID.DATASET.MODEL_NAME`
+                         [, STRUCT(TRUE AS standardize)]);
+```
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `MODEL` | model ref | Yes | — | Linear or binary logistic regression model trained with the required options above. |
+| `STANDARDIZE` | BOOL (in STRUCT) | No | `FALSE` | Standardize weights (mean 0 / std 1). Set TRUE to obtain a standard error and p-value for the intercept. |
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `processed_input` | STRING | Feature input column name. |
+| `weight` | FLOAT64 | Weight for a numeric feature; NULL for categorical (see `category`). |
+| `category` | STRING | Category name for non-numeric/dummy-encoded inputs (NULL for numeric). |
+| `standard_error` | FLOAT64 | Standard error of the weight. `NULL` for the intercept unless `standardize=TRUE`; **`0.0`** (not `NaN`) for a dropped dummy category. |
+| `p_value` | FLOAT64 | p-value of the weight. `NULL` for the intercept unless `standardize=TRUE`; **`NaN`** for a dropped dummy category. |
+
+The intercept arrives as a final row with `processed_input = '__INTERCEPT__'` and `category = NULL`. Note the two sentinel values differ: a baseline row is `weight = 0.0`, `standard_error = 0.0`, `p_value = NaN`, so `NOT IS_NAN(p_value)` is the reliable filter — reading `standard_error = 0.0` as a measured value will silently corrupt any average.
+
+**Best practices:**
+- Train with `CALCULATE_P_VALUES = TRUE` up front — p-values/standard errors are computed at CREATE MODEL time and cannot be added later.
+- Set `STANDARDIZE => TRUE` when you need inference statistics for the intercept.
+
+**Limitations:**
+- Not available for multiclass logistic regression, matrix factorization, or any non-GLM model.
+- Requires `L1_REG = 0` and `DUMMY_ENCODING`; incompatible with L1 regularization.
+- Dropped dummy categories report `weight = 0.0` and `standard_error = 0.0` with a `NaN` p-value.
+
+**Gotchas (verified live):**
+- **Each precondition fails differently, and three of the four fail at `CREATE MODEL` time, not at query time.** Verified error text:
+
+  | What you got wrong | Where it fails | Message |
+  |---|---|---|
+  | `calculate_p_values` omitted | the TVF | `Model is not supported by ML.ADVANCED_WEIGHTS because it was not trained with calculate_p_values = true.` |
+  | left the default `ONE_HOT_ENCODING` | `CREATE MODEL` | `Please specify CATEGORY_ENCODING_METHOD=DUMMY_ENCODING to enable p_values calculation.` |
+  | `l1_reg > 0` | `CREATE MODEL` | `L1_REG must be zero if CALCULATE_P_VALUES=TRUE.` |
+  | multiclass label | `CREATE MODEL` | `Option(s) calculate_p_values are found. P-values can only be calculated for linear regression and binary logistic regression models.` |
+
+- **A model with categorical features emits a warning at `CREATE MODEL` time:** `Since model contains categorical values, regression statistics will not be calculated for unstandardized intercept.` This is expected, not a failure — it is why `__INTERCEPT__` returns `NULL` statistics and why `STRUCT(TRUE AS standardize)` exists.
+- **The dropped `DUMMY_ENCODING` baseline is the most frequent category, not the first alphabetically.** On `penguins` the baselines are Adelie (146 rows), Biscoe (163), and `MALE` (168) — `MALE` wins despite `FEMALE` sorting earlier.
+- **`standardize` does not change any p-value.** It scales `weight` and `standard_error` by the same factor, so the t-statistic is invariant. Use it to compare effect sizes on a common scale, never to change which coefficients are significant.
+- **The standardized intercept is not the label mean.** Categorical features stay dummy-coded rather than centered, so it is the prediction at mean numeric features *and* baseline categories. On `penguins` it is 4110.72 while mean `body_mass_g` is 4207.06.
+- **`p_value` bottoms out near `1e-15`** (double-precision floor). Below that, differences are numerical noise — do not rank features by p-value down there.
+- **`category` values inherit whatever is in the data, including leading whitespace.** `census_adult_income` returns `' White'` and `' Female'`; `category = 'White'` matches nothing. Use `TRIM(category)`.
+
+**BigFrames API:** No dedicated wrapper; call the TVF via SQL / `read_gbq`.
+
+**Repo example (tested):** `bigquery-ml/models/linear_regression/` (Example 7) and `bigquery-ml/models/logistic_regression/` (Example 8) — the two model types that satisfy the constraint, each training its main model with `calculate_p_values = TRUE`, `DUMMY_ENCODING`, and `l1_reg = 0` from the start, since the options cannot be added later.
+
+The linear example is the clean statistical read: on `penguins`, `island` is the only feature whose categories are insignificant (p ≈ 0.82 and 0.42) because Gentoo appears only on Biscoe and Chinstrap only on Dream, so `island` merely restates `species`. `ML.WEIGHTS` reports non-zero island weights and gives you no way to see this. It also runs the `STRUCT(TRUE AS standardize)` variant to recover intercept statistics and to show that `flipper_length_mm` moves from 16.24 g/mm to 227.60 g per standard deviation while its p-value is unchanged at 2.6e-06.
+
+The logistic example is the scale read: on `census_adult_income` the model expands to 106 rows (105 coefficients plus `__INTERCEPT__`); eight are dropped baselines, leaving 97 real tests, and **54 of those come back with p > 0.05**. Verified breakdown — `native_country` 38/41 insignificant, `education` 5/15, `workclass` 4/8, `occupation` 4/14, `race` 2/4, `marital_status` 1/6, and every numeric feature significant. More than half of what a 32k-row model learned is undefendable, concentrated almost entirely in one sparse high-cardinality column, and neither `ML.WEIGHTS` nor `ML.GLOBAL_EXPLAIN` would surface it.
+
+> Note: For boosted trees / random forest see `ML.FEATURE_IMPORTANCE` and `ML.GLOBAL_EXPLAIN`; for k-means see `ML.CENTROIDS`; for PCA/autoencoder see `ML.PRINCIPAL_COMPONENTS` / `ML.PRINCIPAL_COMPONENT_INFO`; for ARIMA_PLUS see `ML.ARIMA_COEFFICIENTS`. The tree, clustering, and forecasting model files (`models/boosted_tree_classifier/`, `models/pca/`, `models/kmeans/`, `models/autoencoder/`, `models/arima_plus/`) do NOT use `ML.WEIGHTS`. Neither do the DNN and wide-and-deep types — `models/dnn_classifier/` (Example 7) records why: no coefficients exist, so Integrated Gradients via `ML.GLOBAL_EXPLAIN` is the only mechanism available there.
+
+
+---

bigquery_data_science/_skills/bigquery-ml/syntax/mlarima_coefficients.md

@@ -0,0 +1,30 @@
+# `ML.ARIMA_COEFFICIENTS`
+
+> **Source:** [`bigquery-ml/reference/model-lifecycle-functions.md#mlarima_coefficients`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-lifecycle-functions.md#mlarima_coefficients)
+
+- **Description:** Returns the fitted ARIMA coefficients per time series — the "weights" function for ARIMA_PLUS models.
+- **Use cases:** Inspect AR/MA coefficient vectors and the intercept/drift; audit/export fitted model parameters.
+- **documentation:** https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-arima-coefficients
+- **Type:** Table-valued (model-weights family). **Applies to models:** `ARIMA_PLUS`, `ARIMA_PLUS_XREG`. **Status:** GA.
+
+**Syntax:**
+```sql
+SELECT *
+FROM ML.ARIMA_COEFFICIENTS(MODEL `PROJECT_ID.DATASET.MODEL_NAME`);
+```
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `\<time_series_id_col\>` | (varies) | Series id (when trained with `time_series_id_col`). |
+| `ar_coefficients` | ARRAY\<FLOAT64\> | Autoregressive (AR) coefficients; length = `non_seasonal_p`. Empty `[]` when p=0. |
+| `ma_coefficients` | ARRAY\<FLOAT64\> | Moving-average (MA) coefficients; length = `non_seasonal_q`. Empty `[]` when q=0. |
+| `intercept_or_drift` | FLOAT64 | Constant (intercept) or drift term of the model. |
+
+**Best practices:** Join with `ML.ARIMA_EVALUATE` on the series id to pair `(p,d,q)` with the coefficient vectors.
+**Limitations:** Output is the ARIMA-specific analog of `ML.WEIGHTS`; standard `ML.WEIGHTS` does not apply to ARIMA_PLUS.
+**BigFrames API:** `bigframes.ml.forecasting.ARIMAPlus().coef_` (or `.summary()`).
+**Repo example (tested):** `bigquery-ml/models/arima_plus/arima_plus.sql` (Example 8) — per-series `ar_coefficients` / `ma_coefficients` / `intercept_or_drift`, sitting directly after `ML.ARIMA_EVALUATE` (Example 3) so the `(p,d,q)` order and the coefficient vector lengths can be read against each other.
+
+---

bigquery_data_science/_skills/bigquery-ml/syntax/mlarima_evaluate.md

@@ -0,0 +1,49 @@
+# `ML.ARIMA_EVALUATE`
+
+> **Source:** [`bigquery-ml/reference/model-lifecycle-functions.md#mlarima_evaluate`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-lifecycle-functions.md#mlarima_evaluate)
+
+- **Description:** Returns the auto-ARIMA model selection results and quality metrics, one row per time series (the candidate chosen by auto.ARIMA, or all candidates with `show_all_candidate_models`).
+- **Use cases:**
+  - Inspect the selected `(p,d,q)` order, drift, and AIC per series.
+  - See which decomposition features fired (`has_holiday_effect`, `has_spikes_and_dips`, `has_step_changes`) and detected seasonalities.
+  - Surface per-series training failures via `error_message`.
+- **documentation:** https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-arima-evaluate
+- **Type:** Table-valued. **Applies to models:** `ARIMA_PLUS`, `ARIMA_PLUS_XREG`. **Status:** GA.
+
+**Syntax:**
+```sql
+SELECT *
+FROM ML.ARIMA_EVALUATE(
+  MODEL `PROJECT_ID.DATASET.MODEL_NAME`,
+  STRUCT(FALSE AS show_all_candidate_models)
+);
+```
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `show_all_candidate_models` | BOOL | No | FALSE | If TRUE, returns every candidate ARIMA model evaluated by auto.ARIMA; if FALSE, only the selected model per series. |
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `\<time_series_id_col\>` | (varies) | Series id (when trained with `time_series_id_col`). |
+| `non_seasonal_p` / `non_seasonal_d` / `non_seasonal_q` | INT64 | ARIMA order (AR lags, differencing degree, MA terms). |
+| `has_drift` | BOOL | Whether a drift term is included. |
+| `log_likelihood` | FLOAT64 | Model log-likelihood. |
+| `AIC` | FLOAT64 | Akaike Information Criterion (lower is better; selection criterion). |
+| `variance` | FLOAT64 | Variance of the model residuals. |
+| `seasonal_periods` | ARRAY\<STRING\> | Detected seasonalities, e.g. `[WEEKLY]`, `[WEEKLY, YEARLY]`, `[NO_SEASONALITY]`. |
+| `has_holiday_effect` | BOOL | Holiday effect detected (when `holiday_region` set). |
+| `has_spikes_and_dips` | BOOL | Spike/dip outliers detected. |
+| `has_step_changes` | BOOL | Step (level) changes detected. |
+| `error_message` | STRING | Per-series training error, empty when successful. |
+
+**Best practices:** Order by the series id column for stable review; check `error_message` for short/failed series.
+**Limitations:** `seasonal_periods`, `has_holiday_effect`, etc. depend on `CREATE MODEL` options (e.g. `holiday_region`).
+**BigFrames API:** No direct equivalent.
+**Repo example (tested):** `bigquery-ml/models/arima_plus/arima_plus.sql` (Example 3) — the full per-series model-selection table for a multi-series model. Example 10 in the same file uses it as the *verification* of a manual order: training with `auto_arima = FALSE` and `non_seasonal_order = STRUCT(2 AS p, 1 AS d, 1 AS q)`, then reading `non_seasonal_p/d/q` back to confirm auto.ARIMA was actually bypassed.
+
+---

bigquery_data_science/_skills/bigquery-ml/syntax/mlbag_of_words.md

@@ -0,0 +1,47 @@
+# `ML.BAG_OF_WORDS`
+
+> **Source:** [`bigquery-ml/reference/model-free-functions.md#mlbag_of_words`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-free-functions.md#mlbag_of_words)
+
+- **Description:** Computes a bag-of-words (per-document term-frequency) representation of tokenized documents.
+- **Use cases:**
+  - Produce sparse count features for text classification when raw frequency (not IDF weighting) is wanted.
+  - Quick baseline text features feeding `LOGISTIC_REG` / `BOOSTED_TREE_*`.
+- **documentation:** https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-bag-of-words
+- **Type:** Analytic — **requires `OVER ()`** (dictionary is built across the document window).
+- **Applies to models:** Any model type via `TRANSFORM` or precomputed columns.
+
+**Syntax:**
+```sql
+ML.BAG_OF_WORDS(tokenized_document [, top_k] [, frequency_threshold]) OVER ()
+```
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `tokenized_document` | `ARRAY<STRING>` | Yes | — | One document's tokens. |
+| `top_k` | `INT64` | No | 32000 (max 1048576) | Dictionary size excluding the unknown term; keeps the `top_k` terms appearing in the most documents. |
+| `frequency_threshold` | `INT64` | No | 5 | Minimum number of documents a term must appear in to enter the dictionary. |
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| (result) | `ARRAY<STRUCT<index INT64, value INT64>>` | Per-document term counts. `index 0` = unknown term; remaining indices map to the alphabetically ordered dictionary; `value` is the count of that term in the document. |
+
+**Best practices:** Use inside `TRANSFORM` so the dictionary is frozen with the model and re-applied at predict time. Combine with `ML.NGRAMS` upstream for n-gram bags. Use `top_k` to cap dimensionality.
+**Limitations:** Empty `OVER ()` required; counts depend on the analytic window — apply over the full corpus. Index `0` reserved for unknown. Shares the same `frequency_threshold=5` default gotcha as `ML.TF_IDF` above — verified live (see that entry).
+**BigFrames API:** No direct equivalent.
+**Repo example (tested):** `bigquery-ml/functions/text/text.ipynb` — real product-name tokens, the `frequency_threshold=5` gotcha proof, and `ML.BAG_OF_WORDS` embedded in a real `LOGISTIC_REG` `TRANSFORM` classifying product category from the product name alone (~0.95 accuracy).
+
+---
+
+**Status:** All three are **GA**. **Connection required:** No.
+
+**Typical pipeline:** tokenize (GoogleSQL `TEXT_ANALYZE`/`SPLIT`) → optional `ML.NGRAMS` → `ML.BAG_OF_WORDS`
+or `ML.TF_IDF` (in `TRANSFORM`) → train classifier. Note: `ML.TF_IDF`/`ML.BAG_OF_WORDS` are analytic
+(`OVER ()`) and an analytic function cannot be an argument of another analytic function — chain via
+subqueries/CTEs, not nesting (consistent with the repo's note for `ML.STANDARD_SCALER` etc.).
+
+
+---

bigquery_data_science/_skills/bigquery-ml/syntax/mlbucketize.md

@@ -0,0 +1,47 @@
+# `ML.BUCKETIZE`
+
+> **Source:** [`bigquery-ml/reference/model-free-functions.md#mlbucketize`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-free-functions.md#mlbucketize)
+
+- **Description:** Bucketizes a continuous numerical value into a string-named bucket using a manually supplied array of split points (bin boundaries).
+- **Use cases:**
+  - Convert a numeric column (age, price) into categorical bins for linear/logistic models.
+  - Encode domain knowledge as explicit boundaries (e.g. age brackets).
+  - Inside `TRANSFORM` so the same boundaries are reapplied at `ML.PREDICT`.
+- **documentation:** [ML.BUCKETIZE](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-bucketize)
+- **Type:** Scalar (operates row-by-row; no `OVER()`).
+- **Applies to models:** Any model type that supports manual feature preprocessing (linear/logistic regression, boosted trees, DNN, k-means, etc.) when used in `TRANSFORM`; also usable in plain SQL.
+
+**Syntax:**
+```sql
+ML.BUCKETIZE(numerical_expression, array_split_points[, exclude_boundaries[, output_format]])
+```
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `numerical_expression` | NUMERIC/FLOAT64 | Yes | — | The numerical value to bucketize. |
+| `array_split_points` | ARRAY\<numeric\> | Yes | — | Sorted points at which to split into buckets. |
+| `exclude_boundaries` | BOOL | No | `FALSE` | If `TRUE`, drops the implicit lower (`-inf`) and upper (`+inf`) overflow buckets so only interior bins remain. |
+| `output_format` | STRING | No | `"bucket_names"` | `"bucket_names"` → `bin_<i>` (index starts at 1); `"bucket_ranges"` → `[lower, upper)`; `"bucket_ranges_json"` → `{"start":"..","end":".."}`. |
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| (scalar result) | STRING | Bucket label, e.g. `bin_3` or `[2, 3)` depending on `output_format`. |
+
+**Best practices:**
+- Generate boundaries dynamically with `GENERATE_ARRAY(start, end, step)` for evenly spaced bins.
+- Place inside `TRANSFORM` so the bin definition travels with the model to serving/export.
+- Use `exclude_boundaries = TRUE` to suppress overflow buckets when out-of-range values are not expected.
+
+**Limitations:**
+- Split points must be numeric and sorted; behavior with NULL `numerical_expression` returns NULL.
+- Boundaries are fixed (no data-driven balancing) — for equal-frequency bins use `ML.QUANTILE_BUCKETIZE`.
+- **GOTCHA, verified live — `exclude_boundaries=TRUE` does NOT null out-of-range values.** It's easy to misread "drops the implicit lower/upper overflow buckets" as "values outside the split-point range become NULL." What actually happens: the **outermost split points are dropped entirely**, merging the overflow bucket into its nearest interior neighbor. With split points `[10, 20, 30]`: default gives 4 bins `(-inf,10)` `[10,20)` `[20,30)` `[30,+inf)`; with `exclude_boundaries=TRUE` this becomes just 2 bins `(-inf,20)` `[20,+inf)` — the `10` and `30` split points disappear, leaving only `20` as the sole effective boundary. No value ever becomes NULL from this option alone.
+
+**BigFrames API:** `bigframes.ml.preprocessing.KBinsDiscretizer` (strategy-dependent; not a 1:1 of explicit split points).
+**Repo example (tested):** [`functions/bucketizing/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/bucketizing) — Example 1 runs all three `output_format` values against the same `[3000, 4000, 5000]` split points on `penguins.body_mass_g`; Example 2 is the `exclude_boundaries` clarification above, proven live with `[10, 20, 30]`; Example 5 embeds `ML.QUANTILE_BUCKETIZE` and `ML.HASH_BUCKETIZE` together in a real `LOGISTIC_REG` `TRANSFORM` (accuracy ~0.81).
+
+---

bigquery_data_science/_skills/bigquery-ml/syntax/mlcentroids.md

@@ -0,0 +1,46 @@
+# `ML.CENTROIDS`
+
+> **Source:** [`bigquery-ml/reference/model-lifecycle-functions.md#mlcentroids`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-lifecycle-functions.md#mlcentroids)
+
+- **Description:** Returns the learned centroid (cluster center) coordinates of a trained `KMEANS` model — one row per feature per centroid, so you can inspect what each cluster "looks like."
+- **Use cases:**
+  - Profile/interpret clusters by comparing each centroid's feature values.
+  - Compare centroids across hyperparameter-tuning trials (one `trial_id` per tuned model).
+  - Feed centroid coordinates into downstream segmentation/labeling logic.
+- **documentation:** https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-centroids
+- **Type:** Table-valued function (model weights function).
+- **Applies to models:** `KMEANS` only.
+
+**Syntax:**
+```sql
+SELECT *
+FROM ML.CENTROIDS(MODEL `PROJECT_ID.DATASET.MODEL_NAME`
+  [, STRUCT(standardize AS standardize)]);
+```
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `MODEL` | model ref | Yes | — | A trained `KMEANS` model. |
+| `standardize` | BOOL | No | `TRUE` | Whether to return centroid feature values in standardized (`TRUE`) or original (`FALSE`) space. |
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `centroid_id` | INT64 | The cluster (centroid) identifier. |
+| `feature` | STRING | The feature/column name. |
+| `numerical_value` | FLOAT64 | Centroid value for numeric features (else NULL). |
+| `categorical_value` | ARRAY\<STRUCT\<category STRING, value FLOAT64\>\> | Centroid values for categorical (one-hot) features. |
+| `trial_id` | INT64 | Present only for hyperparameter-tuned models; identifies the trial. |
+
+**Best practices:** Use `standardize = FALSE` to read centroids in the original feature units when profiling clusters for business stakeholders.
+**Limitations:** `KMEANS` only — not valid for PCA/autoencoder/MF or supervised models. Numeric vs. categorical features land in separate columns; `UNNEST(categorical_value)` to flatten one-hot categories.
+**BigFrames API:** `bigframes.ml.cluster.KMeans().cluster_centers_` (centroid attribute).
+**Repo example (tested):**
+- `bigquery-ml/models/kmeans/kmeans.sql` (Example 4) — one row per (`centroid_id`, `feature`) `ORDER BY centroid_id, feature`, read as the interpretation step: the centroid with the highest body-mass and flipper-length coordinates is the large-species cluster. Because `KMEANS` is non-deterministic (Example 7), read the profile off the coordinates, not off a remembered `centroid_id`.
+- `bigquery-ml/workflows/customer_segmentation/customer_segmentation.sql` (Step 3) — the same call on RFM features, feeding the aggregated segment profiles in Step 4.
+- On a hyperparameter-tuned model the output also carries `trial_id`, so the row count multiplies by the number of trials (one row per feature x centroid x trial) — see `models/kmeans/kmeans.sql`'s tuning example for the tuned variant.
+
+---

bigquery_data_science/_skills/bigquery-ml/syntax/mlconfusion_matrix.md

@@ -0,0 +1,56 @@
+# `ML.CONFUSION_MATRIX`
+
+> **Source:** [`bigquery-ml/reference/model-lifecycle-functions.md#mlconfusion_matrix`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-lifecycle-functions.md#mlconfusion_matrix)
+
+- **Description:** Returns a confusion matrix for a trained classification model — rows of actual (expected) labels vs. counts of predicted labels — computed over evaluation/input data.
+- **Use cases:**
+  - Inspect per-class correct vs. incorrect predictions beyond a single accuracy number.
+  - Diagnose class imbalance and which classes a model confuses (multiclass).
+  - Validate the effect of a custom decision `threshold` on binary classifiers.
+- **documentation:** https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-confusion
+- **Type:** Table-valued function (TVF).
+- **Applies to models:** Classification models only — `LOGISTIC_REG` (binary + multiclass), `BOOSTED_TREE_CLASSIFIER`, `RANDOM_FOREST_CLASSIFIER`, `DNN_CLASSIFIER`, `DNN_LINEAR_COMBINED_CLASSIFIER` (wide-and-deep), and `AUTOML_CLASSIFIER`. Does **not** support imported TensorFlow models, regression, clustering, MF, PCA, autoencoder, ARIMA_PLUS, or remote/LLM models.
+
+**Syntax:**
+```sql
+ML.CONFUSION_MATRIX(
+  MODEL `PROJECT_ID.DATASET.MODEL_NAME`
+  [, { TABLE `PROJECT_ID.DATASET.TABLE` | (QUERY_STATEMENT) }]
+  [, STRUCT([THRESHOLD AS threshold] [, TRIAL_ID AS trial_id])]
+)
+```
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `MODEL` | model ref | Yes | — | A trained classification model. |
+| `TABLE` / `QUERY_STATEMENT` | table or query | No\* | eval/training split | Input data whose columns match the model's features plus the label column. If omitted, uses the held-out eval split (or the full training data if no split). \*Required for some models, and required if you set `threshold`. |
+| `threshold` | FLOAT64 (in STRUCT) | No | 0.5 | Custom positive-class cutoff. **Binary only** — error if used on a multiclass model. Requires TABLE/QUERY also be supplied. |
+| `trial_id` | INT64 (in STRUCT) | No | optimal trial | Select a specific hyperparameter-tuning trial; defaults to the optimal trial. |
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `expected_label` | (label type) | Actual label value; one row per class. Exact value/type as passed in the label column. |
+| \<one column per class\> | INT64 | Count of rows with that actual label predicted as this class. Column name is the class label (if it conforms to BigQuery column-naming rules). |
+
+**Best practices:**
+- Call without input args right after training to use the held-out eval split; pass your own TABLE for an independent test set.
+- For binary models, sweep `threshold` (e.g. 0.3 / 0.5 / 0.7) to see precision/recall trade-offs as integer counts.
+- Pair with `ML.EVALUATE` (scalar metrics) and `ML.ROC_CURVE` (threshold curve) for a complete classification picture.
+
+**Limitations:**
+- Classification models only; no regression/clustering/imported-TF.
+- `threshold` is binary-only and requires explicit input data.
+- Column names derive from class labels; labels not matching column-naming rules get sanitized names.
+
+**BigFrames API:** `model.confusion_matrix(X, y)` on a `bigframes.ml` classifier (e.g. `LogisticRegression`, `XGBoostClassifier`).
+**Repo example (tested):**
+- `bigquery-ml/models/logistic_regression/logistic_regression.sql` (Example 3) — `SELECT * FROM ML.CONFUSION_MATRIX(MODEL ...)` on the census income binary classifier.
+- `bigquery-ml/models/boosted_tree_classifier/boosted_tree_classifier.sql` (Example 3) — the same call on a `BOOSTED_TREE_CLASSIFIER`; `models/random_forest_classifier/`, `models/dnn_classifier/`, `models/wide_and_deep_classifier/`, and `models/automl_classifier/` each carry it too, so the matrix is directly comparable across classifier types on one dataset.
+- `bigquery-ml/workflows/churn_retention/churn_retention.sql` (Step 3) — uses the matrix to settle *which class* a metric describes: the `expected_label = true` row (FALSE=1997, TRUE=3059) confirms `ML.EVALUATE`'s `recall = 0.605` is recall on the churned class, not the retained one.
+- `bigquery-ml/workflows/cross_validation/cross_validation.sql` — the motivating case for k-fold: a held-out eval split with only ~15 real fraud cases (TP=13, FN=2), where a metric estimated from 15 positives is exactly the high-variance situation cross-validation exists to quantify.
+
+---

bigquery_data_science/_skills/bigquery-ml/syntax/mlconvert_color_space.md

@@ -0,0 +1,53 @@
+# `ML.CONVERT_COLOR_SPACE`
+
+> **Source:** [`bigquery-ml/reference/model-free-functions.md#mlconvert_color_space`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-free-functions.md#mlconvert_color_space)
+
+- **Description:** Converts an RGB image to a different color space.
+- **Use cases:**
+  - Match the color space a vision model was trained on (e.g. grayscale or YIQ input).
+- **documentation:** [ML.CONVERT_COLOR_SPACE](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-convert-color-space)
+- **Type:** Scalar (row-wise; no `OVER()`).
+- **Applies to models:** Same as the family; consumes `ML.DECODE_IMAGE`/`ML.RESIZE_IMAGE` output (input must be RGB).
+
+**Syntax:**
+```sql
+ML.CONVERT_COLOR_SPACE(rgb_image, target_color_space)
+```
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `rgb_image` | STRUCT (RGB, from `ML.DECODE_IMAGE`/`ML.RESIZE_IMAGE`) | Yes | — | Image in RGB color space. Accepts **either** value type — `ARRAY<FLOAT64>` or the `ARRAY<INT64>` produced by `ML.CONVERT_IMAGE_TYPE`. |
+| `target_color_space` | STRING | Yes | — | Target color space: `'HSV'`, `'YIQ'`, `'YUV'`, or `'GRAYSCALE'`. |
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| (result) | STRUCT\<ARRAY\<INT64\> dimensions, ARRAY\<FLOAT64\> values\> | Image in the requested color space. Always `FLOAT64`, whichever value type went in. `'GRAYSCALE'` returns **1 channel**; the other three return 3. |
+
+**Best practices:** Only convert when the model expects a non-RGB color space; input must be RGB.
+**Limitations:**
+- **Requires a reservation** — see the family gotcha above.
+- **`'GRAYSCALE'` changes the channel count to 1** (verified: a 1×2 RGB image converts to `[1, 2, 1]`, two values). This is the reverse of `ML.DECODE_IMAGE`, which replicates a grayscale source *up* to 3 channels — so a decode-then-grayscale pipeline goes 1 → 3 → 1, and the channel count of a feature vector is set by the last function in the chain, not by the source image. `'HSV'`, `'YIQ'`, and `'YUV'` return 3 distinct channels.
+- **`'GRAYSCALE'` and `'YIQ'`'s Y channel are two different lumas.** `'GRAYSCALE'` uses `[0.2989, 0.5870, 0.1140]` — the 4-decimal rounding `tf.image.rgb_to_grayscale` hardcodes. `'YIQ'`'s Y uses the true BT.601 `[0.299, 0.587, 0.114]`. On RGB `(200, 100, 50)` they give `0.48698039215686273` and `0.4870588235294117`, a gap of `0.02` of one 8-bit level. Invisible, and enough to break a bit-for-bit reproducibility check against a reimplementation — pick the right one deliberately.
+- **`'HSV'`'s hue is on `[0, 1]`, not degrees.** `(200, 100, 50)` → hue `0.05555555555555555`, which is 20°/360. A threshold written in degrees is off by 360×.
+- **Integer input is normalized, not misread.** `ML.CONVERT_COLOR_SPACE(ML.CONVERT_IMAGE_TYPE(img), 'HSV')` is bit-identical to the float path — it divides by 255 on the way in rather than treating a `200` as already-scaled.
+- An unsupported target is rejected by name: `Color space LAB is not supported; error in ML.CONVERT_COLOR_SPACE expression`.
+- Output-size/display gotcha as above.
+**BigFrames API:** No direct equivalent.
+**Repo example (tested):** [`functions/image/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/image) Step 5 — the channel-count change, the two lumas, the `[0, 1]` hue, integer normalization, and the rejected target; Step 7 diffs `'HSV'`/`'YIQ'`/`'YUV'` against TensorFlow.
+
+---
+
+**Status:** All four are **GA**. **Reservation required: YES** — see the family gotcha at the top of this section; this is the one thing about them that no official page states. **Connection required:** No for the functions themselves; the *object table* over the image files needs a Cloud Resource Connection whose service account holds `roles/storage.objectViewer` on the bucket, and the vision model may need one too (see the model entry).
+
+**TRANSFORM-clause behavior:** All four are exportable preprocessing functions usable inside `TRANSFORM`, so the decode/resize/type/color-space steps are stored with the model and re-applied automatically at `ML.PREDICT`. Because they are scalar (no `OVER()`), they nest directly and impose no analytic-window constraints — unlike the analytic scalers/encoders/text functions documented in this folder.
+
+**Typical pipeline:** object table (`data` BYTES) → `ML.DECODE_IMAGE` → `ML.RESIZE_IMAGE` → `ML.CONVERT_COLOR_SPACE` → `ML.CONVERT_IMAGE_TYPE` → vision model via `ML.PREDICT` (or all of it inside `TRANSFORM`). **The order is not arbitrary:** `ML.CONVERT_IMAGE_TYPE` is one-way and has no integer→float signature, and `ML.RESIZE_IMAGE` is type-preserving, so converting early quantizes at every later resize. Convert last.
+
+**Sources:** [ML.DECODE_IMAGE](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-decode-image) · [ML.RESIZE_IMAGE](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-resize-image) · [ML.CONVERT_IMAGE_TYPE](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-convert-image-type) · [ML.CONVERT_COLOR_SPACE](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-convert-color-space) · [Run inference on image object tables](https://cloud.google.com/bigquery/docs/object-table-inference)
+
+
+---

bigquery_data_science/_skills/bigquery-ml/syntax/mlconvert_image_type.md

@@ -0,0 +1,39 @@
+# `ML.CONVERT_IMAGE_TYPE`
+
+> **Source:** [`bigquery-ml/reference/model-free-functions.md#mlconvert_image_type`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-free-functions.md#mlconvert_image_type)
+
+- **Description:** Converts the floating-point pixel values produced by `ML.DECODE_IMAGE` into integers on the **closed** range `[0, 255]`, as required by some models.
+- **Use cases:**
+  - Feed integer-input vision models (e.g. SSD MobileNet V2, which expects `tf.uint8`).
+- **documentation:** [ML.CONVERT_IMAGE_TYPE](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-convert-image-type)
+- **Type:** Scalar (row-wise; no `OVER()`).
+- **Applies to models:** Same as the family; consumes `ML.DECODE_IMAGE` (or other image-function) output.
+
+**Syntax:**
+```sql
+ML.CONVERT_IMAGE_TYPE(decoded_image)
+```
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `decoded_image` | STRUCT with `ARRAY<FLOAT64> values` | Yes | — | Image whose float pixel values are converted to integers on `[0, 255]`. **Only the float form is accepted** — see below. |
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| (result) | STRUCT\<ARRAY\<INT64\> dimensions, ARRAY\<INT64\> values\> | Image with integer pixel values. |
+
+**Best practices:** Apply only when the target model requires integer pixels; leave float output for models trained on `[0,1]` floats. Make it the **last** step of the pipeline — `ML.RESIZE_IMAGE` is type-preserving and will re-quantize integers at every subsequent resize.
+**Limitations:**
+- **Requires a reservation** — see the family gotcha above.
+- **It is one-way.** There is no inverse and no overload for integer input; feeding it its own output fails with `No matching signature for function ML.CONVERT_IMAGE_TYPE / Argument types: STRUCT<dimensions ARRAY<INT64>, values ARRAY<INT64>> / Signature: ML.CONVERT_IMAGE_TYPE(STRUCT<dimensions ARRAY<INT64>, values ARRAY<FLOAT64>>)`. (That error is also the authoritative statement of the struct's field names.)
+- The documented range `[0, 255)` is **wrong at the top end** — a pure-white pixel returns exactly `255` (verified). The interval is closed.
+- Matches `tf.image.convert_image_dtype(..., tf.uint8)` exactly (verified against TensorFlow 2.21.0).
+- Output-size/display gotcha as above.
+**BigFrames API:** No direct equivalent.
+**Repo example (tested):** [`functions/image/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/image) Steps 3–4 — the closed range, the one-way signature error, and the pipeline-ordering consequence; Step 8 ends its feature pipeline with it.
+
+---

bigquery_data_science/_skills/bigquery-ml/syntax/mlcorrelation.md

@@ -0,0 +1,78 @@
+# `ML.CORRELATION`
+
+> **Source:** [`bigquery-ml/reference/model-free-functions.md#mlcorrelation`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-free-functions.md#mlcorrelation)
+
+- **Description:** Computes the correlation between one target column and one or more numeric correlation columns, optionally sliced by dimension columns. Three methods: `PEARSON`, `SPEARMAN`, `KENDALL`. With `dimension_cols`, the output is a full `GROUP BY CUBE` over those dimensions. **Preview.**
+- **Use cases:**
+  - Rank candidate features against a target before feature selection.
+  - Detect **Simpson's-paradox-shaped** structure: the whole-table coefficient versus the same coefficient inside every segment, in one query.
+  - Compare linear (`PEARSON`) against monotone (`SPEARMAN`) association to find non-linear-but-ordered relationships.
+  - Produce a correlation matrix over a wide table without writing one `CORR()` per pair.
+- **documentation:** https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-correlation
+- **Type:** Table-valued function. One row per (correlation column × cube cell).
+- **Applies to models:** None — model-free utility.
+
+**Syntax:**
+```sql
+ML.CORRELATION(
+  { TABLE `PROJECT_ID.DATASET.TABLE_NAME` | (QUERY_STATEMENT) },
+  target_col              => 'target',
+  target_correlation_cols => 'metric' | ['metric_1', 'metric_2', ...]
+  [, dimension_cols       => 'dim'    | ['dim_1', 'dim_2', ...] ]
+  [, method               => 'PEARSON' | 'SPEARMAN' | 'KENDALL' ]
+)
+```
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| input data | `TABLE` reference or `(QUERY_STATEMENT)` | Yes | — | Data to correlate. |
+| `target_col` | `STRING` | Yes | — | Name of the numeric target column. |
+| `target_correlation_cols` | `STRING` or `ARRAY<STRING>` | Yes | — | Numeric column(s) to correlate against the target. |
+| `dimension_cols` | `STRING` or `ARRAY<STRING>` | No | none | Groupable column(s) to slice by. **Maximum 12.** |
+| `method` | `STRING` | No | `'PEARSON'` | `'PEARSON'`, `'SPEARMAN'`, or `'KENDALL'`. |
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `target_col` | `STRING` | Echo of the target column name. |
+| `corr_col` | `STRING` | Which correlation column this row is for. |
+| `correlation` | `FLOAT64` | The coefficient. `NULL` for degenerate segments (a single row) — those rows are **not** dropped. |
+| `segment_size` | `INT64` | Rows in the segment. **Not** the pairwise *n* the coefficient used. |
+| `segment_proportion` | `FLOAT64` | `segment_size` as a fraction of the whole input. |
+| `segment` | `ARRAY<STRUCT<dimension_col STRING, dimension_value JSON>>` | Which dimensions this row is *not* rolled up over, and their values. Empty array on the global row. |
+| one column per `dimension_cols` entry | source type | The dimension value, or `NULL` — for either of two reasons; see below. |
+
+**Best practices:**
+- **Round before comparing.** Never compare `correlation` across queries with `=`.
+- **Use `segment` to disambiguate `NULL`.** A dimension column is `NULL` both when the row is a rollup over that dimension and when the segment *is* the genuine-`NULL` group. The rolled-up dimension is **absent** from `segment`; the genuine-`NULL` group is **present** in `segment` with `dimension_value` = JSON `null`. `NOT EXISTS (SELECT 1 FROM UNNEST(segment) s WHERE s.dimension_col = 'col')` is the rollup test.
+- **Bucketize continuous columns before using them as dimensions** — see [`ML.QUANTILE_BUCKETIZE`](mlquantile_bucketize.md) and [`functions/bucketizing/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/bucketizing).
+- **Filter by `segment_size`** to keep the cube readable, but treat it as an upper bound on evidence, not a sample size.
+- Run [`ML.DESCRIBE_DATA`](mldescribe_data.md) first — it tells you the `NULL` rates that make `segment_size` misleading, and catches placeholder-encoded nulls before they become a segment.
+
+**Limitations:**
+- **Preview.**
+- **`SPEARMAN` is not the textbook Spearman.** BigQuery computes it as `CORR()` over SQL `RANK()` — **competition (min) ranks**, where every tied observation gets the smallest rank in its group. SciPy, R, and pandas' default use **mid-ranks** (the average of the ranks the tie spans). On tied data the two disagree in the second decimal place: on `census_adult_income`, `education_num` vs `hours_per_week` reads `0.172612` from `ML.CORRELATION` and `0.167215` from `scipy.stats.spearmanr`. Reproduce BigQuery's number with `pandas.Series.rank(method='min')` or a pure-SQL `CORR()` over `RANK()`; on tie-free data all conventions agree. Not documented.
+- **`KENDALL` *is* tie-corrected — it is tau-b.** So within one function, Kendall corrects for ties and Spearman does not. Matched against `scipy.stats.kendalltau(variant='b')` to floating-point noise; `variant='c'` does not match.
+- **`KENDALL` is O(n²) and the constant is not small.** Measured on a synthetic table with the cache off, every doubling of rows multiplies slot time several-fold and the fitted exponent lands near 2, while `PEARSON` and `SPEARMAN` show no size dependence at all across the same range and return a million rows in about a second. A `KENDALL` on 100,000 rows of that shape did not finish inside 12 minutes. Sample or segment before using it.
+- **Projecting the `segment` column changes `correlation`** in its last three digits, reproducibly, with the query cache off. Column pruning changes the physical plan, which changes the float summation order. Every projection that includes `segment` returns one value; every projection that omits it returns another. The gap lands in the sixteenth decimal place.
+- **`PEARSON` matches `CORR()` but is not bit-identical** — agreement to ~15 significant digits, residual ~4e-16, float summation order again.
+- **`segment_size` is the segment's row count, not the pairwise *n*.** Correlation is computed pairwise: for each correlation column, rows where either that column or the target is `NULL` are dropped. Two `corr_col` rows in the same segment can rest on different sample sizes and nothing in the output says so. `WHERE segment_size >= 30` does not guarantee 30 usable pairs.
+- **`dimension_cols` accepts any groupable type, including `INT64`.** Passing a continuous numeric column silently produces one segment per distinct value (73 distinct ages → 74 output rows, no error).
+- **Maximum 12 `dimension_cols`** — a 13th fails with an explicit error. Output row count is the full CUBE cell count, including degenerate single-row cells, so it grows fast: three dimensions on `census_adult_income` produce 134 rows.
+
+**Related — three ways to correlate in BigQuery:**
+
+| You want | Use | Notes |
+|---|---|---|
+| One pair, Pearson | `CORR(x, y)` | Aggregate function; matches `ML.CORRELATION` `PEARSON` to ~15 digits |
+| One target vs many columns, optionally sliced | `ML.CORRELATION` | Three methods, CUBE slicing, one query |
+| Which features *drive* an outcome, not just co-move | [`AI.KEY_DRIVERS`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/augmented-analytics.md) / [`AI.CAUSAL_EFFECT`](aicausal_effect.md) | Correlation is symmetric and pairwise; these are neither |
+
+**BigFrames API:** No wrapper. `bigframes.pandas.DataFrame.corr()` is a pandas-shaped equivalent compiled to BigQuery SQL — Pearson only, no dimension slicing, and a matrix rather than a target-vs-columns shape. Reach the SQL function via `bigframes.pandas.read_gbq`.
+
+**Repo example (tested):** [`functions/exploration/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/exploration) — establishes `PEARSON` == `CORR()`, reproduces the `SPEARMAN` min-rank convention three ways (pandas, pure SQL, and a tie-free control), confirms `KENDALL` is tau-b against SciPy, measures the Kendall cost curve, matches the `dimension_cols` output row count against `GROUP BY CUBE` exactly, separates the two kinds of dimension `NULL`, and shows `segment_size` overstating the pairwise *n*.
+
+---

bigquery_data_science/_skills/bigquery-ml/syntax/mldecode_image.md

@@ -0,0 +1,44 @@
+# `ML.DECODE_IMAGE`
+
+> **Source:** [`bigquery-ml/reference/model-free-functions.md#mldecode_image`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-free-functions.md#mldecode_image)
+
+- **Description:** Converts image bytes into a multi-dimensional `STRUCT` of dimensions + pixel values that downstream image functions and vision models consume. This is the required entry point of the image pipeline. The argument is **any `BYTES` value** — an object table's `data` pseudocolumn is the usual source, but a `FROM_BASE64(...)` literal or a `BYTES` table column works identically (verified).
+- **Use cases:**
+  - Decode JPEG/PNG/BMP bytes from an object table before inference with an imported/remote vision model.
+  - Produce a reusable decoded-image column to feed `ML.RESIZE_IMAGE` / `ML.CONVERT_*`.
+- **documentation:** [ML.DECODE_IMAGE](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-decode-image) · [Run inference on image object tables](https://cloud.google.com/bigquery/docs/object-table-inference)
+- **Type:** Scalar (row-wise; no `OVER()`).
+- **Applies to models:** Imported/remote vision models via `ML.PREDICT`, or any model's `TRANSFORM` that takes image input. Operates on object-table image bytes, not on a trained BQML model.
+
+**Syntax:**
+```sql
+ML.DECODE_IMAGE(image_bytes)
+```
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `image_bytes` | BYTES | Yes | — | Any `BYTES` value holding a JPEG/PNG/BMP image — typically an object table's `data` pseudocolumn. Must be `<= 10 MB`. |
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| (result) | STRUCT\<ARRAY\<INT64\> dimensions, ARRAY\<FLOAT64\> values\> | Decoded image. `dimensions` is `[height, width, channels]`; `values` is `H × W × C` floats, **row-major HWC**, each `pixel / 255` on the **closed** interval `[0, 1]`. |
+
+**Best practices:**
+- When passing `ML.DECODE_IMAGE` output **directly** into `ML.PREDICT`, alias it with the model's expected input field name (e.g. `... AS input`).
+- For repeated use, persist the decoded column to a table to avoid re-decoding.
+**Limitations:**
+- **Requires a reservation** — see the family gotcha above.
+- **Channels is always 3.** Verified: an RGBA PNG comes back `[1, 2, 3]` with the alpha channel **dropped, not composited** (an `alpha = 128` pixel returns its raw RGB unchanged), and a single-channel grayscale PNG is **replicated into three identical channels**. There is no way to get 1 or 4 channels out of this function; do alpha compositing before the bytes reach BigQuery.
+- The documented range `[0, 1)` is **wrong at the top end** — a pure-white pixel returns exactly `1.0` (verified). The interval is closed.
+- Values are float64-exact: `values` matches `pixel / 255` computed in float64 with a maximum absolute difference of `0.0` (verified).
+- Only JPEG, PNG, and BMP decode. Anything else — including a GIF, a WebP, or non-image bytes — fails with `Input image bytes could not be decoded. Expected valid image of type PNG, JPEG, or BMP; error in ML.DECODE_IMAGE expression`. `content_type` on an object table is upload-time metadata and is **not** validation; this error is the only real check.
+- JPEG is lossy and is not close at small sizes: a 1×2 image of `(200,100,50),(0,128,255)` round-trips through JPEG as `(105,123,159),(86,104,140)` (Pillow's encoder at its default quality; a different encoder or quality gives different numbers, which is the point). PNG round-trips exactly. Use PNG for anything where pixel values matter.
+- Large images can exceed editor display limits — write to a table.
+**BigFrames API:** No direct equivalent (use SQL / object tables).
+**Repo example (tested):** [`functions/image/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/image) — Steps 1–3 decode from `FROM_BASE64` literals and prove the field names, the channel rule, the closed range, the format list, and JPEG loss; Step 8 runs the object-table `data` path end to end.
+
+---