bigquery-data-science 0.7.0__py3-none-any.whl

bigquery_data_science/_skills/bigquery-ai/syntax/mlprocess_document.md

@@ -0,0 +1,125 @@
+# `ML.PROCESS_DOCUMENT`
+
+> **Source:** [`bigquery-ai/reference/document-processing.md#mlprocess_document`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/document-processing.md#mlprocess_document)
+
+- **Description:** Table-valued function that processes unstructured documents from an object table using the Document AI API. Sends documents stored in Cloud Storage (referenced via a BigQuery object table) to a Document AI processor and returns structured extraction results (entities, key-value pairs, parsed content) directly as BigQuery columns.
+- **Use cases:** Invoice parsing, expense/receipt extraction, form key-value extraction, OCR text extraction, document layout analysis and chunking (for RAG pipelines), bank statement parsing, W2/tax form parsing, utility bill parsing, pay slip parsing, US passport and driver license parsing, identity document proofing, custom document classification, custom document splitting, document summarization.
+- [documentation](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-process-document)
+- **Type:** Table-valued function (returns a table)
+
+**Syntax:**
+```sql
+SELECT *
+FROM ML.PROCESS_DOCUMENT(
+  MODEL `PROJECT_ID.DATASET.MODEL`,
+  { TABLE `PROJECT_ID.DATASET.OBJECT_TABLE` | (QUERY_STATEMENT) }
+  [, PROCESS_OPTIONS => (JSON 'PROCESS_OPTIONS')]
+);
+```
+
+**Model creation (prerequisite):**
+```sql
+CREATE OR REPLACE MODEL `PROJECT_ID.DATASET_ID.MODEL_NAME`
+REMOTE WITH CONNECTION {DEFAULT | `PROJECT_ID.REGION.CONNECTION_ID`}
+OPTIONS (
+  REMOTE_SERVICE_TYPE = 'CLOUD_AI_DOCUMENT_V1',
+  DOCUMENT_PROCESSOR = 'PROCESSOR_ID'
+);
+```
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `MODEL` | Model reference | Required | -- | Fully-qualified name of a remote model with `REMOTE_SERVICE_TYPE = 'CLOUD_AI_DOCUMENT_V1'`. |
+| `TABLE` / `(QUERY_STATEMENT)` | Table or subquery | Required | -- | Object table reference or a `SELECT` subquery over the object table. Subquery cannot use `JOIN` or rename columns with aliases. Must include `uri` and `content_type` columns. |
+| `PROCESS_OPTIONS` | JSON (named parameter) | Optional | -- | A `ProcessOptions` resource in JSON format. Configures processing options like OCR config, layout parser chunking, page selection. |
+
+**PROCESS_OPTIONS details:**
+
+| Field | Applicable To | Description |
+|-------|---------------|-------------|
+| `ocrConfig.hints.languageHints[]` | OCR, Form Parser | BCP-47 language codes. Auto-detection if not specified. |
+| `ocrConfig.enableNativePdfParsing` | OCR, Form Parser | Better extraction for PDFs with existing text. |
+| `ocrConfig.enableImageQualityScores` | OCR, Form Parser | Returns quality scores. Adds latency. |
+| `ocrConfig.premiumFeatures.enableSelectionMarkDetection` | OCR 2.0+ | Enable checkbox detection. |
+| `ocrConfig.premiumFeatures.computeStyleInfo` | OCR, Form Parser | Enable font identification and style info. |
+| `ocrConfig.premiumFeatures.enableMathOcr` | OCR, Form Parser | Extract LaTeX math formulas. |
+| `layoutConfig.chunkingConfig.chunkSize` | Layout Parser | Chunk size for splitting documents. |
+| `layoutConfig.chunkingConfig.includeAncestorHeadings` | Layout Parser | Include ancestor headings when splitting. |
+| `layoutConfig.returnImages` | Layout Parser | Include images in response. |
+| `layoutConfig.returnBoundingBoxes` | Layout Parser | Include bounding boxes. |
+| `individualPageSelector.pages[]` | All | 1-indexed list of specific pages to process. |
+| `fromStart` | All | Only process this many pages from the start. |
+| `fromEnd` | All | Only process this many pages from the end. |
+
+Note: `individualPageSelector`, `fromStart`, and `fromEnd` are a union field — only one can be specified. Setting `ocrConfig` on a non-OCR processor or `layoutConfig` on a non-Layout Parser processor returns an error.
+
+**Outputs:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `ml_process_document_result` | JSON | Entities returned by the Document AI API (confidence scores, mention text, page anchors, bounding polygons, properties). |
+| `ml_process_document_status` | STRING | API response status. Empty if successful. Contains error messages (e.g., `RESOURCE EXHAUSTED`) if processing failed. |
+| *(processor-specific fields)* | *(varies)* | Fields specific to the processor (e.g., invoice parser returns `invoice_type`, `currency`, `total_amount`). Visible in the model's Schema tab under Labels. |
+| *(object table columns)* | *(varies)* | All columns from the input object table or query are passed through (e.g., `uri`, `content_type`). |
+
+**Supported Document AI processors:**
+
+| Category | Processor | Type ID |
+|----------|-----------|---------|
+| Digitize Text | Enterprise Document OCR | `OCR_PROCESSOR` |
+| Extract (General) | Custom Extractor | `CUSTOM_EXTRACTION_PROCESSOR` |
+| Extract (General) | Form Parser | `FORM_PARSER_PROCESSOR` |
+| Extract (General) | Layout Parser | `LAYOUT_PARSER_PROCESSOR` |
+| Extract (Pretrained) | Bank Statement Parser | `BANK_STATEMENT_PROCESSOR` |
+| Extract (Pretrained) | W2 Parser | `FORM_W2_PROCESSOR` |
+| Extract (Pretrained) | US Passport Parser | `US_PASSPORT_PROCESSOR` |
+| Extract (Pretrained) | Utility Parser | `UTILITY_PROCESSOR` |
+| Extract (Pretrained) | Identity Document Proofing | `ID_PROOFING_PROCESSOR` |
+| Extract (Pretrained) | Pay Slip Parser | `PAYSTUB_PROCESSOR` |
+| Extract (Pretrained) | US Driver License Parser | `US_DRIVER_LICENSE_PROCESSOR` |
+| Extract (Pretrained) | Expense Parser | `EXPENSE_PROCESSOR` |
+| Extract (Pretrained) | Invoice Parser | `INVOICE_PROCESSOR` |
+| Classify | Custom Classifier | `CUSTOM_CLASSIFICATION_PROCESSOR` |
+| Split | Custom Splitter | `CUSTOM_SPLITTING_PROCESSOR` |
+| Summarize | Summarizer | `SUMMARY_PROCESSOR` |
+
+**Supported file types:**
+
+| Format | Extensions | MIME Type |
+|--------|-----------|-----------|
+| PDF | `.pdf` | `application/pdf` |
+| GIF | `.gif` | `image/gif` |
+| TIFF | `.tiff`, `.tif` | `image/tiff` |
+| JPEG | `.jpg`, `.jpeg` | `image/jpeg` |
+| PNG | `.png` | `image/png` |
+| BMP | `.bmp` | `image/bmp` |
+| WebP | `.webp` | `image/webp` |
+| HTML | `.html` | `text/html` (Layout Parser only) |
+| Word OOXML | `.docx` | `application/vnd.openxmlformats-officedocument.wordprocessingml.document` (Layout Parser only) |
+| PowerPoint OOXML | `.pptx` | `application/vnd.openxmlformats-officedocument.presentationml.presentation` (Layout Parser only) |
+| Excel OOXML | `.xlsx` | `application/vnd.openxmlformats-officedocument.spreadsheetml.sheet` (Layout Parser only) |
+
+**Best practices:**
+- Filter documents with `WHERE`/`LIMIT` in a subquery rather than processing the entire object table.
+- Use `CREATE TABLE ... AS SELECT` to persist results and avoid re-processing.
+- Handle `RESOURCE_EXHAUSTED` errors with BigQuery remote inference SQL scripts or the Dataform package for retry-until-complete iteration.
+- Select only the processor-specific columns you need rather than `SELECT *`.
+- Minimum 200 dpi for document scans; 300 dpi or higher recommended.
+
+**Limitations:**
+- Maximum **130 pages per document**. Rows with larger documents return an error.
+- **120-second timeout per request.** Documents that take longer to process will fail.
+- **Requests are processed in batches of 10.**
+- Custom Splitter only supports PDF, TIFF, TIF, and GIF.
+- See [Cloud AI service functions quotas](https://cloud.google.com/bigquery/quotas#cloud_ai_service_functions) for current rate limits.
+- Some rows may show `RESOURCE EXHAUSTED` errors after job success — BigQuery retries internally, but parallel batch queries can exceed quota limits.
+
+**Locations:** Models can only be created in the **US** and **EU** multi-regions. The dataset, connection, and Document AI processor must all be in the same region.
+
+**Provisioned throughput:** Not specified. Subject to Document AI API quotas.
+
+**BigFrames API:** No direct equivalent. Use `%%bigquery` magics or `session.read_gbq_query()` to execute ML.PROCESS_DOCUMENT SQL from BigFrames.
+
+---

bigquery_data_science/_skills/bigquery-ai/syntax/mltranscribe.md

@@ -0,0 +1,26 @@
+# `ML.TRANSCRIBE`
+
+> **Source:** [`bigquery-ai/reference/cloud-ai-service-models.md#mltranscribe`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/cloud-ai-service-models.md#mltranscribe)
+
+- **Description:** Table-valued function that transcribes audio in an object table with Speech-to-Text V2, returning the finished text in a `STRING` column.
+- [documentation](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-transcribe) · [notebook](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/functions/ml_transcribe/ml_transcribe.ipynb) · [sql](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/functions/ml_transcribe/ml_transcribe.sql)
+
+```sql
+SELECT * FROM ML.TRANSCRIBE(
+  MODEL `PROJECT_ID.DATASET.MODEL_NAME`,
+  { TABLE `PROJECT_ID.DATASET.OBJECT_TABLE` | (QUERY_STATEMENT) },
+  recognition_config => JSON '{"language_codes": ["en-US"], "model": "chirp", "auto_decoding_config": {}}');
+```
+
+**Output:** `transcripts` (`STRING` — the full transcript), `ml_transcribe_result` (JSON), `ml_transcribe_status`, plus the object table columns.
+
+**Worth knowing:**
+- **The third argument is a named argument, not a `STRUCT`** — the only member of the family that breaks the convention. It is also **required** unless the model was created with `SPEECH_RECOGNIZER`. Both mistakes produce errors that print the signature: `ML.TRANSCRIBE(MODEL, TABLE, [recognition_config => JSON])`.
+- **`chirp` is the only accepted `model`.** `chirp_2`, `long`, `short` and `telephony` are all rejected with *"Supported models are: chirp."*
+- **`SPEECH_RECOGNIZER` is regional.** BigQuery accepts a recognizer only from `asia-southeast1`, `us-central1` or `europe-west4` — a constraint to weigh against a multi-region dataset before choosing a recognizer over a per-call config.
+- **`language_codes` is echoed, not enforced.** Forcing `["en-US"]` on French audio still returns French, still reporting `en-US`. `["auto"]` detects and reports bare subtags (`fr`, `es`, `en`).
+- **Some `features` keys void the row.** `enable_automatic_punctuation` and `enable_word_time_offsets` work; `enable_word_confidence`, `diarization_config` and `profanity_filter` each fail the row with *"Config contains unsupported fields"* — naming no field, so you find the culprit by bisecting the object.
+- **The result JSON is keyed by URI.** `ml_transcribe_result.results[uri]…` — the accessor has to index the JSON with the row's own `uri` column, and long audio arrives as several segments, which the `transcripts` column concatenates for you.
+- **It is not reproducible.** Identical calls with the cache off can return different transcripts. Transcribe once, persist, and treat the text as data rather than as a key.
+
+---

bigquery_data_science/_skills/bigquery-ai/syntax/mltranslate.md

@@ -0,0 +1,27 @@
+# `ML.TRANSLATE`
+
+> **Source:** [`bigquery-ai/reference/cloud-ai-service-models.md#mltranslate`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/cloud-ai-service-models.md#mltranslate)
+
+- **Description:** Table-valued function that translates a text column, or detects its language, using the Cloud Translation V3 API.
+- [documentation](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-translate) · [notebook](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/functions/ml_translate/ml_translate.ipynb) · [sql](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/functions/ml_translate/ml_translate.sql)
+
+```sql
+SELECT * FROM ML.TRANSLATE(
+  MODEL `PROJECT_ID.DATASET.MODEL_NAME`,
+  { TABLE `PROJECT_ID.DATASET.TABLE` | (QUERY_STATEMENT) },
+  STRUCT('TRANSLATE_TEXT' AS translate_mode, 'LANGUAGE_CODE' AS target_language_code));
+```
+
+| Argument | Values | Notes |
+|---|---|---|
+| `translate_mode` | `'TRANSLATE_TEXT'`, `'DETECT_LANGUAGE'` | Two modes in one function |
+| `target_language_code` | ISO-639 code | **Required** for `TRANSLATE_TEXT`; `DETECT_LANGUAGE` does not take one |
+
+**Output:** `ml_translate_result` (JSON), `ml_translate_status`, plus the input columns including `text_content`.
+
+**Worth knowing:**
+- **One target language per call, and it cannot come from a column.** `target_language_code` is a scalar in the `STRUCT`, evaluated once for the whole call; a column reference fails to compile with `Unrecognized name`. Fanning out over three languages is three calls unioned together, each re-reading and re-billing the source text.
+- **The source language is detected for free** in `TRANSLATE_TEXT` mode — the result carries `detected_language_code` beside `translated_text`, so there is no reason to run `DETECT_LANGUAGE` first just to route rows. The two modes return different JSON shapes: `translations[]` versus `languages[]` with a confidence.
+- **It is stable across runs.** The notebook compares run-to-run output with the query cache off, and puts `AI.GENERATE` on the same rows for contrast.
+
+---

bigquery_data_science/_skills/bigquery-ai/syntax/mlunderstand_text.md

@@ -0,0 +1,32 @@
+# `ML.UNDERSTAND_TEXT`
+
+> **Source:** [`bigquery-ai/reference/cloud-ai-service-models.md#mlunderstand_text`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/cloud-ai-service-models.md#mlunderstand_text)
+
+- **Description:** Table-valued function that runs one of five Cloud Natural Language analyses over a text column.
+- [documentation](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-understand-text) · [notebook](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/functions/ml_understand_text/ml_understand_text.ipynb) · [sql](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/functions/ml_understand_text/ml_understand_text.sql)
+
+```sql
+SELECT * FROM ML.UNDERSTAND_TEXT(
+  MODEL `PROJECT_ID.DATASET.MODEL_NAME`,
+  { TABLE `PROJECT_ID.DATASET.TABLE` | (QUERY_STATEMENT) },
+  STRUCT('ANALYZE_SENTIMENT' AS nlu_option, TRUE AS flatten_json_output));
+```
+
+| `nlu_option` | What comes back |
+|---|---|
+| `ANALYZE_SENTIMENT` | Document and per-sentence sentiment, as a score and a magnitude |
+| `ANALYZE_ENTITIES` | Named things in the text, with a type and a salience |
+| `ANALYZE_ENTITY_SENTIMENT` | The same entities, each carrying its own sentiment |
+| `ANALYZE_SYNTAX` | Tokens with part of speech, lemma and dependency edges |
+| `CLASSIFY_TEXT` | Categories from a published, fixed taxonomy |
+
+Two more arguments: `flatten_json_output` (`BOOL`) adds unpacked columns beside the JSON, and `encoding_type` (`'UTF8'`, `'UTF16'`, `'UTF32'`) turns on the text-span offsets. Its default is `NONE`, which means `begin_offset` is simply not computed — and when you do ask, the offsets are measured in that encoding's units, so UTF-8 and UTF-16 disagree on any accented text.
+
+**Output:** `ml_understand_text_result` (JSON), `ml_understand_text_status`, the input columns, and — with `flatten_json_output` — the flattened columns.
+
+**Worth knowing:**
+- **Language support differs per analysis**, and an unsupported language fails **the row**, not the job. Measured: `ANALYZE_ENTITY_SENTIMENT` returns `INVALID_ARGUMENT: The language ko is not supported for entity_sentiment analysis.` for Korean and Vietnamese rows while the same call succeeds for the rest — entity sentiment supports fewer languages than sentiment does.
+- **An empty sentiment object reads as `NULL`, not `0`.** An entity can come back with no sentiment, and the accessor cannot tell that apart from neutral unless you coalesce.
+- **The taxonomy and the entity types are closed vocabularies.** That is the feature — the same text yields the same category next quarter — and the limit: if your business needs a category the taxonomy does not have, this is the wrong function.
+
+---

bigquery_data_science/_skills/bigquery-ai/syntax/multimodal-input-patterns-by-function.md

@@ -0,0 +1,128 @@
+# Multimodal Input Patterns by Function
+
+> **Source:** [`bigquery-ai/reference/unstructured-data-infrastructure.md#multimodal-input-patterns-by-function`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/unstructured-data-infrastructure.md#multimodal-input-patterns-by-function)
+
+**Every multimodal AI function takes an `ObjectRef`.** What differs between them is only *where the reference goes in the call*: a field of a STRUCT prompt, the second slot of a tuple, a bare positional argument, or a named `content` parameter. There are four shapes, not four pipelines.
+
+Where the `ObjectRef` comes from is an independent choice:
+
+```sql
+OBJ.MAKE_REF('gs://bucket/doc.pdf', 'PROJECT.REGION.CONNECTION')  -- inline, no table
+ref                                                                -- object table column
+```
+
+### Shape 1: STRUCT Prompt (most generation functions + AI.IF)
+
+Replace the STRING prompt with a STRUCT containing text and an array of `ObjectRef`. Works inline — no object table needed.
+
+**Functions:** `AI.GENERATE`, `AI.GENERATE_TEXT`, `AI.GENERATE_TABLE`, `AI.GENERATE_BOOL`, `AI.GENERATE_DOUBLE`, `AI.GENERATE_INT`, `ML.GENERATE_TEXT`, `AI.IF`
+
+```sql
+-- Scalar function (AI.GENERATE)
+SELECT (AI.GENERATE(
+  STRUCT(
+    'Summarize this document.' AS prompt,
+    [OBJ.MAKE_REF('gs://bucket/doc.pdf', 'PROJECT.REGION.CONNECTION')] AS object_refs
+  )
+)).result;
+
+-- TVF (AI.GENERATE_TEXT)
+SELECT result
+FROM AI.GENERATE_TEXT(
+  MODEL `project.dataset.model`,
+  (SELECT STRUCT(
+    'Summarize this document.' AS prompt,
+    [OBJ.MAKE_REF('gs://bucket/doc.pdf', 'PROJECT.REGION.CONNECTION')] AS object_refs
+  ) AS prompt)
+);
+
+-- Managed function (AI.IF)
+SELECT AI.IF(
+  STRUCT(
+    'This document is a financial invoice' AS prompt,
+    [OBJ.MAKE_REF('gs://bucket/doc.pdf', 'PROJECT.REGION.CONNECTION')] AS object_refs
+  )
+) AS is_invoice;
+```
+
+> The STRUCT's field *names* are not part of the contract — the function concatenates the fields in the order given, the way `CONCAT` would. `object_refs` is this project's convention because it says what the field holds; `object_ref_runtime` or anything else behaves identically.
+
+### Shape 2: Tuple Syntax (AI.SCORE)
+
+`AI.SCORE` does not accept the STRUCT prompt. Pass a tuple of (scoring text, `ObjectRef`).
+
+**Functions:** `AI.SCORE`
+
+```sql
+-- Create an object table first
+CREATE EXTERNAL TABLE `project.dataset.docs`
+WITH CONNECTION `project.region.connection`
+OPTIONS (object_metadata = 'SIMPLE', uris = ['gs://bucket/path/*.pdf']);
+
+-- Score documents using tuple syntax
+SELECT
+  uri,
+  AI.SCORE(
+    ('Rate the professionalism of this document on a scale of 0 to 1', ref)
+  ) AS professionalism
+FROM `project.dataset.docs`;
+```
+
+### Shape 3: Direct Argument (AI.CLASSIFY, AI.AGG)
+
+The reference is just an argument — no prompt STRUCT, no tuple. `AI.AGG` wraps it in a one-field `STRUCT` because that is how it takes any input.
+
+**Functions:** `AI.CLASSIFY`, `AI.AGG`
+
+```sql
+-- Classify straight off the object table's ref column
+SELECT
+  uri,
+  AI.CLASSIFY(ref, ['invoice', 'receipt', 'contract', 'letter']) AS doc_type
+FROM `project.dataset.docs`;
+
+-- AI.AGG wraps the reference in a one-field STRUCT
+SELECT AI.AGG(
+  STRUCT(ref),
+  'Summarize the common themes across these documents.'
+) AS themes
+FROM `project.dataset.docs`;
+```
+
+`EXTERNAL_OBJECT_TRANSFORM(TABLE t, ['SIGNED_URL'])` is not required here — verified live 2026-09-15 against a 10-PDF object table, where the bare `ref` column and the transformed one returned the same classifications.
+
+### Shape 4: `content` Parameter (Embedding and similarity functions)
+
+Embedding and similarity functions take the reference as the content argument rather than in a prompt. Requires a multimodal embedding endpoint (e.g., `multimodalembedding@001`).
+
+**Functions:** `AI.EMBED`, `AI.GENERATE_EMBEDDING`, `ML.GENERATE_EMBEDDING`, `AI.SIMILARITY`
+
+```sql
+-- AI.EMBED with inline ObjectRef
+SELECT AI.EMBED(
+  OBJ.MAKE_REF('gs://bucket/image.jpg', 'PROJECT.REGION.CONNECTION'),
+  endpoint => 'multimodalembedding@001',
+  connection_id => 'PROJECT.REGION.CONNECTION'
+).result AS embedding;
+
+-- AI.SIMILARITY with ObjectRef (returns a FLOAT64 directly, no .result accessor)
+SELECT AI.SIMILARITY(
+  OBJ.MAKE_REF('gs://bucket/img1.jpg', 'PROJECT.REGION.CONNECTION'),
+  OBJ.MAKE_REF('gs://bucket/img2.jpg', 'PROJECT.REGION.CONNECTION'),
+  endpoint => 'multimodalembedding@001',
+  connection_id => 'PROJECT.REGION.CONNECTION'
+) AS similarity;
+```
+
+### Shape Summary
+
+| Shape | Functions | Object Table Required? | Input Method |
+|-------|-----------|----------------------|--------------|
+| STRUCT prompt | AI.GENERATE, AI.GENERATE_TEXT, AI.GENERATE_TABLE, AI.GENERATE_BOOL, AI.GENERATE_DOUBLE, AI.GENERATE_INT, ML.GENERATE_TEXT, AI.IF | No | `STRUCT(text AS prompt, [refs] AS object_refs)` |
+| Tuple | AI.SCORE | No (object table is the convenient source) | `('scoring text', ref)` |
+| Bare argument | AI.CLASSIFY, AI.AGG | No (object table is the convenient source) | `AI.CLASSIFY(ref, categories)`, `AI.AGG(STRUCT(ref), prompt)` |
+| `content` parameter | AI.EMBED, AI.SIMILARITY, AI.GENERATE_EMBEDDING, ML.GENERATE_EMBEDDING | No | Pass the `ObjectRef` as the content argument |
+| Object table (document processing) | ML.PROCESS_DOCUMENT | Yes | Object table rows as input to Document AI processor |
+| Not supported | VECTOR_SEARCH, AI.SEARCH, AI.FORECAST, AI.DETECT_ANOMALIES, AI.PREDICT, AI.EVALUATE | — | Text/numeric input only |
+
+> **`ObjectRefRuntime` still works everywhere.** Nothing in this section is a breaking change — wrapping a reference in `OBJ.GET_ACCESS_URL` is accepted by every function above. It is simply no longer necessary, and `AI.AGG`'s Known Issues give a reason to prefer the unwrapped form.

bigquery_data_science/_skills/bigquery-ai/syntax/object-tables.md

@@ -0,0 +1,109 @@
+# Object Tables
+
+> **Source:** [`bigquery-ai/reference/unstructured-data-infrastructure.md#object-tables`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/unstructured-data-infrastructure.md#object-tables)
+
+- **Description:** Read-only external tables over unstructured data objects in Cloud Storage. Each row corresponds to a Cloud Storage object, with columns for object metadata. Object tables use access delegation via a Cloud resource connection — users need access to the table, not direct access to Cloud Storage.
+- **Use cases:** Indexing Cloud Storage objects for AI function processing, document analysis pipelines, image/video/audio inference, joining unstructured data results with structured BigQuery data.
+- [introduction](https://cloud.google.com/bigquery/docs/object-table-introduction) · [create](https://cloud.google.com/bigquery/docs/object-table-create) · [DDL reference](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_external_table_statement)
+
+**CREATE syntax:**
+```sql
+CREATE [ OR REPLACE ] EXTERNAL TABLE [ IF NOT EXISTS ] `PROJECT_ID.DATASET.TABLE_NAME`
+WITH CONNECTION { `PROJECT_ID.REGION.CONNECTION_ID` | DEFAULT }
+OPTIONS (
+    object_metadata = 'SIMPLE',
+    uris = ['BUCKET_PATH'[, ...]],
+    [max_staleness = STALENESS_INTERVAL,]
+    [metadata_cache_mode = 'CACHE_MODE']
+);
+```
+
+**Example (minimal):**
+```sql
+CREATE EXTERNAL TABLE `myproject.mydataset.documents`
+WITH CONNECTION `myproject.us.myconnection`
+OPTIONS (
+    object_metadata = 'SIMPLE',
+    uris = ['gs://mybucket/documents/*.pdf']
+);
+```
+
+**Example (with metadata caching):**
+```sql
+CREATE EXTERNAL TABLE `myproject.mydataset.documents`
+WITH CONNECTION `myproject.us.myconnection`
+OPTIONS (
+    object_metadata = 'SIMPLE',
+    uris = ['gs://mybucket/*'],
+    max_staleness = INTERVAL 1 DAY,
+    metadata_cache_mode = 'AUTOMATIC'
+);
+```
+
+**Metadata columns:**
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `uri` | STRING | Cloud Storage URI (`gs://bucket/path/object`). |
+| `generation` | INTEGER | Object version identifier. |
+| `content_type` | STRING | MIME type of the object (e.g., `image/jpeg`, `application/pdf`). Defaults to `application/octet-stream` if not set. |
+| `size` | INTEGER | Content length in bytes. |
+| `md5_hash` | STRING | MD5 hash of the data, base64-encoded. |
+| `updated` | TIMESTAMP | Last time the object's metadata was modified. |
+| `metadata` | RECORD (REPEATED) | Custom metadata as key-value pairs (`name` STRING, `value` STRING). |
+| `ref` | STRUCT (Preview) | `ObjectRef` value: `STRUCT<uri STRING, version STRING, authorizer STRING, details JSON>`. Created only if on the multimodal data preview allowlist. |
+
+There is also a `data` pseudocolumn containing raw file bytes, used by `ML.DECODE_IMAGE`. It cannot be directly queried.
+
+> **The image-preprocessing family lives in the sibling project** — `ML.DECODE_IMAGE`, `ML.RESIZE_IMAGE`, `ML.CONVERT_COLOR_SPACE`, `ML.CONVERT_IMAGE_TYPE`, documented in [`bigquery-ml/reference/model-free-functions.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-free-functions.md#image-preprocessing-functions-mldecode_image-mlresize_image-mlconvert_image_type-mlconvert_color_space) with a tested example in [`bigquery-ml/functions/image/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/image). Two things carry over to any object-table work here: those four functions **require a BigQuery Editions reservation** and fail outright on on-demand pricing (verified live 2026-09-11; no official page says so), and `ML.DECODE_IMAGE` accepts **any** `BYTES` value, so the `data` pseudocolumn is the usual source rather than the only one.
+
+**CREATE parameters:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `object_metadata` | STRING | Required | -- | Must be `'SIMPLE'` for object tables. |
+| `uris` | ARRAY\<STRING\> | Required | -- | Cloud Storage URIs. One `*` wildcard allowed per path. Multiple buckets supported. |
+| `max_staleness` | INTERVAL | Optional | `0` (disabled) | Metadata cache staleness. Range: 30 minutes to 7 days. |
+| `metadata_cache_mode` | STRING | Optional (required if `max_staleness > 0`) | -- | `'AUTOMATIC'` (system-refreshed every 30–60 min) or `'MANUAL'` (you call `BQ.REFRESH_EXTERNAL_METADATA_CACHE`). |
+
+**Metadata caching:**
+- Without caching, queries must list files from Cloud Storage — can take several minutes for large tables.
+- With caching, queries use cached metadata. Set `max_staleness` and `metadata_cache_mode`.
+- Automatic refresh: system-defined interval (30–60 min). Recommended: create a `BACKGROUND` reservation for refresh jobs.
+- Manual refresh: `CALL BQ.REFRESH_EXTERNAL_METADATA_CACHE('project.dataset.table');`
+- Cache expires after 7 days if not refreshed.
+
+**Signed URLs (EXTERNAL_OBJECT_TRANSFORM):**
+```sql
+SELECT uri, signed_url
+FROM EXTERNAL_OBJECT_TRANSFORM(TABLE `mydataset.myobjecttable`, ['SIGNED_URL']);
+```
+Signed URLs expire after 6 hours. Row-level access policies restrict which URLs a user can generate.
+
+`EXTERNAL_OBJECT_TRANSFORM` also rewrites the table's `ref` column from an `ObjectRef` into a signed `ObjectRefRuntime`. **No AI function requires that rewrite** — they all take the `ObjectRef` directly. Use the transform when something outside the AI call needs a fetchable URL: a display link, delegated access for another consumer, a shorter-lived credential.
+
+**Connection requirements:**
+- Requires a **Cloud resource connection** (type: "Vertex AI remote models, remote functions, BigLake and Spanner").
+- Connection's service account needs `roles/storage.objectViewer` on the Cloud Storage bucket.
+- Connection must be in the same region as the dataset.
+
+**AI function integration:** every one of these reads the `ref` column as-is — an `ObjectRef` — with no signing step in between.
+- `ML.PROCESS_DOCUMENT` — document extraction via Document AI processors.
+- `AI.GENERATE_TEXT` / `AI.GENERATE` — text generation from object data: `STRUCT(text AS prompt, [ref] AS object_refs)`.
+- `AI.GENERATE_EMBEDDING` — embeddings from image/video data: a `content` column of `ObjectRef`.
+- `AI.SCORE` — score documents using tuple syntax: `AI.SCORE(('text', ref))`.
+- `AI.CLASSIFY` — classify documents: `AI.CLASSIFY(ref, categories)`.
+- `ML.ANNOTATE_IMAGE` — image annotation via Cloud Vision API.
+- `ML.TRANSCRIBE` — audio transcription via Speech-to-Text API.
+
+**Limitations:**
+- Read-only — cannot alter data or use DML.
+- Maximum **60 million rows** (300 million in preview with allowlist).
+- Maximum **10 GB of object metadata** per query.
+- Not available in Legacy SQL, AWS, or Azure.
+- `UNION ALL` combining empty and non-empty object tables may error.
+- Signed URLs expire after 6 hours.
+
+**Locations:** Any BigQuery region or multi-region. Connection must be colocated with the dataset.
+
+---

bigquery_data_science/_skills/bigquery-ai/syntax/objectref-and-objectrefruntime-schema-reference.md

@@ -0,0 +1,50 @@
+# ObjectRef and ObjectRefRuntime Schema Reference
+
+> **Source:** [`bigquery-ai/reference/unstructured-data-infrastructure.md#objectref-and-objectrefruntime-schema-reference`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/unstructured-data-infrastructure.md#objectref-and-objectrefruntime-schema-reference)
+
+**ObjectRef** — a STRUCT stored in BigQuery table columns:
+```sql
+STRUCT<
+  uri STRING,           -- Cloud Storage URI (gs://bucket/path/file)
+  version STRING,       -- Cloud Storage object version
+  authorizer STRING,    -- Cloud resource connection for access
+  details JSON          -- Cloud Storage metadata (content_type, md5_hash, size, updated)
+>
+```
+
+**ObjectRefRuntime** — a JSON value produced by `OBJ.GET_ACCESS_URL`:
+```json
+{
+  "obj_ref": { "uri", "version", "authorizer", "details" },
+  "access_urls": { "read_url", "write_url", "expiry_time" }
+}
+```
+
+**Array support:** `ARRAY<STRUCT<uri STRING, version STRING, authorizer STRING, details JSON>>` columns can store ordered collections of ObjectRef values (e.g., video frames). Use `ARRAY_AGG` with `ORDER BY` to aggregate and `UNNEST` with `WITH OFFSET` to decompose.
+
+**Content requirements** (documented on `AI.GENERATE` and `AI.GENERATE_TEXT`, and applying wherever object references are accepted):
+- Content must be in one of the formats the Gemini API's `mimeType` parameter supports.
+- **At most one video object per input.** More than one is not rejected loudly — plan around the cap.
+- Videos are truncated to **two minutes**; a longer video returns results based on its first two minutes only.
+
+**Typical workflow:**
+```
+URI string
+  → OBJ.MAKE_REF(uri, connection)     → ObjectRef
+  → AI.GENERATE(STRUCT(prompt, ref))  → Generated output
+```
+
+**Shortcut from object tables:** when the `ref` column is available (preview allowlist), skip `OBJ.MAKE_REF` too — the column already *is* an `ObjectRef`.
+
+**Where the other two functions fit.** `OBJ.FETCH_METADATA` and `OBJ.GET_ACCESS_URL` are no longer steps on the path to an AI call; they are tools you reach for on purpose:
+
+| You want | Call |
+|----------|------|
+| Feed an object to an AI function | Nothing — pass the `ObjectRef` |
+| `content_type`, `size`, `md5_hash`, `updated` in SQL | `OBJ.FETCH_METADATA(objectref)` |
+| Display the object in query results | `OBJ.GET_READ_URL(objectref)` |
+| Hand a URL to a consumer outside BigQuery | `OBJ.GET_ACCESS_URL(objectref, 'r')` |
+| Write a transformation output back to GCS | `OBJ.GET_ACCESS_URL(objectref, 'rw')` |
+| An expiry shorter than the 6-hour default | `OBJ.GET_ACCESS_URL(objectref, 'r', INTERVAL 30 MINUTE)` |
+
+---

bigquery_data_science/_skills/bigquery-ai/syntax/objfetch_metadata.md

@@ -0,0 +1,49 @@
+# `OBJ.FETCH_METADATA`
+
+> **Source:** [`bigquery-ai/reference/unstructured-data-infrastructure.md#objfetch_metadata`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/unstructured-data-infrastructure.md#objfetch_metadata)
+
+- **Description:** (Preview) Scalar function that fetches Cloud Storage metadata for a partially populated `ObjectRef` value. Takes an ObjectRef with `uri` and `authorizer` (typically from `OBJ.MAKE_REF`) and fills in the `details` field with `content_type`, `md5_hash`, `size`, and `updated` from Cloud Storage. Still succeeds on error — the `details` field contains an `error` field instead.
+- **Use cases:** Populate ObjectRef columns with full metadata when not using object tables, refresh metadata for existing ObjectRef values after underlying objects change, combine with `OBJ.MAKE_REF` in a single expression: `OBJ.FETCH_METADATA(OBJ.MAKE_REF(uri, 'connection'))`.
+- [documentation](https://cloud.google.com/bigquery/docs/reference/standard-sql/objectref_functions)
+- **Type:** Scalar function (returns fully-populated ObjectRef STRUCT) — Preview
+
+**Syntax:**
+```sql
+OBJ.FETCH_METADATA(
+  objectref
+)
+
+OBJ.FETCH_METADATA(
+  ARRAY<objectref>
+)
+```
+The array overload fetches metadata for a whole `ARRAY<ObjectRef>` column in one call.
+
+**Inputs:**
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `objectref` | ObjectRef STRUCT | Required | Partially populated ObjectRef (with `uri` and `authorizer`). Typically from `OBJ.MAKE_REF()`. |
+
+**Outputs:** Returns a fully populated `ObjectRef` STRUCT. The `details` JSON field contains:
+```json
+{
+  "gcs_metadata": {
+    "content_type": "image/png",
+    "md5_hash": "d9c38814e44028bf7a012131941d5631",
+    "size": 23000,
+    "updated": 1741374857000000
+  }
+}
+```
+On error, `details` contains `{"errors": {"OBJ.FETCH_METADATA": "error message"}}` instead.
+
+**Required permissions:** `bigquery.objectRefs.read` on the connection (from `roles/bigquery.objectRefReader` or `roles/bigquery.objectRefAdmin`). Connection service account needs `roles/storage.objectUser`.
+
+**Limitations:**
+- Same 20-connection limit per project+region.
+- Less scalable than object tables for large numbers of objects (requires per-object metadata retrieval from Cloud Storage).
+
+**Locations:** All BigQuery regions supporting Cloud resource connections.
+
+---

bigquery_data_science/_skills/bigquery-ai/syntax/objget_access_url.md

@@ -0,0 +1,66 @@
+# `OBJ.GET_ACCESS_URL`
+
+> **Source:** [`bigquery-ai/reference/unstructured-data-infrastructure.md#objget_access_url`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/unstructured-data-infrastructure.md#objget_access_url)
+
+- **Description:** (Preview) Scalar function that converts an `ObjectRef` into an `ObjectRefRuntime` JSON value containing signed access URLs, with a caller-chosen mode (`'r'` / `'rw'`) and TTL.
+- **Use cases:** Generate a **writable** signed URL for saving transformation outputs back to Cloud Storage, hand a read URL to a consumer outside BigQuery, pin an explicit expiry shorter than the 6-hour default.
+- **Not needed to call an AI function.** Every multimodal `AI.*` function takes an `ObjectRef` directly; wrapping the reference in `OBJ.GET_ACCESS_URL` buys nothing but a longer query. See [Multimodal Input Patterns](multimodal-input-patterns-by-function.md).
+- [documentation](https://cloud.google.com/bigquery/docs/reference/standard-sql/objectref_functions)
+- **Type:** Scalar function (returns ObjectRefRuntime JSON) — Preview
+
+**Syntax:**
+```sql
+OBJ.GET_ACCESS_URL(
+  objectref,
+  mode
+  [, duration]
+)
+
+OBJ.GET_ACCESS_URL(
+  ARRAY<objectref>,
+  mode
+  [, duration]
+)
+```
+The array overload signs a whole `ARRAY<ObjectRef>` column in one call.
+
+**Inputs:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `objectref` | ObjectRef STRUCT | Required | -- | An ObjectRef value representing a Cloud Storage object. |
+| `mode` | STRING | Required | -- | `'r'` for read-only URL, `'rw'` for read and write URLs. |
+| `duration` | INTERVAL | Optional | `INTERVAL 6 HOUR` | URL validity period. Range: 30 minutes to 6 hours. |
+
+**Outputs:** Returns `ObjectRefRuntime` JSON:
+```json
+{
+  "obj_ref": {
+    "uri": "gs://bucket/file.jpg",
+    "version": "12345",
+    "authorizer": "us.connection1",
+    "details": {"gcs_metadata": {...}}
+  },
+  "access_urls": {
+    "read_url": "https://storage.googleapis.com/...",
+    "write_url": "https://storage.googleapis.com/...",
+    "expiry_time": "2024-01-15T18:30:00Z"
+  }
+}
+```
+On error, `access_urls` is replaced by `runtime_errors`.
+
+**Required permissions:**
+- Read URLs (`'r'`): `bigquery.objectRefs.read` (from `roles/bigquery.objectRefReader` or `roles/bigquery.objectRefAdmin`).
+- Write URLs (`'rw'`): `bigquery.objectRefs.write` (from `roles/bigquery.objectRefAdmin` only).
+
+**Accepted by AI functions:** yes — `ObjectRefRuntime` still works everywhere `ObjectRef` does, and `AI.AGG`'s reference page is the one that still specifies it. It is no longer the required form for any of them.
+
+**Limitations:**
+- Access URLs expire after at most 6 hours. Do not persist `ObjectRefRuntime` values long-term — regenerate from `ObjectRef` values.
+- Same 20-connection limit per project+region.
+- The `AI.AGG` page's Known Issues note that rows carrying arrays of `ObjectRefRuntime` values "might be skipped," and that some image objects created by `OBJ.GET_ACCESS_URL` "might fail to process." Passing the `ObjectRef` avoids both.
+
+**Locations:** All BigQuery regions supporting Cloud resource connections.
+
+---