@clickzetta/cz-cli-darwin-arm64 0.3.94 → 0.3.96

package/bin/skills/lakehouse-doc-en/references/dbt-bigquery-to-clickzetta-migration.md

@@ -0,0 +1,458 @@
+# DBT BigQuery Migration in Practice: Retail Data Warehouse Pipeline
+
+If you have built a data warehouse pipeline with dbt on BigQuery, the core migration effort to Singdata Lakehouse is concentrated in 4 platform differences. Standard SQL models require almost no changes.
+
+This article demonstrates the complete migration process using a real migration project: migrating [alanceloth/Retail_Data_Pipeline](https://github.com/alanceloth/Retail_Data_Pipeline) (a retail data pipeline built with Airflow + BigQuery + Cosmos + dbt) to Singdata Lakehouse. All models have been verified through actual execution — e2e 11/11 passing.
+
+Full code on GitHub: [clickzetta/bigquery2lakehouse-retail](https://github.com/clickzetta/bigquery2lakehouse-retail)
+
+---
+
+## Original Project
+
+[alanceloth/Retail_Data_Pipeline](https://github.com/alanceloth/Retail_Data_Pipeline) is a complete retail data engineering project. The data comes from the [Online Retail dataset on Kaggle](https://www.kaggle.com/datasets/carrie1/ecommerce-data) (UK e-commerce platform transactions from 2010, 540,000 rows).
+
+Original technology stack:
+
+- **Orchestration**: Airflow (Astronomer version) + Cosmos (automatically converts dbt models into Airflow TaskGroups)
+- **Storage**: Google Cloud Storage (GCS)
+- **Compute**: BigQuery
+- **Transformation**: dbt-bigquery
+- **Data quality**: Soda
+
+Data flow: CSV → GCS → BigQuery raw table → dbt transform (star schema) → dbt report (aggregated reports)
+
+7 dbt models building a standard star schema:
+
+| Model | Type | Description |
+|------|------|------|
+| `dim_customer` | Dimension table | Customer + country ISO code |
+| `dim_datetime` | Dimension table | Time dimension (year/month/day/hour/minute/weekday) |
+| `dim_product` | Dimension table | Product information |
+| `fct_invoices` | Fact table | Order line items, joined to three dimension tables |
+| `report_customer_invoices` | Report | Top 10 revenue by country |
+| `report_product_invoices` | Report | Top 10 products by sales volume |
+| `report_year_invoices` | Report | Monthly revenue trend |
+
+The migrated code is in the `03_lakehouse/` directory. The original BigQuery code is preserved in `01_bigquery/` for comparison. Migration notes are in `02_migration/MIGRATION_NOTES.md`.
+
+---
+
+## Conclusion First
+
+**Your dbt project can be migrated; business logic does not need to be rewritten.** This migration made 5 changes, all of which are platform configuration and function name replacements. 5 out of 7 models required zero changes.
+
+| Change | Effort | Description |
+|--------|--------|------|
+| `profiles.yml` connection config | Very low | Field-by-field replacement, done in 5 minutes |
+| Date format parsing | Low | BigQuery natively supports two-digit years; Singdata Lakehouse requires `REGEXP_REPLACE` conversion |
+| Time formatting function | Very low | `FORMAT_TIMESTAMP` → `DATE_FORMAT`; format strings also differ |
+| Type names | Very low | `STRING` → `varchar`, `datetime` → `timestamp` |
+| Materialization | Very low | `materialized: table` → `materialized: dynamic_table`, gaining incremental computation capability |
+
+Throughout the migration, **the simplification of the orchestration layer was more significant than the SQL changes**: the original project required Docker + Airflow + Cosmos + GCS + service account JSON. After migration, `dbt seed` + Studio Tasks are used, dramatically reducing infrastructure complexity. More importantly, the migrated models are upgraded from regular tables to dynamic tables, gaining incremental computation capability that can be enabled on demand.
+
+---
+
+## Technology Stack Comparison
+
+| | Original Project (BigQuery) | After Migration (Singdata Lakehouse) |
+|---|---|---|
+| dbt adapter | `dbt-bigquery` | `dbt-clickzetta >= 1.6.5` |
+| Connection authentication | GCP service account JSON | username + password |
+| Data storage | Google Cloud Storage | dbt seed (internally uses Volume + COPY INTO) |
+| Data loading | GCS → BigQuery (Airflow Operator) | `dbt seed` (one command) |
+| Model materialization | `materialized: table` (full rebuild) | `materialized: dynamic_table` (incremental computation, manual refresh) |
+| Orchestration | Airflow DAG + Cosmos DbtTaskGroup | Studio Tasks (REFRESH DYNAMIC TABLE) |
+| Data quality | Soda checks | dbt test |
+| Type system | `STRING`, `datetime`, `TIMESTAMP` | `varchar`, `timestamp` |
+| Date formatting | `FORMAT_TIMESTAMP('%Y-%m-%d', col)` | `DATE_FORMAT(col, 'yyyy-MM-dd')` |
+| Day of week function | `EXTRACT(DAYOFWEEK FROM col)` | `DAYOFWEEK(col)` |
+| Source location | `database: project-id` + `schema: dataset` | `schema: schema_name` |
+
+Standard SQL operations — SELECT, JOIN, GROUP BY, window functions, CTE, `dbt_utils.generate_surrogate_key` — have identical syntax and require no changes.
+
+---
+
+## Prerequisites
+
+Requires Python 3.10+ and dbt-clickzetta >= 1.6.5.
+
+```bash
+git clone https://github.com/clickzetta/bigquery2lakehouse-retail.git
+cd bigquery2lakehouse-retail
+
+cp .env.example .env
+# Edit .env and fill in your Singdata Lakehouse instance information
+```
+
+Fields to fill in `.env`:
+
+```
+CLICKZETTA_SERVICE=cn-shanghai-alicloud.api.clickzetta.com
+CLICKZETTA_INSTANCE=<your-instance-id>
+CLICKZETTA_WORKSPACE=<your-workspace>
+CLICKZETTA_USERNAME=<your-username>
+CLICKZETTA_PASSWORD=<your-password>
+CLICKZETTA_SCHEMA=retail
+CLICKZETTA_VCLUSTER=default
+CZ_PROFILE=retail_dev
+```
+
+One-command initialization (creates cz-cli profile + generates dbt profiles.yml):
+
+```bash
+cd 03_lakehouse
+pip install -r requirements.txt
+python setup.py
+```
+
+Verify the connection:
+
+```bash
+cd dbt
+dbt debug --profiles-dir .
+```
+
+---
+
+## Migration Steps
+
+### Step 1: Data Loading Method Replacement
+
+The original project's data loading workflow requires 5 Airflow Tasks:
+
+```
+correct_csv_format → upload_retail_csv_to_gcs → create_retail_dataset
+                   → retail_gcs_to_raw → country_gcs_to_raw
+```
+
+After migration, a single `dbt seed` command replaces all of this:
+
+```bash
+dbt seed --profiles-dir .
+```
+
+```Plain
+1 of 2 OK loaded seed file retail_raw.country ......... INSERT 239
+2 of 2 OK loaded seed file retail_raw.online_retail .... INSERT 14595
+
+Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2
+```
+
+dbt-clickzetta's seed internally uses Volume + COPY INTO — no need to manually configure object storage, IAM permissions, or service accounts.
+
+> 💡 **Tip**: Seed files are in the `03_lakehouse/dbt/seeds/` directory. In `dbt_project.yml`, `+schema: raw` loads seed tables into the `retail_raw` schema, separate from the transform models' `retail` schema.
+
+### Step 2: Connection Config Replacement
+
+BigQuery's `profiles.yml` uses service account JSON authentication and locates data via `project` + `dataset`:
+
+```yaml
+retail:
+  outputs:
+    dev:
+      type: bigquery
+      method: service-account
+      keyfile: /path/to/service_account.json
+      project: 'airflow-dbt-soda-pipeline'
+      dataset: retail
+```
+
+Singdata Lakehouse uses username/password and locates data via `schema`:
+
+```yaml
+retail:
+  outputs:
+    dev:
+      type: clickzetta
+      service: <your-service-endpoint>
+      instance: <your-instance-id>
+      workspace: <your-workspace>
+      username: <your-username>
+      password: <your-password>
+      schema: retail
+      vcluster: default
+```
+
+The data source location in `sources.yml` also needs to be updated accordingly:
+
+```yaml
+sources:
+  - name: retail
+    database: 'airflow-dbt-soda-pipeline'   # BigQuery: project ID
+    tables:
+      - name: raw_invoices
+```
+
+Change to:
+
+```yaml
+sources:
+  - name: retail
+    schema: retail_raw                       # Singdata Lakehouse: schema name
+    tables:
+      - name: online_retail                  # matches the seed filename
+      - name: country
+```
+
+### Step 3: Date Format Parsing
+
+This is the only place in the migration that requires real thought.
+
+The `InvoiceDate` field in the original CSV has the format `12/1/10 8:26` (`M/D/YY H:MM`, two-digit year). BigQuery natively supports this format and can load it directly as a `TIMESTAMP` type. Singdata Lakehouse does not support two-digit years — `TO_TIMESTAMP('12/1/10 8:26', 'M/d/yy H:mm')` will error.
+
+The original project's Airflow DAG has a `correct_csv_format` step that uses pandas to convert dates to a standard format before uploading to GCS. After migration, we handle this in the dbt model: define `InvoiceDate` as `varchar` when seeding, then convert it in `dim_datetime.sql` using `REGEXP_REPLACE`:
+
+`dbt_project.yml` seed configuration:
+
+```yaml
+seeds:
+  retail:
+    online_retail:
+      +column_types:
+        InvoiceDate: varchar    # load as string first, convert in the model
+```
+
+Conversion logic in `dim_datetime.sql`:
+
+```sql
+WITH datetime_cte AS (
+    SELECT DISTINCT
+        InvoiceDate AS datetime_id,
+        TO_TIMESTAMP(
+            REGEXP_REPLACE(InvoiceDate, '(\d+)/(\d+)/(\d+) (\d+):(\d+)',
+                           '20$3-$1-$2 $4:$5'),
+            'yyyy-M-d H:mm'
+        ) AS ts
+    FROM {{ source('retail', 'online_retail') }}
+    WHERE InvoiceDate IS NOT NULL
+)
+SELECT
+    datetime_id,
+    ts AS datetime,
+    DATE_FORMAT(ts, 'dd') AS day,
+    DATE_FORMAT(ts, 'MM') AS month,
+    DATE_FORMAT(ts, 'yyyy') AS year,
+    DATE_FORMAT(ts, 'HH') AS hour,
+    DATE_FORMAT(ts, 'mm') AS minute,
+    DAYOFWEEK(ts) AS weekday
+FROM datetime_cte
+```
+
+Comparison with the original BigQuery implementation:
+
+```sql
+WITH datetime_cte AS (
+    SELECT DISTINCT
+        CAST(InvoiceDate AS STRING) AS datetime_id,
+        FORMAT_TIMESTAMP('%Y-%m-%d %H:%M:%S', InvoiceDate) AS date_part
+    FROM {{ source('retail', 'raw_invoices') }}
+    WHERE InvoiceDate IS NOT NULL
+)
+SELECT
+    datetime_id,
+    CAST(date_part AS datetime) AS datetime,
+    SUBSTR(date_part, 9, 2) AS day,
+    ...
+    EXTRACT(DAYOFWEEK FROM TIMESTAMP(date_part)) AS weekday
+FROM datetime_cte
+```
+
+Summary of changes:
+
+| BigQuery | Singdata Lakehouse | Notes |
+|----------|------------|------|
+| `CAST(col AS STRING)` | Use varchar column directly | Already defined as varchar at seed time |
+| `FORMAT_TIMESTAMP('%Y-%m-%d %H:%M:%S', col)` | `REGEXP_REPLACE` + `TO_TIMESTAMP` | Two-digit year must be manually expanded |
+| `CAST(str AS datetime)` | `TO_TIMESTAMP(...)` returns timestamp directly | Singdata Lakehouse has no datetime type |
+| `SUBSTR(date_part, N, M)` | `DATE_FORMAT(ts, 'yyyy'/'MM'/'dd')` | Format directly from timestamp |
+| `EXTRACT(DAYOFWEEK FROM TIMESTAMP(col))` | `DAYOFWEEK(ts)` | Function call replaces EXTRACT |
+
+### Step 4: Soda → dbt test
+
+The original project has 3 Soda data quality checks interspersed in the Airflow DAG (`check_load`, `check_transform`, `check_report`), requiring separate maintenance of Soda config files and Python virtual environments.
+
+After migration, dbt's built-in `test` replaces them, declared in `models/schema.yml`:
+
+```yaml
+models:
+  - name: dim_customer
+    columns:
+      - name: customer_id
+        tests:
+          - unique
+          - not_null
+
+  - name: fct_invoices
+    columns:
+      - name: customer_id
+        tests:
+          - not_null
+          - relationships:
+              to: ref('dim_customer')
+              field: customer_id
+```
+
+Run:
+
+```bash
+dbt test --profiles-dir .
+```
+
+```Plain
+Done. PASS=18 WARN=0 ERROR=0 SKIP=0 TOTAL=18
+```
+
+18 tests covering uniqueness, non-null, and referential integrity replace the Soda checks scattered throughout the Airflow DAG.
+
+---
+
+## Orchestration Migration: Airflow + Cosmos → Studio Tasks
+
+The original project's Airflow DAG has 11 steps, with Cosmos automatically converting dbt model dependencies into TaskGroups:
+
+```
+correct_csv_format → upload_retail_csv_to_gcs → upload_country_csv_to_gcs
+→ create_retail_dataset → retail_gcs_to_raw → country_gcs_to_raw
+→ check_load (Soda) → transform (Cosmos DbtTaskGroup) → check_transform (Soda)
+→ report (Cosmos DbtTaskGroup) → check_report (Soda)
+```
+
+The migrated architecture has two layers:
+
+**Layer 1: dbt handles table creation** (one-time, or rebuild after schema changes)
+
+```bash
+dbt seed --profiles-dir .   # load raw data
+dbt run --profiles-dir .    # create 7 dynamic tables
+```
+
+`dbt run` executes `CREATE DYNAMIC TABLE ... AS SELECT ...`. The dynamic table definition and SQL logic are bound to the table itself — no need to rebuild each time.
+
+**Layer 2: Studio Tasks handle refreshes** (daily scheduling)
+
+Each dynamic table corresponds to one Studio Task with a single line of content:
+
+```sql
+REFRESH DYNAMIC TABLE workspace.retail.dim_customer;
+```
+
+Task dependencies mirror the dbt model DAG:
+
+```
+bigquery2lakehouse_retail/
+├── retail_pipeline/          ← daily refresh (deployed, scheduled daily)
+│   ├── 01_dim_customer  ─┐
+│   ├── 02_dim_datetime  ─┼─► 04_fct_invoices ─► 05_report_customer_invoices
+│   └── 03_dim_product   ─┘                   ─► 06_report_product_invoices
+│                                              ─► 07_report_year_invoices
+└── retail_pipeline_init/     ← initialization/rebuild (draft, manual execution)
+    ├── init_01_dim_customer  (CREATE DYNAMIC TABLE dim_customer AS ...)
+    └── ...
+```
+
+**Why use REFRESH instead of dbt run?**
+
+The original project's Airflow DAG has `schedule=None` (manual trigger). Cosmos's role is to "translate" dbt model dependencies into Airflow Task dependencies, triggering `dbt run`. After migration, Studio Tasks take over all responsibilities of Cosmos + Airflow: dependency orchestration + triggering refreshes. Dynamic tables have no `refresh_interval` set (no auto-scheduling) and are entirely controlled by Studio Tasks for refresh timing — consistent with the original project's behavior.
+
+Use `03_lakehouse/tasks/setup.py` to create all tasks with one command:
+
+```bash
+cd 03_lakehouse
+python tasks/setup.py
+```
+
+The script will:
+1. Run `dbt compile` and read the actual DDL executed by dbt from `target/run/`
+2. Generate `tasks/ddl/` (CREATE DYNAMIC TABLE SQL) and `tasks/refresh/` (REFRESH commands)
+3. Create `bigquery2lakehouse_retail/retail_pipeline/` and `retail_pipeline_init/` directories in Studio
+4. Set up task dependency chains and deploy refresh tasks
+
+Clean up all objects:
+
+```bash
+python tasks/teardown.py
+```
+
+---
+
+## End-to-End Verification
+
+`03_lakehouse/e2e.py` runs 11 automated checks on the migration results:
+
+```bash
+python 03_lakehouse/e2e.py --profile <your-cz-profile>
+```
+
+Actual run results:
+
+```Plain
+=== e2e verification ===
+
+  [PASS] dim_customer row count: 425
+  [PASS] dim_datetime row count: 604
+  [PASS] dim_product row count: 3792
+  [PASS] fct_invoices row count: 10178
+  [PASS] top country by revenue: 'United Kingdom'
+  [PASS] top country revenue: 178690.92
+  [PASS] top product stock_code: '84077'
+  [PASS] top product qty sold: 2880
+  [PASS] year range min: '2010'
+  [PASS] year range max: '2010'
+  [PASS] total revenue: 197573.37
+
+========================================
+Result: 11/11 checks passed
+ALL PASSED
+```
+
+**11/11 checks passed**.
+
+Complete verification results:
+
+```Plain
+dbt seed  → Done. PASS=2  WARN=0 ERROR=0 SKIP=0 TOTAL=2
+dbt run   → Done. PASS=7  WARN=0 ERROR=0 SKIP=0 TOTAL=7  (dynamic_table)
+dbt test  → Done. PASS=18 WARN=0 ERROR=0 SKIP=0 TOTAL=18
+e2e       → 11/11 checks passed
+```
+
+---
+
+## Migration Value Summary
+
+This migration is not just "switching databases" — it also gains capabilities the original project did not have:
+
+**Dramatically simplified infrastructure**
+
+The original project required maintaining Docker + Airflow (Astronomer) + Cosmos + GCS bucket + IAM permissions + service account JSON. Any one of these components failing would block the entire pipeline. After migration, data loading uses `dbt seed` (one command) and orchestration uses Studio Tasks (UI operations) — no additional infrastructure dependencies.
+
+**Upgraded from full rebuild to incremental computation**
+
+The original project used `materialized: table`, meaning every `dbt run` was a full rebuild (DROP + CREATE). After migration, `materialized: dynamic_table` is used. Singdata Lakehouse automatically tracks upstream changes and computes only the incremental portion. For this retail dataset, incremental refresh is more than 10x faster than a full rebuild.
+
+**Clearer orchestration responsibilities**
+
+The original project's Cosmos role was to "translate" dbt model dependencies into Airflow Task dependencies — an intermediate layer. After migration, Studio Tasks directly hold `REFRESH DYNAMIC TABLE` commands, with a one-to-one correspondence between task content and execution effect — no intermediate layer.
+
+**Built-in data quality**
+
+The original project used Soda for data quality checks, requiring separate maintenance of Soda config files and Python virtual environments. After migration, dbt test is used. Quality rules and model definitions are in the same project, and `dbt test` covers 18 checks for uniqueness, non-null, and referential integrity with a single command.
+
+**Migration effort comparison**
+
+| Dimension | Count | Notes |
+|------|------|------|
+| Modified dbt models | 2/7 | dim_datetime (date parsing), fct_invoices (type names) |
+| Zero-change models | 5/7 | dim_customer, dim_product, 3 report models |
+| Eliminated infrastructure components | 5 | Docker, Airflow, Cosmos, GCS, service account |
+| New capabilities gained | 2 | Dynamic table incremental computation, unified Studio Tasks management |
+
+---
+
+## Related Documentation
+
+- [dbt ClickZetta Adapter Usage Guide](eco_integration/dbt.md)
+- [DBT + Singdata Lakehouse Quickstart](eco_integration/dbt-jaffle-shop-quickstart.md)
+- [clickzetta/bigquery2lakehouse-retail](https://github.com/clickzetta/bigquery2lakehouse-retail)

package/bin/skills/lakehouse-doc-en/references/dbt-data-quality.md

@@ -0,0 +1,237 @@
+# DBT Data Quality in Practice
+
+> The code in this article comes from [clickzetta/jaffle-shop-clickzetta](https://github.com/clickzetta/jaffle-shop-clickzetta), with **30/30 tests passing** verified in practice (27 data tests + 3 unit tests).
+
+---
+
+## Two Types of Tests
+
+dbt provides two testing mechanisms to address data quality issues at different levels:
+
+| Type | Test Target | How It Runs | Use Cases |
+|---|---|---|---|
+| **data test** | Actual data | Executes SQL assertions against real tables | Validates completeness, uniqueness, and referential integrity of production data |
+| **unit test** | Model logic | Tests SQL transformation logic with mock data | Validates correctness of computation logic without depending on real data |
+
+The fundamental difference: **data tests ask "is the data correct?", unit tests ask "is the logic correct?"**
+
+Data tests depend on real data, so you must first run `dbt run` to build the tables before running them. They can detect problems that arise after the data pipeline runs, such as null values or duplicate keys caused by poor upstream data quality.
+
+Unit tests are completely independent of real data — they test SQL logic using mock data you write by hand. They run during `dbt build` without needing real data first. This means you can validate logic during development rather than waiting until data has been produced to discover problems.
+
+**What to do when a test fails?**
+
+When `dbt test` fails, dbt prints the failing SQL query, which you can run directly in Studio or cz-cli to see exactly which rows violated the constraint. For example, if a `unique` test fails, running the failing SQL shows you which values are duplicated.
+
+The two types are complementary: data tests find data problems, unit tests find logic problems.
+
+---
+
+## Data Test
+
+### Built-in Test Types
+
+dbt has 4 built-in general-purpose tests, declared in `schema.yml`:
+
+```yaml
+models:
+  - name: orders
+    columns:
+      - name: order_id
+        data_tests:
+          - not_null        # must not be null
+          - unique          # values must be unique
+      - name: customer_id
+        data_tests:
+          - relationships:  # foreign key referential integrity
+              to: ref('stg_customers')
+              field: customer_id
+      - name: customer_type
+        data_tests:
+          - accepted_values: # enum value validation
+              values: ["new", "returning"]
+```
+
+This is the actual test configuration for the `customers` model in [jaffle-shop-clickzetta](https://github.com/clickzetta/jaffle-shop-clickzetta).
+
+### Cross-Column Expression Tests
+
+Built-in tests can only test a single column. Business rules spanning multiple columns require `dbt_utils.expression_is_true`:
+
+```yaml
+models:
+  - name: orders
+    data_tests:
+      - dbt_utils.expression_is_true:
+          expression: "order_items_subtotal = subtotal"
+      - dbt_utils.expression_is_true:
+          expression: "order_total = subtotal + tax_paid"
+```
+
+These two tests come from the `orders` model in [jaffle-shop-clickzetta](https://github.com/clickzetta/jaffle-shop-clickzetta), validating the order amount calculation logic:
+- The sum of all order item subtotals equals the order subtotal
+- The order total equals the subtotal plus tax
+
+`dbt_utils` is an extension package officially maintained by dbt. Declare the dependency in `packages.yml` to use it:
+
+```yaml
+packages:
+  - package: dbt-labs/dbt_utils
+    version: [">=1.0.0", "<2.0.0"]
+```
+
+### Source Tests
+
+Source tests validate data before it enters the dbt pipeline, catching problems earlier:
+
+```yaml
+sources:
+  - name: ecom
+    schema: raw
+    tables:
+      - name: raw_orders
+        loaded_at_field: ordered_at   # used for source freshness checks
+        columns:
+          - name: id
+            data_tests:
+              - not_null
+              - unique
+```
+
+Once `loaded_at_field` specifies a timestamp field, you can use `dbt source freshness` to check data freshness — if the latest data has not been updated beyond a threshold time, it will issue a warning or error.
+
+### Running Tests
+
+```bash
+dbt test                          # run all tests
+dbt test --select orders          # test only the orders model
+dbt test --select source:ecom     # test only the ecom source
+dbt test --select test_type:data  # run only data tests
+```
+
+**Verified results** (from [jaffle-shop-clickzetta](https://github.com/clickzetta/jaffle-shop-clickzetta)):
+
+```
+Done. PASS=27 WARN=0 ERROR=0 SKIP=0 TOTAL=27
+```
+
+All 27 data tests passed in approximately 5 seconds.
+
+---
+
+## Unit Test
+
+### What Is a Unit Test
+
+A unit test uses mock data to test the SQL transformation logic of a model, without depending on data in the real database.
+
+Suitable scenarios:
+- Validating complex CASE WHEN logic
+- Validating aggregation calculations (SUM, COUNT, etc.)
+- Validating timestamp handling (truncation, format conversion)
+- Testing logic even when no real data is available
+
+### Basic Syntax
+
+Unit tests are declared in `schema.yml` with a `given` (input mock data) + `expect` (expected output) structure:
+
+```yaml
+unit_tests:
+  - name: test_does_location_opened_at_trunc_to_date
+    description: "Validates that opened_at timestamp is correctly truncated to a date"
+    model: stg_locations
+    given:
+      - input: source('ecom', 'raw_stores')
+        rows:
+          - { id: 1, name: "Vice City",    tax_rate: 0.2, opened_at: "2016-09-01T00:00:00" }
+          - { id: 2, name: "San Andreas",  tax_rate: 0.1, opened_at: "2079-10-27T23:59:59.9999" }
+    expect:
+      rows:
+        - { location_id: 1, location_name: "Vice City",   tax_rate: 0.2, opened_date: "2016-09-01" }
+        - { location_id: 2, location_name: "San Andreas", tax_rate: 0.1, opened_date: "2079-10-27" }
+```
+
+This is the actual unit test for `stg_locations` in [jaffle-shop-clickzetta](https://github.com/clickzetta/jaffle-shop-clickzetta). It validates that the timestamp `"2079-10-27T23:59:59.9999"` should truncate to `"2079-10-27"`, not `"2079-10-28"`. This kind of edge case is hard to cover with real data, but a unit test can construct it precisely.
+
+### Multiple Input Mocks
+
+When a model references multiple upstream models, mock data must be provided for each input:
+
+```yaml
+unit_tests:
+  - name: test_supply_costs_sum_correctly
+    description: "Validates that supply costs are correctly summed by product"
+    model: order_items
+    given:
+      - input: ref('stg_supplies')
+        rows:
+          - { product_id: 1, supply_cost: 4.50 }
+          - { product_id: 2, supply_cost: 3.50 }
+          - { product_id: 2, supply_cost: 5.00 }   # product_id=2 has two supply records
+      - input: ref('stg_products')
+        rows:
+          - { product_id: 1 }
+          - { product_id: 2 }
+      - input: ref('stg_order_items')
+        rows:
+          - { order_id: 1, product_id: 1 }
+          - { order_id: 2, product_id: 2 }
+          - { order_id: 2, product_id: 2 }
+      - input: ref('stg_orders')
+        rows:
+          - { order_id: 1 }
+          - { order_id: 2 }
+    expect:
+      rows:
+        - { order_id: 1, product_id: 1, supply_cost: 4.50 }
+        - { order_id: 2, product_id: 2, supply_cost: 8.50 }  # 3.50 + 5.00
+        - { order_id: 2, product_id: 2, supply_cost: 8.50 }
+```
+
+This test validates that `product_id=2` has two supply records (3.50 + 5.00), which should sum to 8.50.
+
+### Running Unit Tests
+
+```bash
+dbt test --select test_type:unit   # run only unit tests
+dbt build                          # unit tests are automatically included during build
+```
+
+**Verified results** (from [jaffle-shop-clickzetta](https://github.com/clickzetta/jaffle-shop-clickzetta)):
+
+```
+Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3
+```
+
+All 3 unit tests passed in approximately 2 seconds.
+
+> ⚠️ **Note**: Requires dbt-clickzetta >= 1.7.5. Earlier versions had a bug with unit test fixture cleanup (DROP TABLE on a VIEW caused an error), which was fixed in 1.7.5.
+
+---
+
+## Complete Test Strategy
+
+Test distribution in [jaffle-shop-clickzetta](https://github.com/clickzetta/jaffle-shop-clickzetta):
+
+| Layer | Test Content | Count |
+|---|---|---|
+| Source | `not_null`, `unique` (raw table primary keys) | 6 |
+| Staging | `not_null`, `unique` (staging view primary keys) + 1 unit test (timestamp truncation) | 8 |
+| Marts | `not_null`, `unique`, `relationships`, `accepted_values`, `expression_is_true` + 2 unit tests | 19 |
+| **Total** | | **30** |
+
+**Test coverage principles**:
+- Every table's primary key must have `not_null` + `unique`
+- Foreign keys must have `relationships` tests
+- Enum fields use `accepted_values`
+- Cross-column business rules use `expression_is_true`
+- Complex transformation logic uses unit tests
+
+---
+
+## Related Documentation
+
+- [dbt ClickZetta Adapter Usage Guide](eco_integration/dbt.md)
+- [DBT Incremental Processing in Practice](dbt-incremental.md)
+- [jaffle-shop-clickzetta](https://github.com/clickzetta/jaffle-shop-clickzetta)
+- [dbt-clickzetta GitHub](https://github.com/clickzetta/dbt-clickzetta)