gcp-pipeline-ref-transform 1.0.7__tar.gz

gcp_pipeline_ref_transform-1.0.7/PKG-INFO

@@ -0,0 +1,206 @@
+Metadata-Version: 2.4
+Name: gcp-pipeline-ref-transform
+Version: 1.0.7
+Summary: GCP Pipeline Reference: Generic FDP Transformation — dbt models for ODP to FDP (JOIN + MAP patterns)
+License: MIT
+Project-URL: Homepage, https://github.com/enrichmeai/gcp-pipeline-reference
+Project-URL: Documentation, https://github.com/enrichmeai/gcp-pipeline-reference/tree/main/deployments/bigquery-to-mapped-product
+Project-URL: Repository, https://github.com/enrichmeai/gcp-pipeline-reference
+Keywords: gcp,bigquery,dbt,transformation,reference,example
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: dbt-bigquery>=1.5.0
+Requires-Dist: gcp-pipeline-framework>=1.0.6
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+
+# Generic Transformation
+
+**Unit 2 of Generic 3-Unit Deployment**
+
+FDP Transformation - dbt models for ODP → FDP transformation.
+
+---
+
+## Flow Diagram
+
+```
+                         Generic TRANSFORMATION FLOW
+                         ──────────────────────
+
+  BigQuery ODP                    dbt                      BigQuery FDP
+  ────────────                    ───                      ────────────
+
+  odp_generic.customers ─────┐
+                        │    ┌─────────────────┐
+  odp_generic.accounts  ─────┼───►│   JOIN Logic    │────────► fdp_generic.event_transaction_excess
+                        │    └─────────────────┘
+                        │
+  odp_generic.decision  ─────┼───────────────────────────────► fdp_generic.portfolio_account_excess
+                        │
+                        └───────────────────────
+```
+
+---
+
+## Pattern
+
+**MULTI-TARGET**:
+1. **JOIN**: 2 ODP sources (customers, accounts) → 1 FDP target (`event_transaction_excess`)
+2. **MAP**: 1 ODP source (decision) → 1 FDP target (`portfolio_account_excess`)
+
+| Step | Description |
+|------|-------------|
+| 1 | Staging models clean and type-cast raw ODP data |
+| 2 | `add_audit_columns` macro injects `run_id` and `source_file` |
+| 3 | `mask_pii` macro applies environment-aware masking to sensitive fields |
+| 4 | `event_transaction_excess` performs `INNER JOIN` between Customers and Accounts |
+| 5 | `portfolio_account_excess` maps Decision ODP 1:1 to FDP |
+
+---
+
+## Data Mapping
+
+| Source Table | Key Fields |
+|--------------|------------|
+| `odp_generic.customers` | customer_id, ssn, first_name, last_name, dob, status |
+| `odp_generic.accounts` | account_id, customer_id, account_type, balance, open_date |
+| `odp_generic.decision` | decision_id, customer_id, application_id, decision_code, score, decision_date |
+
+| Target Table | Description |
+|--------------|-------------|
+| `fdp_generic.event_transaction_excess` | Joined customer-account view |
+| `fdp_generic.portfolio_account_excess` | Decision-based portfolio view |
+
+---
+
+## Components
+
+| Directory | Purpose |
+|-----------|---------|
+| `dbt/models/staging/generic/` | Staging models (clean raw data) |
+| `dbt/models/fdp/` | FDP models (JOIN and MAP logic) |
+
+---
+
+## Library-Driven Ease of Use
+
+The Generic transformation unit uses the `gcp-pipeline-transform` library to ensure data privacy and lineage with zero local macro development:
+
+1.  **Zero-Bleed PII Masking**: Uses `{{ mask_pii(column, 'SSN') }}`. The library automatically applies the correct mask (Full in Prod, Partial in Staging) based on the environment.
+2.  **Automated Lineage**: Uses `{{ add_audit_columns() }}` to inject `run_id` and `source_file` variables, maintaining the E2E lineage established in the ingestion layer.
+3.  **Metadata Enrichment**: Replaces hardcoded business logic with generic library macros that interpret rules from the `EntitySchema`.
+
+---
+
+## How to Replicate this JOIN Transformation (3-to-1)
+
+To create a new transformation unit that joins multiple entities, follow the [Creating New Deployment Guide](../../docs/CREATING_NEW_DEPLOYMENT_GUIDE.md).
+
+Key steps for this JOIN pattern:
+1.  **Register Library**: Point your `dbt_project.yml` to the `gcp-pipeline-transform` macro paths.
+2.  **Staging Models**: Create views for your ODP tables. Use `add_audit_columns` for consistency.
+3.  **FDP Models**: Implement your `LEFT JOIN` logic. Apply `mask_pii` to all sensitive fields.
+4.  **Governance**: Run `validate_no_pii_in_export` in your CI/CD to prevent leakage.
+
+---
+
+## Infrastructure & Configurations
+
+### Google Cloud Resources
+This deployment requires the following GCP infrastructure, provisioned via Terraform:
+- **Data Warehouse**: BigQuery datasets `odp_generic` (source) and `fdp_generic` (target).
+- **Processing**: dbt (running on Cloud Composer or as a standalone process) for executing transformations.
+
+For detailed infrastructure definitions, see [infrastructure/terraform/systems/generic/transformation/](../../infrastructure/terraform/systems/generic/transformation/).
+
+### dbt Configuration (`dbt_project.yml`)
+The transformation behavior is controlled by variables and configurations in `dbt_project.yml`:
+
+| Variable | Description | Default / Source |
+|----------|-------------|------------------|
+| `gcp_project_id` | Target GCP Project | `GCP_PROJECT_ID` env var |
+| `source_dataset` | Source ODP dataset | `odp_generic` |
+| `staging_dataset` | Intermediate staging dataset | `stg_generic` |
+| `fdp_dataset` | Target FDP dataset | `fdp_generic` |
+| `marts_dataset` | Marts dataset | `marts_generic` |
+| `analytics_dataset` | Analytics dataset | `analytics_generic` |
+| `extract_date` | Date of data extract | `null` (optional filter) |
+| `generic_entities` | List of entities to process | `['customers', 'accounts', 'decision']` |
+| `masking_level` | PII masking strategy (`FULL`, `PARTIAL`, `NONE`) | `AUTO` |
+
+### Technology Stack & Documentation
+- [Google BigQuery](https://cloud.google.com/bigquery/docs) - Serverless data warehouse
+- [dbt (data build tool)](https://docs.getdbt.com/docs/introduction) - Transformation workflow
+- [dbt-bigquery Adapter](https://docs.getdbt.com/reference/warehouse-setups/bigquery-setup) - dbt to BigQuery connector
+- [Data Modeling in dbt](https://docs.getdbt.com/docs/build/models) - Best practices for models
+
+---
+
+## Dependencies
+
+| Library | Purpose |
+|---------|---------|
+| `dbt-bigquery` | dbt adapter for BigQuery |
+| `gcp-pipeline-transform` | Shared macros (audit columns) |
+
+---
+
+## Execution & Testing
+
+### 1. Local Development Setup
+Initialize the virtual environment:
+```bash
+./scripts/setup_deployment_venv.sh bigquery-to-mapped-product
+source deployments/bigquery-to-mapped-product/venv/bin/activate
+```
+
+### 2. Local dbt Execution
+Run dbt models locally against the development BigQuery dataset:
+```bash
+cd dbt
+dbt run --profiles-dir . --target dev
+```
+
+### 3. Data Quality Validation
+Run dbt tests to verify transformation logic and PII masking:
+```bash
+dbt test --profiles-dir . --target dev
+```
+
+### 4. Governance Verification
+Use the library macro to ensure no unmasked PII exists in your models before deployment:
+```sql
+{{ validate_no_pii_in_export('fdp_generic.event_transaction_excess') }}
+```
+
+### 5. Cloud Execution
+In production, this unit is triggered by the `generic_odp_load_dag` once ingestion is successful. The transformation is executed via a `BashOperator` running `dbt run`.
+
+---
+
+## SQL Example
+
+```sql
+-- fdp_generic.event_transaction_excess
+SELECT
+    c.customer_id,
+    c.first_name,
+    c.last_name,
+    a.account_id,
+    a.current_balance,
+    -- Audit columns
+    c._run_id,
+    CURRENT_TIMESTAMP() as _transformed_at
+FROM {{ ref('stg_generic_customers') }} c
+JOIN {{ ref('stg_generic_accounts') }} a 
+    ON c.customer_id = a.customer_id
+```
+

gcp_pipeline_ref_transform-1.0.7/README.md

@@ -0,0 +1,183 @@
+# Generic Transformation
+
+**Unit 2 of Generic 3-Unit Deployment**
+
+FDP Transformation - dbt models for ODP → FDP transformation.
+
+---
+
+## Flow Diagram
+
+```
+                         Generic TRANSFORMATION FLOW
+                         ──────────────────────
+
+  BigQuery ODP                    dbt                      BigQuery FDP
+  ────────────                    ───                      ────────────
+
+  odp_generic.customers ─────┐
+                        │    ┌─────────────────┐
+  odp_generic.accounts  ─────┼───►│   JOIN Logic    │────────► fdp_generic.event_transaction_excess
+                        │    └─────────────────┘
+                        │
+  odp_generic.decision  ─────┼───────────────────────────────► fdp_generic.portfolio_account_excess
+                        │
+                        └───────────────────────
+```
+
+---
+
+## Pattern
+
+**MULTI-TARGET**:
+1. **JOIN**: 2 ODP sources (customers, accounts) → 1 FDP target (`event_transaction_excess`)
+2. **MAP**: 1 ODP source (decision) → 1 FDP target (`portfolio_account_excess`)
+
+| Step | Description |
+|------|-------------|
+| 1 | Staging models clean and type-cast raw ODP data |
+| 2 | `add_audit_columns` macro injects `run_id` and `source_file` |
+| 3 | `mask_pii` macro applies environment-aware masking to sensitive fields |
+| 4 | `event_transaction_excess` performs `INNER JOIN` between Customers and Accounts |
+| 5 | `portfolio_account_excess` maps Decision ODP 1:1 to FDP |
+
+---
+
+## Data Mapping
+
+| Source Table | Key Fields |
+|--------------|------------|
+| `odp_generic.customers` | customer_id, ssn, first_name, last_name, dob, status |
+| `odp_generic.accounts` | account_id, customer_id, account_type, balance, open_date |
+| `odp_generic.decision` | decision_id, customer_id, application_id, decision_code, score, decision_date |
+
+| Target Table | Description |
+|--------------|-------------|
+| `fdp_generic.event_transaction_excess` | Joined customer-account view |
+| `fdp_generic.portfolio_account_excess` | Decision-based portfolio view |
+
+---
+
+## Components
+
+| Directory | Purpose |
+|-----------|---------|
+| `dbt/models/staging/generic/` | Staging models (clean raw data) |
+| `dbt/models/fdp/` | FDP models (JOIN and MAP logic) |
+
+---
+
+## Library-Driven Ease of Use
+
+The Generic transformation unit uses the `gcp-pipeline-transform` library to ensure data privacy and lineage with zero local macro development:
+
+1.  **Zero-Bleed PII Masking**: Uses `{{ mask_pii(column, 'SSN') }}`. The library automatically applies the correct mask (Full in Prod, Partial in Staging) based on the environment.
+2.  **Automated Lineage**: Uses `{{ add_audit_columns() }}` to inject `run_id` and `source_file` variables, maintaining the E2E lineage established in the ingestion layer.
+3.  **Metadata Enrichment**: Replaces hardcoded business logic with generic library macros that interpret rules from the `EntitySchema`.
+
+---
+
+## How to Replicate this JOIN Transformation (3-to-1)
+
+To create a new transformation unit that joins multiple entities, follow the [Creating New Deployment Guide](../../docs/CREATING_NEW_DEPLOYMENT_GUIDE.md).
+
+Key steps for this JOIN pattern:
+1.  **Register Library**: Point your `dbt_project.yml` to the `gcp-pipeline-transform` macro paths.
+2.  **Staging Models**: Create views for your ODP tables. Use `add_audit_columns` for consistency.
+3.  **FDP Models**: Implement your `LEFT JOIN` logic. Apply `mask_pii` to all sensitive fields.
+4.  **Governance**: Run `validate_no_pii_in_export` in your CI/CD to prevent leakage.
+
+---
+
+## Infrastructure & Configurations
+
+### Google Cloud Resources
+This deployment requires the following GCP infrastructure, provisioned via Terraform:
+- **Data Warehouse**: BigQuery datasets `odp_generic` (source) and `fdp_generic` (target).
+- **Processing**: dbt (running on Cloud Composer or as a standalone process) for executing transformations.
+
+For detailed infrastructure definitions, see [infrastructure/terraform/systems/generic/transformation/](../../infrastructure/terraform/systems/generic/transformation/).
+
+### dbt Configuration (`dbt_project.yml`)
+The transformation behavior is controlled by variables and configurations in `dbt_project.yml`:
+
+| Variable | Description | Default / Source |
+|----------|-------------|------------------|
+| `gcp_project_id` | Target GCP Project | `GCP_PROJECT_ID` env var |
+| `source_dataset` | Source ODP dataset | `odp_generic` |
+| `staging_dataset` | Intermediate staging dataset | `stg_generic` |
+| `fdp_dataset` | Target FDP dataset | `fdp_generic` |
+| `marts_dataset` | Marts dataset | `marts_generic` |
+| `analytics_dataset` | Analytics dataset | `analytics_generic` |
+| `extract_date` | Date of data extract | `null` (optional filter) |
+| `generic_entities` | List of entities to process | `['customers', 'accounts', 'decision']` |
+| `masking_level` | PII masking strategy (`FULL`, `PARTIAL`, `NONE`) | `AUTO` |
+
+### Technology Stack & Documentation
+- [Google BigQuery](https://cloud.google.com/bigquery/docs) - Serverless data warehouse
+- [dbt (data build tool)](https://docs.getdbt.com/docs/introduction) - Transformation workflow
+- [dbt-bigquery Adapter](https://docs.getdbt.com/reference/warehouse-setups/bigquery-setup) - dbt to BigQuery connector
+- [Data Modeling in dbt](https://docs.getdbt.com/docs/build/models) - Best practices for models
+
+---
+
+## Dependencies
+
+| Library | Purpose |
+|---------|---------|
+| `dbt-bigquery` | dbt adapter for BigQuery |
+| `gcp-pipeline-transform` | Shared macros (audit columns) |
+
+---
+
+## Execution & Testing
+
+### 1. Local Development Setup
+Initialize the virtual environment:
+```bash
+./scripts/setup_deployment_venv.sh bigquery-to-mapped-product
+source deployments/bigquery-to-mapped-product/venv/bin/activate
+```
+
+### 2. Local dbt Execution
+Run dbt models locally against the development BigQuery dataset:
+```bash
+cd dbt
+dbt run --profiles-dir . --target dev
+```
+
+### 3. Data Quality Validation
+Run dbt tests to verify transformation logic and PII masking:
+```bash
+dbt test --profiles-dir . --target dev
+```
+
+### 4. Governance Verification
+Use the library macro to ensure no unmasked PII exists in your models before deployment:
+```sql
+{{ validate_no_pii_in_export('fdp_generic.event_transaction_excess') }}
+```
+
+### 5. Cloud Execution
+In production, this unit is triggered by the `generic_odp_load_dag` once ingestion is successful. The transformation is executed via a `BashOperator` running `dbt run`.
+
+---
+
+## SQL Example
+
+```sql
+-- fdp_generic.event_transaction_excess
+SELECT
+    c.customer_id,
+    c.first_name,
+    c.last_name,
+    a.account_id,
+    a.current_balance,
+    -- Audit columns
+    c._run_id,
+    CURRENT_TIMESTAMP() as _transformed_at
+FROM {{ ref('stg_generic_customers') }} c
+JOIN {{ ref('stg_generic_accounts') }} a 
+    ON c.customer_id = a.customer_id
+```
+

gcp_pipeline_ref_transform-1.0.7/dbt/dbt_project.yml

@@ -0,0 +1,57 @@
+name: 'bigquery_mapped_product'
+version: '1.0.0'
+config-version: 2
+
+profile: 'mapped_product_profile'
+
+model-paths: ["models"]
+analysis-paths: ["analyses"]
+test-paths: ["tests"]
+seed-paths: ["seeds"]
+macro-paths: ["macros", "../libs/gcp_pipeline_transform/dbt_shared/macros"]
+snapshot-paths: ["snapshots"]
+
+target-path: "target"
+clean-targets:
+  - "target"
+  - "dbt_packages"
+
+# Documentation
+docs-paths: ["docs"]
+
+models:
+  bigquery_mapped_product:
+    staging:
+      +materialized: view
+      +schema: staging
+      +tags: ["staging"]
+    fdp:
+      +materialized: incremental
+      +schema: fdp
+      +tags: ["fdp"]
+    marts:
+      +materialized: table
+      +schema: marts
+      +tags: ["marts"]
+    analytics:
+      +materialized: table
+      +schema: analytics
+      +tags: ["analytics"]
+
+vars:
+  # GCP project
+  gcp_project_id: "{{ env_var('GCP_PROJECT_ID') }}"
+
+  # Consolidated datasets
+  source_dataset: "odp_generic"
+  staging_dataset: "stg_generic"
+  fdp_dataset: "fdp_generic"
+  marts_dataset: "marts_generic"
+  analytics_dataset: "analytics_generic"
+
+  # Date parameters
+  extract_date: null
+  current_date: "CURRENT_DATE()"
+
+  # Consolidated entities
+  all_entities: ['customers', 'accounts', 'decision', 'applications']

gcp_pipeline_ref_transform-1.0.7/dbt/macros/data_quality_check.sql

@@ -0,0 +1,17 @@
+-- Generic dbt Macro: Data Quality Checks
+-- Validates data quality metrics (completeness, uniqueness, patterns)
+-- Thresholds configurable via dbt variables
+
+{% macro check_data_quality(table_name) %}
+  select
+    '{{ table_name }}' as table_name,
+    count(*) as row_count,
+    count(case when id is not null then 1 end) / count(*) * 100 as completeness_pct,
+    count(distinct id) / count(*) * 100 as uniqueness_pct,
+    max(created_at) as last_updated
+  from {{ table_name }}
+  having
+    count(case when id is not null then 1 end) / count(*) * 100 >= {{ var('quality_completeness_threshold', 95) }}
+    and count(distinct id) / count(*) * 100 >= {{ var('quality_uniqueness_threshold', 100) }}
+{% endmacro %}
+

gcp_pipeline_ref_transform-1.0.7/dbt/macros/incremental_strategy.sql

@@ -0,0 +1,120 @@
+-- Generic Blueprint - dbt Macro: Incremental Strategy
+--
+-- Provides different strategies for incremental loads:
+-- - MERGE: Update existing, insert new (recommended)
+-- - APPEND_ONLY: Always append (simple but slower)
+-- - DELETE_INSERT: Delete+insert partition (good for daily batches)
+--
+-- Usage in dbt model:
+--   {{ config(
+--       materialized = 'incremental',
+--       on_schema_change = 'fail',
+--       incremental_strategy = 'merge',
+--       unique_key = 'id'
+--   ) }}
+--
+
+{% macro build_merge_statement(target_table, source_table, unique_key, update_columns) %}
+    MERGE INTO {{ target_table }} T
+    USING {{ source_table }} S
+    ON T.{{ unique_key }} = S.{{ unique_key }}
+
+    -- Update existing records
+    WHEN MATCHED THEN
+        UPDATE SET
+            {% for column in update_columns %}
+                T.{{ column }} = S.{{ column }}
+                {%- if not loop.last %}, {% endif %}
+            {% endfor %}
+
+    -- Insert new records
+    WHEN NOT MATCHED THEN
+        INSERT (*)
+        VALUES (*)
+{% endmacro %}
+
+
+{% macro build_append_strategy() %}
+    -- Simple append: just SELECT all new records
+    -- This is the least efficient but simplest approach
+    -- Use when:
+    --   - Data never changes (immutable)
+    --   - Speed is not critical
+    --   - Simplicity is preferred
+
+    SELECT *
+    FROM {{ source }}
+    WHERE run_id = '{{ var("run_id") }}'
+{% endmacro %}
+
+
+{% macro build_delete_insert_strategy(unique_key, date_column) %}
+    -- Delete+insert for current partition, then insert new records
+    -- Use when:
+    --   - Processing daily batches
+    --   - Current day's data might be updated
+    --   - Historical data is immutable
+
+    DELETE FROM {{ this }}
+    WHERE DATE({{ date_column }}) = '{{ run_date }}';
+
+    INSERT INTO {{ this }}
+    SELECT * FROM {{ source }}
+    WHERE run_id = '{{ var("run_id") }}'
+{% endmacro %}
+
+
+-- Implementation of merge strategy for incremental loads
+{% if execute %}
+    {% if flags.FULL_REFRESH %}
+        -- Full refresh: truncate and reload
+        TRUNCATE TABLE {{ this }};
+    {% elif var("incremental_strategy") == "merge" %}
+        -- Use merge for incremental loads
+        {{ build_merge_statement(this, source_data, var("unique_key"), var("update_columns")) }}
+    {% elif var("incremental_strategy") == "delete_insert" %}
+        -- Use delete+insert for daily batches
+        {{ build_delete_insert_strategy(var("unique_key"), var("date_column")) }}
+    {% else %}
+        -- Default: append only
+        INSERT INTO {{ this }}
+        {{ build_append_strategy() }}
+    {% endif %}
+{% endif %}
+
+
+-- Macro to handle late-arriving data
+{% macro handle_late_arriving_data(table, days_lookback) %}
+    {% set sql %}
+        CREATE OR REPLACE TEMP TABLE late_arrivals AS
+        SELECT *
+        FROM {{ table }}
+        WHERE DATE(processed_timestamp) >= DATE_SUB(CURRENT_DATE(), INTERVAL {{ days_lookback }} DAY)
+          AND run_id != '{{ var("run_id") }}';
+
+        MERGE INTO {{ table }} T
+        USING late_arrivals S
+        ON T.id = S.id
+        WHEN MATCHED THEN
+            UPDATE SET T.* = S.*
+        WHEN NOT MATCHED THEN
+            INSERT *;
+    {% endset %}
+
+    {% do run_query(sql) %}
+{% endmacro %}
+
+
+-- Macro for SCD Type 2 (Slowly Changing Dimension)
+{% macro scd2_logic(business_key, change_columns) %}
+    SELECT
+        {% for col in change_columns %}
+            {{ col }},
+        {% endfor %}
+        CURRENT_TIMESTAMP() as valid_from,
+        NULL as valid_to,
+        TRUE as is_current,
+        ROW_NUMBER() OVER (PARTITION BY {{ business_key }} ORDER BY {{ change_columns | join(", ") }}) as version
+    FROM {{ source }}
+{% endmacro %}
+

gcp_pipeline_ref_transform-1.0.7/dbt/models/fdp/_fdp_generic_models.yml

@@ -0,0 +1,93 @@
+version: 2
+
+models:
+  - name: event_transaction_excess
+    description: |
+      Generic FDP: Event Transaction Excess
+
+      JOIN of 2 ODP sources (customers, accounts) into a single
+      table for transaction-level analysis.
+
+    config:
+      tags: ["fdp", "generic", "join"]
+      materialized: table
+      partition_by:
+        field: _extract_date
+        data_type: date
+      cluster_by: [customer_id, account_id]
+
+    columns:
+      - name: event_key
+        tests: [not_null, unique]
+      - name: customer_id
+        tests: [not_null]
+      - name: account_id
+        tests: [not_null]
+      - name: ssn_masked
+      - name: first_name
+      - name: last_name
+      - name: current_balance
+      - name: account_type_desc
+      - name: _run_id
+        tests: [not_null]
+      - name: _extract_date
+        tests: [not_null]
+      - name: _transformed_ts
+
+  - name: portfolio_account_excess
+    description: |
+      Generic FDP: Portfolio Account Excess
+
+      MAP of 1 ODP source (decision) into FDP target for portfolio analysis.
+
+    config:
+      tags: ["fdp", "generic", "map"]
+      materialized: table
+      partition_by:
+        field: _extract_date
+        data_type: date
+      cluster_by: [customer_id, _run_id]
+
+    columns:
+      - name: portfolio_key
+        tests: [not_null, unique]
+      - name: decision_id
+        tests: [not_null]
+      - name: customer_id
+        tests: [not_null]
+      - name: decision_outcome
+      - name: score
+      - name: _run_id
+        tests: [not_null]
+      - name: _extract_date
+        tests: [not_null]
+      - name: _transformed_ts
+
+  - name: portfolio_account_facility
+    description: |
+      Generic FDP: Portfolio Account Facility
+
+      MAP of 1 ODP source (applications) into FDP target for facility analysis.
+
+    config:
+      tags: ["fdp", "generic", "map"]
+      materialized: table
+      partition_by:
+        field: _extract_date
+        data_type: date
+      cluster_by: [application_id, customer_id]
+
+    columns:
+      - name: facility_key
+        tests: [not_null, unique]
+      - name: application_id
+        tests: [not_null]
+      - name: customer_id
+      - name: loan_amount
+      - name: application_date
+      - name: application_status
+      - name: _run_id
+        tests: [not_null]
+      - name: _extract_date
+        tests: [not_null]
+      - name: _transformed_at