ga4-export-fixer 0.9.0-dev.1 → 0.9.0-dev.2

package/README.md

@@ -334,6 +334,7 @@ All fields are optional except `sourceTable`. Default values are applied automat
 | `preOperations`        | object                  | [See details](#preOperations)       | Date range and incremental refresh configuration                                                                                                                                                                                                                                                                             |
 | `eventParamsToColumns` | object[]                | `[]`                                | Event parameters to promote to columns. [See item schema](#eventParamsToColumns)                                                                                                                                                                                                                                             |
 | `customSteps`          | object[]                | `[]`                                | User-defined CTEs appended to the pipeline after `enhanced_events`. [See Custom CTEs](#custom-ctes)                                                                                                                                                                                                                          |
+| `enrichments`          | object[]                | `[]`                                | Declarative external-data enrichments joined into `enhanced_events`. [See Data Enrichments](#data-enrichments)                                                                                                                                                                                                               |
 
 <a id="default-dataformtableconfig"></a>
 <details>
@@ -473,7 +474,8 @@ itemListAttribution: { lookbackType: 'TIME', lookbackTimeMs: 86400000 }
 | `session_data`           | yes                                   | Session-level aggregations (grouped by `session_id`).                                                                                                |
 | `items_unnested`         | only when `itemListAttribution` is on | Per-event item rows (one row per item per ecommerce event), with attribution window function applied.                                                |
 | `items_rebuilt`          | only when `itemListAttribution` is on | Re-aggregated items with attributed list fields, joined back to events via `_item_row_id`.                                                           |
-| `enhanced_events`        | yes                                   | The package's standard output shape (joined event_data + session_data + items_rebuilt, columns ordered, incremental date filter applied). The natural starting point for most custom CTEs. |
+| `enrich_<name>`          | only when configured via `enrichments` | One CTE per [enrichment](#data-enrichments) entry, providing dim data for joining into `enhanced_events`.                                              |
+| `enhanced_events`        | yes                                   | The package's standard output shape (joined event_data + session_data + items_rebuilt + enrich_*, columns ordered, incremental date filter applied). The natural starting point for most custom CTEs. |
 
 Example custom step using the raw SQL format:
 
@@ -521,6 +523,77 @@ end`,
 
 > **Note:** Built-in assertions assume the package's standard schema. If your custom CTEs rename, drop, or filter rows in ways that break those assumptions, disable the affected assertions explicitly via the `assertions` config option.
 
+<a id="data-enrichments"></a>
+
+**`enrichments`** — declaratively join external dimension data into `enhanced_events` (cohort labels, page metadata, marketing attribution, etc.). Each entry describes one dim source plus the join — the package generates the source CTE, the `LEFT JOIN`, and column descriptions automatically.
+
+For typical use cases this is the right tool; reach for `customSteps` only when you need a transformation that doesn't fit a flat dim join.
+
+**Per-enrichment shape:**
+
+| Field | Type | Required | Description |
+| --- | --- | --- | --- |
+| `name` | string | Yes | Used in the generated `enrich_<name>` CTE name. Unique within `enrichments`. |
+| `level` | `'event'` | No, defaults to `'event'` | Join grain. Currently only `'event'` is supported (item-level enrichments will arrive in a later release). |
+| `source` | Dataform `ref()` / string | Yes | Source dim table. Use `ref()` in Dataform or a backtick-quoted ``` `project.dataset.table` ``` string. |
+| `joinKey` | string / string[] | Yes | Column name(s) on `enhanced_events` to join on. Composite keys (array) compile to `USING(col1, col2, ...)`. |
+| `columns` | string[] | Yes | Source columns to add to the output (excluding `joinKey`). Names matching existing columns REPLACE them. |
+| `dedupe` | boolean | No, defaults to `false` | When `true`, wraps the source CTE in `qualify row_number() over (partition by <joinKey>) = 1` for non-unique-key dim sources. Non-deterministic which row wins; for strict needs, pre-aggregate in source SQL. |
+
+**Replace-or-add semantics.** If an enrichment column name matches an existing column on `enhanced_events` (a column promoted via `eventParamsToColumns`, a package-generated column, or a default GA4 column from the export), the enrichment value REPLACES it. If there is no overlap, the column is added.
+
+**Example** — attach user cohort labels by `user_pseudo_id`:
+
+```javascript
+enrichments: [
+    {
+        name: 'cohorts',
+        level: 'event',
+        source: ctx.ref('user_cohorts'),
+        joinKey: 'user_pseudo_id',
+        columns: ['cohort_label', 'lifecycle_stage'],
+    },
+],
+```
+
+**Example** — composite key (date + user) for daily-varying dim data, with dedupe safety net:
+
+```javascript
+enrichments: [
+    {
+        name: 'segments',
+        level: 'event',
+        source: ctx.ref('daily_user_segments'),
+        joinKey: ['event_date', 'user_pseudo_id'],
+        columns: ['segment'],
+        dedupe: true,
+    },
+],
+```
+
+**Example** — fix a promoted event parameter via enrichment (replacement case):
+
+```javascript
+{
+    eventParamsToColumns: [{ name: 'page_title', type: 'string' }],
+    enrichments: [
+        {
+            name: 'titles',
+            level: 'event',
+            source: ctx.ref('page_title_overrides'),
+            joinKey: 'page_location',
+            columns: ['page_title'],   // overlaps the promoted column → replaces it
+        },
+    ],
+}
+```
+
+> **Note:** Each enrichment generates a CTE named `enrich_<name>` at the top of the pipeline. The `enrich_*` namespace is part of the reserved-names contract — `customSteps` cannot use these names. The active reserved set includes only the names of enrichments actually configured.
+
+> **Note:** Enrichment columns get auto-generated descriptions (`Added by enrichment '<name>' (joined on <joinKey> from <source>).` for new columns; `Replaced by enrichment '<name>' (...). Original: <description>` for replacements). User-supplied `dataformTableConfig.columns` overrides win — the auto-generated description is the default.
+
+> **Note:** `joinKey` and `columns` entries must be plain SQL identifiers — inline aliases like `'id as user_id'` are rejected at validation time. If your dim source uses a different column name, alias it in an upstream Dataform view and point `source` at that view.
+
 <br>
 
 ---

package/documentation.js

@@ -1,223 +1,272 @@
-const constants = require('./constants');
-const { version } = require('./package.json');
-
-/**
- * Composes a multi-section column description string from individual sections.
- * Sections with null/undefined/empty values are omitted.
- * Sections are separated by line breaks for readability.
- *
- * @param {Object} sections - { base, lineage, typicalUse, config }
- * @returns {string} Composed description
- */
-const composeDescription = (sections) => {
-    const parts = [];
-
-    if (sections.base) {
-        parts.push(sections.base);
-    }
-
-    if (sections.lineage) {
-        parts.push(`Lineage: ${sections.lineage}`);
-    }
-
-    if (sections.typicalUse) {
-        parts.push(`Typical use: ${sections.typicalUse}`);
-    }
-
-    if (sections.config) {
-        parts.push(`Config: ${sections.config}`);
-    }
-
-    return parts.join('\n\n');
-};
-
-/**
- * Returns a formatted lineage text string for a column, or null if no lineage data exists.
- *
- * @param {string} columnName - The column name to look up.
- * @param {Object} columnLineage - The lineage data object mapping column names to { source, note }.
- * @returns {string|null} Formatted lineage string, e.g. "Derived -- Concatenation of ..."
- */
-const getLineageText = (columnName, columnLineage) => {
-    const entry = columnLineage[columnName];
-    if (!entry) return null;
-
-    const sourceLabels = {
-        'ga4_export': 'Standard GA4 export field',
-        'ga4_export_modified': 'GA4 export field (modified)',
-        'derived': 'Derived',
-    };
-
-    const label = sourceLabels[entry.source] || entry.source;
-    return entry.note ? `${label} -- ${entry.note}` : label;
-};
-
-/**
- * Builds a map of config-specific notes for columns based on the provided configuration.
- * Extracts the configuration-dependent description suffixes into a { columnName: "note" } map.
- *
- * @param {Object} config - The merged configuration object.
- * @returns {Object} Map of column names to config note strings.
- */
-const buildConfigNotes = (config) => {
-    const notes = {};
-
-    if (!config) return notes;
-
-    const append = (key, text) => {
-        notes[key] = notes[key] ? `${notes[key]}. ${text}` : text;
-    };
-
-    // timezone
-    if (config.timezone) {
-        append('event_datetime', `Timezone: ${config.timezone}`);
-    }
-
-    // customTimestampParam
-    if (config.customTimestampParam) {
-        append('event_datetime', `Custom timestamp parameter: '${config.customTimestampParam}'`);
-        append('event_custom_timestamp', `Source parameter: '${config.customTimestampParam}'`);
-    }
-
-    // data_is_final
-    if (config.dataIsFinal) {
-        const method = config.dataIsFinal.detectionMethod;
-        if (method === 'DAY_THRESHOLD') {
-            append('data_is_final', `Detection method: DAY_THRESHOLD (${config.dataIsFinal.dayThreshold} days)`);
-        } else {
-            append('data_is_final', `Detection method: EXPORT_TYPE`);
-        }
-    }
-
-    // excludedEvents
-    if (config.excludedEvents && config.excludedEvents.length > 0) {
-        append('event_name', `Excluded events: ${config.excludedEvents.join(', ')}`);
-    }
-
-    // excludedEventParams
-    if (config.excludedEventParams && config.excludedEventParams.length > 0) {
-        append('event_params', `Excluded parameters: ${config.excludedEventParams.join(', ')}`);
-    }
-
-    // sessionParams
-    if (config.sessionParams && config.sessionParams.length > 0) {
-        append('session_params', `Configured parameters: ${config.sessionParams.join(', ')}`);
-    }
-
-    // includedExportTypes
-    if (config.includedExportTypes) {
-        const types = Object.entries(config.includedExportTypes)
-            .filter(([, enabled]) => enabled)
-            .map(([type]) => type);
-        if (types.length > 0) {
-            append('export_type', `Included export types: ${types.join(', ')}`);
-        }
-    }
-
-    return notes;
-};
-
-/**
- * Returns a deep copy of the column descriptions, enriched with
- * lineage, typical use, and configuration-specific sections composed into
- * multi-section descriptions.
- *
- * @param {Object} config - The merged configuration object.
- * @param {Object} columnMetadata - Column metadata provided by the table module.
- * @param {Object} columnMetadata.descriptions - Column descriptions (Dataform ITableConfig columns format).
- * @param {Object} columnMetadata.lineage - Column lineage data mapping column names to { source, note }.
- * @param {Object} columnMetadata.typicalUse - Column typical use mapping column names to description strings.
- * @returns {Object} Column descriptions object in Dataform ITableConfig columns format.
- */
-const getColumnDescriptions = (config, columnMetadata) => {
-    const descriptions = JSON.parse(JSON.stringify(columnMetadata.descriptions));
-
-    const configNotes = buildConfigNotes(config);
-
-    // Compose multi-section descriptions for each top-level column
-    for (const key of Object.keys(descriptions)) {
-        const isStruct = typeof descriptions[key] === 'object' && descriptions[key].description;
-        const baseDesc = isStruct ? descriptions[key].description : (typeof descriptions[key] === 'string' ? descriptions[key] : null);
-
-        if (!baseDesc) continue;
-
-        const composed = composeDescription({
-            base: baseDesc,
-            lineage: getLineageText(key, columnMetadata.lineage),
-            typicalUse: columnMetadata.typicalUse[key] || null,
-            config: configNotes[key] || null,
-        });
-
-        if (isStruct) {
-            descriptions[key].description = composed;
-        } else {
-            descriptions[key] = composed;
-        }
-    }
-
-    // Add descriptions for dynamically promoted event parameter columns
-    if (config && config.eventParamsToColumns && config.eventParamsToColumns.length > 0) {
-        config.eventParamsToColumns.forEach(p => {
-            const columnName = p.columnName || p.name;
-            const type = p.type ? ` (${p.type})` : ' (any data type)';
-            descriptions[columnName] = composeDescription({
-                base: `Promoted from event parameter '${p.name}'${type}`,
-                lineage: `Derived -- Promoted from the event_params array`,
-                typicalUse: 'Promoted event parameter available as a top-level column for direct filtering and aggregation',
-                config: null,
-            });
-        });
-    }
-
-    return descriptions;
-};
-
-/**
- * Checks whether a column (or its parent struct) is excluded by the config.
- *
- * @param {string[]} dependsOn - Column names this entry depends on.
- * @param {string[]} excludedColumns - Combined excluded columns from config.
- * @returns {boolean} True if ALL dependsOn columns are excluded.
- */
-const isExcluded = (dependsOn, excludedColumns) => {
-    if (!dependsOn || dependsOn.length === 0) return false;
-    return dependsOn.every(col => excludedColumns.includes(col));
-};
-
-/**
- * Builds the full table description by combining table-specific sections
- * with shared sections (package attribution, config JSON dump).
- *
- * @param {Object} config - The merged configuration object.
- * @param {string[]} tableSections - Table-specific description sections (provided by the table module).
- * @returns {string} The composed table description.
- */
-const buildTableDescription = (config, tableSections) => {
-    const sections = [...tableSections];
-
-    // Package Attribution
-    sections.push(`${constants.TABLE_DESCRIPTION_SUFFIX} Version: ${version}\n${constants.TABLE_DESCRIPTION_DOCUMENTATION_LINK}`);
-
-    // Config JSON dump
-    const configForDump = Object.fromEntries(
-        Object.entries(config).filter(([key]) => !key.startsWith('default'))
-    );
-    // Strip description and columns from dataformTableConfig to avoid circular reference and bloat
-    if (configForDump.dataformTableConfig) {
-        const { description, columns, ...rest } = configForDump.dataformTableConfig;
-        configForDump.dataformTableConfig = rest;
-    }
-    const configJson = JSON.stringify(configForDump, null, 2);
-    sections.push(`The last full table refresh was done using this configuration:\n${configJson}`);
-
-    return sections.join('\n\n');
-};
-
-module.exports = {
-    getColumnDescriptions,
-    buildTableDescription,
-    composeDescription,
-    getLineageText,
-    buildConfigNotes,
-    isExcluded,
-};
+const constants = require('./constants');
+const { version } = require('./package.json');
+
+/**
+ * Composes a multi-section column description string from individual sections.
+ * Sections with null/undefined/empty values are omitted.
+ * Sections are separated by line breaks for readability.
+ *
+ * @param {Object} sections - { base, lineage, typicalUse, config }
+ * @returns {string} Composed description
+ */
+const composeDescription = (sections) => {
+    const parts = [];
+
+    if (sections.base) {
+        parts.push(sections.base);
+    }
+
+    if (sections.lineage) {
+        parts.push(`Lineage: ${sections.lineage}`);
+    }
+
+    if (sections.typicalUse) {
+        parts.push(`Typical use: ${sections.typicalUse}`);
+    }
+
+    if (sections.config) {
+        parts.push(`Config: ${sections.config}`);
+    }
+
+    return parts.join('\n\n');
+};
+
+/**
+ * Returns a formatted lineage text string for a column, or null if no lineage data exists.
+ *
+ * @param {string} columnName - The column name to look up.
+ * @param {Object} columnLineage - The lineage data object mapping column names to { source, note }.
+ * @returns {string|null} Formatted lineage string, e.g. "Derived -- Concatenation of ..."
+ */
+const getLineageText = (columnName, columnLineage) => {
+    const entry = columnLineage[columnName];
+    if (!entry) return null;
+
+    const sourceLabels = {
+        'ga4_export': 'Standard GA4 export field',
+        'ga4_export_modified': 'GA4 export field (modified)',
+        'derived': 'Derived',
+    };
+
+    const label = sourceLabels[entry.source] || entry.source;
+    return entry.note ? `${label} -- ${entry.note}` : label;
+};
+
+/**
+ * Builds a map of config-specific notes for columns based on the provided configuration.
+ * Extracts the configuration-dependent description suffixes into a { columnName: "note" } map.
+ *
+ * @param {Object} config - The merged configuration object.
+ * @returns {Object} Map of column names to config note strings.
+ */
+const buildConfigNotes = (config) => {
+    const notes = {};
+
+    if (!config) return notes;
+
+    const append = (key, text) => {
+        notes[key] = notes[key] ? `${notes[key]}. ${text}` : text;
+    };
+
+    // timezone
+    if (config.timezone) {
+        append('event_datetime', `Timezone: ${config.timezone}`);
+    }
+
+    // customTimestampParam
+    if (config.customTimestampParam) {
+        append('event_datetime', `Custom timestamp parameter: '${config.customTimestampParam}'`);
+        append('event_custom_timestamp', `Source parameter: '${config.customTimestampParam}'`);
+    }
+
+    // data_is_final
+    if (config.dataIsFinal) {
+        const method = config.dataIsFinal.detectionMethod;
+        if (method === 'DAY_THRESHOLD') {
+            append('data_is_final', `Detection method: DAY_THRESHOLD (${config.dataIsFinal.dayThreshold} days)`);
+        } else {
+            append('data_is_final', `Detection method: EXPORT_TYPE`);
+        }
+    }
+
+    // excludedEvents
+    if (config.excludedEvents && config.excludedEvents.length > 0) {
+        append('event_name', `Excluded events: ${config.excludedEvents.join(', ')}`);
+    }
+
+    // excludedEventParams
+    if (config.excludedEventParams && config.excludedEventParams.length > 0) {
+        append('event_params', `Excluded parameters: ${config.excludedEventParams.join(', ')}`);
+    }
+
+    // sessionParams
+    if (config.sessionParams && config.sessionParams.length > 0) {
+        append('session_params', `Configured parameters: ${config.sessionParams.join(', ')}`);
+    }
+
+    // includedExportTypes
+    if (config.includedExportTypes) {
+        const types = Object.entries(config.includedExportTypes)
+            .filter(([, enabled]) => enabled)
+            .map(([type]) => type);
+        if (types.length > 0) {
+            append('export_type', `Included export types: ${types.join(', ')}`);
+        }
+    }
+
+    return notes;
+};
+
+/**
+ * Returns a deep copy of the column descriptions, enriched with
+ * lineage, typical use, and configuration-specific sections composed into
+ * multi-section descriptions.
+ *
+ * @param {Object} config - The merged configuration object.
+ * @param {Object} columnMetadata - Column metadata provided by the table module.
+ * @param {Object} columnMetadata.descriptions - Column descriptions (Dataform ITableConfig columns format).
+ * @param {Object} columnMetadata.lineage - Column lineage data mapping column names to { source, note }.
+ * @param {Object} columnMetadata.typicalUse - Column typical use mapping column names to description strings.
+ * @returns {Object} Column descriptions object in Dataform ITableConfig columns format.
+ */
+const getColumnDescriptions = (config, columnMetadata) => {
+    const descriptions = JSON.parse(JSON.stringify(columnMetadata.descriptions));
+
+    const configNotes = buildConfigNotes(config);
+
+    // Compose multi-section descriptions for each top-level column
+    for (const key of Object.keys(descriptions)) {
+        const isStruct = typeof descriptions[key] === 'object' && descriptions[key].description;
+        const baseDesc = isStruct ? descriptions[key].description : (typeof descriptions[key] === 'string' ? descriptions[key] : null);
+
+        if (!baseDesc) continue;
+
+        const composed = composeDescription({
+            base: baseDesc,
+            lineage: getLineageText(key, columnMetadata.lineage),
+            typicalUse: columnMetadata.typicalUse[key] || null,
+            config: configNotes[key] || null,
+        });
+
+        if (isStruct) {
+            descriptions[key].description = composed;
+        } else {
+            descriptions[key] = composed;
+        }
+    }
+
+    // Add descriptions for dynamically promoted event parameter columns
+    if (config && config.eventParamsToColumns && config.eventParamsToColumns.length > 0) {
+        config.eventParamsToColumns.forEach(p => {
+            const columnName = p.columnName || p.name;
+            const type = p.type ? ` (${p.type})` : ' (any data type)';
+            descriptions[columnName] = composeDescription({
+                base: `Promoted from event parameter '${p.name}'${type}`,
+                lineage: `Derived -- Promoted from the event_params array`,
+                typicalUse: 'Promoted event parameter available as a top-level column for direct filtering and aggregation',
+                config: null,
+            });
+        });
+    }
+
+    // Add descriptions for columns added or replaced by data enrichments.
+    // Item-level enrichments are not yet supported and throw at SQL gen time — skip here.
+    if (config && Array.isArray(config.enrichments) && config.enrichments.length > 0) {
+        config.enrichments.forEach(e => {
+            const level = e.level ?? 'event';
+            if (level !== 'event') return;
+            const joinKeys = Array.isArray(e.joinKey) ? e.joinKey : [e.joinKey];
+            const joinKeyText = joinKeys.join(', ');
+            const sourceText = renderEnrichmentSource(e.source);
+            for (const c of e.columns) {
+                const existing = descriptions[c];
+                const existingText = typeof existing === 'string'
+                    ? existing
+                    : (existing && typeof existing === 'object' && existing.description)
+                        ? existing.description
+                        : null;
+                const newDesc = existingText
+                    ? `Replaced by enrichment '${e.name}' (joined on ${joinKeyText} from ${sourceText}). Original: ${existingText}`
+                    : `Added by enrichment '${e.name}' (joined on ${joinKeyText} from ${sourceText}).`;
+                // If the original was a struct-shaped entry, preserve the structure but replace the description.
+                // Otherwise, set as a plain string.
+                if (existing && typeof existing === 'object' && !Array.isArray(existing)) {
+                    descriptions[c] = { ...existing, description: newDesc };
+                } else {
+                    descriptions[c] = newDesc;
+                }
+            }
+        });
+    }
+
+    return descriptions;
+};
+
+/**
+ * Renders an enrichment source for inclusion in column descriptions.
+ *
+ * - Backtick-quoted string: passed through as-is.
+ * - Dataform table reference object: rendered as `<dataset>.<name>` (project not available
+ *   at description-generation time; resolved later via ctx.ref()).
+ *
+ * @param {string|Object} source - The enrichment's source field.
+ * @returns {string} Backtick-quoted source identifier for display.
+ */
+const renderEnrichmentSource = (source) => {
+    if (typeof source === 'string') return source;
+    if (source && typeof source === 'object') {
+        const dataset = source.dataset || source.schema;
+        if (dataset && source.name) return '`' + dataset + '.' + source.name + '`';
+    }
+    return String(source);
+};
+
+/**
+ * Checks whether a column (or its parent struct) is excluded by the config.
+ *
+ * @param {string[]} dependsOn - Column names this entry depends on.
+ * @param {string[]} excludedColumns - Combined excluded columns from config.
+ * @returns {boolean} True if ALL dependsOn columns are excluded.
+ */
+const isExcluded = (dependsOn, excludedColumns) => {
+    if (!dependsOn || dependsOn.length === 0) return false;
+    return dependsOn.every(col => excludedColumns.includes(col));
+};
+
+/**
+ * Builds the full table description by combining table-specific sections
+ * with shared sections (package attribution, config JSON dump).
+ *
+ * @param {Object} config - The merged configuration object.
+ * @param {string[]} tableSections - Table-specific description sections (provided by the table module).
+ * @returns {string} The composed table description.
+ */
+const buildTableDescription = (config, tableSections) => {
+    const sections = [...tableSections];
+
+    // Package Attribution
+    sections.push(`${constants.TABLE_DESCRIPTION_SUFFIX} Version: ${version}\n${constants.TABLE_DESCRIPTION_DOCUMENTATION_LINK}`);
+
+    // Config JSON dump
+    const configForDump = Object.fromEntries(
+        Object.entries(config).filter(([key]) => !key.startsWith('default'))
+    );
+    // Strip description and columns from dataformTableConfig to avoid circular reference and bloat
+    if (configForDump.dataformTableConfig) {
+        const { description, columns, ...rest } = configForDump.dataformTableConfig;
+        configForDump.dataformTableConfig = rest;
+    }
+    const configJson = JSON.stringify(configForDump, null, 2);
+    sections.push(`The last full table refresh was done using this configuration:\n${configJson}`);
+
+    return sections.join('\n\n');
+};
+
+module.exports = {
+    getColumnDescriptions,
+    buildTableDescription,
+    composeDescription,
+    getLineageText,
+    buildConfigNotes,
+    isExcluded,
+};

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ga4-export-fixer",
-  "version": "0.9.0-dev.1",
+  "version": "0.9.0-dev.2",
   "description": "",
   "main": "index.js",
   "files": [
@@ -17,7 +17,7 @@
     "createTable.js"
   ],
   "scripts": {
-    "test": "node tests/ga4EventsEnhanced.test.js && node tests/assertions.test.js && node tests/mergeSQLConfigurations.test.js && node tests/preOperations.test.js && node tests/documentation.test.js && node tests/inputValidation.test.js && node tests/createTable.test.js && node tests/queryBuilder.test.js && node tests/customSteps.test.js",
+    "test": "node tests/ga4EventsEnhanced.test.js && node tests/assertions.test.js && node tests/mergeSQLConfigurations.test.js && node tests/preOperations.test.js && node tests/documentation.test.js && node tests/inputValidation.test.js && node tests/createTable.test.js && node tests/queryBuilder.test.js && node tests/customSteps.test.js && node tests/enrichments.test.js",
     "test:summary": "node tests/testRunner.js",
     "test:docs": "node tests/documentation.test.js",
     "test:preops": "node tests/preOperations.test.js",
@@ -28,6 +28,7 @@
     "test:createTable": "node tests/createTable.test.js",
     "test:queryBuilder": "node tests/queryBuilder.test.js",
     "test:customSteps": "node tests/customSteps.test.js",
+    "test:enrichments": "node tests/enrichments.test.js",
     "test:integration": "node tests/integration/integration.test.js",
     "release:dev": "./scripts/release-dev.sh",
     "readme": "node scripts/updateReadme.js",

package/tables/ga4EventsEnhanced/config.js

@@ -68,6 +68,10 @@ const ga4EventsEnhancedConfig = {
     // user-defined CTEs appended to the pipeline after enhanced_events
     // each entry is a queryBuilder step (raw {name, query} or structured {name, select, from, ...})
     customSteps: [],
+    // declarative external-data enrichments joined into the pipeline
+    // each entry: { name, level: 'event' | 'item', source, joinKey, columns, dedupe? }
+    // 'item' level is accepted at config time but throws at SQL gen — not yet implemented
+    enrichments: [],
 };
 
 module.exports = { ga4EventsEnhancedConfig };

package/tables/ga4EventsEnhanced/index.js

@@ -268,8 +268,7 @@ ${excludedEventsSQL}`,
         'group by': 'session_id',
     };
 
-    // Shared item-array CTEs (currently used by item-list attribution; will also be used by
-    // item-level data enrichments — see design_docs/planned/data-enrichments.md, Q16):
+    // Shared item-array CTEs:
     // 1. items_unnested: unnest items from ecommerce events, compute attribution via window function
     // 2. items_rebuilt: re-aggregate items with attributed list fields
     const itemListSteps = itemListAttribution ? (() => {
@@ -326,6 +325,66 @@ ${excludedEventsSQL}`,
     } : {};
     const itemListExcludedColumns = itemListSteps ? ['_item_row_id'] : [];
 
+    // Build enrichment-source CTEs and gather event-level join/column data.
+    // Item-level enrichments throw "not yet supported" — they will arrive in a later release.
+    const enrichments = mergedConfig.enrichments ?? [];
+    const enrichmentSteps = [];
+    const enrichmentJoins = [];
+    const enrichmentColumns = {};         // column name → SQL expression for select.columns
+    const enrichmentColumnNames = new Set();  // column names for excludedColumns of wildcards
+    const enrichmentColumnOwner = {};     // column name → { i, name } for collision errors
+    for (const [i, e] of enrichments.entries()) {
+        const level = e.level ?? 'event';
+        if (level === 'item') {
+            throw new Error(
+                `config.enrichments[${i}] uses level: 'item', which is not yet supported in this version. ` +
+                `Item-level enrichments will ship in a future release; see design_docs/planned/data-enrichments.md.`
+            );
+        }
+        const joinKeys = Array.isArray(e.joinKey) ? e.joinKey : [e.joinKey];
+        const cteName = `enrich_${e.name}`;
+        // Source CTE selects joinKey columns plus the requested columns. key === value
+        // shape skips the alias clause in queryBuilder's columnsToSQL.
+        const cteCols = {};
+        for (const k of joinKeys) cteCols[k] = k;
+        for (const c of e.columns) cteCols[c] = c;
+        const sourceStep = {
+            name: cteName,
+            select: { columns: cteCols },
+            from: e.source,
+        };
+        // Opt-in dedupe: which row wins is non-deterministic — users with strict needs
+        // pre-aggregate in their source SQL.
+        if (e.dedupe) {
+            sourceStep.qualify = `row_number() over (partition by ${joinKeys.join(', ')}) = 1`;
+        }
+        enrichmentSteps.push(sourceStep);
+
+        enrichmentJoins.push({
+            type: 'left',
+            table: cteName,
+            on: `using(${joinKeys.join(', ')})`,
+        });
+
+        // Replace-or-add: each enrichment column overrides explicit select columns via JS object
+        // spread, AND joins the excludedColumns set so it suppresses overlap with the wildcard
+        // event_data.* / session_data.* expansions below.
+        for (const c of e.columns) {
+            if (enrichmentColumnNames.has(c)) {
+                const owner = enrichmentColumnOwner[c];
+                throw new Error(
+                    `config.enrichments[${i}] (name: '${e.name}') and config.enrichments[${owner.i}] ` +
+                    `(name: '${owner.name}') both target column '${c}'. ` +
+                    `Two enrichments cannot write the same column; rename one in source SQL or pick a different name.`
+                );
+            }
+            enrichmentColumns[c] = `${cteName}.${c}`;
+            enrichmentColumnNames.add(c);
+            enrichmentColumnOwner[c] = { i, name: e.name };
+        }
+    }
+    const enrichmentExcludedColumns = [...enrichmentColumnNames];
+
     // Join event_data and session_data, include additional logic
     // Named 'enhanced_events' so user-supplied customSteps can reference it as a stable handle.
     const enhancedEventsStep = {
@@ -335,6 +394,9 @@ ${excludedEventsSQL}`,
                 // get the most important columns in the correct order
                 ...finalColumnOrder,
                 ...itemListOverrides,
+                // event-level enrichment columns: override matching explicit columns; new columns added.
+                // Wildcard-column overlap is handled below via excludedColumns.
+                ...enrichmentColumns,
                 // get the rest of the event_data columns
                 '[sql]event_data': utils.selectOtherColumns(
                     eventDataStep,
@@ -345,13 +407,14 @@ ${excludedEventsSQL}`,
                         'data_is_final',
                         'export_type',
                         ...itemListExcludedColumns,
+                        ...enrichmentExcludedColumns,
                     ]
                 ),
                 // get the rest of the session_data columns
                 '[sql]session_data': utils.selectOtherColumns(
                     sessionDataStep,
                     Object.keys(finalColumnOrder),
-                    []
+                    [...enrichmentExcludedColumns],
                 ),
                 // include additional columns
                 row_inserted_timestamp: 'current_timestamp()',
@@ -370,12 +433,15 @@ ${excludedEventsSQL}`,
                 type: 'left',
                 table: 'session_data',
                 on: 'using(session_id)'
-            }
+            },
+            // Event-level enrichment joins go last so they apply on top of the package's own joins.
+            ...enrichmentJoins,
         ],
         where: helpers.incrementalDateFilter(mergedConfig)
     };
 
     const packageSteps = [
+        ...enrichmentSteps,
         eventDataStep,
         ...(itemListSteps ?? []),
         sessionDataStep,
@@ -384,7 +450,8 @@ ${excludedEventsSQL}`,
 
     // Layer 2 validation: customSteps name must not collide with package step names.
     // Reserved set is derived from packageSteps at runtime (single source of truth) — what
-    // is reserved depends on config (e.g. item_list_* exist only when itemListAttribution is on).
+    // is reserved depends on config (e.g. item_list_* exist only when itemListAttribution is on,
+    // and enrich_* names exist only when enrichments are configured).
     const customSteps = mergedConfig.customSteps ?? [];
     if (customSteps.length > 0) {
         const reservedNames = new Set(packageSteps.map(s => s.name));

package/tables/ga4EventsEnhanced/validation.js

@@ -225,6 +225,101 @@ const validateEnhancedEventsConfig = (config, options = {}) => {
             seenNames.add(step.name);
         }
     }
+
+    // enrichments - optional array of declarative external-data enrichment specs.
+    // This block performs Layer 1 (config-shape) checks. Layer 2 checks (reserved-name collision
+    // + item-level deferral throw) live in _generateEnhancedEventsSQL — the reserved set is
+    // config-dependent and the item-level deferral throws there once the SQL is built.
+    if (config.enrichments !== undefined) {
+        if (!Array.isArray(config.enrichments)) {
+            throw new Error(`config.enrichments must be an array. Received: ${JSON.stringify(config.enrichments)}`);
+        }
+        const validLevels = ['event', 'item'];
+        const seenNames = new Set();
+        for (let i = 0; i < config.enrichments.length; i++) {
+            const entry = config.enrichments[i];
+            if (!entry || typeof entry !== 'object' || Array.isArray(entry)) {
+                throw new Error(`config.enrichments[${i}] must be a non-null object. Received: ${JSON.stringify(entry)}`);
+            }
+            if (typeof entry.name !== 'string' || !entry.name.trim()) {
+                throw new Error(`config.enrichments[${i}].name must be a non-empty string. Received: ${JSON.stringify(entry.name)}`);
+            }
+            if (seenNames.has(entry.name)) {
+                throw new Error(`config.enrichments contains duplicate name '${entry.name}'. Each enrichments entry must have a unique name.`);
+            }
+            seenNames.add(entry.name);
+            if (entry.level !== undefined && !validLevels.includes(entry.level)) {
+                throw new Error(`config.enrichments[${i}].level must be one of: ${validLevels.join(', ')}. Received: ${JSON.stringify(entry.level)}`);
+            }
+            // source: Dataform table reference object or backtick-quoted string
+            if (entry.source === undefined || entry.source === null) {
+                throw new Error(`config.enrichments[${i}].source is required.`);
+            }
+            if (isDataformTableReferenceObject(entry.source)) {
+                // Valid Dataform reference
+            } else if (typeof entry.source === 'string') {
+                if (!entry.source.trim()) {
+                    throw new Error(`config.enrichments[${i}].source must be a non-empty string. Received empty string.`);
+                }
+                if (!/^`[^\.]+\.[^\.]+\.[^\.]+`$/.test(entry.source.trim())) {
+                    throw new Error(`config.enrichments[${i}].source must be in the format '\`project.dataset.table\`' (with backticks) or a Dataform table reference. Received: ${JSON.stringify(entry.source)}`);
+                }
+            } else {
+                throw new Error(`config.enrichments[${i}].source must be a Dataform table reference object or a string in format '\`project.dataset.table\`'. Received: ${JSON.stringify(entry.source)}`);
+            }
+            // joinKey: required, plain SQL identifier OR non-empty array of plain SQL identifiers.
+            // Plain identifier = ^[a-zA-Z_][a-zA-Z0-9_]*$ — no aliases (`id as user_id`), no backticks,
+            // no dotted paths. Users with mismatched dim-column names alias in an upstream Dataform view.
+            const sqlIdentifier = /^[a-zA-Z_][a-zA-Z0-9_]*$/;
+            const aliasingHint = ' Aliases like \'id as user_id\' are not supported here; alias in an upstream Dataform view if your dim has a different column name.';
+            if (entry.joinKey === undefined || entry.joinKey === null) {
+                throw new Error(`config.enrichments[${i}].joinKey is required.`);
+            }
+            if (typeof entry.joinKey === 'string') {
+                if (!entry.joinKey.trim()) {
+                    throw new Error(`config.enrichments[${i}].joinKey must be a non-empty string. Received empty string.`);
+                }
+                if (!sqlIdentifier.test(entry.joinKey)) {
+                    throw new Error(`config.enrichments[${i}].joinKey must be a plain SQL identifier. Received: ${JSON.stringify(entry.joinKey)}.${aliasingHint}`);
+                }
+            } else if (Array.isArray(entry.joinKey)) {
+                if (entry.joinKey.length === 0) {
+                    throw new Error(`config.enrichments[${i}].joinKey must be a non-empty array when provided as an array.`);
+                }
+                for (let j = 0; j < entry.joinKey.length; j++) {
+                    const k = entry.joinKey[j];
+                    if (typeof k !== 'string' || !k.trim()) {
+                        throw new Error(`config.enrichments[${i}].joinKey[${j}] must be a non-empty string. Received: ${JSON.stringify(k)}`);
+                    }
+                    if (!sqlIdentifier.test(k)) {
+                        throw new Error(`config.enrichments[${i}].joinKey[${j}] must be a plain SQL identifier. Received: ${JSON.stringify(k)}.${aliasingHint}`);
+                    }
+                }
+            } else {
+                throw new Error(`config.enrichments[${i}].joinKey must be a string or a non-empty array of strings. Received: ${JSON.stringify(entry.joinKey)}`);
+            }
+            // columns: required, non-empty array of plain SQL identifiers (no aliasing).
+            if (!Array.isArray(entry.columns)) {
+                throw new Error(`config.enrichments[${i}].columns must be an array. Received: ${JSON.stringify(entry.columns)}`);
+            }
+            if (entry.columns.length === 0) {
+                throw new Error(`config.enrichments[${i}].columns must be non-empty. List the source columns to add to the output (excluding joinKey).`);
+            }
+            for (let j = 0; j < entry.columns.length; j++) {
+                const c = entry.columns[j];
+                if (typeof c !== 'string' || !c.trim()) {
+                    throw new Error(`config.enrichments[${i}].columns[${j}] must be a non-empty string. Received: ${JSON.stringify(c)}`);
+                }
+                if (!sqlIdentifier.test(c)) {
+                    throw new Error(`config.enrichments[${i}].columns[${j}] must be a plain SQL identifier. Received: ${JSON.stringify(c)}.${aliasingHint}`);
+                }
+            }
+            // dedupe: optional boolean
+            if (entry.dedupe !== undefined && typeof entry.dedupe !== 'boolean') {
+                throw new Error(`config.enrichments[${i}].dedupe must be a boolean when provided. Received: ${JSON.stringify(entry.dedupe)}`);
+            }
+        }
+    }
   } catch (e) {
     e.message = `Config validation: ${e.message}`;
     throw e;

package/utils.js

@@ -389,6 +389,16 @@ const setDataformContext = (ctx, config) => {
         }
     }
 
+    // resolve Dataform refs in enrichments[].source the same way as sourceTable
+    if (Array.isArray(config.enrichments)) {
+        config.enrichments = config.enrichments.map(e => {
+            if (isDataformTableReferenceObject(e.source)) {
+                return { ...e, source: ctx.ref(e.source) };
+            }
+            return e;
+        });
+    }
+
     config.self = ctx.self();
     config.incremental = ctx.incremental();
 
@@ -479,23 +489,35 @@ const selectOtherColumns = (step, alreadyDefinedColumns = [], excludedColumns =
     const stepName = step.name;
     const stepColumns = Object.keys(step.select.columns);
 
-    // Determine which columns to exclude: those already defined or explicitly excluded
-    const exceptColumns = stepColumns.filter(
+    // Columns in step.select.columns that should be excluded (already-defined or explicitly listed)
+    const internalExcept = stepColumns.filter(
         column => alreadyDefinedColumns.includes(column) || excludedColumns.includes(column)
     );
 
-    // If none of the columns have been defined or excluded, select them all
-    if (exceptColumns.length === 0) {
+    // Columns in excludedColumns that aren't enumerated in step.select.columns. These are
+    // wildcard-sourced columns (e.g. default GA4 export columns coming through `event_data.*`
+    // inside event_data's own select). The caller knows what to exclude; trust them.
+    // BigQuery throws at dry-run if the column doesn't exist in the source — surfaces typos.
+    // Filter out undefined/null entries (callers can pass conditional values like
+    // `cond ? 'col' : undefined` for ergonomics).
+    const externalExcept = excludedColumns.filter(
+        c => typeof c === 'string' && c.length > 0 && !stepColumns.includes(c)
+    );
+
+    const allExcept = [...internalExcept, ...externalExcept];
+
+    // If nothing is excluded, select everything
+    if (allExcept.length === 0) {
         return `${stepName}.*`;
     }
 
-    // If all columns have been defined or excluded, do not select any
-    if (exceptColumns.length === stepColumns.length) {
+    // If every enumerated column is excluded and there are no external excepts to apply,
+    // there's nothing to select via the wildcard
+    if (internalExcept.length === stepColumns.length && externalExcept.length === 0) {
         return;
     }
 
-    // Otherwise, select all except the excluded/defined ones
-    return `${stepName}.* except (${exceptColumns.join(', ')})`;
+    return `${stepName}.* except (${allExcept.join(', ')})`;
 };