@revos/cli 0.1.2 → 0.1.3

package/dist/templates/skills/create-semantic-model/SKILL.md

@@ -1,1611 +1,305 @@
 ---
 name: create-semantic-model
-description: Create semantic models (Cube.dev cubes) from existing RevOS dbt gold models. Use when asked to build a semantic layer, create cubes, or generate Cube definitions from dbt.
+description: >
+  Create semantic models (Cube.dev cubes) from existing RevOS dbt gold models.
+  Use when asked to: build a semantic layer, create cubes, generate Cube definitions from dbt,
+  create a semantic overlay, or create a semantic model from gold models.
 ---
 
-@.claude/skills/create_dbt_transformations/SKILL.md
-
-# Create Semantical Model
-
-Use this skill when the user asks to create a semantic model or semantic overlay.
-
-Typical user requests:
-
-> Create a new semantic overlay for `<transformation>`.
-
-> Create a semantic model from the available gold models.
-
-> Build a Cube semantic layer for these gold tables.
-
-This skill analyzes existing dbt gold models, asks the user which models should participate, detects keys and relationships, validates join candidates with SQL before proposing them to the user, asks the user to confirm the semantic structure and measures, and generates Cube.dev semantic overlays under `semantic/cubes/`.
+# Create Semantic Model
 
 ## Skill Dependencies
 
-This skill delegates dbt-related knowledge to `create_dbt_transformations` (loaded above):
+This skill delegates dbt-related knowledge to `create-dbt-transformations`:
+project layout, finding gold models, resolving dbt model names to BigQuery table references,
+creating bridge models from JSON arrays, and dbt validation commands.
 
-- Project layout and how to find gold models
-- Resolving a dbt model name to its physical BigQuery table reference
-- Creating new dbt support models (bridge models from JSON arrays)
-- dbt validation commands
+If `create-dbt-transformations` is not installed: discover gold models directly via
+`find dbt/models/gold -name "*.sql"`, run dbt commands without the `revos` wrapper
+(`dbt parse`, `dbt run`, `dbt test`), and skip bridge model delegation.
+Warn the user: "The `create-dbt-transformations` skill is not installed — bridge model creation and dbt validation are limited."
 
-If BigQuery exploration is needed (listing tables, inspecting schemas, previewing rows, checking null rates beyond what is required for join validation), use the `explore-lakehouse` skill at `.claude/skills/explore-lakehouse/SKILL.md`. Load it only when those capabilities are actually required.
+If BigQuery exploration is needed (listing tables, inspecting schemas, previewing rows),
+load the `explore-lakehouse` skill on demand.
 
 ---
 
 ## Purpose
 
-Expose existing dbt gold models as queryable Cube.dev semantic models without manually writing YAML boilerplate.
+Expose existing dbt gold models as queryable Cube.dev semantic models without manually writing YAML boilerplate. Gold models may be tables or views.
 
-Gold models may be dbt tables or dbt views. Treat both as valid semantic sources.
-
-This skill does not build gold models from silver. If a needed gold model is missing, hand off to `create_dbt_transformations`.
-
-The expected flow is:
-
-```text
-dbt/models/gold/
-  -> discover available gold models (via create_dbt_transformations)
-  -> ask the user which gold models should participate
-  -> inspect selected model schemas from the lakehouse
-  -> detect primary keys, secondary keys, foreign keys, JSON/array keys, and candidate relationships
-  -> if selected models are not directly connected, search remaining gold models for connector/intermediate models
-  -> ask the user to approve connector or bridge/support models when needed
-  -> validate candidate joins with SQL before proposing them as relationships
-  -> present validated relationships and join directions to the user
-  -> ask the user to confirm relationships
-  -> generate dimensions from all selected columns by default
-  -> generate default count measure
-  -> suggest useful additional measures
-  -> ask the user to confirm or define custom measures
-  -> generate Cube.dev semantic overlays in semantic/cubes/
-  -> validate generated files
-```
+This skill does not build gold models. If a needed gold model is missing, hand off to `create-dbt-transformations`.
 
 ---
 
-## Execution Order
-
-1. Discover available gold models (use `create_dbt_transformations` for navigation).
-2. If the user named a specific transformation/model, find that gold model first.
-3. Ask the user which gold models should participate, unless the request clearly targets one specific model only.
-4. Resolve `$GOOGLE_CLOUD_PROJECT` and `$REVOS_BQ_DATASET` to literal values via `echo` and keep them for the rest of the session.
-5. Inspect schemas and column types for selected gold models.
-6. Detect primary keys, secondary keys, foreign keys, JSON/array keys, and candidate relationships.
-7. If selected models are disconnected, search for connector models among the remaining gold models.
-8. Ask the user to approve connector or bridge/support models before adding them to the working scope.
-9. Validate candidate joins with SQL before proposing them as relationships.
-10. Present validated relationships, join directions, cardinality, and validation evidence to the user.
-11. Ask the user to confirm or modify relationships.
-12. Generate dimensions from all selected gold model columns by default.
-13. Create the default `count` measure.
-14. Suggest useful additional measures.
-15. Ask the user to confirm suggested measures or define custom measures.
-16. Generate Cube semantic overlays in `semantic/cubes/`.
-17. Validate generated files.
-
-Do not skip ahead.
-
----
+## Naming Convention
 
-## Naming Convention: Cube Files and Cube Names
-
-Gold models in `dbt/models/gold/` use the `gold_` prefix in their file name (for example, `gold_hubspot_companies.sql`). The materialized table in BigQuery keeps the same name.
-
-The semantic overlay layer drops the `gold_` prefix:
-
-1. The gold SQL file: `gold_<entity>` (for example, `gold_hubspot_companies.sql`).
-2. The materialized BigQuery table: `gold_<entity>` (for example, `gold_hubspot_companies`).
-3. The semantic overlay YAML file: `<entity>.yml` (for example, `hubspot_companies.yml`).
-4. The cube `name` inside that YAML file: `<entity>` (for example, `name: hubspot_companies`).
-5. References to that cube from joins: `${<entity>}` (for example, `${hubspot_companies}`).
-
-Mapping example:
+Strip the `gold_` prefix for cube names and file names. Keep `gold_` in `sql_table` (physical table).
 
 ```text
-gold SQL file:        dbt/models/gold/gold_hubspot_companies.sql
-BigQuery table:       gold_hubspot_companies
-overlay YAML file:    semantic/cubes/hubspot_companies.yml
-cube name:            hubspot_companies
-join reference:       ${hubspot_companies}
+gold SQL file:     dbt/models/gold/gold_hubspot_companies.sql
+BigQuery table:    gold_hubspot_companies
+overlay file:      semantic/hubspot_companies.yml
+cube name:         hubspot_companies
+join reference:    ${hubspot_companies}
+sql_table:         "`<dataset>.gold_hubspot_companies`"
 ```
 
-Naming rules:
-
-1. Always strip the `gold_` prefix when generating the overlay YAML file name.
-2. The cube `name` must match the overlay YAML file name (without extension).
-3. The cube `name` is what appears in `${...}` references inside join SQL.
-4. The physical table reference in `sql_table` always uses the full BigQuery name with `gold_` prefix.
-5. Bridge and support cube names follow the same rule. If the gold support model is `gold_deals_companies`, the cube name is `deals_companies` and the file is `deals_companies.yml`.
-
----
+Same rule for bridge cubes: `gold_deals_companies` -> cube name `deals_companies`, file `deals_companies.yml`.
 
 ## Cube `sql_table` Reference
 
-Cube overlays must reference the physical warehouse table directly, not through `dbt ref()`. Cube does not understand Jinja.
+Reference the physical warehouse table directly. Cube does not understand Jinja — never use `dbt ref()`.
 
-Use the literal `project` and `dataset` values captured during the "Resolve Environment Variables First" step at the start of Phase 2 (see Phase 2 below). If you reach Phase 8 without those literals already in memory, go back and run the resolution step first.
-
-Wrap the literal in BigQuery backticks and use it as `sql_table`:
-
-```yaml
-sql_table: "`<resolved_project>.<resolved_dataset>.<gold_model_name>`"
-```
-
-Concrete example:
+Use the literal `dataset` value resolved in Phase 2. Wrap in BigQuery backticks:
 
 ```yaml
-sql_table: "`revos-dev.revos_1737556292084.gold_hubspot_companies`"
+sql_table: "`revos_1737556292084.gold_hubspot_companies`"
 ```
 
-The cube `name` is `hubspot_companies` (no `gold_` prefix), but `sql_table` keeps the `gold_` prefix because that is the physical table.
-
-Apply the same fully qualified format anywhere a physical table name is needed in a Cube overlay (notably in `refresh_key.sql`).
+Apply the same fully qualified format in `refresh_key.sql`.
 
 ---
 
-## Non-Negotiable Rules
-
-1. Always start from existing dbt gold models in `dbt/models/gold/`. If no usable gold models exist, hand off to `create_dbt_transformations` and stop.
-2. Gold models may be tables or views.
-3. Always discover available gold models before doing anything else.
-4. Always ask the user which discovered gold models should participate, unless the user clearly requested one specific model only.
-5. Do not inspect selected schemas, analyze joins, generate overlays, or create support models until the model scope is clear.
-6. Only selected models may be used for the main semantic model.
-7. If selected models are not directly connected, search remaining gold models for connector/intermediate models.
-8. During connector search, Claude may inspect column names and key-like fields of non-selected gold models only for relationship discovery.
-9. Inspecting a non-selected model does not add it to the semantic model.
-10. Connector models can only be added after explicit user approval.
-11. If a many-to-many relationship requires a bridge/support model and no suitable gold model exists, ask the user whether to create one. If the user approves, hand off the bridge model creation to `create_dbt_transformations`.
-12. Do not invent joins. If models cannot be connected directly or through approved connector/bridge models, generate separate semantic subgraphs and clearly report that they are disconnected.
-13. Validate candidate joins with SQL before proposing them to the user as confirmed relationship options.
-14. Ask the user to confirm validated relationships before generating semantic overlays.
-15. For every confirmed relationship, generate joins in both directions where both cubes exist.
-16. Validate joins in both directions where possible.
-17. Relationship direction must be correct from the perspective of the current cube.
-18. Preserve every business column from each selected gold model as a Cube dimension by default.
-19. From technical Airbyte metadata columns (`_airbyte_*`), include only `_airbyte_extracted_at` as a dimension named `airbyte_extracted_at`. Exclude all other `_airbyte_*` columns by default.
-20. Do not remove non-Airbyte columns just because they do not look immediately useful.
-21. Measures are not all numeric columns by default. Always create `count`, suggest useful additional measures, and ask the user to confirm or define custom measures.
-22. Keys may be stored in normal columns, JSON strings, JSON arrays, repeated fields, or nested structures. Always check for hidden relationship keys.
-23. For JSON arrays, use `UNNEST(JSON_VALUE_ARRAY(...))`.
-24. Bridge and junction cubes should use `public: false` where the project convention supports it.
-25. Every generated Cube overlay must include `refresh_key`.
-26. Prefer `_airbyte_extracted_at` for refresh keys when available. Otherwise fall back to `every: 1 hour`.
-27. New Cube overlays must reference the physical warehouse table directly using the fully qualified name. Do not use `dbt ref()` in Cube overlays.
-28. Cube `name` and overlay file name must drop the `gold_` prefix. The physical table reference in `sql_table` keeps the `gold_` prefix.
-29. Follow the existing `semantic/cubes/` style in the repository. If existing overlays use map-style `dimensions`, `measures`, and `joins`, use map-style.
-
----
-
-## Mandatory User Checkpoints
+## User Checkpoints
 
 ### Checkpoint 1: Gold Model Selection
 
-After discovering gold models, show the available models and ask which ones should participate.
-
-Do not proceed until the user selects models.
-
-Exception: if the user explicitly requested one specific model, and that model exists in `dbt/models/gold/`, treat that model as selected. If the user appears to want a multi-model semantic layer, still ask whether related models should be included.
+After discovering gold models, show available models and ask which should participate. Do not proceed until the user selects. Exception: if the user named one specific model, treat it as selected.
 
 ### Checkpoint 2a: Connector Model Approval
 
-Triggered when selected models are not directly connected.
-
-Search remaining gold models for connector/intermediate models. During this search, Claude may inspect column names and key-like fields of non-selected gold models only for relationship discovery. Inspecting a model does not add it to the semantic model.
-
-If a connector path is found, explain why it is needed and ask whether to add it.
-
-Example:
-
-```text
-You selected products and addresses.
-
-I do not see a direct relationship between products and addresses.
-
-However, I found a possible connector path:
-
-products -> clients -> addresses
-
-The clients model appears to connect products to addresses.
-
-Should I include gold_clients as a connector model in this semantic model?
-```
-
-Do not use connector models until the user approves them.
+When selected models are not directly connected, search remaining gold models for connectors. Present the connector path and ask approval before adding.
 
 ### Checkpoint 2b: Bridge / Support Model Approval
 
-Triggered when a many-to-many relationship is detected and no suitable existing gold bridge model exists.
-
-Ask whether to create one:
-
-```text
-The relationship between deals and contacts appears to be many-to-many,
-likely stored as a JSON array on hubspot_deals.
-
-I did not find an existing gold bridge model for this relationship.
-
-Should I create a support model named gold_deals_contacts?
-If yes, I will hand off bridge model creation to the create_dbt_transformations skill,
-then continue with the semantic model.
-```
-
-If approved, invoke `create_dbt_transformations` with the bridge model template (it knows the standard JSON-array bridge pattern). Once that skill confirms the bridge model is built and tested, return here and continue.
-
-Do not create or use bridge/support models until the user approves them.
+When a many-to-many relationship is detected and no suitable bridge model exists, ask whether to create one. If approved, delegate to `create-dbt-transformations`.
 
 ### Checkpoint 3: Relationship Confirmation
 
-After candidate relationships are detected and validated with SQL, present the validated relationship options, join directions, cardinality, and validation evidence.
-
-Ask the user to confirm or modify relationships before generating semantic overlays.
-
-Do not present unvalidated joins as confirmed relationships.
-
-If validation could not be executed, clearly mark the relationship as `validation pending` and ask the user whether to proceed with that assumption. If the user proceeds, the join is generated but flagged inside the YAML with a comment (see Phase 8).
+Present validated relationships with join directions, cardinality, and match rates. Ask the user to confirm before generating overlays. Do not present unvalidated joins as confirmed — mark them as `validation pending`.
 
 ### Checkpoint 4: Measures Confirmation
 
-After schema and relationship analysis, generate default dimensions and default `count` measure, then propose useful additional measures.
-
-Ask the user to confirm suggested measures or define custom measures.
-
-Do not create ambiguous custom measures. If the user asks for a custom measure but the definition is unclear, ask for more detail.
+Generate default `count` measure, suggest additional measures, and ask the user to confirm or define custom measures.
 
 ---
 
 # Workflow
 
-Follow these phases in order.
+Follow these phases in order. Do not skip ahead.
 
 ---
 
 ## Phase 1: Discover Gold Models and Select Scope
 
-### Goal
-
-Find available prepared gold models, detect existing overlay conventions, and determine which models participate in the semantic model.
-
-### Steps
-
-1. Discover gold models. Use the navigation commands documented in `create_dbt_transformations` ("Model Navigation"). Specifically:
-
-```bash
-find dbt/models/gold -name "*.sql"
-```
-
-2. If no usable gold models exist, stop and tell the user:
-
-```text
-No gold models were found in dbt/models/gold/. The required dbt transformation
-must exist before I can build a semantic model on top of it.
-
-Use the create_dbt_transformations skill to create the gold model first, then
-run semantic model generation again.
-```
-
-3. Inspect existing overlays in `semantic/cubes/` to detect conventions used in this repository.
-
-Look at one or two existing files and check for:
-
-- Whether `extends:` is used to inherit from base cubes.
-- Whether existing cubes use map-style or list-style for `dimensions`, `measures`, and `joins`.
-- The `public:` flag pattern (true / false / omitted).
-- The `refresh_key` style (`sql:` based or `every:` based).
-- Naming patterns for dimensions and measures.
-- Whether there are base cubes intended for `extends:`.
-
-Apply the detected conventions to the new overlays. If the repository is empty or conventions are inconsistent, follow the defaults defined in this skill.
-
-4. If the user named a specific transformation/model, search for it. If the model exists, treat it as the selected model. If it does not exist, stop:
-
-```text
-I could not find the requested gold model in dbt/models/gold/. The gold model
-must exist before I can create a semantic overlay for it.
-```
-
-5. If the user did not name a specific model, list all discovered gold models.
-
-6. Infer likely business entities from file names.
-
-Examples:
-
-```text
-gold_hubspot_companies -> companies
-gold_hubspot_deals -> deals
-gold_hubspot_contacts -> contacts
-gold_hubspot_users -> users
-gold_products -> products
-gold_clients -> clients
-gold_addresses -> addresses
-gold_invoices -> invoices
-gold_payments -> payments
-gold_active_users_last_month -> active users
-```
-
-7. Present discovered models and ask the user which ones should be included.
-
-8. Stop and wait for user selection if no specific model was provided.
-
-9. Confirm selected models back to the user.
-
-10. Keep the full discovered gold model list available for connector search.
-
-11. If the user asks to include all models, warn that this may create a larger and more complex semantic model before proceeding.
+1. Discover gold models via `find dbt/models/gold -name "*.sql"`.
+2. If none exist, stop and tell the user to create gold models first via `create-dbt-transformations`.
+3. Inspect 1-2 existing overlays in `semantic/` to detect conventions (map-style vs list-style, `extends:`, `public:`, `refresh_key` style). Apply detected conventions to new overlays.
+4. If the user named a specific model, find it. If not found, stop.
+5. Otherwise list all discovered gold models and ask which should participate (Checkpoint 1).
+6. Keep the full discovered list available for connector search in Phase 3.
 
 ---
 
 ## Phase 2: Analyze Selected Model Schemas and Keys
 
-### Goal
-
-Inspect selected gold models and identify primary keys, secondary keys, foreign keys, JSON keys, array keys, time columns, and metric-like columns.
-
 ### Resolve Environment Variables First
 
-Before running any SQL or generating any YAML in this session, resolve `$GOOGLE_CLOUD_PROJECT` and `$REVOS_BQ_DATASET` to their literal values **once**, and use those literals everywhere downstream. SQL placeholders like `<project>` are not valid BigQuery syntax, and Cube YAML does not interpolate env variables — both contexts need real values.
-
-Run:
+Before any SQL or YAML generation, resolve `$REVOS_BQ_DATASET` to a literal:
 
 ```bash
-echo "PROJECT=$GOOGLE_CLOUD_PROJECT"
 echo "DATASET=$REVOS_BQ_DATASET"
 ```
 
-Record the two literal values returned (for example, `revos-dev` and `revos_1737556292084`). For the rest of this session, every time you see `<project>` or `<dataset>` in a SQL template or YAML example, substitute these literals.
-
-If either variable is empty, stop and tell the user:
-
-```text
-The environment variable $GOOGLE_CLOUD_PROJECT or $REVOS_BQ_DATASET is not set.
-I cannot resolve physical BigQuery table references without them. Please set
-them and try again.
-```
-
-Do not proceed to schema inspection or join validation until the variables are resolved.
+If empty, stop and ask the user to set it. Use this literal everywhere downstream.
 
 ### Schema Discovery
 
-For each selected gold model, inspect available columns and types. If BigQuery exploration commands are needed (listing tables, inspecting schemas, previewing rows), refer to the `explore-lakehouse` skill at `.claude/skills/explore-lakehouse/SKILL.md`.
-
-When inspecting schema output, check whether `_airbyte_extracted_at` exists (it will be needed for `refresh_key` and as the only Airbyte dimension). Other `_airbyte_*` columns can be noted but are not used by default.
-
-Validation queries reference physical tables using the resolved literal values (see "Resolve Environment Variables First" above). Substitute `<project>` and `<dataset>` placeholders in the SQL templates throughout this skill with the literals captured at the start of Phase 2.
-
-### Primary Key Detection
-
-Look for stable unique identifiers.
-
-Common primary key patterns:
-
-```text
-id
-<entity>_id
-<model_name>_id
-uuid
-unique_id
-external_id
-source_id
-```
-
-Examples:
-
-```text
-companies.id
-companies.company_id
-companies.office_unique_id
-hubspot_companies.properties_company_unique_id
-```
-
-Primary key rules:
-
-1. Prefer a known business or platform identifier over a generated row number.
-2. Prefer stable IDs over names or labels.
-3. Do not mark a column as primary key only because it looks unique by name.
-4. Validate uniqueness where possible.
-
-Validation:
-
-```sql
-SELECT
-  COUNT(*) AS total_rows,
-  COUNT(DISTINCT <candidate_pk>) AS distinct_keys,
-  COUNT(*) - COUNT(DISTINCT <candidate_pk>) AS duplicate_count
-FROM `<project>.<dataset>.<gold_model>`;
-```
-
-A primary key should normally have `total_rows = distinct_keys`, or a clearly explained reason why duplicates are expected.
-
-### Secondary Key Detection
-
-Secondary keys are identifiers that are not the table primary key but can be used for joins, grouping, lookup, or `count_distinct` measures.
-
-Common secondary key patterns:
-
-```text
-office_unique_id
-company_id
-customer_id
-client_id
-deal_id
-contact_id
-owner_id
-user_id
-account_id
-product_id
-address_id
-external_id
-source_id
-```
+For each selected gold model, inspect columns and types. Use `explore-lakehouse` if needed.
 
-Secondary key rules:
-
-1. Track secondary keys explicitly.
-2. Secondary keys may be foreign keys to another entity.
-3. Secondary keys should usually become Cube dimensions.
-4. Secondary keys may support `count_distinct` measures if analytically useful, but only inside the cube that owns the FK (see Phase 7 caution about fan-out).
-
-### JSON / Array Key Detection
-
-Keys may be hidden inside JSON strings, JSON arrays, repeated fields, or nested structures. This is especially common for one-to-many and many-to-many relationships.
-
-Common JSON / array relationship patterns:
-
-```text
-companies
-deals
-contacts
-users
-owners
-clients
-products
-addresses
-associations
-associated_companies
-associated_deals
-associated_contacts
-associated_clients
-associated_products
-company_ids
-deal_ids
-contact_ids
-client_ids
-product_ids
-address_ids
-```
-
-Example: `gold_hubspot_deals.companies` may contain an array of company IDs.
-
-For JSON arrays, use `UNNEST(JSON_VALUE_ARRAY(...))`.
-
-JSON / array rules:
-
-1. Always inspect JSON, array, repeated, and nested fields for hidden relationship keys.
-2. Do not assume relationship keys only exist as flat columns.
-3. If JSON structure is unknown, inspect sample values first:
-
-```sql
-SELECT
-  <json_or_array_column>
-FROM `<project>.<dataset>.<gold_model>`
-WHERE <json_or_array_column> IS NOT NULL
-LIMIT 20;
-```
+Check whether `_airbyte_extracted_at` exists (needed for `refresh_key` and as the only Airbyte dimension to expose).
 
-4. If a relationship is stored as an array of IDs, use or create an approved bridge/support model. Bridge model creation is delegated to `create_dbt_transformations` (which has the standard JSON-array bridge template).
-5. Bridge models should preserve both sides of the relationship as keys.
-6. Bridge and junction cubes should use `public: false` where the project convention supports it.
+### Key Detection
 
-### Foreign Key Detection
+Detect primary keys, secondary keys, foreign keys, and JSON/array keys.
+See [references/key-patterns.md](references/key-patterns.md) for common patterns and detection rules.
 
-Look for columns that reference other entities.
+Validate primary key uniqueness with SQL. See [references/validation-queries.md](references/validation-queries.md), section 1.
 
-Common patterns:
-
-```text
-<entity>_id
-<entity>Id
-fk_<entity>
-associated_<entity>_id
-parent_<entity>_id
-owner_id
-created_by_user_id
-updated_by_user_id
-```
-
-Also check JSON and array-based foreign keys:
-
-```text
-deals.companies -> companies.id
-companies.deals -> deals.id
-contacts.associated_company_ids -> companies.id
-```
-
-### Schema Summary Output
-
-After analysis, summarize each selected model.
-
-Example:
-
-```text
-Model: gold_hubspot_deals
-Columns: 18
-Candidate primary key: deal_id
-Secondary keys: company_id, owner_id
-JSON / array relationship columns: companies, contacts
-Time columns: created_at, updated_at, closed_at
-Numeric metric-like columns: amount
-Airbyte columns present: _airbyte_extracted_at (will be exposed), _airbyte_raw_id, _airbyte_meta, _airbyte_generation_id (will be excluded by default)
-```
+Output a schema summary for each model (format in key-patterns.md).
 
 ---
 
 ## Phase 3: Detect Candidate Relationships
 
-### Goal
-
-Build a candidate relationship graph between selected models and approved connector/bridge models.
-
-These relationships are candidates only. They must be validated with SQL before being proposed to the user as relationship options.
+Build a candidate relationship graph. These are candidates only — they must be validated in Phase 4.
 
 ### Single-Model Case
 
-If only one gold model was selected in Phase 1 **and** that model has no JSON/array relationship columns identified in Phase 2, skip Phase 3 entirely. There is nothing to relate. Proceed directly to Phase 6 (dimensions) and Phase 7 (measures). Phase 4 (validation) is also skipped since there are no candidate joins.
-
-If only one model is selected but it has JSON/array relationship columns (for example, `gold_hubspot_deals` with a `companies` array), Phase 3 still applies — but only the bridge/junction detection branch. Treat the JSON-array target entity as a candidate model and ask the user whether to add it to the scope (and, if needed, to create a bridge support model).
-
-### Relationship Types
+If only one model was selected and it has no JSON/array relationship columns, skip to Phase 6. If it has JSON/array columns, check for bridge/junction needs.
 
-Use Cube relationship types: `one_to_one`, `one_to_many`, `many_to_one`, `many_to_many`.
+### Relationship Types and Direction
 
-### Relationship Direction
-
-Relationship direction is from the perspective of the current cube.
-
-```text
-Deal belongs to one company:
-deals.company_id -> companies.id
-relationship from deals to companies: many_to_one
-
-Company has many deals:
-companies.id -> deals.company_id
-relationship from companies to deals: one_to_many
-```
+Use Cube types: `one_to_one`, `one_to_many`, `many_to_one`, `many_to_many`. Direction is always from the perspective of the current cube.
 
 ### Cardinality Rules
 
-1. If source foreign key can repeat and target key is unique → `many_to_one` / reverse `one_to_many`.
-2. If both sides are unique → `one_to_one`.
-3. If both sides can repeat or the relationship is represented through a bridge/junction model → `many_to_many` through bridge/junction model.
-4. If cardinality is unclear, report uncertainty and validate before proposing the relationship.
+1. Source FK can repeat, target key unique -> `many_to_one` / reverse `one_to_many`.
+2. Both sides unique -> `one_to_one`.
+3. Both can repeat or through bridge -> `many_to_many`.
+4. If unclear, validate before proposing.
 
 ### Direct Join Detection
 
-Analyze direct joins between selected models first.
-
-Look for:
-
-1. Foreign key to primary key matches.
-2. Secondary key matches.
-3. Known entity conventions.
-4. Existing junction/bridge models among selected models.
-5. JSON / array relationship fields among selected models.
-6. Nested association structures.
+Look for FK-to-PK matches, secondary key matches, existing bridge models, and JSON/array relationship fields.
 
 ### Connector Model Search
 
-Run connector search when selected models are not directly connected.
-
-Rules:
-
-1. Search all discovered gold models, including non-selected models.
-2. During connector search, inspect column names and key-like fields of non-selected gold models only for relationship discovery.
-3. This lightweight schema discovery does not add the connector model to the semantic model.
-4. Do not automatically add connector models.
-5. Look for paths of length 2 first.
-6. If needed, look for paths of length 3.
-7. Do not create long or speculative chains without user confirmation.
-8. Use connector models only after user approval.
+When selected models are disconnected:
 
-Example. User selected `gold_products` and `gold_addresses`. No direct relationship found. Search remaining gold models and find that both have `client_id` matching `gold_clients.id`. Proposed connector path: `products -> clients -> addresses`.
+1. Search all discovered gold models for connector paths (length 2, then 3).
+2. Inspect non-selected models for relationship discovery only — this does not add them to the scope.
+3. Present connector path and ask user approval (Checkpoint 2a).
+4. If approved, add to scope and run full schema discovery. If rejected, document disconnected models.
 
-If the user approves the connector model:
-
-1. Add it to the working model scope.
-2. Run full schema discovery for it.
-3. Analyze its keys and relationships.
-4. Validate the connector path with SQL before proposing it as a relationship.
-5. Generate a semantic overlay for it.
-
-If the user rejects the connector model:
-
-1. Continue only with directly connected selected models.
-2. Clearly document disconnected selected models.
-3. Do not invent joins.
-
-### Bridge / Junction Relationship Detection
-
-Use existing bridge/junction gold models when available — but only after confirming they are usable.
+### Bridge / Junction Detection
 
 If an existing bridge model is found in `dbt/models/gold/`:
 
-1. Inspect its schema (use `explore-lakehouse` for schema commands).
-2. Verify it contains the expected key columns for the relationship — typically `<entity_a>_id` and `<entity_b>_id`. The exact column names may differ from the convention (for example, `company_uuid` instead of `company_id`).
-3. Check whether `_airbyte_extracted_at` is preserved. If not, the bridge cube will have to fall back to `every: 1 hour` for `refresh_key`.
-4. Run a quick row-count and sample-rows check to confirm the bridge has data.
-
-If the existing bridge model fits the relationship cleanly, use it.
-
-If it does not fit (key columns named unexpectedly, missing one side of the relationship, has unrelated extra columns that change cardinality, or is empty), present the situation to the user and ask:
+1. Inspect its schema and verify it has the expected key columns.
+2. If it fits, use it. If it doesn't fit cleanly, present the mismatch and offer options (use as-is, create new, or abort).
 
-```text
-I found an existing bridge model `<model_name>` in dbt/models/gold/, but it does not fit the relationship cleanly:
-- <specific issue, e.g. "key column is named company_uuid, not company_id">
-- <specific issue>
-
-Options:
-- use as-is (I will adapt the cube join SQL to match the existing column names)
-- create a new bridge model via create_dbt_transformations (I will use the standard JSON-array template and a new name)
-- abort and let me revisit this later
-```
-
-Wait for the user's decision. Do not silently adapt to a mismatched bridge — surfacing the mismatch is more important than the fix being automatic.
-
-If no existing bridge model is available and the user approves creating one (Checkpoint 2b), delegate the bridge model creation to `create_dbt_transformations`. That skill has the standard JSON-array bridge template and the naming convention (`gold_<entity_a>_<entity_b>`, with cube name `<entity_a>_<entity_b>` after dropping `gold_`).
-
-Once the bridge model exists and is materialized, return here and continue:
-
-- Generate a Cube overlay for the bridge as a junction cube with `public: false`.
-- Generate joins in both directions through the bridge.
+If no bridge exists and user approves creating one (Checkpoint 2b), delegate to `create-dbt-transformations`. Once built, generate a bridge cube with `public: false`.
 
 ---
 
-## Phase 4: Validate Candidate Relationships Before Proposal
-
-### Goal
-
-Verify with SQL that candidate joins actually work and that relationship direction is correct before presenting them to the user for confirmation.
-
-This validation runs against BigQuery directly. It is not a YAML-level check. The point is to verify, with real data, that:
-
-- Candidate primary keys are unique.
-- Foreign keys actually match target primary keys at acceptable rates.
-- Reverse aggregations produce sensible counts.
-- Many-to-many bridge edges resolve correctly on both sides.
-- JSON / array relationships extract to keys that exist in the target table.
-- Join column types are compatible.
-
-Validation must check both directions of every candidate relationship where possible.
-
-If validation cannot be executed because the environment is incomplete, clearly mark validation as pending and explain what must be run later.
-
-Validation queries reference physical tables using the literal `project` and `dataset` values resolved at the start of Phase 2. Substitute `<project>` and `<dataset>` placeholders in the SQL templates below with those literals before executing.
-
-Do not present an unvalidated join as a confirmed relationship.
-
-### 4.1 Validate Key Uniqueness
-
-```sql
-SELECT
-  COUNT(*) AS total_rows,
-  COUNT(DISTINCT <candidate_pk>) AS distinct_keys,
-  COUNT(*) - COUNT(DISTINCT <candidate_pk>) AS duplicate_count
-FROM `<project>.<dataset>.<gold_model>`;
-```
-
-For a primary key, `duplicate_count` should normally be `0`. If duplicates exist, do not mark the column as `primary_key: true` unless there is a clearly documented reason.
-
-### 4.2 Validate Many-to-One Direction
-
-Example relationship: `deals.company_id -> companies.id`.
-
-Expected direction: `deals -> companies: many_to_one`, `companies -> deals: one_to_many`.
-
-Validate the many-to-one side:
-
-```sql
-SELECT
-  COUNT(*) AS total_rows_with_fk,
-  COUNT(c.id) AS matched_rows,
-  COUNT(*) - COUNT(c.id) AS unmatched_rows,
-  ROUND(100.0 * COUNT(c.id) / COUNT(*), 2) AS match_percentage
-FROM `<project>.<dataset>.gold_hubspot_deals` d
-LEFT JOIN `<project>.<dataset>.gold_hubspot_companies` c
-  ON d.company_id = c.id
-WHERE d.company_id IS NOT NULL;
-```
-
-Also check whether a single source row matches multiple target rows:
-
-```sql
-SELECT
-  d.deal_id,
-  COUNT(c.id) AS matched_companies
-FROM `<project>.<dataset>.gold_hubspot_deals` d
-LEFT JOIN `<project>.<dataset>.gold_hubspot_companies` c
-  ON d.company_id = c.id
-WHERE d.company_id IS NOT NULL
-GROUP BY d.deal_id
-HAVING COUNT(c.id) > 1
-LIMIT 20;
-```
-
-For a valid many-to-one relationship, this result should normally be empty.
-
-### 4.3 Validate Reverse One-to-Many Direction
-
-For the same relationship, validate the reverse direction using aggregation:
-
-```sql
-SELECT
-  c.id AS company_id,
-  COUNT(d.deal_id) AS deal_count
-FROM `<project>.<dataset>.gold_hubspot_companies` c
-LEFT JOIN `<project>.<dataset>.gold_hubspot_deals` d
-  ON c.id = d.company_id
-GROUP BY c.id
-ORDER BY deal_count DESC
-LIMIT 20;
-```
-
-Cross-check sampled counts directly from the child/source table:
-
-```sql
-SELECT
-  company_id,
-  COUNT(*) AS expected_deal_count
-FROM `<project>.<dataset>.gold_hubspot_deals`
-WHERE company_id IN (<sample_company_ids>)
-GROUP BY company_id;
-```
-
-The counts must match.
-
-### 4.4 Validate One-to-One Relationships
-
-For a one-to-one relationship, validate uniqueness on both sides:
-
-```sql
-SELECT
-  COUNT(*) AS total_rows,
-  COUNT(DISTINCT <left_key>) AS distinct_left_keys
-FROM `<project>.<dataset>.<left_model>`;
-```
-
-Same for the right side. Then validate the join:
-
-```sql
-SELECT
-  COUNT(*) AS total_rows,
-  COUNT(r.<right_key>) AS matched_rows,
-  COUNT(*) - COUNT(r.<right_key>) AS unmatched_rows,
-  ROUND(100.0 * COUNT(r.<right_key>) / COUNT(*), 2) AS match_percentage
-FROM `<project>.<dataset>.<left_model>` l
-LEFT JOIN `<project>.<dataset>.<right_model>` r
-  ON l.<left_key> = r.<right_key>
-WHERE l.<left_key> IS NOT NULL;
-```
-
-Also validate the reverse direction. If either side has duplicate keys, the relationship is not one-to-one.
-
-### 4.5 Validate Many-to-Many Through Bridge/Junction Models
-
-For a many-to-many relationship, validate both bridge edges.
-
-Example: `companies <-> deals through gold_companies_deals`.
-
-Validate bridge to companies:
+## Phase 4: Validate Candidate Relationships
 
-```sql
-SELECT
-  COUNT(*) AS total_bridge_rows,
-  COUNT(c.id) AS matched_companies,
-  COUNT(*) - COUNT(c.id) AS unmatched_companies,
-  ROUND(100.0 * COUNT(c.id) / COUNT(*), 2) AS match_percentage
-FROM `<project>.<dataset>.gold_companies_deals` b
-LEFT JOIN `<project>.<dataset>.gold_hubspot_companies` c
-  ON b.company_id = c.id
-WHERE b.company_id IS NOT NULL;
-```
-
-Validate bridge to deals (analogous query, swapping `c.id` for `d.id` and the source table).
-
-Validate reverse aggregations from each parent through the bridge:
-
-```sql
-SELECT
-  c.id AS company_id,
-  COUNT(b.deal_id) AS related_deals
-FROM `<project>.<dataset>.gold_hubspot_companies` c
-LEFT JOIN `<project>.<dataset>.gold_companies_deals` b
-  ON c.id = b.company_id
-GROUP BY c.id
-ORDER BY related_deals DESC
-LIMIT 20;
-```
-
-Same query swapped for deals → bridge → companies.
-
-Report sampled counts to the user.
+Verify with SQL against BigQuery that candidate joins work and direction is correct.
 
-### 4.6 Validate JSON / Array Relationships
+For each candidate relationship, validate:
 
-For JSON or array-based relationships, validate extracted keys:
+- Key uniqueness
+- FK match rates (LEFT JOIN + match percentage)
+- Reverse direction aggregation counts
+- Bridge edge integrity (both sides)
+- JSON array extraction match rates
+- Type compatibility via INFORMATION_SCHEMA
 
-```sql
-WITH extracted AS (
-  SELECT DISTINCT
-    src.<source_pk> AS source_id,
-    extracted_id
-  FROM `<project>.<dataset>.<source_model>` src,
-  UNNEST(JSON_VALUE_ARRAY(src.<json_array_column>)) AS extracted_id
-)
+See [references/validation-queries.md](references/validation-queries.md) for all SQL templates.
 
-SELECT
-  COUNT(*) AS total_relationships,
-  COUNT(tgt.<target_pk>) AS matched_relationships,
-  COUNT(*) - COUNT(tgt.<target_pk>) AS unmatched_relationships,
-  ROUND(100.0 * COUNT(tgt.<target_pk>) / COUNT(*), 2) AS match_percentage
-FROM extracted e
-LEFT JOIN `<project>.<dataset>.<target_model>` tgt
-  ON e.extracted_id = tgt.<target_pk>;
-```
-
-Sample the extracted join to inspect actual matched values:
-
-```sql
-WITH extracted AS (
-  SELECT DISTINCT
-    src.<source_pk> AS source_id,
-    extracted_id
-  FROM `<project>.<dataset>.<source_model>` src,
-  UNNEST(JSON_VALUE_ARRAY(src.<json_array_column>)) AS extracted_id
-)
-
-SELECT
-  e.source_id,
-  e.extracted_id,
-  tgt.<target_pk>,
-  tgt.<display_column>
-FROM extracted e
-LEFT JOIN `<project>.<dataset>.<target_model>` tgt
-  ON e.extracted_id = tgt.<target_pk>
-LIMIT 10;
-```
-
-### 4.7 Validate Type Compatibility
-
-Check that join columns have compatible types using INFORMATION_SCHEMA:
-
-```sql
-SELECT column_name, data_type
-FROM `<project>.<dataset>.INFORMATION_SCHEMA.COLUMNS`
-WHERE table_name IN ('<source_model>', '<target_model>')
-  AND column_name IN ('<foreign_key>', '<target_pk>');
-```
+Validate both directions of every candidate relationship where possible.
 
-For JSON / array extracted keys, check the extracted key type against the target key type.
-
-If types differ:
-
-1. Report the mismatch.
-2. Prefer fixing type alignment in the dbt model or approved support model.
-3. Only cast in Cube join SQL when necessary.
-4. Prefer casting the foreign-key side to match the primary-key side.
-
-### 4.8 Validation Rules
-
-Do not present relationships to the user as valid options until:
-
-1. Candidate joins have been validated, or validation is explicitly marked as pending.
-2. Both directions of every candidate relationship have been validated where possible.
-3. Primary key uniqueness has been checked, or marked as pending.
-4. Type compatibility has been checked, or marked as pending.
-5. Match rates are reported, or marked as pending.
-6. Reverse one-to-many aggregations are checked with sampled counts where applicable.
-7. Many-to-many bridge edges are validated in both directions where applicable.
-8. JSON / array relationships have been extracted and validated where present, or marked as pending.
-9. Low match rates or suspicious results are explained.
-10. Sample joined data or sampled aggregate counts look reasonable, or limitations are documented.
+Do not present unvalidated joins as confirmed. If validation cannot run, mark as `validation pending`.
 
 ---
 
-## Phase 5: Present Validated Relationships and Ask for Confirmation
-
-### Goal
-
-Show the detected and validated semantic structure before generating files.
+## Phase 5: Present Validated Relationships
 
-Present selected models, approved connector models, candidate keys, JSON/array keys, connector paths, bridge models, validated joins, cardinality, and validation evidence.
+Present all validated relationships to the user: selected models, approved connectors, keys, joins with cardinality, match rates, and validation evidence.
 
-Example:
-
-```text
-Selected gold models:
-- gold_hubspot_companies
-- gold_hubspot_deals
-
-Approved connector models:
-- gold_hubspot_users
-
-Entity: deals
-Source model: gold_hubspot_deals
-Cube name (overlay): hubspot_deals
-Candidate primary key: deal_id
-Secondary keys: owner_id
-JSON / array keys:
-  - companies -> company_id
-
-Validated joins:
-  - deals.owner_id -> users.user_id (many_to_one)
-    Match rate: 99.8%
-    Reverse direction: users -> deals (one_to_many)
-    Sample reverse counts checked.
-
-  - deals.companies[] -> companies.id through bridge gold_companies_deals
-    Relationship: many_to_many
-    Bridge edges validated.
-```
+Ask user to confirm or modify (Checkpoint 3). Do not generate files until confirmed.
 
-Then ask:
-
-```text
-Please confirm these selected models, connector models, bridge/support models, and validated relationships, or tell me what to change before I generate the semantic overlays.
-```
-
-Do not generate final files until the user confirms or corrects the relationship model.
-
-If a relationship could not be validated but the user still wants to proceed, mark it clearly as an assumption in the generated summary, and tag it with a YAML comment in the generated overlay (see Phase 8).
+If a relationship could not be validated but user proceeds, tag it with `# UNVALIDATED: <reason>` in the generated YAML.
 
 ---
 
 ## Phase 6: Generate Dimensions
 
-### Goal
-
-Expose all selected gold model business columns as Cube dimensions, plus `_airbyte_extracted_at`.
-
-### Type Mapping
-
-```text
-STRING / VARCHAR / TEXT -> string
-INTEGER / FLOAT / NUMERIC / DECIMAL -> number
-BOOLEAN / BOOL -> boolean
-DATE / DATETIME / TIMESTAMP -> time
-JSON / ARRAY / STRUCT -> string or skip only if not queryable directly
-```
-
-### Dimension Rules
-
-1. Include primary keys as dimensions with `primary_key: true`.
-2. Include secondary keys as dimensions.
-3. Include human-readable names and statuses as string dimensions.
-4. Include timestamps as time dimensions.
-5. Include numeric attributes as number dimensions.
-6. Expose every business column from each selected gold model as a dimension by default.
-7. From technical Airbyte columns, include only `_airbyte_extracted_at`. Name the dimension `airbyte_extracted_at` (without the leading underscore) and reference the column as `${CUBE}._airbyte_extracted_at`.
-8. Exclude all other `_airbyte_*` columns by default (`_airbyte_raw_id`, `_airbyte_meta`, `_airbyte_generation_id`, etc.). Do not include them unless the user explicitly asks.
-9. Skip or transform only columns that cannot be represented safely in Cube, and document why.
-10. JSON / array columns used for relationships should usually be represented through bridge/support models.
-
-### Composite Primary Keys
-
-Cube allows exactly one dimension flagged with `primary_key: true` per cube. When a gold model has a composite primary key (multiple columns that together uniquely identify a row, for example `(office_unique_id, month)` in a monthly snapshot model), do not flag any of the component columns as primary key directly. Instead:
-
-1. Keep each component column as a regular dimension (no `primary_key` flag).
-2. Add an additional synthetic primary-key dimension that concatenates the components, using the same `CONCAT` pattern used for bridge cubes.
-
-Example for a monthly active users model with composite PK `(office_unique_id, month)`:
-
-```yaml
-dimensions:
-  id:
-    sql: "CONCAT(${CUBE}.office_unique_id, '-', ${CUBE}.month)"
-    type: string
-    primary_key: true
-
-  office_unique_id:
-    sql: "${CUBE}.office_unique_id"
-    type: string
-
-  month:
-    sql: "${CUBE}.month"
-    type: time
-```
-
-Choose a separator that does not appear in either component value. `-` is usually safe for IDs and dates; if components may contain `-`, use `||` or another unambiguous separator.
-
-The synthetic `id` dimension is the cube's primary key for Cube's purposes. Joins to this cube must reference that synthetic `id`, not individual components — unless the joining cube has the same composite key columns and uses a parallel `CONCAT` in the join SQL.
-
-### Large Column Count Warning
-
-If a model has more than 50 business columns, inform the user before generating:
-
-```text
-This model has <N> business columns. I will generate <N> dimensions by default,
-plus airbyte_extracted_at if the column exists.
-Other _airbyte_* columns will be excluded by default.
-
-Proceed with all business columns, or should I skip any groups?
-```
+Expose all business columns from each selected gold model as Cube dimensions.
 
-Do not skip business columns without explicit user instruction.
+Key rules:
 
-Example dimensions block:
+1. Include PKs with `primary_key: true`. For composite PKs, use a synthetic `CONCAT` dimension — see [references/cube-examples.md](references/cube-examples.md), Composite Primary Key section.
+2. Include secondary keys, names, statuses, timestamps, and numeric attributes as dimensions.
+3. From `_airbyte_*` columns, include only `_airbyte_extracted_at` as dimension `airbyte_extracted_at` (reference as `${CUBE}._airbyte_extracted_at`). Exclude all other `_airbyte_*` columns.
+4. Do not remove business columns just because they don't look immediately useful.
+5. JSON/array columns used for relationships should be represented through bridge models.
+6. If a model has >50 business columns, ask user before generating.
 
-```yaml
-dimensions:
-  deal_id:
-    sql: "${CUBE}.deal_id"
-    type: string
-    primary_key: true
-
-  company_id:
-    sql: "${CUBE}.company_id"
-    type: string
-
-  deal_name:
-    sql: "${CUBE}.deal_name"
-    type: string
-
-  amount:
-    sql: "${CUBE}.amount"
-    type: number
-
-  created_at:
-    sql: "${CUBE}.created_at"
-    type: time
-
-  airbyte_extracted_at:
-    sql: "${CUBE}._airbyte_extracted_at"
-    type: time
-```
+See [references/cube-examples.md](references/cube-examples.md) for type mapping and dimension examples.
 
 ---
 
 ## Phase 7: Suggest and Confirm Measures
 
-### Goal
-
-Create default and useful measures without inventing ambiguous business logic.
-
-### Default Measure
-
-Always include a row count measure unless project convention says otherwise.
-
-```yaml
-measures:
-  count:
-    type: count
-```
-
-### Suggested Measures
-
-Suggest useful measures based on model schema and column names.
-
-Common suggestions:
-
-```text
-amount -> total_amount (sum), average_amount (avg), min_amount, max_amount
-revenue -> total_revenue (sum)
-price -> total_price or average_price depending on context
-cost -> total_cost (sum)
-quantity / qty -> total_quantity (sum)
-duration -> total_duration or average_duration
-deal_id / company_id / user_id -> count_distinct
-created_at -> first_created_at (min), last_created_at (max)
-closed_at -> first_closed_at (min), last_closed_at (max)
-updated_at -> last_updated_at (max)
-```
-
-Examples:
-
-```yaml
-measures:
-  total_amount:
-    sql: "${CUBE}.amount"
-    type: sum
-
-  unique_companies:
-    sql: "${CUBE}.company_id"
-    type: count_distinct
-
-  last_closed_at:
-    sql: "${CUBE}.closed_at"
-    type: max
-```
-
-### `count_distinct` on Foreign Keys
-
-`count_distinct` on a foreign-key column counts unique values within the cube that owns the FK. For example, `count_distinct(company_id)` inside the `hubspot_deals` cube answers "how many distinct companies have deals."
-
-Be careful when this measure is used in queries that join multiple cubes. Joins can produce row fan-out, which inflates or distorts distinct counts. Rules:
-
-1. Define `count_distinct` on FK columns inside the cube that owns the FK.
-2. Add a brief description in the measure (or note it for the user) so consumers know the cube boundary.
-3. Avoid suggesting `count_distinct` on FK columns inside the parent cube (the cube that owns the PK) — `count` of rows there usually answers the same question more reliably.
-
-### Measure Confirmation
-
-After suggesting measures, ask the user:
-
-```text
-I will create the default count measure.
-
-I also found these possible additional measures:
-- total_amount: sum(amount)
-- average_amount: avg(amount)
-- unique_companies: count_distinct(company_id)
-- last_closed_at: max(closed_at)
-
-Which of these should I include?
-
-Do you want to define any custom measures?
-```
-
-### Custom Measures
-
-If the user requests a custom measure, ask for a precise definition if needed.
-
-A custom measure definition should include:
-
-```text
-measure name
-source column(s)
-aggregation type
-filters or conditions
-business meaning
-```
-
-Custom measure rules:
-
-1. Do not create ambiguous custom measures.
-2. If the definition is clear, generate the measure.
-3. If the definition is unclear, ask for clarification.
-4. If the requested measure cannot be created from available columns, explain why and list the missing data.
+1. Always include a default `count` measure.
+2. Suggest useful additional measures based on column names. See [references/cube-examples.md](references/cube-examples.md), Measure Suggestions section.
+3. `count_distinct` on FK columns: define inside the cube that owns the FK, not the parent cube. Joins produce fan-out that distorts distinct counts.
+4. Ask user to confirm suggested measures or define custom ones (Checkpoint 4).
+5. If custom measure definition is unclear, ask for clarification.
 
 ---
 
 ## Phase 8: Generate Cube Semantic Overlays
 
-### Goal
-
-Create Cube.dev semantic overlay YAML files for selected and approved gold models.
-
-### Output Location and File Naming
-
-Create files under `semantic/cubes/`. The overlay file name strips the `gold_` prefix. The cube `name` matches the file name (without extension).
-
-Examples:
-
-```text
-gold_hubspot_deals.sql      -> semantic/cubes/hubspot_deals.yml      (name: hubspot_deals)
-gold_hubspot_companies.sql  -> semantic/cubes/hubspot_companies.yml  (name: hubspot_companies)
-gold_companies_deals.sql    -> semantic/cubes/companies_deals.yml    (name: companies_deals)
-gold_clients.sql            -> semantic/cubes/clients.yml            (name: clients)
-```
-
-### Required Source Reference
-
-Use the fully qualified BigQuery table reference. Resolve env variables to literals via `echo` (see "Cube `sql_table` Reference" section above).
-
-Example:
-
-```yaml
-sql_table: "`revos-dev.revos_1737556292084.gold_hubspot_deals`"
-```
-
-### Overlay Style
+Create Cube.dev YAML files in `semantic/`. Follow the existing style detected in Phase 1.
 
-Follow the existing `semantic/cubes/` style detected in Phase 1.
+Key rules:
 
-If existing overlays use map-style `dimensions`, `measures`, and `joins`, use map-style.
+1. File name = cube `name` (no `gold_` prefix) + `.yml`.
+2. `sql_table` uses fully qualified BigQuery reference with `gold_` prefix.
+3. Every confirmed relationship gets joins in both directions.
+4. Bridge/junction cubes use `public: false`.
+5. Every overlay includes `refresh_key`. Prefer `_airbyte_extracted_at`, fall back to other timestamps, then `every: 1 hour`.
+6. `refresh_key.sql` references the same table as `sql_table`.
+7. Tag unvalidated joins with `# UNVALIDATED: <reason>`.
 
-If existing overlays use `extends:` to inherit from base cubes, follow that convention. Otherwise, generate self-contained cubes.
-
-### Canonical Example: Standard Cube
-
-A complete example of a standard (non-bridge) cube:
-
-```yaml
-cubes:
-  - name: hubspot_companies
-    sql_table: "`revos-dev.revos_1737556292084.gold_hubspot_companies`"
-
-    joins:
-      companies_deals:
-        sql: "${CUBE}.id = ${companies_deals}.company_id"
-        relationship: one_to_many
-
-    measures:
-      count:
-        type: count
-
-      total_deal_value:
-        sql: "${CUBE}.properties_hs_total_deal_value"
-        type: sum
-
-      num_open_deals:
-        sql: "${CUBE}.properties_hs_num_open_deals"
-        type: sum
-
-    dimensions:
-      id:
-        sql: "${CUBE}.id"
-        type: string
-        primary_key: true
-
-      airbyte_extracted_at:
-        sql: "${CUBE}._airbyte_extracted_at"
-        type: time
-
-    refresh_key:
-      sql: "SELECT MAX(_airbyte_extracted_at) FROM `revos-dev.revos_1737556292084.gold_hubspot_companies`"
-```
-
-Notes:
-
-1. Top-level `cubes:` array is required.
-2. Cube `name` is `hubspot_companies` (no `gold_` prefix).
-3. `sql_table` references `gold_hubspot_companies` (with `gold_` prefix), in backticks.
-4. The join references `${companies_deals}` — the cube name of a bridge cube defined in `semantic/cubes/companies_deals.yml`.
-5. Only `_airbyte_extracted_at` is exposed from Airbyte metadata, as `airbyte_extracted_at`.
-6. `refresh_key.sql` uses the same fully qualified table name as `sql_table`.
-
-### Unvalidated Joins
-
-If the user chose to proceed with a join that could not be validated, generate the join but tag it with a YAML comment:
-
-```yaml
-joins:
-  hubspot_companies:
-    # UNVALIDATED: match rate could not be measured because gold_hubspot_companies was not yet materialized in BigQuery
-    sql: "${CUBE}.company_id = ${hubspot_companies}.id"
-    relationship: many_to_one
-```
-
-Use a short, factual reason after `UNVALIDATED:`.
-
----
-
-## Cube Overlay Requirements
-
-### Joins
-
-Every confirmed relationship must be represented in both directions where both cubes exist.
-
-Direct many-to-one example:
-
-```yaml
-# In hubspot_deals.yml
-joins:
-  hubspot_companies:
-    sql: "${CUBE}.company_id = ${hubspot_companies}.id"
-    relationship: many_to_one
-```
-
-Reverse one-to-many:
-
-```yaml
-# In hubspot_companies.yml
-joins:
-  hubspot_deals:
-    sql: "${CUBE}.id = ${hubspot_deals}.company_id"
-    relationship: one_to_many
-```
-
-Connector path. For `products -> clients -> addresses`, create joins for each edge in both directions:
-
-```yaml
-# In products.yml
-joins:
-  clients:
-    sql: "${CUBE}.client_id = ${clients}.id"
-    relationship: many_to_one
-```
-
-```yaml
-# In clients.yml
-joins:
-  products:
-    sql: "${CUBE}.id = ${products}.client_id"
-    relationship: one_to_many
-
-  addresses:
-    sql: "${CUBE}.id = ${addresses}.client_id"
-    relationship: one_to_many
-```
-
-```yaml
-# In addresses.yml
-joins:
-  clients:
-    sql: "${CUBE}.client_id = ${clients}.id"
-    relationship: many_to_one
-```
-
-Bridge join. The bridge cube joins to both parents:
-
-```yaml
-# In companies_deals.yml
-joins:
-  hubspot_companies:
-    relationship: many_to_one
-    sql: "${CUBE}.company_id = ${hubspot_companies}.id"
-
-  hubspot_deals:
-    relationship: many_to_one
-    sql: "${CUBE}.deal_id = ${hubspot_deals}.id"
-```
-
-Reverse joins from each parent to the bridge:
-
-```yaml
-# In hubspot_companies.yml
-joins:
-  companies_deals:
-    sql: "${CUBE}.id = ${companies_deals}.company_id"
-    relationship: one_to_many
-```
-
-```yaml
-# In hubspot_deals.yml
-joins:
-  companies_deals:
-    sql: "${CUBE}.id = ${companies_deals}.deal_id"
-    relationship: one_to_many
-```
-
-Join rules:
-
-1. Use validated keys.
-2. Use the correct relationship direction from the current cube.
-3. Generate both directions for every confirmed relationship.
-4. Reference other cubes by their cube `name` (without `gold_` prefix) in `${...}`.
-5. Prefer joins between gold cubes.
-6. Keep join SQL readable and explicit.
-7. If key casting is required, prefer fixing it in the dbt model or support model first.
-8. Tag unvalidated joins with `# UNVALIDATED: <reason>` instead of silently emitting them.
-9. For JSON / array relationships, use or create approved bridge/support models (creation delegated to `create_dbt_transformations`).
-10. For connector paths, join through the connector model instead of inventing a direct join.
-11. Bridge and junction cubes should use `public: false` where the project convention supports it.
-
-### Bridge / Junction Cubes
-
-Bridge and junction cubes should use `public: false` where the project convention supports it.
-
-Example:
-
-```yaml
-cubes:
-  - name: companies_deals
-    sql_table: "`revos-dev.revos_1737556292084.gold_companies_deals`"
-    public: false
-
-    joins:
-      hubspot_companies:
-        relationship: many_to_one
-        sql: "${CUBE}.company_id = ${hubspot_companies}.id"
-
-      hubspot_deals:
-        relationship: many_to_one
-        sql: "${CUBE}.deal_id = ${hubspot_deals}.id"
-
-    measures:
-      count:
-        type: count
-
-    dimensions:
-      id:
-        sql: "CONCAT(${CUBE}.deal_id, '-', ${CUBE}.company_id)"
-        type: string
-        primary_key: true
-
-      deal_id:
-        sql: "${CUBE}.deal_id"
-        type: string
-
-      company_id:
-        sql: "${CUBE}.company_id"
-        type: string
-
-      airbyte_extracted_at:
-        sql: "${CUBE}._airbyte_extracted_at"
-        type: time
-
-    refresh_key:
-      sql: "SELECT MAX(_airbyte_extracted_at) FROM `revos-dev.revos_1737556292084.gold_companies_deals`"
-```
-
-If the bridge model does not have `_airbyte_extracted_at`, omit that dimension and use the default time-based refresh key:
-
-```yaml
-refresh_key:
-  every: 1 hour
-```
-
-### Refresh Key
-
-Every generated Cube overlay must include `refresh_key`.
-
-Priority order:
-
-1. If the gold model has `_airbyte_extracted_at`, use it for a SQL-based refresh key.
-2. If the gold model has another reliable ingestion or update timestamp (`updated_at`, `modified_at`, `loaded_at`, `synced_at`), use that.
-3. Otherwise use the default time-based refresh key.
-
-`refresh_key.sql` must reference the same fully qualified BigQuery table as the cube's own `sql_table`. Never a different table.
-
-Airbyte refresh key:
-
-```yaml
-refresh_key:
-  sql: "SELECT MAX(_airbyte_extracted_at) FROM `<project>.<dataset>.<gold_model>`"
-```
-
-Other timestamp-based refresh key:
-
-```yaml
-refresh_key:
-  sql: "SELECT MAX(updated_at) FROM `<project>.<dataset>.<gold_model>`"
-```
-
-Default:
-
-```yaml
-refresh_key:
-  every: 1 hour
-```
-
-Refresh key rules:
-
-1. Always include `refresh_key` in every generated Cube overlay.
-2. Prefer `_airbyte_extracted_at` when it exists.
-3. Prefer reliable timestamp-based refresh keys over fixed interval refresh keys.
-4. Use `every: 1 hour` as the default fallback.
-5. If a timestamp column exists but is not reliable, explain the assumption and use the default fallback.
-6. Bridge and junction cubes must also include `refresh_key`.
-7. `refresh_key.sql` must reference the same fully qualified BigQuery table as the cube's own `sql_table`.
+See [references/cube-examples.md](references/cube-examples.md) for canonical standard cube, bridge cube, join direction examples, and refresh key variants.
 
 ---
 
 ## Phase 9: Validate Generated Files
 
-### Goal
-
-Validate generated semantic files and any approved support models.
-
-### dbt Validation
-
-If `create_dbt_transformations` was invoked during this run (for example, to create a bridge model), it has already validated the new dbt models with `revos dbt run` and `revos dbt test`. No re-validation is needed here for those.
-
-If only existing gold models were used, run a basic syntax check via the dbt skill's standard command:
-
-```bash
-revos dbt parse
-```
-
-### Verify Physical Tables Exist in BigQuery
-
-For each generated cube, confirm the physical table referenced in `sql_table` actually exists in BigQuery. Cube does not catch a missing table at YAML parse time; it only fails at first query.
-
-```bash
-bq show <project>:<dataset>.<table_name>
-```
-
-Example:
-
-```bash
-bq show revos-dev:revos_1737556292084.gold_hubspot_companies
-```
-
-If the table does not exist, the gold model is not yet materialized. Either materialize it first (run the gold dbt build via `create_dbt_transformations`), or document this as a pending item before handing the overlay over.
-
-### Semantic Validation
-
-Run available project commands to validate Cube YAML.
-
-Placeholder:
-
-```bash
-<cube-validation-command>
-```
-
-### Manual Validation Checklist
-
-1. Semantic overlays were created in `semantic/cubes/`.
-2. Overlay file names drop the `gold_` prefix (`hubspot_companies.yml`, not `gold_hubspot_companies.yml`).
-3. Each cube `name` matches the overlay file name (without extension) and has no `gold_` prefix.
-4. Cube overlays reference the gold tables using `sql_table: "`<project>.<dataset>.gold_<entity>`"`.
-5. The physical table name in `sql_table` keeps the `gold_` prefix.
-6. No `dbt ref()` syntax is used in new overlays.
-7. Each cube has dimensions.
-8. Every business column from each selected gold model is exposed as a dimension unless documented otherwise.
-9. From `_airbyte_*` columns, only `_airbyte_extracted_at` is exposed (as dimension `airbyte_extracted_at`); other `_airbyte_*` columns are excluded.
-10. Each cube has a `count` measure.
-11. Suggested and approved additional measures are included.
-12. Ambiguous custom measures are not created without clarification.
-13. `count_distinct` measures on FK columns are defined inside the cube that owns the FK, not the parent cube.
-14. Primary keys are marked correctly. For composite primary keys, a synthetic `CONCAT(...)` dimension is the one flagged with `primary_key: true`; individual components are kept as regular dimensions.
-15. Secondary keys are preserved as dimensions.
-16. JSON arrays use `UNNEST(JSON_VALUE_ARRAY(...))`.
-17. JSON / array relationship keys are extracted into approved bridge/support models where needed.
-18. Bridge and junction cubes use `public: false` where the project convention supports it.
-19. Each cube has a `refresh_key`.
-20. Cubes with `_airbyte_extracted_at` use it in a SQL-based `refresh_key` where possible.
-21. Cubes without `_airbyte_extracted_at` but with another reliable timestamp use that timestamp where possible.
-22. Cubes without a reliable timestamp use `every: 1 hour`.
-23. `refresh_key.sql` references the same fully qualified BigQuery table as the cube's own `sql_table`.
-24. Joins use validated columns.
-25. Joins reference other cubes by their cube `name` (without `gold_` prefix), e.g. `${hubspot_companies}`.
-26. Join relationship types are correct from the current cube perspective.
-27. Every confirmed relationship has joins in both directions where both cubes exist.
-28. Candidate joins were validated before being proposed to the user.
-29. Join validation was performed in both directions where possible.
-30. Reverse one-to-many aggregation checks were performed where applicable.
-31. Many-to-many bridge edges were validated in both directions where applicable.
-32. Joins that could not be validated and were generated anyway are tagged with `# UNVALIDATED: <reason>`.
-33. Connector models are only used if the user approved them.
-34. Selected models that remain disconnected are documented.
-35. Physical tables referenced by `sql_table` exist in BigQuery, or the missing tables are clearly listed as pending items.
-36. Placeholder commands or assumptions are clearly marked.
+1. If `create-dbt-transformations` was invoked (bridge model), it already validated dbt models. Otherwise run `revos dbt parse`.
+2. Verify physical tables exist in BigQuery: `bq show <dataset>.<table_name>`. If missing, document as pending.
+3. Verify generated overlays match conventions: flat YAML, correct naming, correct `sql_table`, all dimensions present, `refresh_key` included, joins in both directions.
 
 ---
 
 ## Final Response Format
 
-After generation, summarize what was created:
-
 ```text
 Created semantic model draft.
 
 Selected gold models:
 - dbt/models/gold/<gold_model_1>.sql
-- dbt/models/gold/<gold_model_2>.sql
 
 Approved connector models:
 - dbt/models/gold/<connector_model>.sql
 
-Bridge/support models created during this run (via create_dbt_transformations):
+Bridge/support models created (via create-dbt-transformations):
 - dbt/models/gold/<bridge_model>.sql
 
 Semantic overlays:
-- semantic/cubes/<entity_1>.yml         (cube name: <entity_1>)
-- semantic/cubes/<entity_2>.yml         (cube name: <entity_2>)
-- semantic/cubes/<bridge_entity>.yml    (cube name: <bridge_entity>, public: false)
+- semantic/<entity_1>.yml         (cube name: <entity_1>)
+- semantic/<bridge_entity>.yml    (cube name: <bridge_entity>, public: false)
 
-Detected and validated relationships:
+Validated relationships:
 - <entity_a>.<key> -> <entity_b>.<key> (<relationship_type>)
-- <entity_b>.<key> -> <entity_a>.<key> (<reverse_relationship_type>)
-
-Connector paths:
-- <selected_entity_a> -> <connector_entity> -> <selected_entity_b>
-
-Bridge relationships:
-- <source_entity>.<json_array_column>[] -> <target_entity>.<target_key> through <bridge_entity>
 
 Measures:
 - count
 - <approved_measure_1>
-- <approved_measure_2>
 
 Validation:
-- dbt validation: <passed / failed / pending / not run — only existing models used>
-- physical table existence in BigQuery: <passed / failed / pending>
-- join candidate validation before proposal: <passed / failed / pending>
-- reverse join validation: <passed / failed / pending>
-- semantic validation: <passed / failed / pending>
+- dbt: <passed / pending / not run>
+- physical tables: <passed / pending>
+- join validation: <passed / pending>
+- semantic validation: <passed / pending>
 
-Unvalidated joins (tagged in YAML with # UNVALIDATED):
+Unvalidated joins (tagged with # UNVALIDATED):
 - <cube>.<join_target>: <reason>
 
 Assumptions:
-- <assumption_1>
-- <assumption_2>
+- <assumption>
 
 Pending items:
-- <pending_item_1>
-- <pending_item_2>
+- <pending_item>
+
+Next step:
+  revos overlays push -d ./semantic
 ```
 
 If validation is incomplete, say exactly what remains pending.