@revos/cli 0.1.2 → 0.1.3

package/dist/templates/skills/create-dbt-transformations/references/edge-cases.md

@@ -0,0 +1,46 @@
+# Edge Cases
+
+## User asks for a model but does not provide the SQL
+
+```text
+I can scaffold the model file and schema entry, but I need to know what the
+transformation should produce.
+
+Could you tell me:
+1. What source model(s) or tables does it read from?
+2. What columns should it expose, and what is the primary key?
+3. Any filtering, aggregation, or join logic?
+
+Alternatively, if this is a bridge model from a JSON array, I can apply the
+standard bridge template — just tell me the source model and the JSON column.
+```
+
+## User asks for a "quick" or "simple" model without details
+
+Same response as above. Do not invent business logic.
+
+## Model depends on another model that does not exist yet
+
+```text
+The transformation you described references `<missing_model>`, which does not
+exist in dbt/models/. Should I create that model first?
+```
+
+## Source is a raw Airbyte table not yet declared as a dbt source
+
+Declare it as a source in `dbt/models/bronze/schema.yml` first (see [schema-conventions.md](schema-conventions.md)), then reference it with `{{ source('raw', '<table>') }}` in the bronze model SQL. Do not use fully qualified BigQuery names directly — that bypasses dbt's dependency graph and source freshness tracking.
+
+## run fails
+
+1. Show the error verbatim — do not paraphrase warehouse errors.
+2. Offer to fix the SQL based on the error message.
+3. Do not proceed to `revos dbt test` until run succeeds.
+
+## test fails
+
+Show which test failed and explain the likely cause:
+
+- `unique` on PK fails → real duplicates exist; PK detection was wrong or source has unexpected duplicates needing `DISTINCT` or `ROW_NUMBER` dedup.
+- `not_null` on a column fails → source has nulls; either the column is genuinely nullable (remove the test) or filter them out in SQL.
+
+Ask the user how to proceed.

package/dist/templates/skills/create-dbt-transformations/references/schema-conventions.md

@@ -0,0 +1,128 @@
+# schema.yml Conventions
+
+## Contents
+
+- [Declaring Sources (bronze layer)](#declaring-sources-bronze-layer)
+- [Standard Model Entry](#standard-model-entry)
+- [Composite Primary Keys (Bridge Models)](#composite-primary-keys-bridge-models)
+- [Description Guidelines](#description-guidelines)
+- [Foreign Key Tests](#foreign-key-tests)
+
+---
+
+Each layer has one shared `schema.yml` at `dbt/models/<layer>/schema.yml`. Append new models; do not create per-model files.
+
+If the file does not exist, create it with:
+
+```yaml
+version: 2
+
+models:
+```
+
+## Declaring Sources (bronze layer)
+
+Raw tables must be declared as dbt sources before they can be referenced with `{{ source() }}`. Sources live in `dbt/models/bronze/schema.yml` under a `sources:` block alongside the `models:` block.
+
+`schema` maps to the BigQuery dataset (`REVOS_BQ_DATASET`):
+
+```yaml
+version: 2
+
+sources:
+  - name: raw
+    schema: "{{ env_var('REVOS_BQ_DATASET') }}"
+    tables:
+      - name: hubspot_contacts
+      - name: hubspot_deals
+      - name: stripe_charges
+
+models:
+  - name: bronze_hubspot_contacts
+    ...
+```
+
+Rules:
+
+- Use `raw` as the source name for all Airbyte-ingested tables.
+- Each raw table referenced in bronze SQL needs a corresponding entry under `tables:`.
+- If the source block already exists, append to the `tables:` list only.
+
+## Standard Model Entry
+
+```yaml
+- name: <model_name>
+  description: |
+    <1–2 sentences: what entity/relationship, what source, any non-obvious logic>
+  columns:
+    - name: <pk_column>
+      description: "Primary key."
+      tests:
+        - not_null
+        - unique
+
+    - name: <fk_column>
+      description: "Foreign key to <target_entity>."
+      tests:
+        - not_null
+```
+
+## Composite Primary Keys (Bridge Models)
+
+If `dbt-utils` is available:
+
+```yaml
+- name: <model_name>
+  description: |
+    <description>
+  tests:
+    - dbt_utils.unique_combination_of_columns:
+        combination_of_columns:
+          - <pk_col_1>
+          - <pk_col_2>
+  columns:
+    - name: <pk_col_1>
+      description: "Composite key part."
+      tests:
+        - not_null
+    - name: <pk_col_2>
+      description: "Composite key part."
+      tests:
+        - not_null
+```
+
+If `dbt-utils` is not available, omit `unique_combination_of_columns` and note it in the description:
+
+```yaml
+description: |
+  <description>
+  Note: composite uniqueness on (<pk_col_1>, <pk_col_2>) is not enforced —
+  dbt-utils is not installed in this project.
+```
+
+Check availability:
+
+```bash
+grep -A2 "packages:" dbt/packages.yml 2>/dev/null | grep dbt-utils || echo "dbt-utils not found"
+```
+
+## Description Guidelines
+
+Answer in 1–2 sentences:
+
+1. What entity or relationship this model represents.
+2. What source(s) it reads from.
+3. Any non-obvious filtering or transformation.
+
+Example:
+
+```yaml
+description: |
+  Bridge table linking HubSpot deals to companies, unpacked from the
+  `companies` JSON array on `hubspot_deals`. Excludes deals with no
+  associated companies.
+```
+
+## Foreign Key Tests
+
+Add `not_null` only. Do not add `relationships` tests by default — they require knowing the target model and column, which needs explicit user input.

package/dist/templates/skills/create-dbt-transformations/references/sql-templates.md

@@ -0,0 +1,73 @@
+# SQL Templates
+
+## Bronze Model (source reference)
+
+Bronze models read raw Airbyte-ingested tables via `{{ source() }}`:
+
+```sql
+SELECT
+  <pk_column>,
+  <business_columns>,
+  _airbyte_extracted_at
+FROM {{ source('raw', '<raw_table_name>') }}
+```
+
+Ensure the source is declared in `dbt/models/bronze/schema.yml` before using it (see `references/schema-conventions.md`).
+
+## Standard Silver / Gold Model
+
+```sql
+SELECT
+  <pk_column>,
+  <business_columns>,
+  _airbyte_extracted_at
+FROM {{ ref('<source_model>') }}
+WHERE <filtering_conditions>
+```
+
+## Bridge Model (JSON Array)
+
+When unpacking a JSON array into a many-to-many bridge table:
+
+```sql
+SELECT DISTINCT
+  d.id              AS <entity_a>_id,
+  <entity_b>_id,
+  d._airbyte_extracted_at
+FROM {{ ref('<source_model>') }} d,
+UNNEST(JSON_VALUE_ARRAY(d.<json_array_column>)) AS <entity_b>_id
+WHERE d.<json_array_column> IS NOT NULL
+```
+
+Concrete example (`gold_deals_companies.sql`, unpacking `companies` array on `hubspot_deals`):
+
+```sql
+SELECT DISTINCT
+  d.id                      AS deal_id,
+  company_id,
+  d._airbyte_extracted_at
+FROM {{ ref('hubspot_deals') }} d,
+UNNEST(JSON_VALUE_ARRAY(d.companies)) AS company_id
+WHERE d.companies IS NOT NULL
+```
+
+Notes:
+
+1. `SELECT DISTINCT` — a single source row can produce duplicate combinations under some Airbyte sync patterns.
+2. `WHERE d.<json_array_column> IS NOT NULL` is required — `UNNEST(JSON_VALUE_ARRAY(NULL))` is unsafe.
+3. `_airbyte_extracted_at` is preserved for downstream freshness checks.
+4. Composite PK: `(<entity_a>_id, <entity_b>_id)`.
+
+## Bridge Model Naming
+
+Convention: `<entity_a>_<entity_b>` (no `to_`, no `bridge_` prefix). Alphabetical order unless one entity clearly owns the relationship.
+
+Examples: `gold_deals_companies`, `gold_deals_contacts`, `gold_companies_contacts`.
+
+## SQL Content Rules
+
+1. No `{{ config(materialized=...) }}` unless user asks to override the layer default.
+2. `{{ source('raw', '<table>') }}` for raw source tables in bronze models.
+3. `{{ ref('<model>') }}` for references to other dbt models.
+4. Named CTEs for non-trivial logic, explicit column lists where practical.
+5. Preserve `_airbyte_extracted_at` from Airbyte-ingested sources.