@revos/cli 0.2.1 → 0.2.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (162) hide show
  1. package/README.md +289 -77
  2. package/dist/adapters/oclif/commands/action-runs/get.mjs +1 -1
  3. package/dist/adapters/oclif/commands/action-runs/list.mjs +8 -2
  4. package/dist/adapters/oclif/commands/actions/get-input-schema.mjs +2 -2
  5. package/dist/adapters/oclif/commands/actions/get-params-schema.mjs +2 -2
  6. package/dist/adapters/oclif/commands/actions/get.mjs +1 -1
  7. package/dist/adapters/oclif/commands/actions/list.mjs +8 -4
  8. package/dist/adapters/oclif/commands/ai-instructions/create.mjs +1 -1
  9. package/dist/adapters/oclif/commands/ai-instructions/delete.mjs +1 -1
  10. package/dist/adapters/oclif/commands/ai-instructions/get.mjs +1 -1
  11. package/dist/adapters/oclif/commands/ai-instructions/list.mjs +8 -2
  12. package/dist/adapters/oclif/commands/ai-instructions/update.mjs +1 -1
  13. package/dist/adapters/oclif/commands/api.d.mts +11 -0
  14. package/dist/adapters/oclif/commands/api.mjs +112 -0
  15. package/dist/adapters/oclif/commands/apply.d.mts +29 -0
  16. package/dist/adapters/oclif/commands/apply.mjs +77 -0
  17. package/dist/adapters/oclif/commands/auth/login.d.mts +6 -4
  18. package/dist/adapters/oclif/commands/auth/login.mjs +23 -11
  19. package/dist/adapters/oclif/commands/auth/logout.d.mts +2 -1
  20. package/dist/adapters/oclif/commands/auth/logout.mjs +3 -2
  21. package/dist/adapters/oclif/commands/auth/status.d.mts +4 -2
  22. package/dist/adapters/oclif/commands/auth/status.mjs +23 -3
  23. package/dist/adapters/oclif/commands/connections/create.d.mts +6 -0
  24. package/dist/adapters/oclif/commands/connections/create.mjs +8 -0
  25. package/dist/adapters/oclif/commands/connections/delete.d.mts +6 -0
  26. package/dist/adapters/oclif/commands/connections/delete.mjs +8 -0
  27. package/dist/adapters/oclif/commands/connections/get.d.mts +6 -0
  28. package/dist/adapters/oclif/commands/connections/get.mjs +8 -0
  29. package/dist/adapters/oclif/commands/connections/list.d.mts +6 -0
  30. package/dist/adapters/oclif/commands/connections/list.mjs +14 -0
  31. package/dist/adapters/oclif/commands/connections/update.d.mts +6 -0
  32. package/dist/adapters/oclif/commands/connections/update.mjs +8 -0
  33. package/dist/adapters/oclif/commands/cubes/create.d.mts +6 -0
  34. package/dist/adapters/oclif/commands/cubes/create.mjs +8 -0
  35. package/dist/adapters/oclif/commands/cubes/delete.d.mts +6 -0
  36. package/dist/adapters/oclif/commands/cubes/delete.mjs +8 -0
  37. package/dist/adapters/oclif/commands/cubes/get.d.mts +6 -0
  38. package/dist/adapters/oclif/commands/cubes/get.mjs +8 -0
  39. package/dist/adapters/oclif/commands/cubes/list.d.mts +6 -0
  40. package/dist/adapters/oclif/commands/cubes/list.mjs +13 -0
  41. package/dist/adapters/oclif/commands/cubes/update.d.mts +6 -0
  42. package/dist/adapters/oclif/commands/cubes/update.mjs +8 -0
  43. package/dist/adapters/oclif/commands/diff.d.mts +28 -0
  44. package/dist/adapters/oclif/commands/diff.mjs +66 -0
  45. package/dist/adapters/oclif/commands/gservice-account-keys/get.mjs +1 -1
  46. package/dist/adapters/oclif/commands/gservice-account-keys/reveal.mjs +2 -2
  47. package/dist/adapters/oclif/commands/gservice-accounts/create.mjs +1 -1
  48. package/dist/adapters/oclif/commands/gservice-accounts/delete.mjs +1 -1
  49. package/dist/adapters/oclif/commands/gservice-accounts/get.mjs +1 -1
  50. package/dist/adapters/oclif/commands/gservice-accounts/list.mjs +7 -2
  51. package/dist/adapters/oclif/commands/init.d.mts +3 -1
  52. package/dist/adapters/oclif/commands/init.mjs +27 -23
  53. package/dist/adapters/oclif/commands/org/create.mjs +3 -2
  54. package/dist/adapters/oclif/commands/org/current.d.mts +12 -3
  55. package/dist/adapters/oclif/commands/org/current.mjs +27 -2
  56. package/dist/adapters/oclif/commands/org/get.mjs +3 -2
  57. package/dist/adapters/oclif/commands/org/list.d.mts +3 -11
  58. package/dist/adapters/oclif/commands/org/list.mjs +35 -26
  59. package/dist/adapters/oclif/commands/org/switch.d.mts +4 -2
  60. package/dist/adapters/oclif/commands/org/switch.mjs +16 -3
  61. package/dist/adapters/oclif/commands/pull.d.mts +29 -0
  62. package/dist/adapters/oclif/commands/pull.mjs +88 -0
  63. package/dist/adapters/oclif/commands/score-groups/create.mjs +3 -2
  64. package/dist/adapters/oclif/commands/score-groups/delete.mjs +1 -1
  65. package/dist/adapters/oclif/commands/score-groups/get.mjs +1 -1
  66. package/dist/adapters/oclif/commands/score-groups/list.mjs +3 -2
  67. package/dist/adapters/oclif/commands/score-groups/update.mjs +1 -1
  68. package/dist/adapters/oclif/commands/scores/create.mjs +3 -2
  69. package/dist/adapters/oclif/commands/scores/delete.mjs +1 -1
  70. package/dist/adapters/oclif/commands/scores/list.mjs +3 -2
  71. package/dist/adapters/oclif/commands/scores/update.mjs +1 -1
  72. package/dist/adapters/oclif/commands/segments/create.mjs +1 -1
  73. package/dist/adapters/oclif/commands/segments/delete.mjs +1 -1
  74. package/dist/adapters/oclif/commands/segments/evaluate.mjs +2 -2
  75. package/dist/adapters/oclif/commands/segments/get-evaluation-history.mjs +2 -2
  76. package/dist/adapters/oclif/commands/segments/get-version.mjs +2 -2
  77. package/dist/adapters/oclif/commands/segments/get.mjs +1 -1
  78. package/dist/adapters/oclif/commands/segments/list-versions.mjs +16 -5
  79. package/dist/adapters/oclif/commands/segments/list.mjs +9 -2
  80. package/dist/adapters/oclif/commands/segments/restore-version.mjs +2 -2
  81. package/dist/adapters/oclif/commands/segments/update.mjs +1 -1
  82. package/dist/adapters/oclif/commands/sources/create.d.mts +11 -0
  83. package/dist/adapters/oclif/commands/sources/create.mjs +16 -0
  84. package/dist/adapters/oclif/commands/sources/delete.d.mts +6 -0
  85. package/dist/adapters/oclif/commands/sources/delete.mjs +8 -0
  86. package/dist/adapters/oclif/commands/sources/get.d.mts +6 -0
  87. package/dist/adapters/oclif/commands/sources/get.mjs +8 -0
  88. package/dist/adapters/oclif/commands/sources/list-streams.d.mts +6 -0
  89. package/dist/adapters/oclif/commands/sources/list-streams.mjs +31 -0
  90. package/dist/adapters/oclif/commands/sources/list.d.mts +6 -0
  91. package/dist/adapters/oclif/commands/sources/list.mjs +13 -0
  92. package/dist/adapters/oclif/commands/{integrations/get.d.mts → sources/update.d.mts} +4 -4
  93. package/dist/adapters/oclif/commands/sources/update.mjs +21 -0
  94. package/dist/adapters/oclif/commands/status.d.mts +27 -0
  95. package/dist/adapters/oclif/commands/status.mjs +77 -0
  96. package/dist/adapters/oclif/commands/table-views/create.mjs +3 -2
  97. package/dist/adapters/oclif/commands/table-views/delete.mjs +1 -1
  98. package/dist/adapters/oclif/commands/table-views/list.mjs +3 -2
  99. package/dist/adapters/oclif/commands/table-views/update.mjs +1 -1
  100. package/dist/adapters/oclif/commands/tables/create.mjs +1 -1
  101. package/dist/adapters/oclif/commands/tables/delete.mjs +1 -1
  102. package/dist/adapters/oclif/commands/tables/get.mjs +1 -1
  103. package/dist/adapters/oclif/commands/tables/list.mjs +3 -2
  104. package/dist/adapters/oclif/commands/tables/update.mjs +1 -1
  105. package/dist/{base.command-d7VW6WTp.d.mts → base.command-BmddDbHa.d.mts} +4 -1
  106. package/dist/base.command-D8taHOFF.mjs +83 -0
  107. package/dist/chunk-CfYAbeIz.mjs +13 -0
  108. package/dist/context-D5uelKLe.d.mts +62 -0
  109. package/dist/core-B-IdeRNl.mjs +2448 -0
  110. package/dist/{factory-BrFKT8t-.mjs → factory-CCcimDhl.mjs} +45 -10
  111. package/dist/iac-render-BSZZEP0n.mjs +17 -0
  112. package/dist/index-D0ax2I61.d.mts +581 -0
  113. package/dist/index.d.mts +4 -4
  114. package/dist/index.mjs +2 -2
  115. package/dist/{presets-D9b6IWKy.mjs → presets-Bb9gwgeh.mjs} +40 -8
  116. package/dist/templates/.claude/settings.json +39 -0
  117. package/dist/templates/.devcontainer/Dockerfile +9 -0
  118. package/dist/templates/.devcontainer/devcontainer.json +4 -1
  119. package/dist/templates/.devcontainer/setup.sh +3 -0
  120. package/dist/templates/AGENTS.md +33 -20
  121. package/dist/templates/dbt/dbt_project.yml +2 -2
  122. package/dist/templates/gitignore +3 -1
  123. package/dist/templates/skills/create-connections/SKILL.md +210 -0
  124. package/dist/templates/skills/create-connections/references/mappers.md +152 -0
  125. package/dist/templates/skills/{create-semantic-model → create-cubes}/SKILL.md +20 -18
  126. package/dist/templates/skills/create-cubes/references/bq-pk-fk-conventions.md +183 -0
  127. package/dist/templates/skills/{create-semantic-model → create-cubes}/references/cube-examples.md +2 -2
  128. package/dist/templates/skills/create-cubes/references/hubspot-entities.md +289 -0
  129. package/dist/templates/skills/create-cubes/references/jira-entities.md +201 -0
  130. package/dist/templates/skills/create-cubes/references/netsuite-entities.md +121 -0
  131. package/dist/templates/skills/create-cubes/references/stripe-entities.md +114 -0
  132. package/dist/templates/skills/create-dbt-transformations/SKILL.md +43 -22
  133. package/dist/templates/skills/create-dbt-transformations/references/edge-cases.md +20 -2
  134. package/dist/templates/skills/create-dbt-transformations/references/schema-conventions.md +21 -7
  135. package/dist/templates/skills/create-dbt-transformations/references/sql-templates.md +34 -20
  136. package/dist/templates/skills/explore-lakehouse/SKILL.md +3 -3
  137. package/dist/templates/skills/load-sample-data/SKILL.md +1 -1
  138. package/dist/templates/skills/visualize-semantic-model/SKILL.md +159 -0
  139. package/dist/templates/skills/visualize-semantic-model/scripts/render_graph.py +186 -0
  140. package/dist/{types-Y_ht_ja5.d.mts → types-Bk2Cb5yt.d.mts} +9 -0
  141. package/package.json +44 -7
  142. package/dist/adapters/oclif/commands/integrations/create.d.mts +0 -11
  143. package/dist/adapters/oclif/commands/integrations/create.mjs +0 -16
  144. package/dist/adapters/oclif/commands/integrations/get.mjs +0 -21
  145. package/dist/adapters/oclif/commands/integrations/list.d.mts +0 -11
  146. package/dist/adapters/oclif/commands/integrations/list.mjs +0 -16
  147. package/dist/adapters/oclif/commands/integrations/update.d.mts +0 -15
  148. package/dist/adapters/oclif/commands/integrations/update.mjs +0 -21
  149. package/dist/adapters/oclif/commands/overlays/diff.d.mts +0 -19
  150. package/dist/adapters/oclif/commands/overlays/diff.mjs +0 -80
  151. package/dist/adapters/oclif/commands/overlays/pull.d.mts +0 -15
  152. package/dist/adapters/oclif/commands/overlays/pull.mjs +0 -45
  153. package/dist/adapters/oclif/commands/overlays/push.d.mts +0 -18
  154. package/dist/adapters/oclif/commands/overlays/push.mjs +0 -59
  155. package/dist/adapters/oclif/commands/overlays/status.d.mts +0 -18
  156. package/dist/adapters/oclif/commands/overlays/status.mjs +0 -53
  157. package/dist/base.command-YiwlGlKs.mjs +0 -62
  158. package/dist/core-jpFPylBb.mjs +0 -997
  159. package/dist/index-DD2Vr-pu.d.mts +0 -193
  160. package/dist/types-C_p_6rkj.d.mts +0 -69
  161. /package/dist/templates/skills/{create-semantic-model → create-cubes}/references/key-patterns.md +0 -0
  162. /package/dist/templates/skills/{create-semantic-model → create-cubes}/references/validation-queries.md +0 -0
@@ -0,0 +1,114 @@
1
+ # Stripe Entities Reference
2
+
3
+ ## Table naming
4
+
5
+ Airbyte syncs Stripe tables with a configurable prefix (default: `stripe_`).
6
+ Inspect the BigQuery dataset to confirm:
7
+
8
+ ```sql
9
+ SELECT table_name FROM `<dataset>.INFORMATION_SCHEMA.TABLES`
10
+ WHERE table_name LIKE '%customers%' OR table_name LIKE '%invoices%'
11
+ ORDER BY table_name LIMIT 20;
12
+ ```
13
+
14
+ Throughout this document `<prefix>` is a placeholder for that prefix.
15
+
16
+ ---
17
+
18
+ ## Primary entities
19
+
20
+ | Cube name | BigQuery table | PK | Notes |
21
+ | ----------------------- | ----------------------- | ---- | -------------------------------- |
22
+ | `<prefix>customers` | `<prefix>customers` | `id` | `name` is display name |
23
+ | `<prefix>subscriptions` | `<prefix>subscriptions` | `id` | FK `customer` (→ customers.id) |
24
+ | `<prefix>invoices` | `<prefix>invoices` | `id` | FK `customer`, FK `subscription` |
25
+
26
+ ---
27
+
28
+ ## Relationship graph
29
+
30
+ ```
31
+ customers ──< subscriptions ──< invoices
32
+ └────────────────────────< invoices
33
+ ```
34
+
35
+ - customer → subscriptions: `one_to_many` via `subscriptions.customer = customers.id`
36
+ - customer → invoices: `one_to_many` via `invoices.customer = customers.id`
37
+ - subscription → invoices: `one_to_many` via `invoices.subscription = subscriptions.id`
38
+ - subscription → latest_invoice: `many_to_one` via `subscriptions.latest_invoice = latest_invoice.id`
39
+
40
+ ---
41
+
42
+ ## Standard cube definitions
43
+
44
+ ### customers
45
+
46
+ ```yaml
47
+ name: <prefix>customers
48
+ sql_table: "`<dataset>.<prefix>customers`"
49
+ joins:
50
+ <prefix>subscriptions:
51
+ relationship: one_to_many
52
+ sql: "${CUBE}.id = ${<prefix>subscriptions.customer}"
53
+ <prefix>invoices:
54
+ relationship: one_to_many
55
+ sql: "${CUBE}.id = ${<prefix>invoices.customer}"
56
+ ```
57
+
58
+ ### subscriptions
59
+
60
+ ```yaml
61
+ name: <prefix>subscriptions
62
+ sql_table: "`<dataset>.<prefix>subscriptions`"
63
+ joins:
64
+ <prefix>customers:
65
+ relationship: many_to_one
66
+ sql: "${CUBE}.customer = ${<prefix>customers.id}"
67
+ <prefix>invoices:
68
+ relationship: one_to_many
69
+ sql: "${CUBE}.id = ${<prefix>invoices.subscription}"
70
+ <prefix>latest_invoice:
71
+ relationship: many_to_one
72
+ sql: "${CUBE}.latest_invoice = ${<prefix>latest_invoice.id}"
73
+ ```
74
+
75
+ ### invoices
76
+
77
+ ```yaml
78
+ name: <prefix>invoices
79
+ sql_table: "`<dataset>.<prefix>invoices`"
80
+ joins:
81
+ <prefix>customers:
82
+ relationship: many_to_one
83
+ sql: "${CUBE}.customer = ${<prefix>customers.id}"
84
+ <prefix>subscriptions:
85
+ relationship: many_to_one
86
+ sql: "${CUBE}.subscription = ${<prefix>subscriptions.id}"
87
+ ```
88
+
89
+ ---
90
+
91
+ ## Special cube: latest_invoice
92
+
93
+ `latest_invoice` is an alias for the `invoices` table (public: false) used
94
+ exclusively for the `subscriptions.latest_invoice` FK join. Needed because
95
+ Cube.js does not support two joins to the same table under the same cube name.
96
+
97
+ ```yaml
98
+ name: <prefix>latest_invoice
99
+ sql_table: "`<dataset>.<prefix>invoices`"
100
+ public: false
101
+ joins:
102
+ <prefix>subscriptions:
103
+ relationship: one_to_many
104
+ sql: "${CUBE}.id = ${<prefix>subscriptions.latest_invoice}"
105
+ ```
106
+
107
+ ---
108
+
109
+ ## Common pitfalls
110
+
111
+ 1. **`latest_invoice` must be a separate cube** — subscriptions needs both `invoices` (for all invoices) and `latest_invoice` (for the most recent one). Same physical table, different cube names.
112
+ 2. **FK column names without suffix** — `subscriptions.customer` is the raw Stripe customer ID (not `customer_id`). Same for `invoices.subscription` and `invoices.customer`. Check actual column names in INFORMATION_SCHEMA.
113
+ 3. **Stripe IDs are strings** — all IDs start with a prefix (`cus_`, `sub_`, `in_`, etc.). No casting needed.
114
+ 4. **Timestamps** — Stripe tables from Airbyte use `_airbyte_extracted_at` as the sync timestamp. Use it for `refresh_key`.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: create-dbt-transformations
3
- description: Create new dbt transformations (bronze/silver/gold models) in the RevOS dbt project. Use when asked to create a dbt model, build a transformation, add a new layer model, declare a raw source, or register a new Airbyte-ingested table. Covers dbt project conventions, sources, materialization, schema.yml, and validation commands.
3
+ description: Create new dbt transformations (silver/gold models) in the RevOS dbt project. Use when asked to create a dbt model, build a transformation, add a new layer model, declare a raw source, or register a new raw table. Bronze is source-declarations only — no SQL files. Covers dbt project conventions, sources, materialization, schema.yml, and validation commands.
4
4
  ---
5
5
 
6
6
  # Create dbt Transformations
@@ -22,32 +22,35 @@ Warn the user: "The `explore-lakehouse` skill is not installed — using `bq sho
22
22
  ## Layer Conventions
23
23
 
24
24
  - **gold** — business-ready models exposed for reporting or downstream consumption.
25
- - **silver** — cleaned, deduplicated, type-conformed intermediates.
26
- - **bronze** — thin views over raw source data. References sources via `{{ source() }}`.
25
+ - **silver** — cleaned, deduplicated, type-conformed intermediates. Lowest SQL layer; reads raw data via `{{ source('bronze', '<table>') }}`.
26
+ - **bronze** — **not a SQL layer**. Holds only `dbt/models/bronze/schema.yml`, which declares raw tables as dbt sources. No `.sql` files belong under `dbt/models/bronze/`.
27
27
 
28
28
  When layer is not obvious from context, ask (see Checkpoint 1).
29
29
 
30
30
  ## Sources (bronze layer)
31
31
 
32
- Raw tables ingested by Airbyte are not dbt models. Declare them as dbt sources so bronze models can reference them with `{{ source() }}`.
32
+ Raw tables loaded into the warehouse by your ingestion pipeline are not dbt models. Declare them as dbt sources so silver models can reference them with `{{ source() }}`.
33
33
 
34
34
  Sources are declared in `dbt/models/bronze/schema.yml` under a `sources:` block using `schema` (the BigQuery dataset):
35
35
 
36
36
  ```yaml
37
37
  sources:
38
- - name: raw
38
+ - name: bronze
39
39
  schema: "{{ env_var('REVOS_BQ_DATASET') }}"
40
40
  tables:
41
41
  - name: hubspot_contacts
42
42
  ```
43
43
 
44
- Reference in bronze SQL:
44
+ Reference in silver SQL:
45
45
 
46
46
  ```sql
47
- SELECT * FROM {{ source('raw', 'hubspot_contacts') }}
47
+ -- dbt/models/silver/silver_hubspot_contacts.sql
48
+ SELECT * FROM {{ source('bronze', 'hubspot_contacts') }}
48
49
  ```
49
50
 
50
- See [schema-conventions.md](references/schema-conventions.md) for the full declaration pattern alongside `models:`.
51
+ `{{ source('bronze', 'hubspot_contacts') }}` resolves to `${REVOS_BQ_DATASET}.hubspot_contacts` — the same dataset where raw tables live — so silver has direct access without a bronze SQL view in between.
52
+
53
+ See [schema-conventions.md](references/schema-conventions.md) for the full declaration pattern.
51
54
 
52
55
  ## Materialization
53
56
 
@@ -63,10 +66,12 @@ Materialized table lives at: `$REVOS_BQ_DATASET.<model_name>`
63
66
 
64
67
  **When to use `{{ ref() }}` vs. `{{ source() }}`:**
65
68
 
66
- | Context | Use |
67
- | ----------------------------------- | -------------------------------- |
68
- | dbt SQL → other dbt model | `{{ ref('<model>') }}` |
69
- | dbt SQL → raw source table (bronze) | `{{ source('raw', '<table>') }}` |
69
+ | Context | Use |
70
+ | -------------------------------------------------- | ----------------------------------- |
71
+ | dbt SQL → other dbt model | `{{ ref('<model>') }}` |
72
+ | dbt SQL → raw table (silver reading from `bronze`) | `{{ source('bronze', '<table>') }}` |
73
+
74
+ Silver is the lowest SQL layer — `{{ source('bronze', ...) }}` is used in silver only. Gold reads from silver via `{{ ref() }}`. There are no SQL files in `dbt/models/bronze/`.
70
75
 
71
76
  Always declare raw tables as sources before referencing them. Do not use bare fully qualified names — that bypasses dbt's dependency graph and source freshness tracking.
72
77
 
@@ -89,12 +94,12 @@ dbt build --select path:models/<layer> # entire layer
89
94
 
90
95
  For each transformation (one at a time — do not batch):
91
96
 
92
- 1. Determine the target layer (Checkpoint 1 if unclear).
97
+ 1. Determine the target layer — **silver** or **gold** only (Checkpoint 1 if unclear). Refuse bronze SQL models (see Checkpoint 4).
93
98
  2. Determine the model name.
94
99
  3. Check if that model already exists (Checkpoint 2 if yes).
95
100
  4. Gather source data and transformation logic. For bridge models, apply the bridge template ([sql-templates.md](references/sql-templates.md)).
96
- 5. For bronze models: check if required sources are declared in `dbt/models/bronze/schema.yml`; add them if missing.
97
- 6. Generate `dbt/models/<layer>/<model_name>.sql`.
101
+ 5. If the model reads raw data, ensure each raw table is declared under the `bronze` source in `dbt/models/bronze/schema.yml`; add it if missing.
102
+ 6. Generate `dbt/models/<silver|gold>/<model_name>.sql`. **Never** generate `.sql` files under `dbt/models/bronze/`.
98
103
  7. Detect the primary key (Checkpoint 3 if ambiguous).
99
104
  8. Add model entry to `dbt/models/<layer>/schema.yml` with PK and FK tests. See [schema-conventions.md](references/schema-conventions.md).
100
105
  9. Run `dbt run --select <model_name>` and report result.
@@ -115,8 +120,9 @@ Ask if the layer is not obvious:
115
120
  Which layer should this transformation live in?
116
121
 
117
122
  - gold: business-ready, exposed for reporting or downstream consumption
118
- - silver: cleaned/intermediate, shared across downstream uses
119
- - bronze: close-to-source view over raw data, references sources
123
+ - silver: cleaned/intermediate, reads raw via `{{ source('bronze', ...) }}`
124
+
125
+ (bronze is not a SQL layer — it only holds `schema.yml` source declarations.)
120
126
  ```
121
127
 
122
128
  Layer is obvious when the user explicitly names it.
@@ -148,6 +154,21 @@ I could not unambiguously detect the primary key. Candidates:
148
154
  Which column(s) should be the primary key?
149
155
  ```
150
156
 
157
+ ### Checkpoint 4: Bronze SQL Model Refused
158
+
159
+ If the user explicitly asks to create a bronze SQL model:
160
+
161
+ ```text
162
+ Bronze is not a SQL layer in this project — it only holds source
163
+ declarations in `dbt/models/bronze/schema.yml`. Silver reads raw data
164
+ directly via `{{ source('bronze', '<raw_table>') }}`.
165
+
166
+ Would you like to create this as a silver model instead?
167
+ ```
168
+
169
+ Do not generate any file under `dbt/models/bronze/` other than
170
+ `schema.yml`.
171
+
151
172
  ---
152
173
 
153
174
  ## Primary Key Detection
@@ -169,9 +190,9 @@ A column is a FK candidate if it matches `<entity>_id` where `<entity>` ≠ mode
169
190
 
170
191
  ## Timestamp Column Propagation (Gold Models)
171
192
 
172
- Every gold model **must** propagate at least one timestamp column so downstream Cube overlays can use SQL-based `refresh_key` (see `create-semantic-model` skill). Priority:
193
+ Every gold model **must** propagate at least one timestamp column so downstream cubes can use SQL-based `refresh_key` (see `create-cubes` skill). Priority:
173
194
 
174
- 1. `_airbyte_extracted_at` present on all Airbyte sources; always propagate if available in upstream.
195
+ 1. An ingestion-time column on the raw table (e.g. Airbyte writes `_airbyte_extracted_at`) propagate when present.
175
196
  2. `updated_at` / `modified_at` — CDC-friendly streams.
176
197
  3. `created_at` — insert-only fact tables.
177
198
 
@@ -181,8 +202,8 @@ If the upstream source has none of these, document it in a SQL comment: `-- no t
181
202
 
182
203
  See [sql-templates.md](references/sql-templates.md) for:
183
204
 
184
- - Bronze model template using `{{ source() }}`
185
- - Standard silver/gold model template
205
+ - Standard silver model template (reads raw via `{{ source('bronze', ...) }}`)
206
+ - Standard gold model template (reads silver via `{{ ref() }}`)
186
207
  - Bridge model (JSON array) template with concrete example
187
208
  - Bridge model naming convention and SQL content rules
188
209
 
@@ -201,7 +222,7 @@ See [edge-cases.md](references/edge-cases.md) for: missing SQL details, missing
201
222
  ```text
202
223
  Created dbt transformation: <model_name>
203
224
 
204
- Layer: <bronze | silver | gold>
225
+ Layer: <silver | gold>
205
226
  File: dbt/models/<layer>/<model_name>.sql
206
227
  Materialization: <inherited: table | overridden: <type>>
207
228
  Primary key: <pk_column> (or composite: <col_1>, <col_2>)
@@ -26,9 +26,27 @@ The transformation you described references `<missing_model>`, which does not
26
26
  exist in dbt/models/. Should I create that model first?
27
27
  ```
28
28
 
29
- ## Source is a raw Airbyte table not yet declared as a dbt source
29
+ ## Source is a raw table not yet declared as a dbt source
30
30
 
31
- Declare it as a source in `dbt/models/bronze/schema.yml` first (see [schema-conventions.md](schema-conventions.md)), then reference it with `{{ source('raw', '<table>') }}` in the bronze model SQL. Do not use fully qualified BigQuery names directly — that bypasses dbt's dependency graph and source freshness tracking.
31
+ Declare it under `sources: - name: bronze` in `dbt/models/bronze/schema.yml`
32
+ first (see [schema-conventions.md](schema-conventions.md)), then reference it
33
+ with `{{ source('bronze', '<table>') }}` in the silver model SQL. Do not use
34
+ fully qualified BigQuery names directly — that bypasses dbt's dependency
35
+ graph and source freshness tracking.
36
+
37
+ ## User asks to create a bronze SQL model
38
+
39
+ Refuse and redirect:
40
+
41
+ ```text
42
+ Bronze is not a SQL layer in this project — `dbt/models/bronze/` only
43
+ contains `schema.yml` declaring raw tables as sources. Silver reads raw
44
+ data directly via `{{ source('bronze', '<raw_table>') }}`.
45
+
46
+ Should I create this as a silver model instead?
47
+ ```
48
+
49
+ Do not generate any file under `dbt/models/bronze/` other than `schema.yml`.
32
50
 
33
51
  ## run fails
34
52
 
@@ -10,9 +10,14 @@
10
10
 
11
11
  ---
12
12
 
13
- Each layer has one shared `schema.yml` at `dbt/models/<layer>/schema.yml`. Append new models; do not create per-model files.
13
+ Each SQL layer (silver, gold) has one shared `schema.yml` at
14
+ `dbt/models/<layer>/schema.yml`. Append new models; do not create per-model
15
+ files.
14
16
 
15
- If the file does not exist, create it with:
17
+ The bronze directory is **not** a SQL layer — its `schema.yml` contains only
18
+ source declarations, no `models:` block.
19
+
20
+ If a layer's `schema.yml` does not exist, create it with:
16
21
 
17
22
  ```yaml
18
23
  version: 2
@@ -22,7 +27,9 @@ models:
22
27
 
23
28
  ## Declaring Sources (bronze layer)
24
29
 
25
- Raw tables must be declared as dbt sources before they can be referenced with `{{ source() }}`. Sources live in `dbt/models/bronze/schema.yml` under a `sources:` block alongside the `models:` block.
30
+ `dbt/models/bronze/schema.yml` is the only file in `dbt/models/bronze/`. It
31
+ declares raw tables as dbt sources so that silver models can reference them
32
+ with `{{ source('bronze', '<table>') }}`.
26
33
 
27
34
  `schema` maps to the BigQuery dataset (`REVOS_BQ_DATASET`):
28
35
 
@@ -30,23 +37,30 @@ Raw tables must be declared as dbt sources before they can be referenced with `{
30
37
  version: 2
31
38
 
32
39
  sources:
33
- - name: raw
40
+ - name: bronze
34
41
  schema: "{{ env_var('REVOS_BQ_DATASET') }}"
35
42
  tables:
36
43
  - name: hubspot_contacts
37
44
  - name: hubspot_deals
38
45
  - name: stripe_charges
46
+ ```
47
+
48
+ The corresponding silver model entry lives in `dbt/models/silver/schema.yml`:
49
+
50
+ ```yaml
51
+ version: 2
39
52
 
40
53
  models:
41
- - name: bronze_hubspot_contacts
54
+ - name: silver_hubspot_contacts
42
55
  ...
43
56
  ```
44
57
 
45
58
  Rules:
46
59
 
47
- - Use `raw` as the source name for all Airbyte-ingested tables.
48
- - Each raw table referenced in bronze SQL needs a corresponding entry under `tables:`.
60
+ - Use `bronze` as the source name for all raw tables.
61
+ - Each raw table referenced in silver SQL needs a corresponding entry under `tables:`.
49
62
  - If the source block already exists, append to the `tables:` list only.
63
+ - Do **not** add a `models:` block to `dbt/models/bronze/schema.yml` — bronze contains source declarations only.
50
64
 
51
65
  ## Standard Model Entry
52
66
 
@@ -1,61 +1,74 @@
1
1
  # SQL Templates
2
2
 
3
- ## Bronze Model (source reference)
3
+ ## Bronze: no SQL
4
4
 
5
- Bronze models read raw Airbyte-ingested tables via `{{ source() }}`:
5
+ Bronze is **not** a SQL layer. `dbt/models/bronze/` contains only `schema.yml`,
6
+ which declares raw tables as dbt sources. See
7
+ [schema-conventions.md](schema-conventions.md). Silver models read raw data
8
+ directly via `{{ source('bronze', '<raw_table_name>') }}`.
9
+
10
+ ## Silver Model (reads raw via source)
6
11
 
7
12
  ```sql
8
13
  SELECT
9
14
  <pk_column>,
10
15
  <business_columns>,
11
- _airbyte_extracted_at
12
- FROM {{ source('raw', '<raw_table_name>') }}
16
+ <ingestion_timestamp_column>
17
+ FROM {{ source('bronze', '<raw_table_name>') }}
18
+ WHERE <filtering_conditions>
13
19
  ```
14
20
 
15
- Ensure the source is declared in `dbt/models/bronze/schema.yml` before using it (see `references/schema-conventions.md`).
21
+ The raw table must be declared under `sources: - name: bronze` in
22
+ `dbt/models/bronze/schema.yml` first (see
23
+ [schema-conventions.md](schema-conventions.md)).
16
24
 
17
- ## Standard Silver / Gold Model
25
+ ## Gold Model (reads silver via ref)
18
26
 
19
27
  ```sql
20
28
  SELECT
21
29
  <pk_column>,
22
30
  <business_columns>,
23
- _airbyte_extracted_at
24
- FROM {{ ref('<source_model>') }}
31
+ <ingestion_timestamp_column>
32
+ FROM {{ ref('<silver_model>') }}
25
33
  WHERE <filtering_conditions>
26
34
  ```
27
35
 
28
36
  ## Bridge Model (JSON Array)
29
37
 
30
- When unpacking a JSON array into a many-to-many bridge table:
38
+ When unpacking a JSON array into a many-to-many bridge table, read from the
39
+ silver model that owns the array column:
31
40
 
32
41
  ```sql
33
42
  SELECT DISTINCT
34
43
  d.id AS <entity_a>_id,
35
44
  <entity_b>_id,
36
- d._airbyte_extracted_at
37
- FROM {{ ref('<source_model>') }} d,
45
+ d.<ingestion_timestamp_column>
46
+ FROM {{ ref('<silver_source_model>') }} d,
38
47
  UNNEST(JSON_VALUE_ARRAY(d.<json_array_column>)) AS <entity_b>_id
39
48
  WHERE d.<json_array_column> IS NOT NULL
40
49
  ```
41
50
 
42
- Concrete example (`gold_deals_companies.sql`, unpacking `companies` array on `hubspot_deals`):
51
+ Concrete example (`gold_deals_companies.sql`, unpacking the `companies` array
52
+ on `silver_hubspot_deals`):
43
53
 
44
54
  ```sql
45
55
  SELECT DISTINCT
46
56
  d.id AS deal_id,
47
57
  company_id,
48
58
  d._airbyte_extracted_at
49
- FROM {{ ref('hubspot_deals') }} d,
59
+ FROM {{ ref('silver_hubspot_deals') }} d,
50
60
  UNNEST(JSON_VALUE_ARRAY(d.companies)) AS company_id
51
61
  WHERE d.companies IS NOT NULL
52
62
  ```
53
63
 
64
+ If the silver model for the upstream entity does not exist yet, create it
65
+ first (see edge case: missing upstream model in `edge-cases.md`).
66
+
54
67
  Notes:
55
68
 
56
- 1. `SELECT DISTINCT` — a single source row can produce duplicate combinations under some Airbyte sync patterns.
69
+ 1. `SELECT DISTINCT` — a single source row can produce duplicate combinations under some sync patterns.
57
70
  2. `WHERE d.<json_array_column> IS NOT NULL` is required — `UNNEST(JSON_VALUE_ARRAY(NULL))` is unsafe.
58
- 3. `_airbyte_extracted_at` is preserved for downstream freshness checks.
71
+ 3. Preserve the ingestion timestamp column from upstream for downstream freshness checks.
59
72
  4. Composite PK: `(<entity_a>_id, <entity_b>_id)`.
60
73
 
61
74
  ## Bridge Model Naming
@@ -66,8 +79,9 @@ Examples: `gold_deals_companies`, `gold_deals_contacts`, `gold_companies_contact
66
79
 
67
80
  ## SQL Content Rules
68
81
 
69
- 1. No `{{ config(materialized=...) }}` unless user asks to override the layer default.
70
- 2. `{{ source('raw', '<table>') }}` for raw source tables in bronze models.
71
- 3. `{{ ref('<model>') }}` for references to other dbt models.
72
- 4. Named CTEs for non-trivial logic, explicit column lists where practical.
73
- 5. Preserve `_airbyte_extracted_at` from Airbyte-ingested sources.
82
+ 1. No `{{ config(materialized=...) }}` unless the user asks to override the layer default.
83
+ 2. `{{ source('bronze', '<table>') }}` for raw tables — used **only** in silver models.
84
+ 3. `{{ ref('<model>') }}` for references to other dbt models (gold reads silver this way).
85
+ 4. Never write `.sql` files in `dbt/models/bronze/`.
86
+ 5. Named CTEs for non-trivial logic, explicit column lists where practical.
87
+ 6. Preserve the ingestion timestamp column from raw (e.g. `_airbyte_extracted_at` if Airbyte loaded it) when present.
@@ -4,7 +4,7 @@ description: >
4
4
  Inspect the RevOS BigQuery lakehouse: list datasets and tables, introspect table schemas
5
5
  and column types, preview sample rows, assess data layers (bronze/silver/gold), and check
6
6
  data completeness and null rates. Required companion skill for create-dbt-transformations
7
- and create-semantic-model — load before generating dbt models or semantic overlays to
7
+ and create-cubes — load before generating dbt models or cube definitions to
8
8
  introspect warehouse columns and types. Use when asked to: explore the lakehouse, list
9
9
  BigQuery tables, inspect a table schema, preview data, check raw source tables, assess data
10
10
  quality, check null rates, understand available data, or perform BigQuery schema introspection.
@@ -86,8 +86,8 @@ FROM \`$GOOGLE_CLOUD_PROJECT.$REVOS_BQ_DATASET.<table>\`
86
86
 
87
87
  1. List tables in the org's dataset: `bq ls $REVOS_BQ_DATASET`
88
88
  2. If the dataset is empty (no tables), tell the user:
89
- - They can add data sources by running `revos integrations create` to open the RevOS UI
90
- - They can view existing integrations with `revos integrations list`
89
+ - They can add data sources by running `revos sources create` to open the RevOS UI
90
+ - They can view existing sources with `revos sources list`
91
91
  - Stop here — no further exploration is possible without data
92
92
  3. Infer the data source and domain from table name prefixes (e.g. `salesforce_*`, `stripe_*`, `hubspot_*`)
93
93
  4. Group tables by source/domain
@@ -108,7 +108,7 @@ Tables copied:
108
108
  Next steps:
109
109
  - Run "explore lakehouse" to inspect the data
110
110
  - Run "create dbt transformations" to build bronze/silver/gold models
111
- - Run "create semantic model" to generate Cube overlays
111
+ - Run "create cube" to generate Cube.dev semantic models
112
112
  ```
113
113
 
114
114
  ## Rules
@@ -0,0 +1,159 @@
1
+ ---
2
+ name: visualize-semantic-model
3
+ description: >
4
+ Generate a model-graph.png visualization of Cube.dev semantic models — render the
5
+ cube graph, show relationships between cubes, draw the fact spine, or diagram the
6
+ semantic layer. Use whenever the user mentions visualizing, drawing, diagramming, or
7
+ graphing the semantic model / cube model / cube relationships, even if they don't
8
+ explicitly ask for a PNG. Triggers: "visualize the semantic model", "draw the cube
9
+ graph", "show relationships between cubes", "generate model-graph.png", "diagram the
10
+ semantic layer", "render the cube model". Accepts an optional folder argument
11
+ (defaults to `cubes/`).
12
+ ---
13
+
14
+ # Visualize Semantic Model
15
+
16
+ Render a dark-themed directed graph of a Cube.dev model by parsing cube YAML files,
17
+ detecting the fact spine, then invoking the bundled renderer with a JSON spec.
18
+
19
+ The renderer (`scripts/render_graph.py`) is pure layout + drawing — it takes a graph
20
+ spec on stdin and writes the PNG. Keeping it bundled means every run produces visually
21
+ consistent output and you don't reinvent matplotlib code each time.
22
+
23
+ ---
24
+
25
+ ## Step 1: Resolve the cubes folder
26
+
27
+ If the user passed a folder argument, use it. Otherwise default to `cubes/`. If the
28
+ folder doesn't exist, ask the user where the cube definitions live before proceeding.
29
+
30
+ ```bash
31
+ find <folder> -name "*.yml" -not -name "model-graph.*" | sort
32
+ ```
33
+
34
+ ---
35
+
36
+ ## Step 2: Parse the cube graph
37
+
38
+ For each `.yml` file extract:
39
+
40
+ - `name` — cube name
41
+ - `joins` — map of `target_cube → { relationship, sql }`
42
+ - A short join-key label parsed from each join's `sql`
43
+
44
+ **Join-key label rules.** Strip `${CUBE}.` and `${<target>}.` prefixes. For a single
45
+ equality, use the LHS column. For composite keys (multiple `AND`), join the LHS column
46
+ names with `+`.
47
+
48
+ ```yaml
49
+ sql: "${CUBE}.user_id = ${users}.user_id"
50
+ # → "user_id"
51
+
52
+ sql: "${CUBE}.traffic_source = ${rev}.traffic_source AND ${CUBE}.order_date = ${rev}.order_date"
53
+ # → "traffic_source + order_date"
54
+ ```
55
+
56
+ **Cardinality** (always render edges from the _one_ side to the _many_ side):
57
+
58
+ | Declared on this cube | This cube's side | Other cube's side |
59
+ | --------------------- | ---------------- | ----------------- |
60
+ | `many_to_one` | ∞ | 1 |
61
+ | `one_to_many` | 1 | ∞ |
62
+ | `one_to_one` | 1 | 1 |
63
+
64
+ **Fact-spine detection.** The cube with the most `many_to_one` joins (it holds the FKs)
65
+ is the spine. Tie-break by preferring names containing `enriched`, `fact`, or `items`.
66
+ If no cube has any `many_to_one` joins (e.g. the model is all `one_to_many` from a hub),
67
+ fall back to the cube with the most outgoing joins of any kind. If still ambiguous, ask
68
+ the user which cube to treat as the spine.
69
+
70
+ **Edge cases — stop and tell the user instead of rendering:**
71
+
72
+ - The folder contains fewer than 2 cubes → at least two cubes are needed.
73
+ - No `joins` found anywhere → there are no relationships to draw.
74
+
75
+ ---
76
+
77
+ ## Step 3: Build the graph spec
78
+
79
+ Build a JSON object the renderer understands:
80
+
81
+ ```json
82
+ {
83
+ "title": "Semantic Model — <project_name>",
84
+ "fact_spine": {
85
+ "name": "<fact_cube>",
86
+ "pk": "<pk_col>",
87
+ "fks": ["fk_a", "fk_b", "fk_c", "fk_d"]
88
+ },
89
+ "dimensions": [
90
+ { "name": "<dim_cube>", "pk": "<pk_col>", "extras": ["metric1", "metric2"] }
91
+ ],
92
+ "edges": [
93
+ {
94
+ "from": "<dim_cube>",
95
+ "to": "<fact_cube>",
96
+ "label": "<join_key>",
97
+ "from_card": "1",
98
+ "to_card": "∞"
99
+ }
100
+ ]
101
+ }
102
+ ```
103
+
104
+ `from_card` / `to_card` are the labels rendered at each end of the arrow — `"1"` or
105
+ `"∞"`. For `one_to_one` joins both ends are `"1"`.
106
+
107
+ The arrow always points _from_ dimension _to_ fact, regardless of how the relationship
108
+ was declared on the YAML.
109
+
110
+ ---
111
+
112
+ ## Step 4: Render
113
+
114
+ ```bash
115
+ python3 -c "import matplotlib" 2>/dev/null || python3 -m pip install matplotlib --quiet
116
+
117
+ python3 .claude/skills/visualize-semantic-model/scripts/render_graph.py \
118
+ --output <folder>/model-graph.png \
119
+ <<'EOF'
120
+ { ... the JSON spec from Step 3 ... }
121
+ EOF
122
+ ```
123
+
124
+ If `<folder>/model-graph.png` already exists and the user did not explicitly ask to
125
+ regenerate, ask before overwriting.
126
+
127
+ ---
128
+
129
+ ## Step 5: Show the result
130
+
131
+ Use the Read tool (not a Python `Read()` call — Read is a Claude Code tool) on the PNG
132
+ path so the image renders inline in chat:
133
+
134
+ > Read `<folder>/model-graph.png`
135
+
136
+ ---
137
+
138
+ ## Final response template
139
+
140
+ ```text
141
+ Generated: <folder>/model-graph.png
142
+
143
+ Cubes visualized: <n>
144
+ Fact spine: <cube_name>
145
+ Dimensions: <dim1>, <dim2>, ...
146
+ Edges: <dim1> → <fact> (<join_key>) [1:∞]
147
+ <dim2> → <fact> (<join_key>) [1:∞]
148
+ ```
149
+
150
+ ---
151
+
152
+ ## Rules
153
+
154
+ - Always parse YAML — never hardcode cube names or relationships.
155
+ - Edge direction is always **dimension → fact** (arrow tail at dimension, head at fact).
156
+ - The `1` and `∞` markers are positioned by the renderer based on `from_card`/`to_card`;
157
+ set them correctly per the cardinality table above.
158
+ - Do not overwrite an existing `model-graph.png` without confirming.
159
+ - After saving, always show the image inline using the Read tool.