@oxy-hq/cli 0.7.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/README.md +131 -8
  2. package/dist/main.mjs +4909 -3551
  3. package/dist/main.mjs.map +1 -1
  4. package/dist/{mcp-xxWL3_dz.mjs → mcp-Bdp0rX0l.mjs} +2 -2
  5. package/dist/{mcp-xxWL3_dz.mjs.map → mcp-Bdp0rX0l.mjs.map} +1 -1
  6. package/package.json +5 -5
  7. package/skills/oxy-agents/SKILL.md +318 -0
  8. package/skills/oxy-agents/references/quick-reference.md +222 -0
  9. package/skills/oxy-agents/templates/agentic-template.yml +141 -0
  10. package/skills/oxy-airway-pipelines/SKILL.md +223 -316
  11. package/skills/oxy-automations/SKILL.md +61 -15
  12. package/skills/oxy-cli/SKILL.md +37 -4
  13. package/skills/oxy-custom-apps/SKILL.md +193 -291
  14. package/skills/oxy-custom-apps/references/minimal-starter.md +63 -0
  15. package/skills/oxy-custom-apps/references/performance-and-usage.md +105 -0
  16. package/skills/oxy-custom-apps/references/reading-data.md +106 -0
  17. package/skills/oxy-custom-apps/references/sandbox-agent-loop.md +267 -0
  18. package/skills/oxy-custom-apps/references/verifying-in-the-browser.md +65 -0
  19. package/skills/oxy-data-apps/SKILL.md +362 -0
  20. package/skills/oxy-data-apps/references/examples.md +263 -0
  21. package/skills/oxy-data-apps/references/quick-reference.md +171 -0
  22. package/skills/oxy-data-apps/templates/dashboard-with-controls.app.yml +128 -0
  23. package/skills/oxy-data-apps/templates/simple-dashboard.app.yml +102 -0
  24. package/skills/oxy-data-architecture/SKILL.md +137 -152
  25. package/skills/oxy-evals/SKILL.md +362 -0
  26. package/skills/oxy-evals/references/diagnosis.md +243 -0
  27. package/skills/oxy-evals/references/results-json.md +145 -0
  28. package/skills/oxy-evals/references/writing-expected.md +112 -0
  29. package/skills/oxy-semantic-modeling/SKILL.md +239 -59
  30. package/skills/oxy-semantic-modeling/references/warehouse-casts.md +57 -0
  31. package/skills/oxy-semantic-modeling/templates/topic-template.yml +38 -0
  32. package/skills/oxy-semantic-modeling/templates/view-template.yml +119 -0
  33. package/template/.github/workflows/publish.yaml +15 -13
  34. package/template/.nvmrc +1 -1
  35. package/template/.oxyc-managed +5 -5
  36. package/template/README.md +21 -10
  37. package/template/package.json +3 -2
  38. package/template/pnpm-workspace.yaml +28 -17
@@ -0,0 +1,145 @@
1
+ # Reading `oxy test` results
2
+
3
+ `oxy test <file> --output-json` writes `<file without .yml>.results.json` next to the test
4
+ file (`tests/sales.performance.test.yml` → `tests/sales.performance.test.results.json`).
5
+ Without a file argument it writes `test-results.json` in the current directory.
6
+ `--format json` prints the same document on stdout.
7
+
8
+ ## Shape
9
+
10
+ The file is a JSON array with **one element per test file**:
11
+
12
+ ```jsonc
13
+ [
14
+ {
15
+ "test_name": "sales.performance.test.yml",
16
+ "errors": [ // one string per run that produced no answer
17
+ "agentic pipeline asked a clarifying question: Which region do you mean?"
18
+ ],
19
+ "stats": {
20
+ "total_attempted": 9, // cases × runs
21
+ "answered": 8 // runs that returned an answer (judged or not)
22
+ },
23
+ "metrics": [
24
+ {
25
+ "type": "Correctness", // the only metric `oxy test` produces
26
+ "score": 0.6667, // mean of every record's score in this file
27
+ "records": [
28
+ {
29
+ "prompt": "What was total revenue across all stores in 2025?",
30
+ "expected": "Total revenue across all stores in 2025 was approximately $6.7 billion.",
31
+ "actual_output": "Total revenue in 2025 was $6.74B across 45 stores…",
32
+ "cot": "1. Key facts in the expected answer… Verdict: PASS",
33
+ "choice": "PASS", // PASS or FAIL
34
+ "score": 1.0, // 1.0 for PASS, 0.0 for FAIL — never partial
35
+ "duration_ms": 18234.5,
36
+ "input_tokens": 41210,
37
+ "output_tokens": 1830
38
+ }
39
+ // … one record per run: `runs: 3` gives 3 records per case
40
+ ]
41
+ }
42
+ ]
43
+ }
44
+ ]
45
+ ```
46
+
47
+ What is **not** in it: the SQL, the tool calls, the rows the agent saw. Records for an
48
+ `.agentic.yml` target carry no `references` array. Diagnose the path with the commands under
49
+ "Seeing what the agent did" below.
50
+
51
+ ## Field by field
52
+
53
+ | Field | Use it for |
54
+ | --- | --- |
55
+ | `prompt` | Grouping. A case's runs share the same prompt string. |
56
+ | `expected` | The specification, exactly as written in the test file. |
57
+ | `actual_output` | The agent's final answer. This is your main evidence. `[ERROR] …` means the run failed. |
58
+ | `cot` | The judge's whole response: its reasoning, then its verdict line. |
59
+ | `choice` | The parsed verdict. Compare it with `cot` to spot judge parse problems. |
60
+ | `score` | `choice` as a number. |
61
+ | `duration_ms`, `*_tokens` | Cost and latency of the agent run, not of the judge. Zero on errored runs. |
62
+ | `errors` | The error text of every run that never answered, in run order. |
63
+ | `stats.answered` vs `total_attempted` | The answer rate. A gap means runs ended in errors. |
64
+
65
+ ## Ordering
66
+
67
+ Answered runs come first, in case order. **Errored runs are appended at the end of
68
+ `records`**, each with `choice: "FAIL"`, `actual_output: "[ERROR] <message>"` and
69
+ `cot: "Run failed with error: <message>"`. Don't assume a case's runs are adjacent. Group by
70
+ `prompt`.
71
+
72
+ ## How the verdict is parsed
73
+
74
+ The judge gets one prompt: the question, the expected answer and the agent's answer. It's
75
+ asked to list the expected answer's key facts, check whether the agent's answer contains them,
76
+ look for contradictions or significant omissions, ignore formatting, style, phrasing and level
77
+ of detail, allow extra correct information, and end with `PASS` or `FAIL`.
78
+
79
+ The verdict is read from the **last five lines** of the judge's response:
80
+
81
+ - A line that contains `PASS` and not `FAIL` → PASS.
82
+ - A line that contains `FAIL` → FAIL. That includes a line containing both words, such as an
83
+ echoed `Verdict: PASS or FAIL`.
84
+ - No verdict in the last five lines → **FAIL**.
85
+
86
+ So a judge false negative usually looks like this: the `cot` reasoning concludes the answer
87
+ matches, but the verdict is missing, buried higher up, or written as "PASS or FAIL". The fix
88
+ for that is the judge model, never the agent.
89
+
90
+ ## Recipes
91
+
92
+ Without tools: for a single `--case` run, the file is small, so read it whole. For a full
93
+ suite, search the file for a distinctive phrase from the prompt and read the records around
94
+ each hit.
95
+
96
+ With `jq`:
97
+
98
+ ```bash
99
+ R=tests/sales.performance.test.results.json
100
+
101
+ # Pass count per case
102
+ jq -r '.[].metrics[] | select(.type=="Correctness") | .records
103
+ | group_by(.prompt)[] | "\(map(.score)|add|floor)/\(length) \(.[0].prompt)"' "$R"
104
+
105
+ # Every answer to one case, side by side
106
+ jq -r '.[].metrics[].records[] | select(.prompt | test("total revenue"; "i"))
107
+ | "\(.choice) \(.actual_output)\n"' "$R"
108
+
109
+ # Failing runs with the judge's reasoning
110
+ jq -r '.[].metrics[].records[] | select(.score < 1)
111
+ | "## \(.prompt)\n\(.actual_output)\n--- judge ---\n\(.cot)\n"' "$R"
112
+
113
+ # Runs that never answered
114
+ jq -r '.[].errors[]' "$R"
115
+ ```
116
+
117
+ ## Seeing what the agent did
118
+
119
+ The results only show answers. To see the route an answer took, replay the prompt:
120
+
121
+ ```bash
122
+ oxy agentic run <path/to/agent.agentic.yml> -q "<prompt>" --json
123
+ ```
124
+
125
+ This needs `OXY_DATABASE_URL` (a Postgres URL; the command records the run there). It prints
126
+ one JSON object per line, `{"seq", "event_type", "payload"}`. The events that matter for
127
+ diagnosis:
128
+
129
+ | `event_type` | What to read in `payload` |
130
+ | --- | --- |
131
+ | `semantic_shortcut_attempted` / `semantic_shortcut_resolved` | whether the semantic model answered directly |
132
+ | `spec_resolved` | the tables, joins and assumptions the agent settled on |
133
+ | `query_generated` | SQL the model wrote |
134
+ | `query_executed` | `query`, `row_count`, `rows` (a preview), `error`, and `source`: `semantic`, `llm`, `verified_sql` or `vendor` |
135
+ | `analytics_validation_failed` | `state`, `reason`: a check that sent the agent back to retry |
136
+
137
+ Run it two or three times for a flaky case. Different `source` values or different SQL across
138
+ runs is the flakiness, made visible.
139
+
140
+ ## The pretty report
141
+
142
+ Without `--format json`, each case prints as `PASS (n/n)`, `FLKY (k/n)` or `FAIL (0/n)`,
143
+ followed by the file's Correctness score and a `Failures:` section with the judge's reasoning.
144
+ `-v` adds each failing run's actual answer, and `-q` prints only the summary. The footer gives
145
+ the overall `Tests: x/y passed`, the answer rate, the time taken and the token totals.
@@ -0,0 +1,112 @@
1
+ # Writing `expected` answers
2
+
3
+ The judge PASSes an answer that contains the **core factual content** of `expected`, with no
4
+ contradiction and no significant omission. It ignores formatting, style, phrasing and level of
5
+ detail, and it allows extra correct information. Every word in `expected` is therefore a fact
6
+ the agent must state. Write only the facts that must hold, as the answer a careful analyst
7
+ would give.
8
+
9
+ ## By question type
10
+
11
+ **Single aggregate**
12
+
13
+ ```text
14
+ Total revenue across all locations in 2025 is approximately $6.7 billion.
15
+ ```
16
+
17
+ **Ranking.** Give the criterion once, the top entries with anchor values, and the bottom of the
18
+ list if it matters.
19
+
20
+ ```text
21
+ The top 5 stores by 2025 net sales are led by Store 20 (~$301M) and Store 4 (~$299M);
22
+ the fifth, Store 13, is at ~$287M.
23
+ ```
24
+
25
+ **Trend.** Give the direction, the current value, and one or two notable points.
26
+
27
+ ```text
28
+ Weekly orders in Q4 2025 rose from ~11K in early October to a peak of ~19K in the
29
+ week of 24 November, then settled near ~14K in December.
30
+ ```
31
+
32
+ **Comparison between groups**
33
+
34
+ ```text
35
+ In 2025, stores in the West region averaged ~$1.2M monthly sales, about 15% above the
36
+ ~$1.04M average of the other regions.
37
+ ```
38
+
39
+ **Share or ratio.** Say what the denominator is.
40
+
41
+ ```text
42
+ Returns were ~4.2% of units sold in 2025 (about 52K of 1.24M units).
43
+ ```
44
+
45
+ **Filter on a rare value.** Assert the scope, so an answer that ignores the filter fails.
46
+
47
+ ```text
48
+ Only 3 orders in 2025 used the GIFTCARD50 promotion, totalling ~$410.
49
+ ```
50
+
51
+ **An empty result that is the correct answer**
52
+
53
+ ```text
54
+ There were no refunds issued on 25 December 2025.
55
+ ```
56
+
57
+ **Unanswerable as asked.** Give the reason, and what a good answer offers instead.
58
+
59
+ ```text
60
+ Margin by product can't be computed because cost data is only recorded at the category
61
+ level. A correct answer explains this and offers margin by category instead.
62
+ ```
63
+
64
+ ## Evidence rules
65
+
66
+ - **Trust facts that agree across runs over any single run.** Two or more agreeing runs are
67
+ the minimum for a full `expected`.
68
+ - **A "successful" run can still be a failure.** Treat these as non-answers, not evidence:
69
+ - an answer with no figures where figures were asked for
70
+ - a ranked list whose metric values are all blank or null
71
+ - an answer built on a row-capped or truncated result, when the question needed every row
72
+ - **Separate transient errors from systematic ones.** A connection reset or a timeout can
73
+ vanish on a re-run. The same error on every run is a finding to report, not noise.
74
+ - **Verify before you encode.** If you can query the data yourself, cross-check the headline
75
+ number. A stable answer can be stably wrong: a wrong join, a double-counted fact table, the
76
+ wrong date column. Encoding it turns a bug into the specification.
77
+ - **When evidence is thin, narrow the placeholder instead of guessing:**
78
+
79
+ ```yaml
80
+ expected: >-
81
+ DRAFT: all 3 runs agree the West region leads; the size of the lead varied
82
+ (12%–31%) — needs a verified figure.
83
+ ```
84
+
85
+ ## Relative time
86
+
87
+ If the prompt says "last month" and the runs show the agent resolved it to November 2025,
88
+ rewrite **both** sides: the prompt becomes "…in November 2025?", and `expected` uses the same
89
+ period. Leaving the prompt relative makes the test start failing next month. The agent
90
+ resolves relative dates against today in its `timezone:` (UTC when unset).
91
+
92
+ ## When to split, rewrite, or flag a case
93
+
94
+ - **Split** a prompt that asks two things. The judge fails the whole run if either half is
95
+ missing, so you can't tell which half broke.
96
+ - **Propose a rewrite** when the runs answer different, reasonable readings of the prompt
97
+ (net or gross, calendar or fiscal year), or when the agent stops to ask a clarifying
98
+ question. That ends the run as an error. Show the user the readings you saw, and let them
99
+ choose.
100
+ - **Flag, don't draft,** when no view or table holds the data. That's a gap in the agent, not
101
+ something a test can fix.
102
+
103
+ ## Checklist before saving
104
+
105
+ - [ ] Every case has a `name:` slug.
106
+ - [ ] No relative dates remain in prompts that `expected` depends on.
107
+ - [ ] Numbers are approximate unless they're exact and stable.
108
+ - [ ] No schema, view or column names the user wouldn't say.
109
+ - [ ] No `expected` was copied from a single run without agreement or verification.
110
+ - [ ] Uncertain cases carry a narrowed `DRAFT:` note, not an invented value.
111
+ - [ ] The file parses: run it by path once (`oxy test <file> --case 0`). Discovery skips a
112
+ broken file silently.
@@ -1,6 +1,13 @@
1
1
  ---
2
2
  name: oxy-semantic-modeling
3
- description: Use when writing, reviewing, or debugging an Oxy semantic model — `.view.yml` views, `.topic.yml` topics, measures, dimensions, entities, and the `meta:` freshness block — in a customer workspace, and when one misbehaves: a measure that vanishes after switching topics, "measure not found", a topic returning no rows or missing joins, or an agent reporting a partial day as a complete number. Carries the craft and the traps the reference docs do not.
3
+ description: >-
4
+ Use when writing, reviewing, or debugging an Oxy semantic model — `.view.yml` views,
5
+ `.topic.yml` topics, measures, dimensions, entities, pre-aggregations (rollups), and the
6
+ `meta:` freshness block — in a customer workspace, and when one misbehaves: a measure that
7
+ vanishes after switching topics, "measure not found", a topic returning no rows or missing
8
+ joins, a date filter failing with `TYPE_MISMATCH`, a rollup that never serves, or an agent
9
+ reporting a partial day as a complete number. Carries the craft and the traps the reference
10
+ docs do not.
4
11
  ---
5
12
 
6
13
  # Oxy semantic modeling
@@ -12,8 +19,16 @@ description: Use when writing, reviewing, or debugging an Oxy semantic model —
12
19
 
13
20
  That page owns the field lists. This skill deliberately does **not** restate
14
21
  them — a copied schema stays confidently wrong long after the real one moves.
15
- What follows is the craft and the traps it does not carry, with every example
16
- taken from Poke House's real 39 views and 23 topics.
22
+ What follows is the craft and the traps it does not carry, with examples taken
23
+ from one production workspace's 39 views and 23 topics.
24
+
25
+ **Read next:**
26
+
27
+ - [templates/view-template.yml](templates/view-template.yml) and
28
+ [templates/topic-template.yml](templates/topic-template.yml) — annotated
29
+ starting points for a new view or topic.
30
+ - [references/warehouse-casts.md](references/warehouse-casts.md) — the
31
+ per-warehouse cast for a date column that is not stored as a date.
17
32
 
18
33
  ## View anatomy
19
34
 
@@ -23,7 +38,7 @@ against it.
23
38
  ```yaml
24
39
  name: sales_daily
25
40
  datasource: "clickhouse"
26
- table: "restaurant_analytics.restaurant_analytics___sales_daily_metrics"
41
+ table: "restaurant_analytics.sales_daily_metrics"
27
42
  ```
28
43
 
29
44
  **`entities` are the join graph, not decoration.** The `primary` entity names
@@ -41,20 +56,31 @@ entities:
41
56
  key: restaurant_id
42
57
  ```
43
58
 
44
- **`key` names a dimension, not a column.** In `restaurants.view.yml` the entity
59
+ **`key` names a dimension, not a column.** In a `restaurants` view the entity
45
60
  `restaurant` has `key: restaurant_id`, and the `restaurant_id` *dimension* is
46
61
  `expr: guid` — the physical column is `guid`. So resolve a `key` against the
47
62
  view's `dimensions:` list, never against the table's columns; the reference's
48
63
  `entities` page carries the same warning.
49
64
 
65
+ **Declare a foreign entity for every foreign-key column.** When a fact view
66
+ holds an ID pointing at another view's primary key (`orders.restaurant_id` →
67
+ `restaurants.guid`), give the fact view a `type: foreign` entity with the
68
+ **same `name:`** as the lookup view's primary entity and `key:` set to the FK
69
+ dimension. That edge is what lets a query group orders by
70
+ `restaurants.location_name` instead of by the raw ID; without it, every chart
71
+ and table built on the view renders opaque UUIDs. A foreign entity with no
72
+ matching primary anywhere is only a *warning* — the join silently does not
73
+ exist.
74
+
50
75
  **`dimensions`** are what you slice by, **`measures`** what you aggregate. Both
51
76
  take `synonyms`; measures also take `filters` and `drivers`. House style keeps
52
77
  entity keys single — even a compound natural key is one pre-concatenated
53
- dimension (`sales_day_key`); no view uses the documented composite `keys:`.
78
+ dimension (`sales_day_key`); no view in the corpus uses the documented
79
+ composite `keys:`.
54
80
 
55
81
  ## The description is a contract with the agent, not a label
56
82
 
57
- `sales_daily`'s description, verbatim:
83
+ A real `sales_daily` description, verbatim:
58
84
 
59
85
  > Daily sales metrics aggregated by restaurant and business date. Loaded several
60
86
  > times daily; the current business day is always intraday-partial - the prior
@@ -67,21 +93,34 @@ say when the data is complete and what it excludes, not what the table holds.
67
93
 
68
94
  ## The `meta:` freshness block
69
95
 
70
- Four keys — `freshness_watermark_column`, `freshness_expected_cadence`,
71
- `freshness_complete_through`, `freshness_caveat` — that state a view's own lag
72
- characteristics so the agent can qualify an answer instead of guessing.
96
+ Four keys that state a view's own lag characteristics, so the analytics agent
97
+ can qualify an answer instead of guessing:
98
+
99
+ ```yaml
100
+ meta:
101
+ freshness_watermark_column: ["business_date"]
102
+ freshness_expected_cadence: ["several times daily"]
103
+ freshness_complete_through: ["prior full day (America/Los_Angeles)"]
104
+ freshness_caveat: ["Current business day is intraday-partial."]
105
+ ```
73
106
 
74
107
  **Only 11 of 39 views carry one.** That low number is the point: it is the most
75
108
  under-used high-value feature in a real model, and adding one is the cheapest
76
109
  quality win available on any view fed by a batch pipeline.
77
110
 
78
- - **It is undocumented.** Not one of the four keys appears anywhere in the docs,
79
- and `views.mdx`'s own property table does not list `meta`. It is a working
80
- project convention, not a schema feature you can look up. **Copy the block from
81
- `sales_daily.view.yml`** — that is the canonical exemplar.
82
- - **Every value is wrapped as a single-element list** (`["prior full day
83
- (America/Los_Angeles)"]`) even though each reads as a scalar. "Simplifying" it
84
- to a bare string silently deviates from all 11 existing blocks.
111
+ - **`meta:` is a schema field; the four key names are a convention.** `meta`
112
+ exists on views (and dimensions, measures, entities, segments and topics),
113
+ typed as a map from a key to a **list of strings**. That is why every value
114
+ is a one-element list: a bare string (`freshness_caveat: "…"`) fails to
115
+ parse. The four `freshness_*` names appear in no docs page; the analytics
116
+ agent reads them by name and uses the first element of each.
117
+ - **The watermark resolves against dimensions first.**
118
+ `freshness_watermark_column` may name a dimension (or match its `expr`); if
119
+ it matches neither, it is used as a raw column. **Without the key, the agent
120
+ falls back to the view's first `date`/`datetime` dimension** — which may be
121
+ an order date rather than the load watermark, so declare it.
122
+ - A view-level `refresh_key: { sql: … }` (see Pre-aggregations) doubles as the
123
+ agent's recency probe for the view.
85
124
 
86
125
  ## Craft
87
126
 
@@ -99,6 +138,29 @@ English onto one name; colliding common words across views is normal and wanted.
99
138
  filters on `"PAID"` instead of inventing `"paid"`. Give the *full* set, not a
100
139
  sample — `["OPEN", "PAID", "CLOSED"]`, `["negative", "neutral", "positive"]`.
101
140
 
141
+ **`samples` is always a list of strings — even on a `boolean` or `number`
142
+ dimension.** Write `samples: ["true", "false"]` and `samples: ["5", "10"]`,
143
+ never `[true, false]` or `[5, 10]`. The field only accepts strings, and one
144
+ bare literal makes the view fail to load — and with it the whole semantic
145
+ model, not just that view. The reference's `dimensions` page shows a numeric
146
+ example with bare numbers; do not copy it.
147
+
148
+ **"Aggregate X where Y" is a filtered measure, not a `custom` `CASE WHEN`.**
149
+ Write a typed measure (`sum`, `count`, `average`, …) with a `filters:` list:
150
+
151
+ ```yaml
152
+ - name: delivery_revenue
153
+ type: sum
154
+ expr: order_amount
155
+ filters:
156
+ - expr: "order_type = 'delivery'" # raw column SQL, or {{dimension}} — see Traps
157
+ ```
158
+
159
+ It reads better, and it re-aggregates: a filtered `sum` can be answered from a
160
+ pre-aggregation, while a `custom` measure never can (nor can `median` or
161
+ `number`). Reserve `custom` for what a filtered measure cannot express —
162
+ ratios, correlations, cross-period math.
163
+
102
164
  **Derived time dimensions so the agent never writes date arithmetic.** Ship
103
165
  `business_year`, `business_month`, `business_quarter`, `day_of_week`,
104
166
  `is_weekend` as dimensions of their own, each re-derived from the raw column
@@ -109,8 +171,7 @@ rather than referencing the base date dimension:
109
171
  expr: "toQuarter(toDate(parseDateTimeBestEffort(toString(business_date))))"
110
172
  ```
111
173
 
112
- **`drivers` must state a mechanism, not a vibe.** The quality bar, from
113
- `sales_daily`:
174
+ **`drivers` must state a mechanism, not a vibe.** The quality bar:
114
175
 
115
176
  ```yaml
116
177
  - name: total_net_sales
@@ -124,11 +185,30 @@ rather than referencing the base date dimension:
124
185
 
125
186
  Note what the `description` does: it names the causal path *and* says why the
126
187
  obvious alternative (discount dollars) was rejected. The weak examples in the
127
- corpus — in `quickbooks_pl.view.yml` — carry `direction`, `strength`,
128
- `confidence`, and a `lag`, but no `description` at all, so the mechanism is never
129
- stated and a reader cannot tell whether the edge was reasoned or guessed. A
130
- driver without a `description` is not finished. Only 8 of 39 views use
131
- `drivers`, and the field is absent from the semantic-model docs — see the traps.
188
+ corpus carry `direction`, `strength`, `confidence`, and a `lag`, but no
189
+ `description` at all, so the mechanism is never stated and a reader cannot tell
190
+ whether the edge was reasoned or guessed. A driver without a `description` is
191
+ not finished. Only 8 of 39 views use `drivers`, and the field is absent from
192
+ the semantic-model docs — see the traps.
193
+
194
+ ## Dates: declare them as dates
195
+
196
+ A dimension whose meaning is a date or timestamp (`*_date`, `*_at`,
197
+ `business_date`, `created`, `event_time`, …) must be `type: date` (or
198
+ `type: datetime`), and its `expr` must produce a real date — not the raw
199
+ column, if the column is stored as a string, an integer like `20240131`, or
200
+ epoch seconds. Declare it `type: number` or `type: string` instead and nothing
201
+ complains when the view loads; the first query that filters on it compiles a
202
+ date literal the column rejects and fails with **`TYPE_MISMATCH`**.
203
+
204
+ Sample before you cast — stored formats differ between warehouses and between
205
+ tables in one warehouse, and one `SELECT <col> FROM <table> LIMIT 1` settles it.
206
+ Then wrap the `expr` with the cast for your warehouse:
207
+ [references/warehouse-casts.md](references/warehouse-casts.md). A column
208
+ already typed `DATE` / `TIMESTAMP` needs no cast.
209
+
210
+ Casting in the view is the fallback, not the goal: when you own the pipeline,
211
+ fix the column type there once (see *Repeated defensive casting* below).
132
212
 
133
213
  ## Topics are question areas, not tables
134
214
 
@@ -146,23 +226,120 @@ views:
146
226
  - restaurants
147
227
  - dining_options
148
228
  default_filters:
149
- - field: "is_voided"
229
+ - field: "orders.is_voided"
150
230
  eq:
151
231
  value: false
152
232
  ```
153
233
 
154
234
  Name topics after the questions people ask, and let `default_filters` carry the
155
235
  "obviously we don't count voided orders" rules so no one has to remember them.
236
+ A topic usually spans several views; "one topic per view" is not the pattern.
237
+
238
+ **Qualify `default_filters` fields as `view.dimension`.** A bare `field:` is
239
+ resolved against a view *named like the topic*, not against `base_view`. It
240
+ works when the topic shares its base view's name (as above, where
241
+ `is_voided` would also work) and points at nothing when it does not — a topic
242
+ `sales` over `base_view: sales_daily` needs `sales_daily.is_voided`.
243
+
244
+ ## Pre-aggregations (rollups)
245
+
246
+ A rollup is a materialized partial aggregate of one view. When a query is fully
247
+ covered by one, the engine answers it from the materialized copy instead of
248
+ scanning the warehouse — a few hundred rows instead of millions. They are
249
+ worth adding to views behind dashboards and frequently asked KPIs. The
250
+ reference documents the `config.yml` block in one line and the view-level
251
+ block not at all, so the shape is here.
252
+
253
+ **Opt-in, per view.** A view with no `pre_aggregations:` block gets **no
254
+ rollups at all** — there is no implicit default (an all-dimensions rollup of a
255
+ wide view is about as large as the table itself).
256
+
257
+ ```yaml
258
+ # in a .view.yml
259
+ refresh_key: # view-level default for every rollup below
260
+ every: "1h"
261
+
262
+ pre_aggregations:
263
+ - name: orders_by_month
264
+ dimensions: [order_status] # names declared in THIS view, never raw columns
265
+ measures: [total_orders, total_order_value]
266
+ time_dimension: order_date # a date/datetime dimension of this view
267
+ granularity: month # second|minute|hour|day|week|month|quarter|year
268
+ - name: orders_summary # no dimensions, no time: one summary row
269
+ measures: [total_orders]
270
+ refresh_key: # per-rollup override
271
+ sql: "SELECT MAX(updated_at) FROM orders"
272
+ ```
273
+
274
+ - **`refresh_key` takes exactly one of `every` or `sql`** — both, neither, or
275
+ any other key is a parse error. `sql` re-runs against the warehouse every
276
+ cycle and rebuilds when its result changes, so keep it a cheap `MAX`.
277
+ - **A rollup that names a member the view does not declare is refused**, not
278
+ quietly trimmed down to the members that resolve.
279
+
280
+ **When a rollup serves a query.** All of these must hold, or the query falls
281
+ back to the warehouse — correct, just not accelerated:
282
+
283
+ - every requested dimension, and every filtered dimension, is in the rollup's
284
+ `dimensions` (a filter on the `time_dimension` is also fine);
285
+ - every requested measure is in its `measures` **and is re-aggregatable** —
286
+ `count`, `count_distinct`, `sum`, `average`, `min`, `max`. A `custom`,
287
+ `number` or `median` measure never serves from a rollup, listed or not;
288
+ - the time dimension matches, and the requested granularity is the stored one
289
+ or coarser (a `day` rollup serves `month`; a `month` rollup cannot serve
290
+ `day`);
291
+ - **except `week`**: a `week` rollup does not serve `month`, `quarter` or
292
+ `year`, because a week straddling a month boundary would land in the wrong
293
+ month. Store `day` if you need both weekly and monthly answers.
294
+
295
+ **Limits.** A rollup belongs to one view and holds only that view's members,
296
+ and granularities are the fixed list above — no 15-minute buckets. For a
297
+ multi-view aggregate or a custom bucket, build a rollup *table* with an
298
+ automation instead (`oxy-data-architecture` owns that pattern).
299
+
300
+ **`config.yml` block** — every key optional:
301
+
302
+ ```yaml
303
+ pre_aggregations:
304
+ schema: AIRLAYER # where rollup tables are built (default AIRLAYER)
305
+ database: warehouse # connector for builds (default: each view's own datasource)
306
+ refresh_worker:
307
+ enabled: true # default true
308
+ heartbeat: "10m" # how often staleness is checked (default 10m, clamped 30s–24h)
309
+ renewal_threshold: "120s" # how long a refresh_key result is trusted (default 120s, clamped 10s–1h)
310
+ ```
311
+
312
+ ## Validating a change
313
+
314
+ - **`oxy validate --file semantics/views/orders.view.yml`** parses one view or
315
+ topic file and reports where it fails — a bare boolean in `samples`, a
316
+ missing `entities:` or `dimensions:` list, a removed feature. Run it on every
317
+ file you touch. It does **not** look inside `pre_aggregations:` or
318
+ `refresh_key:`; mistakes there surface only when the model is loaded for a
319
+ query.
320
+ - Bare **`oxy validate`** (every file) only looks in `semantics/views/` and
321
+ `semantics/topics/`, while views and topics anywhere in the workspace
322
+ compile. A view kept elsewhere is only checked with `--file`.
323
+ - Neither checks **cross-file** wiring — an entity join that matches nothing,
324
+ a topic naming a view that does not exist, a measure that a twin lacks. Run
325
+ a real query through the topic before calling it done.
326
+ - **`oxy build` does not validate the semantic model** — it rebuilds search
327
+ embeddings and syncs integrations. `oxy sync` writes table and column
328
+ metadata to `.databases/`, which is the place to read real column names
329
+ from.
330
+ - `inherits_from:` and `{{ globals.* }}` were removed from views; a file using
331
+ either is refused at load. Inline the values.
156
332
 
157
333
  ## Traps
158
334
 
159
- **The `airhouse_*` twins are a migration artifact, and they are not mirrors.**
160
- The ClickHouse → Airhouse migration is in flight, so **11 of the 39 views exist
161
- twice** (22 views, 11 twin pairs) — plus 3 further `airhouse_*` views with no
162
- ClickHouse counterpart at all, where discount- and payment-level detail lives
163
- only on the new side. Three consequences, in order of how much time they cost:
335
+ **Twin views from a backend migration are not mirrors.** In the reference
336
+ workspace a ClickHouse → Airhouse migration is in flight, so **11 of the 39
337
+ views exist twice** (22 views, 11 twin pairs) — plus 3 further `airhouse_*`
338
+ views with no ClickHouse counterpart at all, where discount- and payment-level
339
+ detail lives only on the new side. Three consequences, in order of how much
340
+ time they cost:
164
341
 
165
- 1. **The twins have diverged.** `sales_daily` defines a `discount_rate` measure;
342
+ 1. **The twins diverge.** `sales_daily` defines a `discount_rate` measure;
166
343
  `airhouse_sales_daily` does not. Switch topics mid-conversation from one
167
344
  family to the other and a query that worked seconds ago fails with "measure
168
345
  not found", with nothing anywhere saying the two views were ever supposed to
@@ -172,30 +349,27 @@ only on the new side. Three consequences, in order of how much time they cost:
172
349
  the edges — as `airhouse_restaurant`, `airhouse_order`, `airhouse_check`,
173
350
  while its twin uses bare `restaurant`, `order`, `check`. Joins resolve by
174
351
  matching entity *names*, so a topic mixing families does not error — it just
175
- fails to auto-join. You get missing rows, not a config failure. No real topic
176
- mixes them; keep it that way.
352
+ fails to auto-join. You get missing rows, not a config failure. Keep each
353
+ topic within one family.
177
354
 
178
355
  **The prefix is not universal, and the exception runs the other way.** The
179
- three twin-less views from above leave their **primary** entity bare:
180
- `airhouse_order_discounts` → `discount_application`,
181
- `airhouse_order_payments` → `payment`, `airhouse_selection_discounts` →
182
- `selection_discount_application`. Their foreign entities are still prefixed,
183
- so rule 2 holds — but the latent failure inverts. A bare primary is exactly
184
- what a ClickHouse-family view would declare, so if anyone ever adds a
185
- `payment` or `discount_application` entity on the old side, those two views
186
- start auto-joining *across* families rather than failing to join: wrong rows
187
- silently, instead of missing rows silently. No such collision exists in the
188
- corpus today (grep: each of the three names is declared in exactly one
189
- view). Prefix the primary too on any new `airhouse_*` view.
356
+ three twin-less views leave their **primary** entity bare
357
+ (`discount_application`, `payment`, `selection_discount_application`).
358
+ Their foreign entities are still prefixed, so rule 2 holds — but the latent
359
+ failure inverts. A bare primary is exactly what an old-family view would
360
+ declare, so if anyone adds a `payment` entity on the old side, the two
361
+ families start auto-joining *across* the divide: wrong rows silently,
362
+ instead of missing rows silently. Prefix the primary too on any new view in
363
+ a migrated family.
190
364
  3. **Do not copy the pattern.** Duplicating a view to change backends is a
191
365
  migration tactic with an end date, not a modelling technique.
192
366
 
193
- **Repeated defensive casting is a smell.** `toDate(parseDateTimeBestEffort(
194
- toString(business_date)))` appears in **6 derived dimensions of `sales_daily`** —
195
- the same three-deep cast chain re-typed six times because the underlying column
196
- is not a date. Fix the column type once, upstream in the pipeline, rather than in
197
- every `expr`. (Counting these by grep over-reports: a seventh occurrence is
198
- inside `measures:`, not `dimensions:`. The dimension count is 6.)
367
+ **Repeated defensive casting is a smell.** In the reference `sales_daily`,
368
+ `toDate(parseDateTimeBestEffort(toString(business_date)))` appears in **6
369
+ derived dimensions** — the same three-deep cast chain re-typed six times
370
+ because the underlying column is not a date. Fix the column type once,
371
+ upstream in the pipeline, rather than in every `expr`. (Counting by grep
372
+ over-reports: a seventh occurrence is inside `measures:`, not `dimensions:`.)
199
373
 
200
374
  **`drivers` and `parent:` are documented, but not where you would look.** Neither
201
375
  appears in the semantic-model pages — `drivers` is explained under
@@ -205,12 +379,18 @@ the metric-tree page links back to `measures` claiming it defines the measures
205
379
  "and their drivers", which it does not. Read only the semantic-model section and
206
380
  you will conclude both fields do not exist and model without them.
207
381
 
208
- **View-qualified `{{ }}` references are the one-off, not the norm.** Across all
209
- 39 views the `{{ view.measure }}` form appears in exactly **one expression** —
210
- the `discount_rate` measure at `sales_daily.view.yml:129`, which uses it twice
211
- in that one line (`{{ sales_daily.total_discounts }} / NULLIF({{
212
- sales_daily.total_gross_sales }}, 0)`). So a grep returns 2 hits from 1 site;
213
- neither number is "the norm". Every other `{{ }}` in the corpus is a bare,
214
- unqualified dimension name inside a measure's `filters:` — `{{is_voided}}`. For
215
- an ordinary same-view filter, write the bare form; copying the qualified one
216
- means copying the single outlier in the codebase.
382
+ **`{{ }}` references: bare for the same view, qualified for another.** Inside a
383
+ measure's `expr` or `filters:`, `{{is_voided}}` names a dimension or measure
384
+ of the *same* view and is expanded for you — that is the norm in the corpus.
385
+ `{{view.member}}` reaches another view's member and is the rare case: across
386
+ 39 views it appears in exactly one expression (`discount_rate`, which uses it
387
+ twice on one line, so a grep shows 2 hits from 1 site). A `{{ }}` that names
388
+ nothing resolvable fails the query with an "unresolved reference" error rather
389
+ than reaching the warehouse. Older guidance that says never to put `{{ }}` in a
390
+ filter predates this and is wrong; raw column SQL (`status = 'completed'`)
391
+ remains fine too.
392
+
393
+ **Do not add a `# yaml-language-server: $schema=…` header.** No JSON schema is
394
+ published for `.view.yml` or `.topic.yml`; a header pointing at one makes an
395
+ editor validate against nothing — or against the wrong file — while looking as
396
+ if it works.