@oxy-hq/cli 0.8.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +89 -7
- package/dist/main.mjs +4914 -3760
- package/dist/main.mjs.map +1 -1
- package/dist/{mcp-xxWL3_dz.mjs → mcp-Bdp0rX0l.mjs} +2 -2
- package/dist/{mcp-xxWL3_dz.mjs.map → mcp-Bdp0rX0l.mjs.map} +1 -1
- package/package.json +5 -5
- package/skills/oxy-agents/SKILL.md +318 -0
- package/skills/oxy-agents/references/quick-reference.md +222 -0
- package/skills/oxy-agents/templates/agentic-template.yml +141 -0
- package/skills/oxy-airway-pipelines/SKILL.md +223 -316
- package/skills/oxy-automations/SKILL.md +61 -15
- package/skills/oxy-cli/SKILL.md +32 -5
- package/skills/oxy-custom-apps/SKILL.md +193 -291
- package/skills/oxy-custom-apps/references/minimal-starter.md +63 -0
- package/skills/oxy-custom-apps/references/performance-and-usage.md +105 -0
- package/skills/oxy-custom-apps/references/reading-data.md +106 -0
- package/skills/oxy-custom-apps/references/sandbox-agent-loop.md +267 -0
- package/skills/oxy-custom-apps/references/verifying-in-the-browser.md +65 -0
- package/skills/oxy-data-apps/SKILL.md +362 -0
- package/skills/oxy-data-apps/references/examples.md +263 -0
- package/skills/oxy-data-apps/references/quick-reference.md +171 -0
- package/skills/oxy-data-apps/templates/dashboard-with-controls.app.yml +128 -0
- package/skills/oxy-data-apps/templates/simple-dashboard.app.yml +102 -0
- package/skills/oxy-data-architecture/SKILL.md +137 -152
- package/skills/oxy-evals/SKILL.md +362 -0
- package/skills/oxy-evals/references/diagnosis.md +243 -0
- package/skills/oxy-evals/references/results-json.md +145 -0
- package/skills/oxy-evals/references/writing-expected.md +112 -0
- package/skills/oxy-semantic-modeling/SKILL.md +239 -59
- package/skills/oxy-semantic-modeling/references/warehouse-casts.md +57 -0
- package/skills/oxy-semantic-modeling/templates/topic-template.yml +38 -0
- package/skills/oxy-semantic-modeling/templates/view-template.yml +119 -0
- package/template/.github/workflows/publish.yaml +15 -13
- package/template/.nvmrc +1 -1
- package/template/.oxyc-managed +5 -5
- package/template/README.md +21 -10
- package/template/package.json +3 -2
- package/template/pnpm-workspace.yaml +28 -17
|
@@ -0,0 +1,145 @@
|
|
|
1
|
+
# Reading `oxy test` results
|
|
2
|
+
|
|
3
|
+
`oxy test <file> --output-json` writes `<file without .yml>.results.json` next to the test
|
|
4
|
+
file (`tests/sales.performance.test.yml` → `tests/sales.performance.test.results.json`).
|
|
5
|
+
Without a file argument it writes `test-results.json` in the current directory.
|
|
6
|
+
`--format json` prints the same document on stdout.
|
|
7
|
+
|
|
8
|
+
## Shape
|
|
9
|
+
|
|
10
|
+
The file is a JSON array with **one element per test file**:
|
|
11
|
+
|
|
12
|
+
```jsonc
|
|
13
|
+
[
|
|
14
|
+
{
|
|
15
|
+
"test_name": "sales.performance.test.yml",
|
|
16
|
+
"errors": [ // one string per run that produced no answer
|
|
17
|
+
"agentic pipeline asked a clarifying question: Which region do you mean?"
|
|
18
|
+
],
|
|
19
|
+
"stats": {
|
|
20
|
+
"total_attempted": 9, // cases × runs
|
|
21
|
+
"answered": 8 // runs that returned an answer (judged or not)
|
|
22
|
+
},
|
|
23
|
+
"metrics": [
|
|
24
|
+
{
|
|
25
|
+
"type": "Correctness", // the only metric `oxy test` produces
|
|
26
|
+
"score": 0.6667, // mean of every record's score in this file
|
|
27
|
+
"records": [
|
|
28
|
+
{
|
|
29
|
+
"prompt": "What was total revenue across all stores in 2025?",
|
|
30
|
+
"expected": "Total revenue across all stores in 2025 was approximately $6.7 billion.",
|
|
31
|
+
"actual_output": "Total revenue in 2025 was $6.74B across 45 stores…",
|
|
32
|
+
"cot": "1. Key facts in the expected answer… Verdict: PASS",
|
|
33
|
+
"choice": "PASS", // PASS or FAIL
|
|
34
|
+
"score": 1.0, // 1.0 for PASS, 0.0 for FAIL — never partial
|
|
35
|
+
"duration_ms": 18234.5,
|
|
36
|
+
"input_tokens": 41210,
|
|
37
|
+
"output_tokens": 1830
|
|
38
|
+
}
|
|
39
|
+
// … one record per run: `runs: 3` gives 3 records per case
|
|
40
|
+
]
|
|
41
|
+
}
|
|
42
|
+
]
|
|
43
|
+
}
|
|
44
|
+
]
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
What is **not** in it: the SQL, the tool calls, the rows the agent saw. Records for an
|
|
48
|
+
`.agentic.yml` target carry no `references` array. Diagnose the path with the commands under
|
|
49
|
+
"Seeing what the agent did" below.
|
|
50
|
+
|
|
51
|
+
## Field by field
|
|
52
|
+
|
|
53
|
+
| Field | Use it for |
|
|
54
|
+
| --- | --- |
|
|
55
|
+
| `prompt` | Grouping. A case's runs share the same prompt string. |
|
|
56
|
+
| `expected` | The specification, exactly as written in the test file. |
|
|
57
|
+
| `actual_output` | The agent's final answer. This is your main evidence. `[ERROR] …` means the run failed. |
|
|
58
|
+
| `cot` | The judge's whole response: its reasoning, then its verdict line. |
|
|
59
|
+
| `choice` | The parsed verdict. Compare it with `cot` to spot judge parse problems. |
|
|
60
|
+
| `score` | `choice` as a number. |
|
|
61
|
+
| `duration_ms`, `*_tokens` | Cost and latency of the agent run, not of the judge. Zero on errored runs. |
|
|
62
|
+
| `errors` | The error text of every run that never answered, in run order. |
|
|
63
|
+
| `stats.answered` vs `total_attempted` | The answer rate. A gap means runs ended in errors. |
|
|
64
|
+
|
|
65
|
+
## Ordering
|
|
66
|
+
|
|
67
|
+
Answered runs come first, in case order. **Errored runs are appended at the end of
|
|
68
|
+
`records`**, each with `choice: "FAIL"`, `actual_output: "[ERROR] <message>"` and
|
|
69
|
+
`cot: "Run failed with error: <message>"`. Don't assume a case's runs are adjacent. Group by
|
|
70
|
+
`prompt`.
|
|
71
|
+
|
|
72
|
+
## How the verdict is parsed
|
|
73
|
+
|
|
74
|
+
The judge gets one prompt: the question, the expected answer and the agent's answer. It's
|
|
75
|
+
asked to list the expected answer's key facts, check whether the agent's answer contains them,
|
|
76
|
+
look for contradictions or significant omissions, ignore formatting, style, phrasing and level
|
|
77
|
+
of detail, allow extra correct information, and end with `PASS` or `FAIL`.
|
|
78
|
+
|
|
79
|
+
The verdict is read from the **last five lines** of the judge's response:
|
|
80
|
+
|
|
81
|
+
- A line that contains `PASS` and not `FAIL` → PASS.
|
|
82
|
+
- A line that contains `FAIL` → FAIL. That includes a line containing both words, such as an
|
|
83
|
+
echoed `Verdict: PASS or FAIL`.
|
|
84
|
+
- No verdict in the last five lines → **FAIL**.
|
|
85
|
+
|
|
86
|
+
So a judge false negative usually looks like this: the `cot` reasoning concludes the answer
|
|
87
|
+
matches, but the verdict is missing, buried higher up, or written as "PASS or FAIL". The fix
|
|
88
|
+
for that is the judge model, never the agent.
|
|
89
|
+
|
|
90
|
+
## Recipes
|
|
91
|
+
|
|
92
|
+
Without tools: for a single `--case` run, the file is small, so read it whole. For a full
|
|
93
|
+
suite, search the file for a distinctive phrase from the prompt and read the records around
|
|
94
|
+
each hit.
|
|
95
|
+
|
|
96
|
+
With `jq`:
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
R=tests/sales.performance.test.results.json
|
|
100
|
+
|
|
101
|
+
# Pass count per case
|
|
102
|
+
jq -r '.[].metrics[] | select(.type=="Correctness") | .records
|
|
103
|
+
| group_by(.prompt)[] | "\(map(.score)|add|floor)/\(length) \(.[0].prompt)"' "$R"
|
|
104
|
+
|
|
105
|
+
# Every answer to one case, side by side
|
|
106
|
+
jq -r '.[].metrics[].records[] | select(.prompt | test("total revenue"; "i"))
|
|
107
|
+
| "\(.choice) \(.actual_output)\n"' "$R"
|
|
108
|
+
|
|
109
|
+
# Failing runs with the judge's reasoning
|
|
110
|
+
jq -r '.[].metrics[].records[] | select(.score < 1)
|
|
111
|
+
| "## \(.prompt)\n\(.actual_output)\n--- judge ---\n\(.cot)\n"' "$R"
|
|
112
|
+
|
|
113
|
+
# Runs that never answered
|
|
114
|
+
jq -r '.[].errors[]' "$R"
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
## Seeing what the agent did
|
|
118
|
+
|
|
119
|
+
The results only show answers. To see the route an answer took, replay the prompt:
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
oxy agentic run <path/to/agent.agentic.yml> -q "<prompt>" --json
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
This needs `OXY_DATABASE_URL` (a Postgres URL; the command records the run there). It prints
|
|
126
|
+
one JSON object per line, `{"seq", "event_type", "payload"}`. The events that matter for
|
|
127
|
+
diagnosis:
|
|
128
|
+
|
|
129
|
+
| `event_type` | What to read in `payload` |
|
|
130
|
+
| --- | --- |
|
|
131
|
+
| `semantic_shortcut_attempted` / `semantic_shortcut_resolved` | whether the semantic model answered directly |
|
|
132
|
+
| `spec_resolved` | the tables, joins and assumptions the agent settled on |
|
|
133
|
+
| `query_generated` | SQL the model wrote |
|
|
134
|
+
| `query_executed` | `query`, `row_count`, `rows` (a preview), `error`, and `source`: `semantic`, `llm`, `verified_sql` or `vendor` |
|
|
135
|
+
| `analytics_validation_failed` | `state`, `reason`: a check that sent the agent back to retry |
|
|
136
|
+
|
|
137
|
+
Run it two or three times for a flaky case. Different `source` values or different SQL across
|
|
138
|
+
runs is the flakiness, made visible.
|
|
139
|
+
|
|
140
|
+
## The pretty report
|
|
141
|
+
|
|
142
|
+
Without `--format json`, each case prints as `PASS (n/n)`, `FLKY (k/n)` or `FAIL (0/n)`,
|
|
143
|
+
followed by the file's Correctness score and a `Failures:` section with the judge's reasoning.
|
|
144
|
+
`-v` adds each failing run's actual answer, and `-q` prints only the summary. The footer gives
|
|
145
|
+
the overall `Tests: x/y passed`, the answer rate, the time taken and the token totals.
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
# Writing `expected` answers
|
|
2
|
+
|
|
3
|
+
The judge PASSes an answer that contains the **core factual content** of `expected`, with no
|
|
4
|
+
contradiction and no significant omission. It ignores formatting, style, phrasing and level of
|
|
5
|
+
detail, and it allows extra correct information. Every word in `expected` is therefore a fact
|
|
6
|
+
the agent must state. Write only the facts that must hold, as the answer a careful analyst
|
|
7
|
+
would give.
|
|
8
|
+
|
|
9
|
+
## By question type
|
|
10
|
+
|
|
11
|
+
**Single aggregate**
|
|
12
|
+
|
|
13
|
+
```text
|
|
14
|
+
Total revenue across all locations in 2025 is approximately $6.7 billion.
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
**Ranking.** Give the criterion once, the top entries with anchor values, and the bottom of the
|
|
18
|
+
list if it matters.
|
|
19
|
+
|
|
20
|
+
```text
|
|
21
|
+
The top 5 stores by 2025 net sales are led by Store 20 (~$301M) and Store 4 (~$299M);
|
|
22
|
+
the fifth, Store 13, is at ~$287M.
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
**Trend.** Give the direction, the current value, and one or two notable points.
|
|
26
|
+
|
|
27
|
+
```text
|
|
28
|
+
Weekly orders in Q4 2025 rose from ~11K in early October to a peak of ~19K in the
|
|
29
|
+
week of 24 November, then settled near ~14K in December.
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
**Comparison between groups**
|
|
33
|
+
|
|
34
|
+
```text
|
|
35
|
+
In 2025, stores in the West region averaged ~$1.2M monthly sales, about 15% above the
|
|
36
|
+
~$1.04M average of the other regions.
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
**Share or ratio.** Say what the denominator is.
|
|
40
|
+
|
|
41
|
+
```text
|
|
42
|
+
Returns were ~4.2% of units sold in 2025 (about 52K of 1.24M units).
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
**Filter on a rare value.** Assert the scope, so an answer that ignores the filter fails.
|
|
46
|
+
|
|
47
|
+
```text
|
|
48
|
+
Only 3 orders in 2025 used the GIFTCARD50 promotion, totalling ~$410.
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
**An empty result that is the correct answer**
|
|
52
|
+
|
|
53
|
+
```text
|
|
54
|
+
There were no refunds issued on 25 December 2025.
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
**Unanswerable as asked.** Give the reason, and what a good answer offers instead.
|
|
58
|
+
|
|
59
|
+
```text
|
|
60
|
+
Margin by product can't be computed because cost data is only recorded at the category
|
|
61
|
+
level. A correct answer explains this and offers margin by category instead.
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## Evidence rules
|
|
65
|
+
|
|
66
|
+
- **Trust facts that agree across runs over any single run.** Two or more agreeing runs are
|
|
67
|
+
the minimum for a full `expected`.
|
|
68
|
+
- **A "successful" run can still be a failure.** Treat these as non-answers, not evidence:
|
|
69
|
+
- an answer with no figures where figures were asked for
|
|
70
|
+
- a ranked list whose metric values are all blank or null
|
|
71
|
+
- an answer built on a row-capped or truncated result, when the question needed every row
|
|
72
|
+
- **Separate transient errors from systematic ones.** A connection reset or a timeout can
|
|
73
|
+
vanish on a re-run. The same error on every run is a finding to report, not noise.
|
|
74
|
+
- **Verify before you encode.** If you can query the data yourself, cross-check the headline
|
|
75
|
+
number. A stable answer can be stably wrong: a wrong join, a double-counted fact table, the
|
|
76
|
+
wrong date column. Encoding it turns a bug into the specification.
|
|
77
|
+
- **When evidence is thin, narrow the placeholder instead of guessing:**
|
|
78
|
+
|
|
79
|
+
```yaml
|
|
80
|
+
expected: >-
|
|
81
|
+
DRAFT: all 3 runs agree the West region leads; the size of the lead varied
|
|
82
|
+
(12%–31%) — needs a verified figure.
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
## Relative time
|
|
86
|
+
|
|
87
|
+
If the prompt says "last month" and the runs show the agent resolved it to November 2025,
|
|
88
|
+
rewrite **both** sides: the prompt becomes "…in November 2025?", and `expected` uses the same
|
|
89
|
+
period. Leaving the prompt relative makes the test start failing next month. The agent
|
|
90
|
+
resolves relative dates against today in its `timezone:` (UTC when unset).
|
|
91
|
+
|
|
92
|
+
## When to split, rewrite, or flag a case
|
|
93
|
+
|
|
94
|
+
- **Split** a prompt that asks two things. The judge fails the whole run if either half is
|
|
95
|
+
missing, so you can't tell which half broke.
|
|
96
|
+
- **Propose a rewrite** when the runs answer different, reasonable readings of the prompt
|
|
97
|
+
(net or gross, calendar or fiscal year), or when the agent stops to ask a clarifying
|
|
98
|
+
question. That ends the run as an error. Show the user the readings you saw, and let them
|
|
99
|
+
choose.
|
|
100
|
+
- **Flag, don't draft,** when no view or table holds the data. That's a gap in the agent, not
|
|
101
|
+
something a test can fix.
|
|
102
|
+
|
|
103
|
+
## Checklist before saving
|
|
104
|
+
|
|
105
|
+
- [ ] Every case has a `name:` slug.
|
|
106
|
+
- [ ] No relative dates remain in prompts that `expected` depends on.
|
|
107
|
+
- [ ] Numbers are approximate unless they're exact and stable.
|
|
108
|
+
- [ ] No schema, view or column names the user wouldn't say.
|
|
109
|
+
- [ ] No `expected` was copied from a single run without agreement or verification.
|
|
110
|
+
- [ ] Uncertain cases carry a narrowed `DRAFT:` note, not an invented value.
|
|
111
|
+
- [ ] The file parses: run it by path once (`oxy test <file> --case 0`). Discovery skips a
|
|
112
|
+
broken file silently.
|
|
@@ -1,6 +1,13 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: oxy-semantic-modeling
|
|
3
|
-
description:
|
|
3
|
+
description: >-
|
|
4
|
+
Use when writing, reviewing, or debugging an Oxy semantic model — `.view.yml` views,
|
|
5
|
+
`.topic.yml` topics, measures, dimensions, entities, pre-aggregations (rollups), and the
|
|
6
|
+
`meta:` freshness block — in a customer workspace, and when one misbehaves: a measure that
|
|
7
|
+
vanishes after switching topics, "measure not found", a topic returning no rows or missing
|
|
8
|
+
joins, a date filter failing with `TYPE_MISMATCH`, a rollup that never serves, or an agent
|
|
9
|
+
reporting a partial day as a complete number. Carries the craft and the traps the reference
|
|
10
|
+
docs do not.
|
|
4
11
|
---
|
|
5
12
|
|
|
6
13
|
# Oxy semantic modeling
|
|
@@ -12,8 +19,16 @@ description: Use when writing, reviewing, or debugging an Oxy semantic model —
|
|
|
12
19
|
|
|
13
20
|
That page owns the field lists. This skill deliberately does **not** restate
|
|
14
21
|
them — a copied schema stays confidently wrong long after the real one moves.
|
|
15
|
-
What follows is the craft and the traps it does not carry, with
|
|
16
|
-
|
|
22
|
+
What follows is the craft and the traps it does not carry, with examples taken
|
|
23
|
+
from one production workspace's 39 views and 23 topics.
|
|
24
|
+
|
|
25
|
+
**Read next:**
|
|
26
|
+
|
|
27
|
+
- [templates/view-template.yml](templates/view-template.yml) and
|
|
28
|
+
[templates/topic-template.yml](templates/topic-template.yml) — annotated
|
|
29
|
+
starting points for a new view or topic.
|
|
30
|
+
- [references/warehouse-casts.md](references/warehouse-casts.md) — the
|
|
31
|
+
per-warehouse cast for a date column that is not stored as a date.
|
|
17
32
|
|
|
18
33
|
## View anatomy
|
|
19
34
|
|
|
@@ -23,7 +38,7 @@ against it.
|
|
|
23
38
|
```yaml
|
|
24
39
|
name: sales_daily
|
|
25
40
|
datasource: "clickhouse"
|
|
26
|
-
table: "restaurant_analytics.
|
|
41
|
+
table: "restaurant_analytics.sales_daily_metrics"
|
|
27
42
|
```
|
|
28
43
|
|
|
29
44
|
**`entities` are the join graph, not decoration.** The `primary` entity names
|
|
@@ -41,20 +56,31 @@ entities:
|
|
|
41
56
|
key: restaurant_id
|
|
42
57
|
```
|
|
43
58
|
|
|
44
|
-
**`key` names a dimension, not a column.** In `restaurants
|
|
59
|
+
**`key` names a dimension, not a column.** In a `restaurants` view the entity
|
|
45
60
|
`restaurant` has `key: restaurant_id`, and the `restaurant_id` *dimension* is
|
|
46
61
|
`expr: guid` — the physical column is `guid`. So resolve a `key` against the
|
|
47
62
|
view's `dimensions:` list, never against the table's columns; the reference's
|
|
48
63
|
`entities` page carries the same warning.
|
|
49
64
|
|
|
65
|
+
**Declare a foreign entity for every foreign-key column.** When a fact view
|
|
66
|
+
holds an ID pointing at another view's primary key (`orders.restaurant_id` →
|
|
67
|
+
`restaurants.guid`), give the fact view a `type: foreign` entity with the
|
|
68
|
+
**same `name:`** as the lookup view's primary entity and `key:` set to the FK
|
|
69
|
+
dimension. That edge is what lets a query group orders by
|
|
70
|
+
`restaurants.location_name` instead of by the raw ID; without it, every chart
|
|
71
|
+
and table built on the view renders opaque UUIDs. A foreign entity with no
|
|
72
|
+
matching primary anywhere is only a *warning* — the join silently does not
|
|
73
|
+
exist.
|
|
74
|
+
|
|
50
75
|
**`dimensions`** are what you slice by, **`measures`** what you aggregate. Both
|
|
51
76
|
take `synonyms`; measures also take `filters` and `drivers`. House style keeps
|
|
52
77
|
entity keys single — even a compound natural key is one pre-concatenated
|
|
53
|
-
dimension (`sales_day_key`); no view uses the documented
|
|
78
|
+
dimension (`sales_day_key`); no view in the corpus uses the documented
|
|
79
|
+
composite `keys:`.
|
|
54
80
|
|
|
55
81
|
## The description is a contract with the agent, not a label
|
|
56
82
|
|
|
57
|
-
`sales_daily`
|
|
83
|
+
A real `sales_daily` description, verbatim:
|
|
58
84
|
|
|
59
85
|
> Daily sales metrics aggregated by restaurant and business date. Loaded several
|
|
60
86
|
> times daily; the current business day is always intraday-partial - the prior
|
|
@@ -67,21 +93,34 @@ say when the data is complete and what it excludes, not what the table holds.
|
|
|
67
93
|
|
|
68
94
|
## The `meta:` freshness block
|
|
69
95
|
|
|
70
|
-
Four keys
|
|
71
|
-
|
|
72
|
-
|
|
96
|
+
Four keys that state a view's own lag characteristics, so the analytics agent
|
|
97
|
+
can qualify an answer instead of guessing:
|
|
98
|
+
|
|
99
|
+
```yaml
|
|
100
|
+
meta:
|
|
101
|
+
freshness_watermark_column: ["business_date"]
|
|
102
|
+
freshness_expected_cadence: ["several times daily"]
|
|
103
|
+
freshness_complete_through: ["prior full day (America/Los_Angeles)"]
|
|
104
|
+
freshness_caveat: ["Current business day is intraday-partial."]
|
|
105
|
+
```
|
|
73
106
|
|
|
74
107
|
**Only 11 of 39 views carry one.** That low number is the point: it is the most
|
|
75
108
|
under-used high-value feature in a real model, and adding one is the cheapest
|
|
76
109
|
quality win available on any view fed by a batch pipeline.
|
|
77
110
|
|
|
78
|
-
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
111
|
+
- **`meta:` is a schema field; the four key names are a convention.** `meta`
|
|
112
|
+
exists on views (and dimensions, measures, entities, segments and topics),
|
|
113
|
+
typed as a map from a key to a **list of strings**. That is why every value
|
|
114
|
+
is a one-element list: a bare string (`freshness_caveat: "…"`) fails to
|
|
115
|
+
parse. The four `freshness_*` names appear in no docs page; the analytics
|
|
116
|
+
agent reads them by name and uses the first element of each.
|
|
117
|
+
- **The watermark resolves against dimensions first.**
|
|
118
|
+
`freshness_watermark_column` may name a dimension (or match its `expr`); if
|
|
119
|
+
it matches neither, it is used as a raw column. **Without the key, the agent
|
|
120
|
+
falls back to the view's first `date`/`datetime` dimension** — which may be
|
|
121
|
+
an order date rather than the load watermark, so declare it.
|
|
122
|
+
- A view-level `refresh_key: { sql: … }` (see Pre-aggregations) doubles as the
|
|
123
|
+
agent's recency probe for the view.
|
|
85
124
|
|
|
86
125
|
## Craft
|
|
87
126
|
|
|
@@ -99,6 +138,29 @@ English onto one name; colliding common words across views is normal and wanted.
|
|
|
99
138
|
filters on `"PAID"` instead of inventing `"paid"`. Give the *full* set, not a
|
|
100
139
|
sample — `["OPEN", "PAID", "CLOSED"]`, `["negative", "neutral", "positive"]`.
|
|
101
140
|
|
|
141
|
+
**`samples` is always a list of strings — even on a `boolean` or `number`
|
|
142
|
+
dimension.** Write `samples: ["true", "false"]` and `samples: ["5", "10"]`,
|
|
143
|
+
never `[true, false]` or `[5, 10]`. The field only accepts strings, and one
|
|
144
|
+
bare literal makes the view fail to load — and with it the whole semantic
|
|
145
|
+
model, not just that view. The reference's `dimensions` page shows a numeric
|
|
146
|
+
example with bare numbers; do not copy it.
|
|
147
|
+
|
|
148
|
+
**"Aggregate X where Y" is a filtered measure, not a `custom` `CASE WHEN`.**
|
|
149
|
+
Write a typed measure (`sum`, `count`, `average`, …) with a `filters:` list:
|
|
150
|
+
|
|
151
|
+
```yaml
|
|
152
|
+
- name: delivery_revenue
|
|
153
|
+
type: sum
|
|
154
|
+
expr: order_amount
|
|
155
|
+
filters:
|
|
156
|
+
- expr: "order_type = 'delivery'" # raw column SQL, or {{dimension}} — see Traps
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
It reads better, and it re-aggregates: a filtered `sum` can be answered from a
|
|
160
|
+
pre-aggregation, while a `custom` measure never can (nor can `median` or
|
|
161
|
+
`number`). Reserve `custom` for what a filtered measure cannot express —
|
|
162
|
+
ratios, correlations, cross-period math.
|
|
163
|
+
|
|
102
164
|
**Derived time dimensions so the agent never writes date arithmetic.** Ship
|
|
103
165
|
`business_year`, `business_month`, `business_quarter`, `day_of_week`,
|
|
104
166
|
`is_weekend` as dimensions of their own, each re-derived from the raw column
|
|
@@ -109,8 +171,7 @@ rather than referencing the base date dimension:
|
|
|
109
171
|
expr: "toQuarter(toDate(parseDateTimeBestEffort(toString(business_date))))"
|
|
110
172
|
```
|
|
111
173
|
|
|
112
|
-
**`drivers` must state a mechanism, not a vibe.** The quality bar
|
|
113
|
-
`sales_daily`:
|
|
174
|
+
**`drivers` must state a mechanism, not a vibe.** The quality bar:
|
|
114
175
|
|
|
115
176
|
```yaml
|
|
116
177
|
- name: total_net_sales
|
|
@@ -124,11 +185,30 @@ rather than referencing the base date dimension:
|
|
|
124
185
|
|
|
125
186
|
Note what the `description` does: it names the causal path *and* says why the
|
|
126
187
|
obvious alternative (discount dollars) was rejected. The weak examples in the
|
|
127
|
-
corpus
|
|
128
|
-
`
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
188
|
+
corpus carry `direction`, `strength`, `confidence`, and a `lag`, but no
|
|
189
|
+
`description` at all, so the mechanism is never stated and a reader cannot tell
|
|
190
|
+
whether the edge was reasoned or guessed. A driver without a `description` is
|
|
191
|
+
not finished. Only 8 of 39 views use `drivers`, and the field is absent from
|
|
192
|
+
the semantic-model docs — see the traps.
|
|
193
|
+
|
|
194
|
+
## Dates: declare them as dates
|
|
195
|
+
|
|
196
|
+
A dimension whose meaning is a date or timestamp (`*_date`, `*_at`,
|
|
197
|
+
`business_date`, `created`, `event_time`, …) must be `type: date` (or
|
|
198
|
+
`type: datetime`), and its `expr` must produce a real date — not the raw
|
|
199
|
+
column, if the column is stored as a string, an integer like `20240131`, or
|
|
200
|
+
epoch seconds. Declare it `type: number` or `type: string` instead and nothing
|
|
201
|
+
complains when the view loads; the first query that filters on it compiles a
|
|
202
|
+
date literal the column rejects and fails with **`TYPE_MISMATCH`**.
|
|
203
|
+
|
|
204
|
+
Sample before you cast — stored formats differ between warehouses and between
|
|
205
|
+
tables in one warehouse, and one `SELECT <col> FROM <table> LIMIT 1` settles it.
|
|
206
|
+
Then wrap the `expr` with the cast for your warehouse:
|
|
207
|
+
[references/warehouse-casts.md](references/warehouse-casts.md). A column
|
|
208
|
+
already typed `DATE` / `TIMESTAMP` needs no cast.
|
|
209
|
+
|
|
210
|
+
Casting in the view is the fallback, not the goal: when you own the pipeline,
|
|
211
|
+
fix the column type there once (see *Repeated defensive casting* below).
|
|
132
212
|
|
|
133
213
|
## Topics are question areas, not tables
|
|
134
214
|
|
|
@@ -146,23 +226,120 @@ views:
|
|
|
146
226
|
- restaurants
|
|
147
227
|
- dining_options
|
|
148
228
|
default_filters:
|
|
149
|
-
- field: "is_voided"
|
|
229
|
+
- field: "orders.is_voided"
|
|
150
230
|
eq:
|
|
151
231
|
value: false
|
|
152
232
|
```
|
|
153
233
|
|
|
154
234
|
Name topics after the questions people ask, and let `default_filters` carry the
|
|
155
235
|
"obviously we don't count voided orders" rules so no one has to remember them.
|
|
236
|
+
A topic usually spans several views; "one topic per view" is not the pattern.
|
|
237
|
+
|
|
238
|
+
**Qualify `default_filters` fields as `view.dimension`.** A bare `field:` is
|
|
239
|
+
resolved against a view *named like the topic*, not against `base_view`. It
|
|
240
|
+
works when the topic shares its base view's name (as above, where
|
|
241
|
+
`is_voided` would also work) and points at nothing when it does not — a topic
|
|
242
|
+
`sales` over `base_view: sales_daily` needs `sales_daily.is_voided`.
|
|
243
|
+
|
|
244
|
+
## Pre-aggregations (rollups)
|
|
245
|
+
|
|
246
|
+
A rollup is a materialized partial aggregate of one view. When a query is fully
|
|
247
|
+
covered by one, the engine answers it from the materialized copy instead of
|
|
248
|
+
scanning the warehouse — a few hundred rows instead of millions. They are
|
|
249
|
+
worth adding to views behind dashboards and frequently asked KPIs. The
|
|
250
|
+
reference documents the `config.yml` block in one line and the view-level
|
|
251
|
+
block not at all, so the shape is here.
|
|
252
|
+
|
|
253
|
+
**Opt-in, per view.** A view with no `pre_aggregations:` block gets **no
|
|
254
|
+
rollups at all** — there is no implicit default (an all-dimensions rollup of a
|
|
255
|
+
wide view is about as large as the table itself).
|
|
256
|
+
|
|
257
|
+
```yaml
|
|
258
|
+
# in a .view.yml
|
|
259
|
+
refresh_key: # view-level default for every rollup below
|
|
260
|
+
every: "1h"
|
|
261
|
+
|
|
262
|
+
pre_aggregations:
|
|
263
|
+
- name: orders_by_month
|
|
264
|
+
dimensions: [order_status] # names declared in THIS view, never raw columns
|
|
265
|
+
measures: [total_orders, total_order_value]
|
|
266
|
+
time_dimension: order_date # a date/datetime dimension of this view
|
|
267
|
+
granularity: month # second|minute|hour|day|week|month|quarter|year
|
|
268
|
+
- name: orders_summary # no dimensions, no time: one summary row
|
|
269
|
+
measures: [total_orders]
|
|
270
|
+
refresh_key: # per-rollup override
|
|
271
|
+
sql: "SELECT MAX(updated_at) FROM orders"
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
- **`refresh_key` takes exactly one of `every` or `sql`** — both, neither, or
|
|
275
|
+
any other key is a parse error. `sql` re-runs against the warehouse every
|
|
276
|
+
cycle and rebuilds when its result changes, so keep it a cheap `MAX`.
|
|
277
|
+
- **A rollup that names a member the view does not declare is refused**, not
|
|
278
|
+
quietly trimmed down to the members that resolve.
|
|
279
|
+
|
|
280
|
+
**When a rollup serves a query.** All of these must hold, or the query falls
|
|
281
|
+
back to the warehouse — correct, just not accelerated:
|
|
282
|
+
|
|
283
|
+
- every requested dimension, and every filtered dimension, is in the rollup's
|
|
284
|
+
`dimensions` (a filter on the `time_dimension` is also fine);
|
|
285
|
+
- every requested measure is in its `measures` **and is re-aggregatable** —
|
|
286
|
+
`count`, `count_distinct`, `sum`, `average`, `min`, `max`. A `custom`,
|
|
287
|
+
`number` or `median` measure never serves from a rollup, listed or not;
|
|
288
|
+
- the time dimension matches, and the requested granularity is the stored one
|
|
289
|
+
or coarser (a `day` rollup serves `month`; a `month` rollup cannot serve
|
|
290
|
+
`day`);
|
|
291
|
+
- **except `week`**: a `week` rollup does not serve `month`, `quarter` or
|
|
292
|
+
`year`, because a week straddling a month boundary would land in the wrong
|
|
293
|
+
month. Store `day` if you need both weekly and monthly answers.
|
|
294
|
+
|
|
295
|
+
**Limits.** A rollup belongs to one view and holds only that view's members,
|
|
296
|
+
and granularities are the fixed list above — no 15-minute buckets. For a
|
|
297
|
+
multi-view aggregate or a custom bucket, build a rollup *table* with an
|
|
298
|
+
automation instead (`oxy-data-architecture` owns that pattern).
|
|
299
|
+
|
|
300
|
+
**`config.yml` block** — every key optional:
|
|
301
|
+
|
|
302
|
+
```yaml
|
|
303
|
+
pre_aggregations:
|
|
304
|
+
schema: AIRLAYER # where rollup tables are built (default AIRLAYER)
|
|
305
|
+
database: warehouse # connector for builds (default: each view's own datasource)
|
|
306
|
+
refresh_worker:
|
|
307
|
+
enabled: true # default true
|
|
308
|
+
heartbeat: "10m" # how often staleness is checked (default 10m, clamped 30s–24h)
|
|
309
|
+
renewal_threshold: "120s" # how long a refresh_key result is trusted (default 120s, clamped 10s–1h)
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
## Validating a change
|
|
313
|
+
|
|
314
|
+
- **`oxy validate --file semantics/views/orders.view.yml`** parses one view or
|
|
315
|
+
topic file and reports where it fails — a bare boolean in `samples`, a
|
|
316
|
+
missing `entities:` or `dimensions:` list, a removed feature. Run it on every
|
|
317
|
+
file you touch. It does **not** look inside `pre_aggregations:` or
|
|
318
|
+
`refresh_key:`; mistakes there surface only when the model is loaded for a
|
|
319
|
+
query.
|
|
320
|
+
- Bare **`oxy validate`** (every file) only looks in `semantics/views/` and
|
|
321
|
+
`semantics/topics/`, while views and topics anywhere in the workspace
|
|
322
|
+
compile. A view kept elsewhere is only checked with `--file`.
|
|
323
|
+
- Neither checks **cross-file** wiring — an entity join that matches nothing,
|
|
324
|
+
a topic naming a view that does not exist, a measure that a twin lacks. Run
|
|
325
|
+
a real query through the topic before calling it done.
|
|
326
|
+
- **`oxy build` does not validate the semantic model** — it rebuilds search
|
|
327
|
+
embeddings and syncs integrations. `oxy sync` writes table and column
|
|
328
|
+
metadata to `.databases/`, which is the place to read real column names
|
|
329
|
+
from.
|
|
330
|
+
- `inherits_from:` and `{{ globals.* }}` were removed from views; a file using
|
|
331
|
+
either is refused at load. Inline the values.
|
|
156
332
|
|
|
157
333
|
## Traps
|
|
158
334
|
|
|
159
|
-
**
|
|
160
|
-
|
|
161
|
-
twice** (22 views, 11 twin pairs) — plus 3 further `airhouse_*`
|
|
162
|
-
ClickHouse counterpart at all, where discount- and payment-level
|
|
163
|
-
only on the new side. Three consequences, in order of how much
|
|
335
|
+
**Twin views from a backend migration are not mirrors.** In the reference
|
|
336
|
+
workspace a ClickHouse → Airhouse migration is in flight, so **11 of the 39
|
|
337
|
+
views exist twice** (22 views, 11 twin pairs) — plus 3 further `airhouse_*`
|
|
338
|
+
views with no ClickHouse counterpart at all, where discount- and payment-level
|
|
339
|
+
detail lives only on the new side. Three consequences, in order of how much
|
|
340
|
+
time they cost:
|
|
164
341
|
|
|
165
|
-
1. **The twins
|
|
342
|
+
1. **The twins diverge.** `sales_daily` defines a `discount_rate` measure;
|
|
166
343
|
`airhouse_sales_daily` does not. Switch topics mid-conversation from one
|
|
167
344
|
family to the other and a query that worked seconds ago fails with "measure
|
|
168
345
|
not found", with nothing anywhere saying the two views were ever supposed to
|
|
@@ -172,30 +349,27 @@ only on the new side. Three consequences, in order of how much time they cost:
|
|
|
172
349
|
the edges — as `airhouse_restaurant`, `airhouse_order`, `airhouse_check`,
|
|
173
350
|
while its twin uses bare `restaurant`, `order`, `check`. Joins resolve by
|
|
174
351
|
matching entity *names*, so a topic mixing families does not error — it just
|
|
175
|
-
fails to auto-join. You get missing rows, not a config failure.
|
|
176
|
-
|
|
352
|
+
fails to auto-join. You get missing rows, not a config failure. Keep each
|
|
353
|
+
topic within one family.
|
|
177
354
|
|
|
178
355
|
**The prefix is not universal, and the exception runs the other way.** The
|
|
179
|
-
three twin-less views
|
|
180
|
-
`
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
so
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
silently, instead of missing rows silently. No such collision exists in the
|
|
188
|
-
corpus today (grep: each of the three names is declared in exactly one
|
|
189
|
-
view). Prefix the primary too on any new `airhouse_*` view.
|
|
356
|
+
three twin-less views leave their **primary** entity bare
|
|
357
|
+
(`discount_application`, `payment`, `selection_discount_application`).
|
|
358
|
+
Their foreign entities are still prefixed, so rule 2 holds — but the latent
|
|
359
|
+
failure inverts. A bare primary is exactly what an old-family view would
|
|
360
|
+
declare, so if anyone adds a `payment` entity on the old side, the two
|
|
361
|
+
families start auto-joining *across* the divide: wrong rows silently,
|
|
362
|
+
instead of missing rows silently. Prefix the primary too on any new view in
|
|
363
|
+
a migrated family.
|
|
190
364
|
3. **Do not copy the pattern.** Duplicating a view to change backends is a
|
|
191
365
|
migration tactic with an end date, not a modelling technique.
|
|
192
366
|
|
|
193
|
-
**Repeated defensive casting is a smell.** `
|
|
194
|
-
toString(business_date)))` appears in **6
|
|
195
|
-
the same three-deep cast chain re-typed six times
|
|
196
|
-
is not a date. Fix the column type once,
|
|
197
|
-
every `expr`. (Counting
|
|
198
|
-
inside `measures:`, not `dimensions:`.
|
|
367
|
+
**Repeated defensive casting is a smell.** In the reference `sales_daily`,
|
|
368
|
+
`toDate(parseDateTimeBestEffort(toString(business_date)))` appears in **6
|
|
369
|
+
derived dimensions** — the same three-deep cast chain re-typed six times
|
|
370
|
+
because the underlying column is not a date. Fix the column type once,
|
|
371
|
+
upstream in the pipeline, rather than in every `expr`. (Counting by grep
|
|
372
|
+
over-reports: a seventh occurrence is inside `measures:`, not `dimensions:`.)
|
|
199
373
|
|
|
200
374
|
**`drivers` and `parent:` are documented, but not where you would look.** Neither
|
|
201
375
|
appears in the semantic-model pages — `drivers` is explained under
|
|
@@ -205,12 +379,18 @@ the metric-tree page links back to `measures` claiming it defines the measures
|
|
|
205
379
|
"and their drivers", which it does not. Read only the semantic-model section and
|
|
206
380
|
you will conclude both fields do not exist and model without them.
|
|
207
381
|
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
the
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
382
|
+
**`{{ }}` references: bare for the same view, qualified for another.** Inside a
|
|
383
|
+
measure's `expr` or `filters:`, `{{is_voided}}` names a dimension or measure
|
|
384
|
+
of the *same* view and is expanded for you — that is the norm in the corpus.
|
|
385
|
+
`{{view.member}}` reaches another view's member and is the rare case: across
|
|
386
|
+
39 views it appears in exactly one expression (`discount_rate`, which uses it
|
|
387
|
+
twice on one line, so a grep shows 2 hits from 1 site). A `{{ }}` that names
|
|
388
|
+
nothing resolvable fails the query with an "unresolved reference" error rather
|
|
389
|
+
than reaching the warehouse. Older guidance that says never to put `{{ }}` in a
|
|
390
|
+
filter predates this and is wrong; raw column SQL (`status = 'completed'`)
|
|
391
|
+
remains fine too.
|
|
392
|
+
|
|
393
|
+
**Do not add a `# yaml-language-server: $schema=…` header.** No JSON schema is
|
|
394
|
+
published for `.view.yml` or `.topic.yml`; a header pointing at one makes an
|
|
395
|
+
editor validate against nothing — or against the wrong file — while looking as
|
|
396
|
+
if it works.
|