@oxy-hq/cli 0.7.0 → 0.9.0

package/skills/oxy-semantic-modeling/references/warehouse-casts.md

@@ -0,0 +1,57 @@
+# Casting a date column per warehouse
+
+Use when a dimension whose meaning is a date or timestamp sits on a column that
+is **not** stored as `DATE` / `DATETIME` / `TIMESTAMP`. Declare the dimension
+`type: date` (or `type: datetime`) and wrap its `expr` with a cast, so filters
+compile against a real date instead of failing with `TYPE_MISMATCH` on the
+first query that filters on it.
+
+**Sample first.** Stored formats differ between warehouses and between tables in
+one warehouse — ISO strings (`"2024-01-31"`), compact strings (`"20240131"`),
+integers (`20240131`), epoch seconds or milliseconds. One
+`SELECT <col> FROM <table> LIMIT 1` tells you which you have; guessing is cheap
+to get wrong and the failure only shows at query time.
+
+| Warehouse  | Cast functions |
+| ---------- | -------------- |
+| ClickHouse | `toDate(<col>)`, `toDateTime(<col>)`, `parseDateTimeBestEffort(<col>)`, `toDate(toString(<col>))` for integer-encoded dates |
+| BigQuery   | `CAST(<col> AS DATE)`, `PARSE_DATE('<format>', <col>)`, `PARSE_TIMESTAMP('<format>', <col>)`, `TIMESTAMP_SECONDS(<col>)` |
+| Snowflake  | `TO_DATE(<col>[, '<format>'])`, `TO_TIMESTAMP(<col>)`, `TRY_TO_DATE(<col>)` |
+| Postgres   | `(<col>)::date`, `(<col>)::timestamp`, `to_date(<col>, '<format>')`, `to_timestamp(<col>)` |
+| DuckDB / Airhouse | `CAST(<col> AS DATE)`, `strptime(<col>, '<format>')`, `to_timestamp(<col>)`, `epoch_ms(<col>)` |
+
+A column already typed as a date or timestamp needs no cast — `expr: <col>` is
+enough.
+
+## Examples
+
+```yaml
+# ClickHouse: business_date stored as Int32 like 20240131
+- name: business_date
+  type: date
+  expr: "toDate(toString(business_date))"
+
+# BigQuery: order_date stored as STRING like "2024-01-31"
+- name: order_date
+  type: date
+  expr: "PARSE_DATE('%Y-%m-%d', order_date)"
+
+# Snowflake: created_at already a TIMESTAMP — no cast
+- name: created_at
+  type: datetime
+  expr: created_at
+```
+
+## Derived parts re-derive from the raw column
+
+A derived dimension (`business_month`, `day_of_week`, …) repeats the cast
+inside its own `expr` rather than referencing the base date dimension:
+
+```yaml
+- name: business_quarter
+  type: number
+  expr: "toQuarter(toDate(toString(business_date)))"
+```
+
+If the same cast chain appears in many dimensions, the column type is the real
+problem — fix it once in the pipeline that loads the table, then drop the casts.

package/skills/oxy-semantic-modeling/templates/topic-template.yml

@@ -0,0 +1,38 @@
+# Oxy topic template — copy, rename to <name>.topic.yml, and fill in.
+# A topic is a question area: a base view plus the views it may join to.
+# Name it after the questions people ask, not after a table; most topics
+# span several views that share entity names.
+
+name: my_topic_name
+description: "Which questions this topic answers, and what it always excludes"
+base_view: primary_view_name  # the view queries start from
+views:
+  - primary_view_name
+  # - related_view_name  # joins resolve through matching entity names
+
+# Optional: filters applied to every query in this topic, ANDed with the
+# user's own. Write `field:` as view.dimension — a bare name is looked up in a
+# view named like the TOPIC, not the base view, so it only works when the two
+# share a name.
+# default_filters:
+#   - field: primary_view_name.is_voided
+#     eq:
+#       value: false
+#
+#   - field: primary_view_name.status
+#     not_in:
+#       values: ["cancelled", "test"]
+#
+#   - field: primary_view_name.amount
+#     gte:
+#       value: 0
+#
+#   - field: primary_view_name.created_date
+#     in_date_range:
+#       from: "2024-01-01"  # a date, or a relative expression
+#       to: "now"
+
+# Operators (the operator name is the key; its value sits under it):
+# - eq / neq / gt / gte / lt / lte:      value: <scalar>
+# - in / not_in:                         values: [<scalar>, ...]
+# - in_date_range / not_in_date_range:   from: <date>, to: <date>

package/skills/oxy-semantic-modeling/templates/view-template.yml

@@ -0,0 +1,119 @@
+# Oxy view template — copy, rename to <name>.view.yml, and fill in.
+# One view = one table (or `sql:`) plus the vocabulary an agent may use on it.
+# No `# yaml-language-server: $schema=` header: no schema is published for views.
+
+name: my_view_name
+description: "What a row is, when the data is complete, and what it excludes"
+datasource: "database_name"  # a database name from config.yml
+table: "schema_name.table_name"
+
+entities:
+  # Primary entity: the grain of one row (one per view).
+  - name: primary_entity_name
+    type: primary
+    description: "What one row represents"
+    key: primary_key_dimension  # a DIMENSION name below, never a raw column
+
+  # Foreign entity: one per foreign-key column. Use the SAME name as the other
+  # view's primary entity — matching names are what create the join.
+  # - name: related_entity_name
+  #   type: foreign
+  #   description: "The related row in another view"
+  #   key: foreign_key_dimension
+
+dimensions:
+  # Key dimensions (entity keys reference these by name)
+  - name: primary_key_dimension
+    type: string
+    description: "Primary key"
+    expr: primary_key_column
+
+  # Regular dimensions (filterable / groupable)
+  - name: dimension_name
+    type: string  # string | number | date | datetime | boolean
+    description: "Business-friendly description"
+    expr: column_name_or_sql_expression
+    samples: ["VALUE_A", "VALUE_B"]  # the FULL set of valid values for enum-like fields
+    synonyms: ["alternative name", "abbreviation"]
+
+  # Date dimensions: type date/datetime, and expr must yield a real date.
+  # If the column is a string, integer or epoch, wrap it in a cast (sample one
+  # row first) — otherwise date filters fail with TYPE_MISMATCH.
+  - name: created_date
+    type: date
+    description: "Date the record was created"
+    expr: created_at
+
+  # Derived date parts, so the agent never writes date arithmetic
+  - name: created_year
+    type: number
+    description: "Year the record was created"
+    expr: "EXTRACT(YEAR FROM created_at)"
+    synonyms: ["year"]
+
+  # Boolean dimensions
+  - name: is_active
+    type: boolean
+    description: "Whether the record is active"
+    expr: "status = 'active'"
+    samples: ["true", "false"]  # samples are ALWAYS strings, even for boolean/number dimensions
+
+measures:
+  - name: total_records
+    type: count
+    description: "Number of records"
+    synonyms: ["record count"]
+
+  - name: total_amount
+    type: sum
+    description: "Sum of amounts"
+    expr: amount_column
+    synonyms: ["total"]
+
+  - name: average_amount
+    type: average
+    description: "Average amount per record"
+    expr: amount_column
+
+  # Filtered measure: "aggregate X where Y". Each filter expr is a SQL
+  # condition over this view's columns, or a bare {{dimension}} naming a
+  # dimension of THIS view. Prefer this over a custom CASE WHEN: it can be
+  # answered from a pre-aggregation, a custom measure never can.
+  - name: completed_amount
+    type: sum
+    description: "Sum of amounts for completed records only"
+    expr: amount_column
+    filters:
+      - expr: "status = 'completed'"
+
+  # Custom measure: only for what a filtered measure cannot express (ratios,
+  # correlations, cross-period math). The expr is already an aggregate.
+  # Never served from a pre-aggregation (nor are median or number measures).
+  - name: completion_rate
+    type: custom
+    description: "Share of records that are completed"
+    expr: "SUM(CASE WHEN status = 'completed' THEN 1 ELSE 0 END) / NULLIF(COUNT(*), 0)"
+
+# Freshness — OPTIONAL but cheap. Every value is a one-element LIST of strings.
+# Without freshness_watermark_column the agent uses the first date dimension.
+# meta:
+#   freshness_watermark_column: ["created_date"]
+#   freshness_expected_cadence: ["hourly"]
+#   freshness_complete_through: ["previous full hour (UTC)"]
+#   freshness_caveat: ["The current hour is partial."]
+
+# Pre-aggregations (rollups) — OPTIONAL and opt-in: with no block, the view has
+# no rollups at all. Members are names declared above, never raw columns.
+#
+# refresh_key:                   # view-level default; exactly ONE of every / sql
+#   every: "1h"
+#
+# pre_aggregations:
+#   - name: by_month
+#     dimensions: [dimension_name]
+#     measures: [total_records, total_amount]  # count/count_distinct/sum/average/min/max
+#     time_dimension: created_date             # a date/datetime dimension above
+#     granularity: day   # second|minute|hour|day|week|month|quarter|year; serves this
+#                        # grain or coarser — but a `week` rollup never serves month+
+#     refresh_key:       # optional per-rollup override
+#       sql: "SELECT MAX(updated_at) FROM schema_name.table_name"

package/template/.github/workflows/publish.yaml

@@ -160,7 +160,7 @@ env:
   # service account are honoured from 0.7.0, which tries them first and falls
   # back to the publisher. A pin older than that still publishes, through the
   # publisher alone.
-  OXYC_VERSION: "0.7.0"
+  OXYC_VERSION: "0.9.0"
 
 jobs:
   build:
@@ -281,25 +281,27 @@ jobs:
 
       # pnpm BEFORE setup-node: setup-node's `cache: pnpm` shells out to
       # `pnpm store path`, so it fails if pnpm is not already installed.
+      #
+      # v6, the first major that supports pnpm 11; v4 does not. And NO
+      # `version:` input, on purpose: the action installs the pnpm that
+      # package.json's `packageManager` pins, and it fails this step
+      # ("Multiple versions of pnpm specified") when a `version:` differs from
+      # that pin — `11` against `11.22.0` included. A package.json with no pin
+      # fails here too ("No pnpm version is specified"); the fix is the
+      # template's `packageManager` line, which `oxyc update` shows in its
+      # package.json diff, not a `version:` in this managed file.
       - name: Install pnpm
         if: steps.discover.outputs.count != '0'
-        uses: pnpm/action-setup@v4
-        with:
-          # package.json floors pnpm at >=10 and there is no `packageManager`
-          # field for the action to read, so the major goes here. 10 is not
-          # cosmetic: pnpm 9 reads no settings at all from
-          # pnpm-workspace.yaml, which silently makes its
-          # `onlyBuiltDependencies` block inert and blocks an app's
-          # esbuild/sharp postinstall with no diagnostic.
-          version: 10
+        uses: pnpm/action-setup@v6
 
       - name: Install Node
         if: steps.discover.outputs.count != '0'
         uses: actions/setup-node@v4
         with:
-          # .nvmrc is the single source of truth (20). This org's CI examples
-          # disagree — one says 26 — and 20 is what the real working monorepo
-          # pins, so read the file rather than restating a number here.
+          # .nvmrc is the single source of truth (26, what the customer-apps
+          # monorepo pins), so read the file rather than restating a number
+          # here. Whatever it says must be >= 22.13: that is the Node pnpm 11
+          # requires, and package.json's `engines.node` says so too.
           node-version-file: .nvmrc
           cache: pnpm
           cache-dependency-path: pnpm-lock.yaml

package/template/.nvmrc

@@ -1 +1 @@
-20
+26

package/template/.oxyc-managed

@@ -239,11 +239,11 @@ mixed     package.json
 # which is `mixed`, not a judgement call between the two.
 mixed     .gitignore
 
-# Same shape, and the file says so itself: the `onlyBuiltDependencies` block
-# carries the three entries the shipped app bundles need, with an instruction
-# to ADD to the list rather than replace it. The `packages:` globs are the
-# tool's; a fourth dependency that needs its install script run is the
-# customer's.
+# Same shape, and the file says so itself: the `allowBuilds` map carries the
+# three entries the shipped app bundles need, with an instruction to ADD to
+# the map rather than replace it. The `packages:` globs and
+# `confirmModulesPurge` are the tool's; a fourth dependency whose install
+# script somebody decided on is the customer's.
 mixed     pnpm-workspace.yaml
 
 # --- scope: which tree each file describes ----------------------------------

package/template/README.md

@@ -468,13 +468,24 @@ should.
 
 ## Node and pnpm
 
-Node **20** (`.nvmrc`), matching the working app monorepo this build config is
-derived from. Use **pnpm**, never npm or yarn — there is no `package-lock.json`
-here and there should not be one.
-
-pnpm is floored at **>= 10**, one major above that monorepo's `>= 9`, and the
-difference is deliberate: pnpm 9 reads no settings at all from
-`pnpm-workspace.yaml`, so on pnpm 9 the `onlyBuiltDependencies` block there is
-silently inert and an app's `esbuild` / `sharp` postinstall is blocked with no
-diagnostic. A floor that admits a version where the config does nothing is
-worse than no floor.
+Node **26** (`.nvmrc`), matching the customer-apps monorepo this build config
+is derived from. Anything from **22.13** up works (`engines.node`): that is the
+floor pnpm 11 itself requires, so a Node below it cannot run pnpm here at all.
+Use **pnpm**, never npm or yarn — there is no `package-lock.json` here and
+there should not be one.
+
+pnpm is **pinned**, not just floored: `packageManager` in `package.json` names
+pnpm 11.22.0, and that one line decides the pnpm everywhere. `corepack enable
+pnpm` installs exactly it; a pnpm 10 already on your PATH switches itself to
+it; CI's `pnpm/action-setup` installs it — which is why the workflow carries
+no `version:` of its own: the action refuses to run with two. To move to
+another pnpm, change that line (`corepack use pnpm@<version>`).
+
+Dependencies' install scripts are approved in `pnpm-workspace.yaml`, in the
+`allowBuilds` map. pnpm 11 **fails the install** (`ERR_PNPM_IGNORED_BUILDS`)
+on a dependency with a build script the map does not decide, so the first
+install after adding such a dependency stops until somebody sets it to `true`
+(run the script) or `false` (skip it) there and commits that. A repo that
+still carries the older `onlyBuiltDependencies` list gets that failure even
+for the dependencies it lists: pnpm 11 removed the key and ignores it without
+a word.

package/template/package.json

@@ -1,9 +1,10 @@
 {
   "name": "__SLUG__",
   "private": true,
+  "packageManager": "pnpm@11.22.0+sha512.1ff870c4c6133dfd88fb2afc46dd13d47f09c9794b438c6fdb47ca98caf3bc16381ee0be93a091b8e3824cf01f889f46d7d9e20910fb0be1ab0fb5baa80dd621",
   "engines": {
-    "node": ">=20",
-    "pnpm": ">=10"
+    "node": ">=22.13",
+    "pnpm": ">=11"
   },
   "scripts": {
     "dev": "./scripts/dev.sh",

package/template/pnpm-workspace.yaml

@@ -12,21 +12,32 @@ packages:
   - "apps/*"
   - "apps/*/*"
 
-# pnpm 10 does not run a dependency's install/postinstall script unless the
-# package is named here. These three are what the real app bundles need
-# (esbuild and unrs-resolver fetch a platform binary, sharp builds one), so
-# without this block the first app added here fails to build. Add to the list
-# rather than replacing it.
+# Which dependencies may run their install/postinstall scripts. pnpm 11 runs
+# none it has not been told about, and FAILS the install
+# (`ERR_PNPM_IGNORED_BUILDS`) on any dependency with a build script this map
+# does not decide. These three are what the real app bundles need: esbuild and
+# unrs-resolver check for, or fetch, a platform binary, sharp builds one when
+# no prebuilt fits. esbuild is in every app that has Oxy Functions — `oxyc
+# publish` bundles them with `pnpm exec esbuild`. Add to the map rather than
+# replacing it: a new dependency with a build script stops the install until
+# somebody writes `true` (run it) or `false` (skip it) for it here.
 #
-# The key is `onlyBuiltDependencies`, NOT the `allowBuilds` that customer-apps
-# carries. Verified against pnpm 10.8.0: a wrong-typed `onlyBuiltDependencies`
-# crashes inside `createAllowBuildFunction`, proving pnpm reads it, while a
-# wrong-typed `allowBuilds` installs cleanly — pnpm never looks at that key at
-# all, with no warning. customer-apps' block is therefore inert; ours is not.
-#
-# pnpm 9 ignores this file's settings entirely (same probe, pnpm 9.15.9:
-# no crash), which is why package.json floors pnpm at >=10 rather than >=9.
-onlyBuiltDependencies:
-  - esbuild
-  - sharp
-  - unrs-resolver
+# The key is `allowBuilds`. pnpm 11 removed `onlyBuiltDependencies`, the list
+# this file used to carry, and ignores it without a word: with esbuild listed
+# there, pnpm 11.22.0 still fails an app's install on `Ignored build scripts:
+# esbuild@0.28.2`. pnpm 10.33 reads `allowBuilds` too, though it rarely gets
+# the chance: package.json pins pnpm 11.22.0 (`packageManager`), and a pnpm 10
+# switches itself to that version before installing anything.
+allowBuilds:
+  esbuild: true
+  sharp: true
+  unrs-resolver: true
+
+# pnpm 11 lays `node_modules` out for a new store version, so its first install
+# over a tree pnpm 10 wrote has to delete the tree and recreate it — and it
+# asks first. With no terminal to ask on — an agent or a script running
+# `pnpm install`, or `oxyc publish`, which runs one — it aborts instead, with
+# `ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY`. Nothing is lost by not asking:
+# the tree is rebuilt from the lockfile. CI never hits it — a runner starts
+# with no `node_modules`.
+confirmModulesPurge: false