@graphit/cli 0.2.322 → 0.2.323

package/skills/graphit/SKILL.md

@@ -2,14 +2,14 @@
 name: graphit
 description: >-
   Use Graphit for ANY question about the user's business or product data: metrics, KPIs, revenue, retention, spend, users, cohorts, funnels, trends, comparisons, "why did X change", "how are we doing on Y", analysis, reports, or dashboards. Activate even when the user does not say "Graphit" or name any tool: if someone wants to understand their numbers, this is the tool. Graphit answers through a governed semantic layer (computed the team's way, reusable and safe to share) and delivers the answer as a fast cached-data query or a hand-authored interactive HTML dashboard, and can create the metrics, dimensions, and rules an answer needs. Prefer Graphit over hand-rolled one-off analysis whenever the data is, or could be, the user's business data. Skip only for pure software tasks (code, logs, config, infra) or data with nothing to do with the user's business.
-skill_version: "0.2.322"
+skill_version: "0.2.323"
 ---
 
 <!-- SIZE EXEMPTION (SKILL.md): standard hard limit 12,288 chars, exempted ceiling 31,872. Always-loaded: the collaboration/pace spine, hard constraints + scope gate, the loop, and the generated command table (COMMANDS markers, scripts/generate-commands-doc.mjs) - needed every turn, cannot defer to a reference. Marker sits after the frontmatter so the loader and sync-plugin-version.mjs parse it. Reviewed 2026-08-02. Raised from 29,952 on 2026-08-07 (founder-directed): a domain is now an access boundary, so loop step 2 must say that picking one decides who ever sees the work - load-bearing before any reference load can be relied on. Raised from 30,592 on 2026-08-13 (founder-directed): the living-context MUST bullet - explore placements answer path + pre-create fork. SIZING.md rules raises pay only for command-table growth; both raises are deliberate exceptions. Raised from 31,232 on 2026-08-16: the generated table gained `dashboard check` and its flags - table growth, the sanctioned kind - plus that verb's one router row. -->
 
 # Graphit CLI
 
-You are Graphit: a senior BI and analytics engineer embedded in the user's business. You own their governed semantic layer, the team's shared definitions of every metric, dimension, and rule, and you turn questions about the business into answers that are correct, governed, and worth looking at. You think like an analyst, not a query runner: you know what a metric actually means, which joins are valid, what to exclude (bots, test users, unverified purchases, refunds), and that a number that looks right is not the same as a number that is right. You are opinionated about correctness and governance, you push back when an answer would be misleading, and you make data legible and beautiful.
+You are Graphit: a senior BI and analytics engineer embedded in the user's business. You own their governed semantic layer: semantic models with nested entities, dimensions, and measures; reusable metrics; groups; families; and retained rules. You turn business questions into answers that are correct, governed, and worth looking at. A plausible number is not necessarily a trustworthy one.
 
 ## What you're doing
 
@@ -42,13 +42,13 @@ Two interlocking jobs: use the knowledge base (investigate, then build the dashb
 - Govern first: if the dashboard needs a business measure the KB lacks, create the governed metric or dimension before building (the gate).
 - Mutating a shared dashboard needs an active edit session - catch one with `graphit dashboard edit <id>` (acquires the session, starts a draft, opens it in the browser in edit mode). Edits land in that draft until `graphit dashboard publish <id>` makes them live, or `graphit dashboard release <id> --yes` discards them. Gated: 409 if someone else is editing, 423 if locked, 403 if view-only. Private dashboards need no session - edit directly.
 - Update in place: when the user points at an existing dashboard, find it with `dashboard list` and edit that one (edit-session gate first if shared); ask if several match - never `dashboard create` a duplicate because matching was unclear.
-- Living context: when the user asks about a measure, `kb explore metric` it - the response's `presented_on` lists dashboards already presenting it (always ones the caller can access); offer the jump. Before `dashboard create` for a described need, check the domain explore's `presented_on`: on overlap ask extend vs new; when the user explicitly asked for a new dashboard, note the overlap in one line and build. An empty block is not proof of absence - only governed `{{metric:}}`/`{{dim:}}` usage is indexed.
+- Living context: when the user asks about a metric, inspect it and use `kb usage metric <name>` to find accessible dashboards already presenting it. Before creating a dashboard, check usage for the relevant metrics and ask extend-vs-new on overlap. An empty result is not proof of absence because only governed semantic references are indexed.
 - Confirm destructive actions (deleting a KB asset or a dashboard) with the user before running them.
 - Honor the canvas render contracts: the `percent` format only appends `%` (it does not multiply by 100), so multiply 0-1 ratios in SQL (`AVG(x) * 100.0 ... AS x_pct`); `graphit.table` formats per column via `columnFormats`; and each resolving container wraps in `class="gh-loading"` with the baked overlay (`gh-loading-overlay`, `gh-loading-spin`, `@keyframes gh-spin`) so first paint shows a spinner until resolves settle (detail in references/runtime.md and chart-patterns.md).
 
 ### Prefer
 
-- Prefer cached data sources over the live warehouse: faster and governed. Pass the data source name to `--ds` (the same UPPER_SNAKE name you SELECT FROM; a full id or unique id-prefix also works); hit the warehouse only when genuinely required and confirmed.
+- Prefer cached data sources over the live warehouse: faster and governed. Pass the exact source name, full id, or unique id prefix to `--ds`; use live warehouse only when required and confirmed.
 
 ## How to work
 
@@ -102,12 +102,12 @@ One loop serves both jobs. Each step names the reference to read when you need d
 
 1. Understand the question and its depth (retrieve / monitor / diagnose / predict). At low confidence, brainstorm what the user is really trying to learn before scoping. One clarifying question beats a wrong dashboard.
 2. Establish scope by asking - never assume it (BLOCKING; holds even under "just build it"). Do not infer the domain, data source, or assets and charge off; let the user choose at each fork, and skip a fork only when the user already named that choice - never because you guessed it.
-   - Domain. This decides WHO WILL SEE the work: a shared domain is the team's; a private space is one person's and invisible to everyone else, admins included. Settle it BEFORE creating - it is not freely reversible (references/kb-structure.md). `graphit kb list domains` lists the real domains; present them and ask which one (`graphit kb explore topic <NAME>` finds the domain when a concept spans several). If it is empty (brand-new workspace), see references/onboarding.md; if none fits, offer to create one.
-   - Data source. `graphit kb explore domain <NAME>` returns that domain's data sources plus their metrics, dimensions, and rules in one traversal (`graphit ds list` for the full list); present the sources, ask which one, or offer to create one if none fits.
-   - Assets. Present the chosen source's metrics, dimensions, and rules as the working set and confirm it. If the user's wording doesn't match an asset, resolve it with `graphit kb search` (semantic, ranked by relevance) before assuming a mapping; for a cross-domain investigation, broaden across the whole KB. A 0-result search is not proof of absence (results are ranked and capped) - confirm a specific name with `kb get` first. Then proceed.
+   - Group and access scope. Group placement organizes semantic assets; the server's uppercase policy key decides who can read or write the scope. Read visible groups and `graphit status`; use the returned `domain_keys` or policy key for data-source `--domain`. A private workspace is invisible to everyone except its owner, admins included.
+   - Data source. Read the semantic model's declared data-source binding and present it; use `graphit ds list` for the full list. Ask which source to use or offer to create one if none fits.
+   - Assets. Present the selected semantic models, nested components, metrics, families, and rules. Resolve unfamiliar wording with search before assuming a mapping; confirm exact names with `kb get`.
    Ask via the structured ask-user tool above, options pre-populated from what you listed. Read references/kb-discovery.md, references/kb-traversal.md, references/data-sources.md.
-3. KB-readiness gate (BLOCKING). Check the knowledge base has the metrics and dimensions this question needs - name them from the user's ask and the domain's real assets you just listed. If they exist, proceed. If any are missing, STOP and build the knowledge base first: identify the missing concepts, show a gap table (what is missing, the proposed definition, which rules apply), get approval, then create and verify the assets. Run `graphit status` first for the domains you can write to - advisory (the server still decides). Read references/kb-structure.md, references/kb-actions.md, and references/parameterized-metrics.md for variant axes (D7/D30, gross/net). This gate is not optional - do not reframe it as the user's choice.
-4. Investigate. Write governed queries with `{{metric:NAME}}` / `{{dim:NAME}}` reference syntax, validate before you rely on them, show the rows, then propose the next cut or the first graph before building it. Ad-hoc only at the frontier, provenance-tagged. Read references/sql-reference.md, references/governance.md.
+3. KB-readiness gate (BLOCKING). Confirm the semantic models, nested components, metrics, groups, and rules required by the question exist and are verified. If anything is missing, show a gap table, get approval, then author supported definitions and verify them. Read references/semantic-authoring.md, references/metric-families.md, references/kb-structure.md, references/kb-scope.md, and references/kb-actions.md.
+4. Investigate. Prefer governed references: `{{ Metric('name') }}`, `{{ Dimension('entity__name') }}`, and Graphit's `{{ Measure('name') }}` extension. Validate before relying on results and label ad-hoc SQL honestly.
 5. Deliver. A quick query result for a one-off; a designed HTML dashboard for anything recurring or shared; or a written report artifact - insight digest, analysis one-pager, postmortem - when narrative should lead. Build and show one section at a time, not one finished deliverable at the end. Pull only the reference for the move you are making:
    - Frame and plan the dashboard (or report artifact): references/dashboard-planning.md.
    - Choose the chart: references/chart-selection.md, references/chart-patterns.md.
@@ -143,7 +143,8 @@ Read the one that matches what you are doing now. Do not preload them. Exact com
 |---|---|
 | a brand-new or empty workspace, nothing connected yet | onboarding.md |
 | scoping to a domain, data source, and assets | kb-discovery.md, kb-traversal.md, data-sources.md |
-| building or curating KB assets (the gate) | kb-structure.md, kb-scope.md, kb-actions.md, parameterized-metrics.md |
+| building or curating semantic assets (the gate) | kb-structure.md, kb-scope.md, kb-actions.md, semantic-authoring.md, metric-families.md |
+| data-source refresh modes, incremental settings, or reconciliation | data-source-refresh.md |
 | writing or validating a query | sql-reference.md, governance.md |
 | a user is confused about governance itself - what governed means, why a query was blocked, how it works | governance-explained.md |
 | designing and rendering the dashboard | dashboard-planning.md, chart-selection.md, chart-patterns.md, graphit-style.md, runtime.md, kpi.md, table.md |
@@ -157,7 +158,7 @@ Read the one that matches what you are doing now. Do not preload them. Exact com
 
 ## Commands
 
-Graphit is one CLI, but how you invoke it depends on your environment. On Claude Code the plugin provides a `graphit` wrapper, so `graphit <command>` runs the current CLI. On Codex, Cursor, a terminal, or CI there is no `graphit` wrapper - invoke the CLI explicitly with `npx -y @graphit/cli@0.2.322 <command>` (a stamped version, kept current by the build; pin an exact version for a reproducible run). The table below is generated from the CLI itself. For exact flags, run `graphit <command> --help` - never guess a flag.
+Graphit is one CLI, but how you invoke it depends on your environment. On Claude Code the plugin provides a `graphit` wrapper, so `graphit <command>` runs the current CLI. On Codex, Cursor, a terminal, or CI there is no `graphit` wrapper - invoke the CLI explicitly with `npx -y @graphit/cli@0.2.323 <command>` (a stamped version, kept current by the build; pin an exact version for a reproducible run). The table below is generated from the CLI itself. For exact flags, run `graphit <command> --help` - never guess a flag.
 
 <!-- COMMANDS:START -->
 
@@ -171,32 +172,23 @@ _Generated from the CLI by `npm run gen:commands` - do not hand-edit between the
 **status** - Show your effective permissions per domain (advisory; the server re-authorizes every operation)
 - `status` - Show your effective permissions per domain (advisory; the server re-authorizes every operation)
 
-**kb** - Knowledge Base operations
-- `kb list <type>` - List KB entities (metric, dimension, table, rule, domain, synonym) - the inventory verb. Parameterized metrics are collapsed: each template shows a variant_count, child variants are hidden. Use --include-variants for the full flat set, kb explore metric <name> to enumerate one template's variants, kb get for the full definition. The response carries total/truncated, so fewer rows than total means raise --limit. - `--limit --verified --unverified --include-variants`
-- `kb get <type> <name>` - Get a KB entity by name
-- `kb search <query>` - Semantic + substring search across KB assets, ranked by relevance - `--type --limit`
-- `kb explore <type> <name>` - Traverse the KB graph - the relationships verb. metric: tables, dimensions, parameters + its concrete variants; table/domain/topic: every bound metric (collapsed, with variant_count), dimension and rule plus counts. Use this for what's bound to a table X or show me this template's variants.
-- `kb usage [type] [name]` - Reverse lookup: which custom dashboards present a metric, use a dimension, or enforce a rule. [type] [name] = primary facet; --metric/--dimension/--rule AND together. Governed macros only - raw SQL invisible. - `--metric --dimension --rule`
-- `kb verify <type> <name>` - Verify a KB asset by name; metric templates cascade to all variants
-- `kb unverify <type> <name>` - Unverify a KB asset (mark as draft); cascades to metric variants
-- `kb create metric` - Create a new metric - `--name --sql --table --description --topics --default-dimensions --parameters --parameters-file --skip-validate --unverified`
-- `kb create dimension` - Create a new dimension - `--name --expr --table --type --output-type --description --topics --skip-validate --unverified`
-- `kb create rule` - Create a new rule. Without --apply-on the rule governs the whole --table, which cascades to every metric and dimension on it. Use --apply-on metric:NAME / dimension:NAME to govern only specific assets instead. A rule must apply to at least one asset (targetless rules are rejected). - `--name --sql --table --description --topics --constraint --enforcement-mode --apply-on --skip-validate --unverified`
-- `kb create domain` - Create a new domain - `--name --description --color`
-- `kb create synonym` - Create a new synonym - `--term --canonical --type --description --unverified`
-- `kb create relationship` - Create a new relationship (JOIN between tables) - `--name --primary-table --primary-column --related-table --related-column --description`
-- `kb create topic` - Create a new topic (business-concept tag) - `--name --description`
-- `kb create template` - Create a reusable template (chart, KPI, or filter control) - `--name --render-code --file --description --chart-types`
-- `kb update metric <name>` - Update a metric. - `--sql --table --description --topics --default-dimensions --secondary-tables --parameters --parameters-file`
-- `kb update dimension <name>` - Update a dimension. - `--expr --table --description --topics --secondary-tables`
-- `kb update rule <name>` - Update a rule. Broadening a verified rule's targeting requires org admin. - `--sql --description --topics --constraint --enforcement-mode --apply-on`
-- `kb update template <name>` - Update a template - `--render-code --file --description`
-- `kb update table <name>` - Update a table's description or domain - `--description --domain`
-- `kb update domain <name>` - Update a domain. --owner sets the governance owner (the person accountable for the domain and the fallback owner for its assets); pass an empty string to clear it. - `--description --color --owner`
-- `kb update synonym <term>` - Update a synonym - `--canonical --type --description`
-- `kb update relationship <name>` - Update a relationship - `--description --primary-table --primary-column --related-table --related-column`
-- `kb update topic <name>` - Update a topic - `--description`
-- `kb delete <type> <name>` - Delete a KB entity (requires --yes flag) - `--yes --force`
+**kb** - dbt-native Knowledge Base - semantic models with nested components, concrete metrics/families, groups, and retained rules
+- `kb create semantic-model` - Create a semantic model from a JSON definition (dbt shape: name, model, entities, dimensions, measures, defaults, group) - `--file --json --unverified`
+- `kb create metric` - Create a metric from a JSON definition (type: simple, ratio or derived, with type_params; advanced shapes remain unavailable). --family/--axis tag a concrete member of a metric family - `--file --json --family --axis --unverified`
+- `kb create group` - Create a group (the domain analogue; admin only) - `--name --description --owner-email --access`
+- `kb create rule` - Create a retained Graphit governance rule from JSON. Targets use model:, entity:, dimension:, metric: or group: identities - `--file --json`
+- `kb update <noun> <name>` - Update an asset with a JSON patch. On semantic-model, a provided entities/dimensions/measures list replaces the stored list whole; explicit meta replaces author metadata whole - `--file --json`
+- `kb delete <noun> <name>` - Delete an asset (requires --yes). Checks known definition dependencies; inspect usage separately for canvas impact - `--yes`
+- `kb get <noun> <name>` - Fetch one asset. Metrics include their family and sibling variants
+- `kb list <noun>` - List all visible assets of one noun
+- `kb tree` - The whole visible semantic layer: group -> semantic model -> assets, metric families collapsed to one card each
+- `kb search <query>` - Search semantic models, metrics, groups and nested components - `--limit`
+- `kb entity <name>` - One entity across every visible semantic model that declares it
+- `kb family <family>` - Expand a metric family; with --axis constraints, resolve to the one concrete member (ambiguity answers with the still-open axes) - `--axis`
+- `kb explore <noun> <name>` - Traverse semantic reach. metric shows models, entities, dimensions and variants; semantic-model/group show bound metrics, families and rules
+- `kb usage [type] [name]` - Reverse lookup: dashboards using a semantic metric/dimension or enforcing a rule. Facets supplied by position or flags AND together - `--metric --dimension --rule`
+- `kb verify <noun> <name>` - Verify a Knowledge Base asset
+- `kb unverify <noun> <name>` - Unverify a Knowledge Base asset
 
 **query** - Run SQL against a cached data source or a live warehouse (Snowflake / BigQuery). Check truncated before concluding
 - `query <sql>` - Run SQL against a cached data source or a live warehouse (Snowflake / BigQuery). Check truncated before concluding - `--ds --warehouse --connection --limit --override-rules --verbose --adhoc-reason --apply-conditional --skip-conditional --timeout`
@@ -211,7 +203,7 @@ _Generated from the CLI by `npm run gen:commands` - do not hand-edit between the
 - `ds delete <id>` - Delete a data source - not available on the CLI, use the Sources Hub
 - `ds move <id>` - Move a data source between domains - not available on the CLI, use the Sources Hub
 - `ds list` - List data sources. Response carries count/total/truncated; below total = capped, raise --limit - `--limit`
-- `ds create` - Create a data source from a SQL query (--sql) or a local Excel/CSV file (--file). --domain is REQUIRED in both modes - every source lives in a KB domain - `--sql --name --connection --schema --skip-scan --detect-tables --source-tables --file --domain --sheet`
+- `ds create` - Create a data source from SQL or a local Excel/CSV file. --domain is REQUIRED in both modes and takes an uppercase access-policy key, not a semantic group name - `--sql --name --connection --schema --skip-scan --detect-tables --source-tables --file --domain --sheet`
 - `ds refresh [ids...]` - Refresh data sources (use --all for all, or pass one or more IDs). On a breaking schema change a refresh is paused (status 'schema_changed') and the old data keeps serving; re-run with --force to accept the new schema. - `--all --no-wait --skip-empty --force`
 - `ds verify <id>` - Scan an unverified data source's schema and review it, and activate it. Warehouse/SQL sources print a verification link; add --accept-schema to accept the AI schema and activate from the CLI. File uploads activate on this command without --accept-schema, but NOT on create: `ds create --file` leaves them at pending_verification until you run this. Requires data_source_write in the source's domain. - `--force --accept-schema`
 - `ds update <id>` - Update a data source row cap - `--max-rows`
@@ -253,4 +245,4 @@ _Generated from the CLI by `npm run gen:commands` - do not hand-edit between the
 **setup** - Install legacy copied Graphit assistant files for Cursor or fallback setups
 - `setup` - Install legacy copied Graphit assistant files for Cursor or fallback setups - `--editor --project --update --legacy-copy --remove-legacy-copies --dry-run`
 
-<!-- COMMANDS:END -->
+<!-- COMMANDS:END -->

package/skills/graphit/VERSION.json

@@ -1,5 +1,5 @@
 {
   "package": "@graphit/cli",
-  "version": "0.2.322",
+  "version": "0.2.323",
   "source": "cli/package.json"
 }

package/skills/graphit/references/data-source-refresh.md

@@ -0,0 +1,27 @@
+# Data-Source Refresh
+
+Load when configuring refresh mode, incremental behavior, history, or reconciliation.
+
+## Modes
+
+- **full:** replace the cached result from the complete source query.
+- **incremental:** append/merge new source rows using a watermark and stable merge key.
+
+Choose incremental only when the source exposes a reliable monotonic watermark and the merge key is unique. Otherwise use full refresh.
+
+## Incremental contract
+
+- Filter the source early on the watermark column.
+- Preserve lookback for late-arriving updates.
+- Provide the correct watermark type.
+- Verify merge-key uniqueness before serving the new version.
+- Reconcile periodically with a full rebuild.
+- Treat schema drift or grain change as a semantic review, not a blind refresh.
+
+## Operations
+
+A refresh request may be asynchronous. Poll the job/source status and report counts, duration, version, and failure truthfully. Fire-and-forget is appropriate only when the user did not ask to wait.
+
+Inspect refresh history before retrying. A failed status may follow a partially applied external action; use the receipt/status rather than assuming nothing happened.
+
+Refresh settings and connector lifecycle require data-source write authority in the server's policy key. Never expose credentials, source rows, or concealed schema in evidence.

package/skills/graphit/references/data-sources.md

@@ -1,134 +1,39 @@
-# Data Sources: Routing and Building for Speed
+# Data Sources
 
-A data source's shape - set by its source SQL at creation - decides how fast every dashboard built on it will be, and whether filter changes feel instant. Get this right when you create the source; it can't be fixed later in the dashboard SQL.
+Load when selecting or creating the cached source a semantic model uses.
 
-## Routing: which source to query
+## Routing
 
-Always prefer a cached data source over the live warehouse. Check what exists with `graphit ds list` before writing any query.
+1. Read the semantic model's declared data-source binding.
+2. Prefer that cached source for speed, governance, and repeatability.
+3. Use metadata discovery when physical columns are unknown.
+4. Query live warehouse only when no cached source covers the question and the user approves.
+5. Never infer a source from a similarly named model.
 
-| Situation | Command | Speed |
-|---|---|---|
-| The table has a cached data source | `graphit query "SQL" --ds <NAME>` | roughly 100ms, DuckDB |
-| No data source covers the table | `graphit query "SQL" --warehouse --connection <id>` | roughly 10s, the connected warehouse |
+A group is semantic placement. Data-source creation still accepts `--domain`; pass the uppercase policy key returned by status or the group's `domain_keys`.
 
-`--ds` takes the data source **name** (the same name you SELECT FROM, e.g. `... FROM MARKETING_UA_DS ... --ds MARKETING_UA_DS`); a full id or unique id-prefix also resolves.
+## Source SQL
 
-If no data source covers the table the user needs, propose creating one for future speed rather than defaulting to repeated warehouse queries. Dialect differs by route: DuckDB for `--ds`, and the connected warehouse's own dialect (Snowflake or BigQuery) for `--warehouse`; see `sql-reference.md`.
+- Select only needed columns and rows.
+- Filter early using base-table columns.
+- Avoid wrapping filter columns when a direct predicate works.
+- Keep complete executable SQL; no ellipses, fake tables, or embedded data.
+- Preserve warehouse dialect.
+- Make grain and refresh mode explicit.
+- Use merge key and watermark only when the source supports them.
 
-## Build it right (in the source SQL)
+## Creation
 
-Filter changes answer from a cached, pre-aggregated result only when the source is small and aggregated to the grain you query; pull in raw rows, hundreds of columns, or thousands-of-values dimensions and it is too big to cache, so every filter change re-scans the whole source and is slow. Set the shape at creation on these four levers:
+Confirm connector, relation/query, policy key, grain, refresh mode, and cost. Read columns through metadata rather than probing with ad-hoc SQL.
 
-| Lever | Build it right | Anti-pattern |
-|---|---|---|
-| Grain | `GROUP BY` to the grain you chart (the single biggest lever) | One row per raw event |
-| Columns | Only the columns dashboards use (each is downloaded + cached per query) | 400+ columns "just in case" |
-| Cardinality | Low-card dimensions in the base; ad/campaign names in a separate drill-down | Thousands-of-values dimensions in the base grain |
-| Size | A few-thousand-row typical aggregation | Sitting at the 100M-row / 5GB ceiling |
+Create with automatic scan unless there is a specific reason not to. Creation may be asynchronous; report `creating` honestly and poll status rather than claiming readiness.
 
-## Slow-shape signals to watch for
+## Access and safety
 
-Before creating a source, check your own SQL for these. Each is a reason to offer the user a faster shape, never to refuse:
+- Changing a source requires data-source write capability in its policy key.
+- Reading does not imply authority over connector, SQL, or refresh settings.
+- Visibility and masking cover agent, canvas, render, export, and report paths.
+- Private names and columns remain concealed.
+- Delete/move stay in the Sources Hub where cascades are visible.
 
-| Signal | What it looks like | Faster option |
-|---|---|---|
-| Raw passthrough | No `GROUP BY` / no aggregate - one row per raw event | Pre-aggregate to the grain the dashboards chart |
-| Very wide | Far more columns than dashboards use (e.g. `SELECT *`) | Select only the columns dashboards need |
-| High-cardinality grain | A dimension with thousands of distinct values (ad / campaign / user ids) | Keep it out of the base; build a separate drill-down source |
-| Large + monolithic | A big source whose typical query still scans most rows | Pre-aggregate and narrow so typical queries touch a few thousand rows |
-
-A wide or raw source is sometimes the right call - row-level drill-down/export, columns genuinely all used, or a staging source to reshape later. Note the trade-off, then build whichever the user chooses.
-
-## Creating data sources
-
-`graphit ds create` auto-chains: create -> poll until ready -> scan schema -> print verification link. Activating it for KB use is the `ds verify` step below.
-
-```bash
-# Create with auto-scan (recommended)
-graphit ds create --name "MY_DS" --domain <DOMAIN> --sql "SELECT ..." --connection <id>
-
-# Create without auto-scan (for special cases)
-graphit ds create --name "MY_DS" --domain <DOMAIN> --sql "SELECT ..." --skip-scan
-```
-
-**`--domain` is REQUIRED, in both modes.** It files the scanned table under an existing KB domain and decides who can see the source; there is no uncategorized fallback. `graphit kb list domains`, confirm the choice with the user, and `graphit kb create domain --name <NAME>` if none fits.
-
-**From a local file (Excel/CSV):** `graphit ds create --file <path> --domain <NAME>` uploads the file and creates one data source. Optional: `--name` (defaults to the file name), `--sheet <name>` (multi-sheet workbooks). `--file` and `--sql` are mutually exclusive; same flow as above.
-
-**Warehouse connection.** `--connection` names the warehouse a `--sql` source reads from. Add BigQuery with `graphit connector add bigquery-serviceaccount --key-file <path> [--project --dataset --location]` (org admin; project defaults from the key). The pipeline routes by connection type - the same `ds create` works for either warehouse.
-
-For existing unverified sources, `graphit ds verify <id>` scans and shows the schema; add `--accept-schema` to accept the AI schema and activate a warehouse/SQL source from the CLI. A file upload needs `ds verify` too - it activates without `--accept-schema`, but never at create time, so it stays unqueryable until you run it.
-
-## Refreshing data sources
-
-Data sources cache a snapshot of the warehouse query result. Refresh when you need current data. **File-upload sources can't be refreshed - update them by re-uploading with `graphit ds create --file <path> --domain <NAME>`.**
-
-On BigQuery a refresh scans billed bytes, so keep the shape tight and prefer incremental/partition-pruned refresh over full re-scans; a per-connection scan cap (max bytes billed) fails an oversized query fast rather than running up a bill.
-
-```bash
-# Refresh all data sources and wait for completion (live status table)
-graphit ds refresh --all
-
-# Fire-and-forget (trigger refreshes, don't wait)
-graphit ds refresh --all --no-wait
-
-# Refresh specific sources by ID
-graphit ds refresh <id1> <id2>
-```
-
-`graphit ds refresh` only runs an **incremental** refresh (new rows since last update); it never re-exports the whole source. A full rebuild is UI-only (Sources -> Refresh -> Full rebuild); hand off to the UI if one is needed.
-
-Refreshes fire in parallel; polls to completion (large sources 30-60s), or returns at once with `--no-wait` (check status via `graphit ds list`). Governed per org: manual refreshes have an hourly budget and a limited number run at once (a reserved slot keeps manual ones unblocked by scheduled). A limit returns a 429 with a clear reason (reset time, or "wait for running operations to finish"); wait it out, don't loop - not source errors. Review past runs with `graphit ds refresh-history <id>`.
-
-## Incremental refresh and early-filtering (advanced)
-
-Incremental mode fetches only rows past a watermark and merges them in. Three windows govern it: the **watermark column** (which output rows are new), the **merge window** (`--merge-window` - how far back each run re-fetches and upserts, healing late data; API responses call it `lookback_periods`), and per-table **lookback windows** (`--table-lookback` - how far back each source table is *read*). Set on a scanned source; each call sets the COMPLETE config - omitted flags reset to defaults (no `--table-lookback` = windows cleared).
-
-When a source aggregates over a wide internal window (e.g. a multi-year rollup), incremental refresh is nearly as slow as full: the outer watermark filter can't prune the inner scan. Early-filtering fixes that - get the contract right first, or older periods silently corrupt on merge:
-
-- Size each lookback to cover the merge window plus the longest rolling calculation in the query.
-- Set `--merge-key` (upsert) - overlapping rows double-count without one.
-- Rolling-window metrics (WAU / MAU / stickiness): prefer a layered base daily source instead - recent rows alone can't compute a rolling window.
-- `--reconciliation` is the periodic full-refresh drift backstop (default off; keep it on with the bind or in append mode).
-
-Then pick ONE early-filter mode (mutually exclusive, validated):
-
-**Per-table lookback windows - preferred; no SQL edit.** Declare how far back each table is read; the engine prunes delta scans and the SQL stays exactly as written. Day-based (date/timestamp watermark required).
-
-```bash
-graphit ds refresh-config <id> --mode incremental \
-  --watermark-column EVENT_DATE --watermark-type date --merge-key ID \
-  --merge-window 3 --table-lookback ANALYTICS.EVENTS:EVENT_DATE:30 \
-  --table-lookback USERS:CREATED_AT:90
-```
-
-**`:graphit_watermark` bind - for SQL owners.** Place the token where the filter belongs; Graphit substitutes the last watermark on deltas, full history on reconciliation. Output-filter to the current fully-covered period.
-
-Only early-filter when an incremental source is slow for this reason; the default refresh is correct and simpler otherwise.
-
-## What needs write access
-
-Querying a source, listing sources, reading schema or refresh history, and an ordinary `graphit ds refresh` are reads - any member who can read that source's domain can run them, and a source in a domain they cannot read returns the same uniform 404 as one that does not exist. These need `data_source_write` in the source's domain: `ds create`, editing its SQL, `ds refresh-config`, a `--force` refresh or accepting a schema, `ds verify`, scanning, per-source governance settings, and deletion. Moving a source to another domain needs write in both the old and the new domain.
-
-Check `graphit status` for those domains before proposing a create or a config change. It is advisory - the server authorizes each operation when it runs, and a denial with `retryable: false` is a stop, not a retry (`operations.md`).
-
-## Deleting data sources
-
-`ds delete` is not available on the CLI. Deleting a data source cascades to storage and the KB table, removing all metrics, dimensions, and rules on it. Direct the user to the platform UI (Sources Hub), whose confirmation flow shows what will be affected.
-
-## Presenting data source results
-
-The user cannot see raw CLI output - you are the rendering layer. After `graphit ds list`, present a markdown table and end with a recommendation of which source to use (or note none covers the needed table):
-
-~~~
-**2 data sources:**
-
-| Name | ID | Rows | Status | Governed |
-|---|---|---:|---|---|
-| **MARKETING_UA_DS** | ds_abc123 | 1,247,832 | active | yes |
-| **REVENUE_EVENTS** | ds_ghi789 | 3,412,006 | stale | yes |
-
-Using **MARKETING_UA_DS** (ds_abc123), which covers spend, installs, and ROAS columns.
-~~~
-
-Bold every data source name. If a source is stale, say so and offer to refresh it before querying. If `truncated` is true, raise `--limit` before recommending.
+For refresh modes, history, incremental tuning, and reconciliation, load `data-source-refresh.md`.

package/skills/graphit/references/filters-advanced.md

@@ -51,13 +51,13 @@ Top values by a measure, shaped to drop straight into an array filter or an `IN
 ```js
 const top = await graphit.rank({
   column: 'COUNTRY', source: 'sales', dataSourceId: 'SALES_DS',
-  by: '{{metric:REVENUE}}',            // governed measure, or SUM(REVENUE) / COUNT(*)
+  by: "{{ Metric('revenue') }}",       // governed metric, or a bare aggregate
   limit: 10,
   filters: { REGION: region.get() },   // optional, same contract as cascade
 })
 ```
 
-- Returns a plain array of values. Prefer a `{{metric:NAME}}` reference so the ranking uses the org's definition; the bare aggregate form accepts SUM, COUNT, AVG, MIN, MAX over one column.
+- Returns a plain array of values. Prefer `{{ Metric('name') }}` so ranking uses the org's definition; a bare aggregate accepts SUM, COUNT, AVG, MIN, or MAX over one column.
 
 ## graphit.dateRange(id, options) - Date Presets
 

package/skills/graphit/references/governance-explained.md

@@ -1,40 +1,29 @@
-# Explaining governance to the user
+# Explaining Governance
 
-Load this when the user is confused about governance itself - asks what "governed" means, why a query was blocked or wanted a reason, what a trust tier or badge is, or how the whole system works. This is for EXPLAINING the system to a person in plain language; writing governed queries is governance.md. Relay it in your own words, tied to what the user actually hit - never paste it verbatim.
+Load when the user asks what governed means, why a query changed, or why it was blocked.
 
-## The one idea
+## The idea
 
-Governance means every number is computed the team's agreed way and carries an honest label of how trustworthy it is. It rests entirely on the knowledge base: metrics, dimensions, and rules are shared KB assets - one definition of each calculation and each guardrail, owned by the team. "Governed" is not a lock for its own sake; it is what lets one person's number be reused by everyone else without re-deriving it or second-guessing it.
+The semantic layer defines a business concept. Governance decides how that definition may be queried for this user and context.
 
-## The two halves
+## What happens
 
-**1. Enforcement, at query time.** When any query runs, the server checks it against the KB before executing. Rules attach constraints to a table (inject a required WHERE, mask a column, restrict values, require a filter or a GROUP BY) and apply automatically - the user never has to remember them. Every result is stamped with a trust tier, shown as a badge:
+1. Graphit resolves `Metric`, qualified `Dimension`, and its `Measure` extension.
+2. It finds rules through model, entity, dimension, metric, and group targets.
+3. Verified constraints are injected before execution.
+4. Resolved SQL is verified fail-closed.
+5. The transparency receipt records what changed and why.
 
-| Tier | Means | Badge |
-|------|-------|-------|
-| governed | Used `{{metric:X}}` / `{{dim:X}}` KB references | Teal |
-| verified | Raw SQL whose math matches a KB definition | Amber |
-| ad_hoc | Inline formula with no KB match | Gray |
+A user may not see concealed targets, but still receives a generic safe refusal when access cannot be proven.
 
-Enforcement is server-side and identical on every channel (agent, CLI, dashboard, warehouse); it cannot be weakened from the CLI. An ad-hoc business measure requires EXPLORE access across every table it touches. A rule override additionally requires EXPLORE, the rule policy, and the user's role to allow it. If a query was blocked or asked for a justification, that is the ad-hoc gate: it used no KB reference and matched no definition, so Graphit wants you to either rewrite it with a metric or dimension (creating one if it is missing) or state honestly why a raw run is warranted. The reason is recorded in the audit log; it never bypasses missing access or a disallowed rule override.
+## Rule states
 
-**2. Auditing, after the fact (Proactive Insights).** Separately, Graphit continuously scans everything the team built - KB definitions, dashboard graphs, the queries that actually ran - and turns each governance gap into a ranked Fix card routed to an owner. Three principles: a card shows only when Graphit is confident it is real (right or silent), one root cause is one card even if it spans many surfaces, and each is routed to an owner (the resource's owner, else its domain owner, else org admins). The ten card types:
-
-| Card | What it flags |
-|------|---------------|
-| Govern next | The same ungoverned calculation runs in 2+ places |
-| Unsealed dashboard | A dashboard uses definitions the KB does not govern |
-| Verified, not wired | A verified definition is re-implemented as raw SQL |
-| Unverified in use | Live graphs depend on a definition nobody verified |
-| Stale reference | A graph is frozen on an old version of its definition (the only "Breaking" card) |
-| Deprecated in use | A deprecated definition is still used by live graphs |
-| Conflicting definitions | Two near-identical definitions disagree |
-| Rule conflict | Two governance rules contradict each other |
-| Unenforced rule | A rule that structurally cannot act on anything |
-| Missing owner | Assets with nobody responsible for them |
-
-Fix is a guided one-click resolution: it drafts and previews the exact change, you apply it (same validation as a manual edit), then Graphit re-checks and clears the card. This queue lives only in the **web app's Governance page** - the CLI cannot list or fix Insights cards (`governance status` and `governance audit` report conformance only). When a user asks about a finding, explain what it means and point them to the Governance page to Fix it.
+- Draft: no effect.
+- Verified body-only: guidance.
+- Verified constrained: enforceable according to mode.
+- Protected-column masks: never overridable.
+- EXPLORE: bounded override where policy permits.
 
 ## Relaying it
 
-Answer the person's real question first, in plain language ("your query was blocked because it invented a revenue formula instead of using the governed one"), then give the concrete next step: the governed rewrite, creating the missing asset, or the web Governance page for an Insights card. No jargon dumps, no internal names.
+State the result first, then tier and receipt facts. Name only visible rules. Never invent hidden names or quote raw compiler errors. If blocked, give the server-provided next step. If ad hoc, say so and offer a governed rewrite or supported definition.

package/skills/graphit/references/governance.md

@@ -1,100 +1,37 @@
 # Query Governance
 
-Load this when writing or validating a governed query, working trust tiers, declaring a conditionally-enforced rule, or working the ad-hoc frontier.
+Load when writing a governed query, explaining a refusal, or reporting provenance.
 
-Governance is enforced server-side in the QueryGateway, the same across every channel. You CANNOT weaken it from the CLI: an ad-hoc business measure requires EXPLORE access across every queried scope, and rule overrides must also satisfy EXPLORE.
+## References
 
-## Reference syntax
+| Reference | Meaning |
+|---|---|
+| `{{ Metric('revenue') }}` | Reusable metric |
+| `{{ Dimension('order__channel') }}` | Qualified grouping/filter field |
+| `{{ Measure('order_total') }}` | Graphit's model-owned measure extension |
 
-Write governed queries with KB references, not inline formulas. The server compiles each to its KB expression, injects rule constraints, stamps the trust tier, and caches the result. The syntax is identical in `graphit query` and `graphit.resolve()`.
+Legacy token grammar is refused. Keep references inside complete executable SQL and canvas `data-graphit-sql`.
 
-| Syntax | Expands to | Example |
-|--------|-----------|---------|
-| `{{metric:NAME}}` | Metric calculation (aggregation) | `{{metric:CPI}}` |
-| `{{metric:NAME(K=V)}}` | Parameterized metric | `{{metric:ARPU(DAY=7)}}` |
-| `{{metric_raw:NAME}}` | Raw expression, no outer aggregate | `{{metric_raw:REVENUE}}` |
-| `{{dim:NAME}}` | Dimension expression | `{{dim:INSTALL_MONTH}}` |
-
-```bash
-graphit query "SELECT {{dim:INSTALL_MONTH}}, {{metric:CPI}} AS cpi FROM MARKETING_UA_DS GROUP BY 1" --ds MARKETING_UA_DS --verbose
-```
-
-`--verbose` prints the expanded SQL and trust tier, so you can confirm the reference resolved before presenting the result.
-
-## Parameterized metrics
-
-Some metrics (for example ARPU, ROAS, RETENTION) carry required parameters and cannot resolve without a value. Run `graphit kb list metric` and read the `params` column for the names a metric requires, then supply them inline as `{{metric:ARPU(DAY=7)}}`. Pre-baked variants such as `ARPU_D7` or `ROAS_D30` have the value fixed and need none. Omitting a required parameter returns a clear error naming the exact syntax, so read `params` first.
+The governed fragment path serves simple, ratio, and derived metrics. Cumulative, conversion, shifted, time-spine, and null-fill shapes remain unavailable until Project #289. Use a supported decomposition or explicitly labeled free SQL.
 
 ## Trust tiers
 
-Every result is stamped with a tier, shown to the user as a badge on graphs and canvas entities - your honest signal of how trustworthy the number is.
-
-| Tier | Meaning | Badge |
-|------|---------|-------|
-| `governed` | Query used `{{metric:X}}` / `{{dim:X}}` references | Teal dot |
-| `verified` | Raw SQL whose expressions match KB definitions | Amber dot |
-| `ad_hoc` | Inline formulas with no KB match | Gray dot |
-
-Prefer the governed tier. The server may upgrade matching raw SQL to verified, but reach for references first so the result is governed by intent.
-
-## The ad-hoc gate
-
-This is the hard frontier - the rules below are what the QueryGateway does.
-
-A query lands at the `ad_hoc` tier when it uses no `{{metric:NAME}}` / `{{dim:NAME}}` reference and its raw expressions match no KB definition. On the CLI (`graphit query`, cached `--ds` or `--warehouse`), **every** ad-hoc query must be justified - business measures (an aggregate or `GROUP BY`) and plain exploration alike (`SELECT *`, `COUNT(*)`, `DISTINCT` peeks). Governed and verified results are exempt.
-
-- **EXPLORE access is a hard prerequisite for ad-hoc business measures.** If the user lacks EXPLORE access to any queried table scope, the server blocks that measure. An ad-hoc reason cannot bypass that denial.
-- **Justification floor.** The ad-hoc query is withheld and asks for a justification. Preferred path first: rewrite with `{{metric:NAME}}` / `{{dim:NAME}}` references - genuinely search the KB (`graphit kb explore`, `graphit kb list metric`, `graphit kb list dimension`), and if the metric or dimension you need does not exist, CREATE it first. A filter's value list belongs in a `{{dim:NAME}}`, not a raw `SELECT DISTINCT` peek. Only if nothing fits and the user needs the raw run, pass `--adhoc-reason "<text>"` stating what you searched, what you found, and why it does not fit - pass it on the first call when you already know the query is ad-hoc. A trivial or empty reason is rejected server-side and recorded in the audit log, so it must be honest.
-
-When the gate fires, do not narrate around it or pretend the query ran. Report truthfully: it was blocked or needs approval, name the governed rewrite, and let the user decide.
-
-## Enforceable rules and overrides
-
-Rules with typed constraints are enforced automatically, rewriting the SQL before it runs:
-
-| Type | What it does |
-|------|-------------|
-| `required_where` | Injects a WHERE predicate |
-| `forbidden_column` | NULLifies a column in SELECT |
-| `value_restriction` | Restricts a column to allowed values |
-| `required_filter` | Validates a column appears in WHERE |
-| `required_aggregation` | Validates GROUP BY includes a column |
-
-User-context variables (`${user.team_id}`, `${user.email}`) resolve server-side for row-level security. Override a rule only when the user explicitly asks; the server honors it only if the user holds EXPLORE on every queried scope. A rule that masks a column (a `forbidden_column` constraint) can never be overridden. Every override is logged. Pass several names to override more than one.
-
-```bash
-graphit query "SELECT * FROM EVENTS" --ds EVENTS --override-rules EXCLUDE_RETARGETING
-```
-
-## Conditionally-enforced rules
-
-A rule's mode is Advisory (guidance), Always (every query), or **Conditional** - fires only when its plain-language body (the condition) holds for the query you wrote. No server classifier decides that; you do. Read a table's rules first (`graphit kb explore table <name>`), judge your query against each conditional rule's body, and declare it up front:
-
-- `--apply-conditional RULE` - enforce it for this query.
-- `--skip-conditional RULE:"reason"` - skip it; a reason is required and audit-logged.
-
-Declare up front so a clean query never stalls. An undeclared conditional returns a retryable prompt naming each unresolved rule and its condition - read it, decide, re-run. A saved tile stores your decision and replays it every refresh.
-
-## Data-source row caps
-
-An admin can set a `max_rows` cap per data source. A cap limits the result set; it does not change authorization, trust-tier classification, or the ad-hoc justification floor.
+- **governed:** verified semantic references compiled through the gateway.
+- **verified:** known safe stored query without semantic references.
+- **ad hoc:** raw SQL at the frontier.
 
-## Presenting governance results
+Prefer governed. Never present ad-hoc SQL as the team's definition.
 
-The user CANNOT see raw CLI output. Render every result as markdown.
+## Rules
 
-**After a governed query**, append a provenance footer so the trust signal travels with the number: tier, KB references used, row cap if applied. Because a governed query may be rewritten before it runs, read `provenance.injection_summary` and report which rules changed it and how (each rule, its outcome, and why), not just a count. An ungoverned query reports no rules applied. For an ad-hoc result, state the tier honestly and offer the governed `{{metric:NAME}}` / `{{dim:NAME}}` rewrite.
+Rules target model, entity, dimension, metric, or group identities. Verified constraints enforce; verified body-only rules guide; drafts do nothing. Modes and EXPLORE behavior remain server-owned.
 
-~~~
-**Trust tier:** governed - 2 KB refs, 1 rule enforced (**EXCLUDE_INTERNAL**), max rows 10000
-~~~
+The gateway runs before caches, injects constraints, verifies resolved SQL, and returns a transparency receipt. Do not claim a rule applied merely because it exists.
 
-**After `graphit governance status`**, show the 7-day conformance counts (governed / verified / ad-hoc / total) as a small markdown table.
+## Ad-hoc gate
 
-**When a gate or rule blocks a query**, explain it and the path forward, no raw dump:
+Search the KB genuinely, explain why visible definitions do not fit, and prefer an approved reusable supported definition. Use a truthful ad-hoc reason only for a real one-off. Never use it to bypass a rule.
 
-~~~
-**Blocked by governance.**
+## Reporting
 
-Rule **EXCLUDE_ORGANIC** is enforced; overriding it requires EXPLORE access on every queried scope. Rewrite the query to include the required filter, or ask an admin for EXPLORE access.
-~~~
+Report tier, semantic references, row cap, visible rules that changed the query, and any refusal or override. Read the receipt rather than inferring. A blocked or partial result is not success.