dataspring-cli 0.3.0__py3-none-any.whl

cli/__init__.py

@@ -0,0 +1,15 @@
+"""DataSpring CLI - Command-line interface for DataSpring.
+
+This package provides a terminal-based interface to DataSpring's
+metric query and dashboard capabilities.
+
+Usage:
+    dataspring login          # Authenticate via Google OAuth
+    dataspring whoami         # Show current user and org
+    dataspring metrics list   # List available metrics
+    dataspring query -m revenue -g month  # Query metrics
+"""
+
+from .main import app
+
+__all__ = ["app"]

cli/_skills/dataspring-author/SKILL.md

@@ -0,0 +1,401 @@
+---
+name: dataspring-author
+description: Conventions for writing dbt YAML descriptions and the org's business_context doc. Use when editing dbt/models/**.yml, dbt/context/business_context.md, or any file that feeds DataSpring's semantic layer or org context.
+---
+
+# Authoring DataSpring's data-layer context
+
+You are editing files that an analytical agent will read at query time.
+Lazy descriptions silently degrade every query that touches the
+affected entity. This skill exists so the conventions stay tight.
+
+## 1. The contract
+
+The descriptions on metrics, dimensions, and semantic models are
+**authoritative**. Consumer agents trust them as the source of truth
+for unit, computation, source rollup, applicable filter rules, and
+known caveats. They are not commentary.
+
+Two surfaces matter:
+
+- **Per-entity descriptions** (dbt YAML) — ride directly in the tool
+  responses to `list_metrics` / `list_dimensions`. No length cap.
+  This is where 80% of "knowledge" should live.
+- **Per-org `business_context`** — a single markdown document that
+  covers cross-cutting facts. Hard cap: 2000 chars. Wired into the
+  MCP `instructions` blob so external agents see it from turn 1.
+
+If it attaches to one entity, put it on that entity. If it doesn't,
+put it in `business_context`.
+
+## 2. Metric description anatomy
+
+A complete metric description has six ordered elements. Skip an
+element only if it genuinely doesn't apply.
+
+1. **Canonical sentence** — what this metric measures, in one line.
+2. **Unit** — DKK, count, %, hours, customers, etc. Always explicit.
+3. **Source rollup** — which systems / tables contribute.
+4. **Caveats** — known bugs, temporal discontinuities, methodology
+   changes. Date them.
+5. **Standing filter rules** — "for B2B, pair with..."; "exclude
+   internal test orders by setting `status != 'test'`".
+6. **Derived-field pointers** — "use `event_segment`, not raw
+   `event_type`".
+
+### Good example
+
+```yaml
+- name: total_revenue
+  description: |
+    Total invoiced revenue from paid Stripe invoices.
+    Unit: DKK (gross, before VAT).
+    Source: stg_stripe__invoices, joined to dim_customer.
+    Caveat: pre-2024-04-01 numbers exclude refunds (ETL bug
+    fixed 2024-04-15; segment if comparing across that boundary).
+    Filter rule: pair with status='paid' to exclude voided invoices.
+    Use the derived `customer_segment`, not raw `customer.tier`.
+```
+
+### Bad example
+
+```yaml
+- name: total_revenue
+  description: Total revenue.
+```
+
+The bad version forces every consumer agent to guess unit, source,
+caveats, and rules. It also makes the metric look interchangeable
+with similarly-named metrics in other systems.
+
+## 3. Dimension description rules
+
+For each dimension, document:
+
+- **Allowed values** if it's an enum (`["smb", "midmarket", "enterprise"]`).
+- **Derivation pointer** if it's computed (`derived from
+  customer.tier in stg_customers`).
+- **Use-this-not-that steering** when a raw field is more naive than
+  the derived one (`prefer event_segment over event_type — segment
+  layers business logic the raw field doesn't`).
+
+```yaml
+- name: customer_segment
+  description: |
+    SMB / Midmarket / Enterprise segmentation. Allowed values:
+    smb, midmarket, enterprise.
+    Derived in stg_customers from ARR + employee count, not the
+    raw customer.tier field (tier hasn't been kept current).
+```
+
+## 4. Source / model description rules
+
+For sources and semantic models, document:
+
+- **Refresh cadence** (hourly, daily 06:00 UTC, etc.).
+- **Format quirks** ("amount stored as cents in JSON payload, not
+  dollars").
+- **Coverage gaps** ("`customer_id` is 100% NULL pre-2023-06-01 —
+  signups before then aren't tracked").
+
+```yaml
+sources:
+  - name: stripe_events
+    description: |
+      Refreshed hourly from Stripe webhook ingest.
+      `payload` is JSON; amount is in cents.
+      Pre-2023-06-01: `customer_id` is NULL (signup tracking
+      added then). Filter out NULL or use the post-2023 window
+      for customer-level aggregations.
+```
+
+## 5. `business_context` anatomy
+
+A single markdown document, ≤2000 chars (the MCP instructions cap).
+Cover the cross-cutting facts that don't attach to any specific
+entity.
+
+Recommended structure:
+
+```markdown
+# <Org name>
+
+## What we sell
+- One sentence on the business model.
+- Currency, fiscal calendar.
+
+## Systems we sync from
+- Stripe (billing) — refreshed hourly.
+- HubSpot (CRM, contacts).
+- Zendesk (support tickets).
+- We do NOT sync personal staff mailboxes; email-correspondence
+  questions cannot be answered.
+
+## Cross-cutting derivations
+- B2B vs B2C is `event_segment`, NOT raw `event_type`.
+- `customer_segment` (smb/midmarket/enterprise) is derived in
+  `stg_customers` — use it instead of `customer.tier`.
+
+## Operational pointers
+- Region: EU.
+- dbt project lives in `dbt/` of the analytics repo.
+- Per-metric specifics live in metric YAML descriptions.
+```
+
+What does NOT belong in `business_context`:
+- Per-metric meaning, unit, formula — goes on the metric.
+- Per-dimension derivation — goes on the dimension.
+- Findings / analysis results — those aren't context.
+
+## 6. The filing decision rule
+
+| Type of fact | Goes in… |
+|---|---|
+| One metric's meaning, computation, unit | That metric's description |
+| One metric's caveat, filter rule, derived-field pointer | Same |
+| One dimension's allowed values, derivation, steering | That dimension's description |
+| One source's refresh cadence, format quirk, NULL coverage | That source's description |
+| Cross-cutting derivation rule | `business_context` |
+| Org-level system inventory / coverage gap | `business_context` |
+| Region, dbt layout, operational pointer | `business_context` |
+| Findings, analysis, history | NOT in the context layer |
+| Definition logic | dbt model SQL, not descriptions |
+
+When unsure: put it in YAML, accept some duplication.
+
+## 7. Edit surfaces for `business_context`
+
+`business_context` is **DataSpring-owned**: the document lives in
+DataSpring, and an edit made there is live immediately and survives
+manifest uploads. dbt owns computation; it does not own this text.
+
+Edit it on whichever surface you are already on — they are the same
+document:
+
+- Web UI: Settings → Data → Business Context. Has a live byte
+  counter against the 2000-char cap.
+- Tool: `business_context_edit` (admin/owner), actions `get`,
+  `show_size`, `set`, `append`. For agent-authored edits.
+- CLI: `dataspring business-context set` (stdin) or
+  `dataspring business-context edit` ($EDITOR).
+
+`dataspring business-context import FILE` remains available as a
+one-time bootstrap: it is how you seed the document from a file you
+already have, not a sync loop to maintain afterwards.
+
+## 8. Maintenance discipline
+
+- **Reference rot.** When you rename a metric or dimension, search
+  `business_context` and other entity descriptions for stale
+  references. CI doesn't lint this yet.
+- **Caveat sunset.** When a temporal-discontinuity bug gets fixed,
+  remove the caveat — but only after enough time has passed that
+  no useful query window starts before the fix.
+- **Cross-PR review.** A description rewrite is a behavior change
+  to every agent. Treat it like an API change in review.
+
+That's the contract. Boring, dense, useful.
+
+## 9. Parameter reference
+
+<!-- dispatch-params: semantic_model_edit metric_edit import_manifest business_context_edit warehouse_edit datacore_edit datacore_run -->
+_Generated from the dispatch registry (`GET /api/dispatch`); do not edit by hand. Field descriptions are the models' own, verbatim on every surface._
+
+#### `semantic_model_edit`
+
+MCP tool `semantic_model_edit`; `POST /api/dispatch/semantic_model_edit`. Edit semantic models in the org manifest. Admin/owner only for writes. The body is `{"action": {"action": "<verb>", ...}}`, one verb per table below.
+
+**`create`** — 
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `model_data` | object | yes | Full semantic-model definition (name, measures, dimensions, entities) |
+| `expect_version` | string | no | Managed tenants only: the datacore version (draft snapshot id) this edit was written against, from the previous edit's `version` or dataspring://datacore/files; refused with the current one when stale |
+
+**`update`** — 
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `name` | string | yes |  |
+| `updates` | object | yes |  |
+| `confirmed` | boolean | no, default `False` | set true only after the user confirmed a redefinition of a NATIVE entity whose impact you showed them; has no effect on imported entities |
+| `expect_version` | string | no | Managed tenants only: the datacore version (draft snapshot id) this edit was written against, from the previous edit's `version` or dataspring://datacore/files; refused with the current one when stale |
+
+**`delete`** — 
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `name` | string | yes |  |
+| `expect_version` | string | no | Managed tenants only: the datacore version (draft snapshot id) this edit was written against, from the previous edit's `version` or dataspring://datacore/files; refused with the current one when stale |
+
+**`preview`** — Validate a semantic model without writing it or touching the warehouse.
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `model_data` | object | yes | Full semantic-model definition to validate (name, measures, dimensions, entities) |
+
+#### `metric_edit`
+
+MCP tool `metric_edit`; `POST /api/dispatch/metric_edit`. Edit metrics in the org manifest. Admin/owner only for writes. The body is `{"action": {"action": "<verb>", ...}}`, one verb per table below.
+
+**`create`** — 
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `metric_data` | object | yes | Metric definition (name, type, type_params) |
+| `expect_version` | string | no | Managed tenants only: the datacore version (draft snapshot id) this edit was written against, from the previous edit's `version` or dataspring://datacore/files; refused with the current one when stale |
+
+**`update`** — 
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `name` | string | yes |  |
+| `updates` | object | yes |  |
+| `confirmed` | boolean | no, default `False` | set true only after the user confirmed a redefinition of a NATIVE entity whose impact you showed them; has no effect on imported entities |
+| `expect_version` | string | no | Managed tenants only: the datacore version (draft snapshot id) this edit was written against, from the previous edit's `version` or dataspring://datacore/files; refused with the current one when stale |
+
+**`delete`** — 
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `name` | string | yes |  |
+| `expect_version` | string | no | Managed tenants only: the datacore version (draft snapshot id) this edit was written against, from the previous edit's `version` or dataspring://datacore/files; refused with the current one when stale |
+
+**`preview`** — 
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `metric_data` | object | yes | Metric definition to preview |
+| `sample_query` | object | no | Optional query params (start_date, end_date, grain, limit) |
+
+**`impact`** — What applying ``updates`` to this metric would touch, without writing.
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `name` | string | yes |  |
+| `updates` | object | yes | The updates whose impact to assess |
+
+#### `import_manifest`
+
+MCP tool `import_manifest`; `POST /api/dispatch/import_manifest`. Import a semantic manifest using replace semantics.
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `manifest_data` | object | yes | The semantic manifest to import (semantic models, metrics), as a JSON object. |
+| `warehouse` | string | no | Which of the org's warehouses the manifest belongs to (see dataspring://warehouses); omit for the active one. |
+
+#### `business_context_edit`
+
+MCP tool `business_context_edit`; `POST /api/dispatch/business_context_edit`. Edit the org's cross-cutting business_context document. The body is `{"action": {"action": "<verb>", ...}}`, one verb per table below.
+
+**`get`** — Read the current org business_context document.
+
+No parameters.
+
+**`show_size`** — Report current size and remaining room within the 2000-byte UTF-8 cap.
+
+No parameters.
+
+**`set`** — Replace the org's business_context with ``content`` (admin/owner).
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `content` | string | yes | Full document content to save |
+
+**`append`** — Append ``content`` to the existing business_context (admin/owner).
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `content` | string | yes | Text to append to the current document |
+
+#### `warehouse_edit`
+
+MCP tool `warehouse_edit`; `POST /api/dispatch/warehouse_edit`. Manage the org's warehouses (admin/owner). Read them with ``dataspring://warehouses`` / ``read(kind="warehouses")``: each entry has id, kind (external \| managed), label, active flag, manifest counts and import time. The body is `{"action": {"action": "<verb>", ...}}`, one verb per table below.
+
+**`activate`** — Make one warehouse the org's default: every query that names no ``warehouse`` runs against it.
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `id` | string | yes | Warehouse id, e.g. 'managed' or 'external' |
+
+**`add_external`** — Register a customer-owned BigQuery project as an external warehouse.
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `label` | string | yes | Display name, e.g. "Noon's BigQuery" |
+| `project` | string | yes | The customer's GCP project id |
+| `dataset` | string | yes | The dataset the semantic manifest points at |
+| `location` | string | yes | BigQuery location of that dataset, e.g. 'europe-north2' |
+| `key_secret` | string | yes | NAME of the Secret Manager secret holding the customer's service-account key (t-<org>-external-bigquery-key). Never the key itself. |
+| `id` | string | no, default `external` | Warehouse id to create (default 'external') |
+
+**`remove`** — Delete a warehouse and its manifest. Refused for the active warehouse and for the managed one.
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `id` | string | yes | Warehouse id to remove |
+
+#### `datacore_edit`
+
+MCP tool `datacore_edit`; `POST /api/dispatch/datacore_edit`. Edit the datacore workspace: connections, pipelines and the dbt project of a managed tenant, as files. ``kind`` picks the family: The body is `{"action": {"action": "<verb>", ...}}`, one verb per table below.
+
+**`apply`** — Write one workspace file whole (create or replace): a connection or pipeline as a ``document`` (validated against its schema and the catalog), a dbt project file as ``content``.
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `kind` | one of `connection`, `pipeline`, `model` | yes | connection (admin), pipeline (admin) or model (member) |
+| `name` | string | yes | For kind connection or pipeline: the name, i.e. the file name without .yaml (connections/<name>.yaml, pipelines/<name>.yaml). For kind model: the path under models/, the dbt project (models/models/marts/orders.sql, models/models/staging/schema.yml, models/seeds/x.csv, models/dbt_project.yml); no traversal, .sql .yml .yaml .csv .md .txt .json, never target/ or dbt_packages/ |
+| `document` | object | no | kind connection: the whole document (connection.schema.json: kind, base_url, auth with secret NAMES). kind pipeline: the whole document (pipeline.schema.json: a catalog `connector` with config, secrets, overrides; or a `connection` reference, an inline `source` or named `sources`, with `resources`; schedule, checks, then). Secrets are named, never valued; every name must exist |
+| `content` | string | no | kind model: the whole file content |
+| `expect_version` | string | no | The datacore version this edit was written against: the `version` the previous edit returned, or dataspring://datacore/files. An edit against a stale version is refused with the current one. Omit to skip the check. |
+
+**`patch`** — Apply a JSON merge patch (RFC 7386: a key set to null is removed) to an existing YAML file: a connection, a pipeline (a schedule, a window, a check), or a YAML file under models/ (not SQL).
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `kind` | one of `connection`, `pipeline`, `model` | yes | connection (admin), pipeline (admin) or model (member) |
+| `name` | string | yes | For kind connection or pipeline: the name, i.e. the file name without .yaml (connections/<name>.yaml, pipelines/<name>.yaml). For kind model: the path under models/, the dbt project (models/models/marts/orders.sql, models/models/staging/schema.yml, models/seeds/x.csv, models/dbt_project.yml); no traversal, .sql .yml .yaml .csv .md .txt .json, never target/ or dbt_packages/ |
+| `patch` | object | yes | JSON merge patch over the current document |
+| `expect_version` | string | no | The datacore version this edit was written against: the `version` the previous edit returned, or dataspring://datacore/files. An edit against a stale version is refused with the current one. Omit to skip the check. |
+
+**`delete`** — Remove one workspace file.
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `kind` | one of `connection`, `pipeline`, `model` | yes | connection (admin), pipeline (admin) or model (member) |
+| `name` | string | yes | For kind connection or pipeline: the name, i.e. the file name without .yaml (connections/<name>.yaml, pipelines/<name>.yaml). For kind model: the path under models/, the dbt project (models/models/marts/orders.sql, models/models/staging/schema.yml, models/seeds/x.csv, models/dbt_project.yml); no traversal, .sql .yml .yaml .csv .md .txt .json, never target/ or dbt_packages/ |
+| `expect_version` | string | no | The datacore version this edit was written against: the `version` the previous edit returned, or dataspring://datacore/files. An edit against a stale version is refused with the current one. Omit to skip the check. |
+
+#### `datacore_run`
+
+MCP tool `datacore_run`; `POST /api/dispatch/datacore_run`. The datacore's check-then-deploy loop and its operations. The body is `{"action": {"action": "<verb>", ...}}`, one verb per table below.
+
+**`check`** — Check the current draft; no arguments beyond an optional note.
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `note` | string | no | Optional note, recorded on the check run |
+
+**`deploy`** — 
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `snapshot` | string | no | Omit to promote the current draft (needs a green check bound to it). A previously deployed snapshot id rolls back to it |
+| `note` | string | no | Why, recorded on the learned trail |
+
+**`run`** — 
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `pipeline` | string | yes | Pipeline name (a deployed pipelines/<name>.yaml) |
+| `window` | RunWindowParams | no | A date window to load instead of the pipeline's own; wider than backfill_chunk becomes a chunked backfill |
+| `cursor` | RunCursorParams | no | {from: <value>}: a cursor to re-pull from |
+| `backfill_chunk` | string | no | Chunk size for a windowed run: 7d, 2w or 1M (calendar months, the default) |
+
+**`reset`** — 
+
+| Parameter | Type | Required | Description |
+|---|---|---|---|
+| `pipeline` | string | yes | The pipeline whose dlt state and raw tables to remove |
+| `approve` | boolean | no, default `False` | Must be true: the reset is irreversible. Ask the user first |
+<!-- /dispatch-params -->