strata-cli 0.1.19 → 0.1.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +34 -0
- data/README.md +30 -11
- data/lib/strata/cli/configuration.rb +11 -1
- data/lib/strata/cli/credentials.rb +44 -7
- data/lib/strata/cli/descriptions/datasource/auth.txt +9 -10
- data/lib/strata/cli/descriptions/init.txt +8 -7
- data/lib/strata/cli/generators/datasource.rb +6 -2
- data/lib/strata/cli/generators/project.rb +6 -129
- data/lib/strata/cli/generators/templates/AGENTS.md +34 -34
- data/lib/strata/cli/generators/templates/adapters/bigquery.yml +29 -0
- data/lib/strata/cli/generators/templates/adapters/clickhouse.yml +39 -0
- data/lib/strata/cli/generators/templates/adapters/redshift.yml +39 -0
- data/lib/strata/cli/generators/templates/adapters/sqlite.yml +22 -0
- data/lib/strata/cli/generators/templates/project.yml +2 -2
- data/lib/strata/cli/helpers/datasource_helper.rb +17 -1
- data/lib/strata/cli/helpers/project_helper.rb +3 -2
- data/lib/strata/cli/helpers/prompts.rb +2 -2
- data/lib/strata/cli/main.rb +5 -7
- data/lib/strata/cli/sub_commands/audit.rb +18 -3
- data/lib/strata/cli/sub_commands/datasource.rb +14 -3
- data/lib/strata/cli/sub_commands/deploy.rb +1 -30
- data/lib/strata/cli/sub_commands/project.rb +6 -9
- data/lib/strata/cli/version.rb +1 -1
- metadata +7 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 8114ee5b03683396612b2831650670281cdcf3da321c40abc259e2e1a31847c9
|
|
4
|
+
data.tar.gz: e84f43d3831b3cfb28da805c728b9005801aca677c0deeb620639ccaf632d213
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: e6c15b69e6170dd5196f94c31b91cb0d5c229c2703402d588c72b6de6899a07e73f62af5ebd2ef7fb127df2e8832eca594644c947ec917e40bcc3daec91822a6
|
|
7
|
+
data.tar.gz: 81f8cec6b27936af833f2bbfae9eb1b878d14a3fd0e828c7acab5399235cb3a64e8c19b10bb6b5345ad8331e7b0cfbbf4250ff48e23362648386abdd0bcfb32c
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,39 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
## [0.1.21] - 2026-11-04
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- **Default server URL**: commands fall back to `https://app.strata.do` when no `server:` is configured, so `strata deploy` no longer stops to prompt for one.
|
|
8
|
+
- **`-s, --server URL` on `strata init`** (alongside the existing flag on `project sample`), written into `project.yml`. Other commands read it from there; use `-e ENVIRONMENT` with per-environment `server:` blocks to target more than one server.
|
|
9
|
+
|
|
10
|
+
### Removed
|
|
11
|
+
|
|
12
|
+
- **`strata init --source`**: it could never do what it claimed. The clone was gated on a `git_url` field the server does not return (the column is `git`), so the repository was never cloned, and the full project URL was written into `project.yml` as `server:`, breaking every later deploy. Its documented GitHub-URL example raised a `TypeError`. To work on an existing project, clone its repository and run `strata deploy`, or use `strata project link PROJECT_ID`. `-s` now means `--server`.
|
|
13
|
+
|
|
14
|
+
### Changed
|
|
15
|
+
|
|
16
|
+
- `strata init` requires `PROJECT_NAME`, and its `--api-key` is purely an optional convenience for scripted setups — browser sign-in on the first `strata deploy` is now the only interactive auth path.
|
|
17
|
+
- `strata project sample` writes the chosen server to `project.yml` instead of `.strata`, where `project.yml` silently overrode it.
|
|
18
|
+
|
|
19
|
+
### Fixed
|
|
20
|
+
|
|
21
|
+
- **A datasource whose file lives where the server runs no longer fails the audit.** The sample project points DuckDB at `/data/tpcds.duckdb`, which ships inside the Strata image, so the connection could never be opened from a laptop and `strata deploy` refused to run. `strata audit` now reports such a connection as unchecked and names it, instead of failing — the server validates it for real when the model deploys. This replaces a hardcoded skip that disabled connection auditing entirely for one repository, and `strata datasource test` now explains the same situation when it fails.
|
|
22
|
+
- A blank `server:` key in `project.yml` no longer wins the config merge and blank out the default.
|
|
23
|
+
- Saving a server URL to `project.yml` rewrote every `server:` line, including the indented per-environment ones. It now only replaces the root key.
|
|
24
|
+
- Documentation links point at `https://strata.do/docs/...` instead of the old `developer-docs` path.
|
|
25
|
+
|
|
26
|
+
## [0.1.20] - 2026-09-30
|
|
27
|
+
|
|
28
|
+
### Changed
|
|
29
|
+
|
|
30
|
+
- **File-based secrets stored as contents**: `strata ds auth` now reads a BigQuery service account keyfile and saves only its `client_email` and `private_key`, and reads a Snowflake key-pair private key file and saves the PEM text, so a remote Strata server never needs the file. `.strata` blocks are written with YAML serialisation to hold multi-line values. Requires dwh 0.6.2 and a server with encrypted `password`/`private_key` columns.
|
|
31
|
+
- **`strata ds auth --remote` removed**: the flag was documented but never implemented; credentials reach the server through `strata deploy`.
|
|
32
|
+
|
|
33
|
+
### Fixed
|
|
34
|
+
|
|
35
|
+
- `strata init -d ADAPTER` wrote a bare `ADAPTER:` node with no config, which then broke `ds list`, `ds auth`, and `audit`. It now renders the commented adapter template again, and templates were added for sqlite, redshift, clickhouse, and bigquery.
|
|
36
|
+
|
|
3
37
|
## [0.1.19] - 2026-09-30
|
|
4
38
|
|
|
5
39
|
### Added
|
data/README.md
CHANGED
|
@@ -52,22 +52,26 @@ Use `strata datasource meta DS_KEY TABLE_NAME --agent` for column metadata when
|
|
|
52
52
|
<details>
|
|
53
53
|
<summary><strong>Project Management</strong></summary>
|
|
54
54
|
|
|
55
|
-
### `strata init
|
|
55
|
+
### `strata init PROJECT_NAME`
|
|
56
56
|
|
|
57
|
-
Initialize a new Strata project
|
|
57
|
+
Initialize a new Strata project.
|
|
58
58
|
|
|
59
59
|
**Options:**
|
|
60
60
|
- `-d, --datasource ADAPTER` - Add datasource adapters (repeatable)
|
|
61
|
-
- `-s, --
|
|
62
|
-
- `-a, --api-key KEY` - API key (
|
|
61
|
+
- `-s, --server URL` - Server URL, written to `project.yml` (default `https://app.strata.do`)
|
|
62
|
+
- `-a, --api-key KEY` - API key, for scripted setups (optional — the first deploy signs you in via browser)
|
|
63
63
|
|
|
64
64
|
**Examples:**
|
|
65
65
|
```bash
|
|
66
66
|
strata init my-project
|
|
67
67
|
strata init my-project -d postgres -d snowflake
|
|
68
|
-
strata init
|
|
68
|
+
strata init my-project --server http://localhost:8080
|
|
69
69
|
```
|
|
70
70
|
|
|
71
|
+
To work on a project that already exists on the server, clone its repository — `project.yml`
|
|
72
|
+
carries both the server URL and the project ID — then run `strata deploy`. Use
|
|
73
|
+
`strata project link PROJECT_ID` to point an existing local project at a server project.
|
|
74
|
+
|
|
71
75
|
### `strata project link PROJECT_ID`
|
|
72
76
|
|
|
73
77
|
Link local project to an existing project on the Strata server.
|
|
@@ -83,7 +87,7 @@ Clone the Strata TPC-DS sample project and set it up interactively. Prompts for
|
|
|
83
87
|
|
|
84
88
|
**Options:**
|
|
85
89
|
- `-a, --api-key KEY` - API key (optional — skips browser sign-in when paired with `--server`)
|
|
86
|
-
- `-s, --server URL` - Server URL (optional — prompted if omitted)
|
|
90
|
+
- `-s, --server URL` - Server URL (optional — prompted if omitted, default `https://app.strata.do`)
|
|
87
91
|
|
|
88
92
|
**Examples:**
|
|
89
93
|
```bash
|
|
@@ -306,6 +310,19 @@ List all supported data warehouse adapters.
|
|
|
306
310
|
|
|
307
311
|
## Configuration
|
|
308
312
|
|
|
313
|
+
### Server URL
|
|
314
|
+
|
|
315
|
+
The server URL lives in `project.yml`. `strata init` and `strata project sample` accept
|
|
316
|
+
`-s, --server URL` and write it there; every other command reads it, so the flag is set once:
|
|
317
|
+
|
|
318
|
+
1. `server:` under the selected environment in `project.yml` (`-e ENVIRONMENT`)
|
|
319
|
+
2. `server:` at the root of `project.yml`, then `.strata`
|
|
320
|
+
3. `https://app.strata.do` — the default when nothing above is set
|
|
321
|
+
|
|
322
|
+
To deploy to more than one server, give `project.yml` a block per environment and select it
|
|
323
|
+
with `-e ENVIRONMENT` (see [Multi-Environment Configuration](#configuration)) rather than
|
|
324
|
+
passing a URL per command.
|
|
325
|
+
|
|
309
326
|
### Project Configuration (`project.yml`)
|
|
310
327
|
|
|
311
328
|
Main project configuration file:
|
|
@@ -328,10 +345,10 @@ project_id: 123 # Auto-populated after first deployment
|
|
|
328
345
|
server: https://dev.strata.example.com
|
|
329
346
|
|
|
330
347
|
staging:
|
|
331
|
-
server: https://staging.strata.
|
|
348
|
+
server: https://staging.strata.do
|
|
332
349
|
|
|
333
350
|
production:
|
|
334
|
-
server: https://app.strata.
|
|
351
|
+
server: https://app.strata.do
|
|
335
352
|
```
|
|
336
353
|
|
|
337
354
|
`.strata` — API keys per environment (**never committed**):
|
|
@@ -364,6 +381,8 @@ ai_api_key: your-ai-key
|
|
|
364
381
|
|
|
365
382
|
**Security:** The `.strata` file automatically has restrictive permissions (0600) for security.
|
|
366
383
|
|
|
384
|
+
File-based secrets are stored as contents, not paths: a BigQuery keyfile becomes `client_email` + `private_key`, and a Snowflake key-pair `private_key` holds the PEM text. Credentials travel to the server with each deploy and are stored encrypted there.
|
|
385
|
+
|
|
367
386
|
### Authentication
|
|
368
387
|
|
|
369
388
|
You don't need to create an API key manually. When no key is configured, `strata deploy` (and `strata project sample`) asks how you'd like to authenticate:
|
|
@@ -441,7 +460,7 @@ my_bigquery:
|
|
|
441
460
|
dataset: analytics
|
|
442
461
|
```
|
|
443
462
|
|
|
444
|
-
Run `strata ds auth my_bigquery`
|
|
463
|
+
Run `strata ds auth my_bigquery` and give the service account JSON keyfile path: only its `client_email` and `private_key` are saved in `.strata`, so the server never needs the file.
|
|
445
464
|
|
|
446
465
|
## Supported Data Warehouse Adapters
|
|
447
466
|
|
|
@@ -457,12 +476,12 @@ Run `strata ds auth my_bigquery` to save a service account `keyfile` path in `.s
|
|
|
457
476
|
- **Databricks** - Databricks SQL warehouse support
|
|
458
477
|
- **ClickHouse** - ClickHouse HTTP/native SQL support
|
|
459
478
|
- **SQLite** - SQLite database support
|
|
460
|
-
- **BigQuery** - Google BigQuery support (service account
|
|
479
|
+
- **BigQuery** - Google BigQuery support (service account key)
|
|
461
480
|
|
|
462
481
|
## Security
|
|
463
482
|
|
|
464
483
|
- ✅ Credentials stored in `.strata` with restrictive permissions (0600)
|
|
465
|
-
- ✅ API keys never required on the command line — browser sign-in or the `.strata` file supplies them (optional `--api-key` flags exist on `init
|
|
484
|
+
- ✅ API keys never required on the command line — browser sign-in or the `.strata` file supplies them (optional `--api-key` flags exist on `init` and `project sample` for scripting)
|
|
466
485
|
- ✅ File paths validated to prevent path traversal attacks
|
|
467
486
|
- ✅ Git commit hashes validated before use
|
|
468
487
|
- ✅ YAML files loaded safely to prevent code execution
|
|
@@ -7,13 +7,21 @@ module Strata
|
|
|
7
7
|
class Configuration
|
|
8
8
|
STRATA_CONFIG_FILE = ".strata"
|
|
9
9
|
PROJECT_CONFIG_FILE = "project.yml"
|
|
10
|
+
DEFAULT_SERVER = "https://app.strata.do"
|
|
10
11
|
|
|
11
12
|
DEFAULT = {
|
|
12
13
|
"api_key" => "",
|
|
13
|
-
"server" =>
|
|
14
|
+
"server" => DEFAULT_SERVER,
|
|
14
15
|
"log_level" => "info"
|
|
15
16
|
}.freeze
|
|
16
17
|
|
|
18
|
+
# Normalizes a configured or user-supplied server URL, falling back to
|
|
19
|
+
# the hosted default when it is missing or blank.
|
|
20
|
+
def self.resolve_server(value)
|
|
21
|
+
value = DEFAULT_SERVER if value.nil? || value.to_s.strip.empty?
|
|
22
|
+
value.to_s.strip.delete_suffix("/")
|
|
23
|
+
end
|
|
24
|
+
|
|
17
25
|
attr_reader :data
|
|
18
26
|
|
|
19
27
|
def initialize
|
|
@@ -94,6 +102,8 @@ module Strata
|
|
|
94
102
|
project_data = load_yaml_file(project_config_file_path)
|
|
95
103
|
|
|
96
104
|
@data = DEFAULT.merge(strata_data).merge(project_data)
|
|
105
|
+
# project.yml ships a blank `server:` key, which would otherwise win the merge.
|
|
106
|
+
@data["server"] = DEFAULT_SERVER if @data["server"].to_s.strip.empty?
|
|
97
107
|
|
|
98
108
|
@api_key = @data["api_key"]
|
|
99
109
|
@server = @data["server"]
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require "yaml"
|
|
4
|
+
require "json"
|
|
4
5
|
require "tty-prompt"
|
|
5
6
|
|
|
6
7
|
module Strata
|
|
@@ -39,7 +40,7 @@ module Strata
|
|
|
39
40
|
credentials["personal_access_token"] = @prompt.ask("Enter Personal Access Token:")
|
|
40
41
|
when "kp"
|
|
41
42
|
credentials["username"] = @prompt.ask("Enter Username:")
|
|
42
|
-
credentials["private_key"] =
|
|
43
|
+
credentials["private_key"] = read_secret_file("Private key path (the PEM contents are stored):")
|
|
43
44
|
end
|
|
44
45
|
when "athena"
|
|
45
46
|
credentials["access_key_id"] = @prompt.ask("AWS Access Key ID:")
|
|
@@ -51,7 +52,7 @@ module Strata
|
|
|
51
52
|
when "clickhouse"
|
|
52
53
|
credentials["password"] = @prompt.mask("Enter Password (leave blank if none):")
|
|
53
54
|
when "bigquery"
|
|
54
|
-
credentials
|
|
55
|
+
credentials.merge!(bigquery_service_account)
|
|
55
56
|
else
|
|
56
57
|
if required?
|
|
57
58
|
unless %w[postgres mysql trino sqlserver].include?(adapter)
|
|
@@ -70,12 +71,11 @@ module Strata
|
|
|
70
71
|
|
|
71
72
|
def write_local(ds_key, context)
|
|
72
73
|
context.gsub_file Configuration::STRATA_CONFIG_FILE, Utils.yaml_block_matcher(ds_key), ""
|
|
73
|
-
|
|
74
|
-
credentials.each do |key, val|
|
|
75
|
-
creds << "\n #{key}: #{val}" unless val.nil? || val.empty?
|
|
76
|
-
end
|
|
74
|
+
values = credentials.reject { |_, val| val.nil? || val.to_s.empty? }
|
|
77
75
|
|
|
78
|
-
|
|
76
|
+
# YAML.dump so multi-line values (PEM keys) become indented block scalars
|
|
77
|
+
# that yaml_block_matcher can still find and replace on the next auth.
|
|
78
|
+
context.append_to_file Configuration::STRATA_CONFIG_FILE, YAML.dump({ds_key => values}).delete_prefix("---\n")
|
|
79
79
|
|
|
80
80
|
# Set restrictive permissions (read/write for owner only)
|
|
81
81
|
strata_file = Configuration::STRATA_CONFIG_FILE
|
|
@@ -83,6 +83,43 @@ module Strata
|
|
|
83
83
|
rescue Errno::EACCES => e
|
|
84
84
|
raise Strata::CommandError, "Permission denied setting permissions on #{strata_file}: #{e.message}"
|
|
85
85
|
end
|
|
86
|
+
|
|
87
|
+
private
|
|
88
|
+
|
|
89
|
+
# Asks for a file path and returns the file's contents so the server never
|
|
90
|
+
# needs the file itself. Blank returns nil; an answer that is not a path is
|
|
91
|
+
# kept as typed, which allows pasting the value directly.
|
|
92
|
+
def read_secret_file(question)
|
|
93
|
+
answer = @prompt.ask(question).to_s.strip
|
|
94
|
+
return nil if answer.empty?
|
|
95
|
+
|
|
96
|
+
File.exist?(answer) ? File.read(answer) : answer
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
# Reads the Google service account JSON once and keeps only the two fields
|
|
100
|
+
# needed to authenticate. The server has no other way to reach BigQuery,
|
|
101
|
+
# so without the file nothing is saved and the user is told to come back.
|
|
102
|
+
def bigquery_service_account
|
|
103
|
+
path = @prompt.ask("Service account keyfile path:").to_s.strip
|
|
104
|
+
if path.empty?
|
|
105
|
+
say " No keyfile given, so no BigQuery credentials were saved. Run `strata ds auth <key>` with the service account keyfile before deploying.", :yellow
|
|
106
|
+
return {}
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
key = JSON.parse(File.read(path))
|
|
110
|
+
unless key["type"] == "service_account" && key["client_email"] && key["private_key"]
|
|
111
|
+
raise Strata::CommandError, "#{path} is not a service account keyfile (needs type, client_email and private_key)"
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
expected = @datasource_config["project_id"]
|
|
115
|
+
if expected && key["project_id"] && expected != key["project_id"]
|
|
116
|
+
say " Warning: keyfile project '#{key["project_id"]}' differs from datasources.yml project_id '#{expected}'", :yellow
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
key.slice("client_email", "private_key")
|
|
120
|
+
rescue Errno::ENOENT, JSON::ParserError => e
|
|
121
|
+
raise Strata::CommandError, "Could not read service account keyfile #{path}: #{e.message}"
|
|
122
|
+
end
|
|
86
123
|
end
|
|
87
124
|
end
|
|
88
125
|
end
|
|
@@ -2,13 +2,12 @@ Example to set local credentials for a datasource with key games_dwh:
|
|
|
2
2
|
|
|
3
3
|
strata ds auth games_dwh
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
go through the OAuth flow. Not all adapters support OAuth.
|
|
5
|
+
Credentials are saved in the project's .strata file, which must not be
|
|
6
|
+
checked into git. They are sent to the Strata server with each deploy and
|
|
7
|
+
stored encrypted there.
|
|
8
|
+
|
|
9
|
+
File-based secrets are stored as contents, not paths, so the server never
|
|
10
|
+
needs the file: for BigQuery, give the service account JSON keyfile path and
|
|
11
|
+
only its client_email and private_key are kept; for Snowflake key-pair auth, give the
|
|
12
|
+
private key path and the PEM contents are kept. OAuth modes need no local
|
|
13
|
+
credentials; each user goes through the OAuth flow instead.
|
|
@@ -1,19 +1,20 @@
|
|
|
1
|
-
Initialize a new Strata project
|
|
1
|
+
Initialize a new Strata project.
|
|
2
2
|
|
|
3
3
|
Creates an AGENTS.md in the project so a coding agent (Claude Code, Codex, Cursor)
|
|
4
4
|
can build the semantic model for you:
|
|
5
|
-
https://strata.do/
|
|
5
|
+
https://strata.do/docs/cli/building-your-model
|
|
6
6
|
|
|
7
|
-
PROJECT_NAME is
|
|
8
|
-
with that name. If using --source, clones the existing project from the URL.
|
|
7
|
+
PROJECT_NAME is required and becomes the project directory and uid.
|
|
9
8
|
|
|
10
9
|
Options:
|
|
11
10
|
|
|
12
11
|
-d, --datasource ADAPTER Add one or more datasource adapters (repeatable)
|
|
13
12
|
|
|
14
|
-
-s, --
|
|
13
|
+
-s, --server URL Strata server URL, written to project.yml.
|
|
14
|
+
Defaults to https://app.strata.do
|
|
15
15
|
|
|
16
|
-
-a, --api-key KEY API key
|
|
16
|
+
-a, --api-key KEY API key, for scripted setups. Optional — the first
|
|
17
|
+
`strata deploy` signs you in via the browser.
|
|
17
18
|
|
|
18
19
|
Examples:
|
|
19
20
|
|
|
@@ -21,4 +22,4 @@ Examples:
|
|
|
21
22
|
|
|
22
23
|
strata init my-project -d postgres -d snowflake
|
|
23
24
|
|
|
24
|
-
strata init
|
|
25
|
+
strata init my-project --server http://localhost:8080
|
|
@@ -27,8 +27,12 @@ module Strata
|
|
|
27
27
|
@ds_key = get_unique_ds_key
|
|
28
28
|
Output.print_status(:adapter, "adding #{adapter} config to datasources", type: :warning, context: self)
|
|
29
29
|
|
|
30
|
-
#
|
|
31
|
-
config_yaml =
|
|
30
|
+
# `ds add` passes prompted config; `init -d` renders the commented adapter template.
|
|
31
|
+
config_yaml = if options[:config]
|
|
32
|
+
{@ds_key => options[:config]}.to_yaml.sub(/^---\n/, "\n")
|
|
33
|
+
else
|
|
34
|
+
"\n#{render_template("adapters/#{adapter}.yml")}"
|
|
35
|
+
end
|
|
32
36
|
append_to_file pathify("datasources.yml"), config_yaml
|
|
33
37
|
end
|
|
34
38
|
|
|
@@ -3,60 +3,23 @@
|
|
|
3
3
|
require_relative "group"
|
|
4
4
|
require_relative "datasource"
|
|
5
5
|
require_relative "../helpers/project_helper"
|
|
6
|
-
require_relative "../api/connection_error_handler"
|
|
7
|
-
require_relative "../api/response_error_handler"
|
|
8
6
|
require_relative "../output"
|
|
9
7
|
require_relative "../utils/git"
|
|
10
8
|
require_relative "../helpers/prompts"
|
|
11
|
-
require "faraday"
|
|
12
|
-
require "json"
|
|
13
|
-
require "uri"
|
|
14
9
|
|
|
15
10
|
module Strata
|
|
16
11
|
module CLI
|
|
17
12
|
module Generators
|
|
18
13
|
class Project < Group
|
|
19
|
-
|
|
20
|
-
include API::ResponseErrorHandler
|
|
21
|
-
|
|
22
|
-
argument :name, type: :string, required: false, desc: "The name of the project. Optional when using --source."
|
|
14
|
+
argument :name, type: :string, required: false, desc: "The name of the project."
|
|
23
15
|
class_option :datasource, type: :string, repeatable: true
|
|
24
|
-
class_option :
|
|
16
|
+
class_option :server, type: :string
|
|
25
17
|
class_option :api_key, type: :string
|
|
26
18
|
class_option :verbose, type: :boolean, default: false, desc: "Show detailed init output (file-by-file actions)."
|
|
27
19
|
|
|
28
20
|
desc "Generates a new Strata project."
|
|
29
21
|
|
|
30
|
-
def validate_options
|
|
31
|
-
if options.key?(:source)
|
|
32
|
-
unless options.key?(:api_key)
|
|
33
|
-
raise Strata::CommandError,
|
|
34
|
-
"API key is required when using --source option. Use --api-key option."
|
|
35
|
-
end
|
|
36
|
-
else
|
|
37
|
-
unless @name && !@name.to_s.strip.empty?
|
|
38
|
-
raise Strata::CommandError,
|
|
39
|
-
"PROJECT_NAME is required when not using --source option."
|
|
40
|
-
end
|
|
41
|
-
end
|
|
42
|
-
end
|
|
43
|
-
|
|
44
|
-
def fetch_and_clone_if_available
|
|
45
|
-
return unless options.key?(:source)
|
|
46
|
-
|
|
47
|
-
project_info = fetch_project_info
|
|
48
|
-
@project_id = project_info["id"]
|
|
49
|
-
@project_info = project_info
|
|
50
|
-
|
|
51
|
-
return unless project_info["git_url"] && !project_info["git_url"].empty?
|
|
52
|
-
|
|
53
|
-
@cloned_from_git = true
|
|
54
|
-
clone_project(project_info["git_url"])
|
|
55
|
-
end
|
|
56
|
-
|
|
57
22
|
def create_project_structure
|
|
58
|
-
return if cloned_from_git?
|
|
59
|
-
|
|
60
23
|
empty_directory uid unless existing_directory?
|
|
61
24
|
empty_directory File.join(uid, "models")
|
|
62
25
|
empty_directory File.join(uid, "tests")
|
|
@@ -67,30 +30,14 @@ module Strata
|
|
|
67
30
|
end
|
|
68
31
|
|
|
69
32
|
def create_project_file
|
|
70
|
-
return if cloned_from_git?
|
|
71
|
-
|
|
72
33
|
template "project.yml", "#{uid}/project.yml"
|
|
73
34
|
end
|
|
74
35
|
|
|
75
|
-
def persist_project_id_if_needed
|
|
76
|
-
# Persist project_id after project.yml exists (either from clone or creation)
|
|
77
|
-
return unless @project_id
|
|
78
|
-
|
|
79
|
-
project_yml_path = File.join(uid, "project.yml")
|
|
80
|
-
return unless File.exist?(project_yml_path)
|
|
81
|
-
|
|
82
|
-
persist_project_id_to_project_yml
|
|
83
|
-
end
|
|
84
|
-
|
|
85
36
|
def create_agents_file
|
|
86
|
-
return if cloned_from_git?
|
|
87
|
-
|
|
88
37
|
copy_file "AGENTS.md", "#{uid}/AGENTS.md"
|
|
89
38
|
end
|
|
90
39
|
|
|
91
40
|
def create_datasources_file
|
|
92
|
-
return if cloned_from_git?
|
|
93
|
-
|
|
94
41
|
template "datasources.yml", "#{uid}/datasources.yml"
|
|
95
42
|
|
|
96
43
|
return unless options.key?(:datasource)
|
|
@@ -103,8 +50,6 @@ module Strata
|
|
|
103
50
|
end
|
|
104
51
|
|
|
105
52
|
def initialize_git
|
|
106
|
-
return if cloned_from_git?
|
|
107
|
-
|
|
108
53
|
inside uid do
|
|
109
54
|
run "git init", verbose: false, capture: true
|
|
110
55
|
create_file ".gitignore", ".strata\n"
|
|
@@ -112,7 +57,6 @@ module Strata
|
|
|
112
57
|
end
|
|
113
58
|
|
|
114
59
|
def setup_datasource
|
|
115
|
-
return if cloned_from_git?
|
|
116
60
|
return if options.key?(:datasource)
|
|
117
61
|
return if existing_directory?
|
|
118
62
|
|
|
@@ -129,8 +73,6 @@ module Strata
|
|
|
129
73
|
end
|
|
130
74
|
|
|
131
75
|
def initial_commit
|
|
132
|
-
return if cloned_from_git?
|
|
133
|
-
|
|
134
76
|
inside uid do
|
|
135
77
|
case Utils::Git.initial_commit("chore: initialize Strata project")
|
|
136
78
|
when :committed
|
|
@@ -151,83 +93,18 @@ module Strata
|
|
|
151
93
|
|
|
152
94
|
private
|
|
153
95
|
|
|
154
|
-
def cloned_from_git?
|
|
155
|
-
@cloned_from_git ||= false
|
|
156
|
-
end
|
|
157
|
-
|
|
158
96
|
def existing_directory?
|
|
159
97
|
File.directory?(name.to_s)
|
|
160
98
|
end
|
|
161
99
|
|
|
162
|
-
def fetch_project_info
|
|
163
|
-
conn = Faraday.new(url: options[:source]) do |f|
|
|
164
|
-
f.request :authorization, "Bearer", options[:api_key]
|
|
165
|
-
f.response :json
|
|
166
|
-
end
|
|
167
|
-
|
|
168
|
-
response = with_connection_error_handling(options[:source]) do
|
|
169
|
-
conn.get
|
|
170
|
-
end
|
|
171
|
-
|
|
172
|
-
unless response.success?
|
|
173
|
-
raise Strata::CommandError,
|
|
174
|
-
response_error_message(
|
|
175
|
-
response,
|
|
176
|
-
default_message: "Failed to fetch project info from #{options[:source]}. Status: #{response.status}."
|
|
177
|
-
)
|
|
178
|
-
end
|
|
179
|
-
|
|
180
|
-
response.body
|
|
181
|
-
end
|
|
182
|
-
|
|
183
|
-
def clone_project(git_url)
|
|
184
|
-
Output.print_status(:clone, "Cloning project from #{git_url}", type: :success, context: self)
|
|
185
|
-
run "git clone #{git_url} #{uid}", verbose: false, capture: true
|
|
186
|
-
|
|
187
|
-
raise Strata::CommandError, "Failed to clone repository from #{git_url}" unless File.directory?(uid)
|
|
188
|
-
|
|
189
|
-
Output.print_status(:success, "Project cloned successfully", type: :success, context: self)
|
|
190
|
-
end
|
|
191
|
-
|
|
192
|
-
def persist_project_id_to_project_yml
|
|
193
|
-
return unless @project_id
|
|
194
|
-
|
|
195
|
-
project_yml_path = File.join(uid, "project.yml")
|
|
196
|
-
Helpers::ProjectHelper.persist_project_id_to_yml(@project_id, project_yml_path: project_yml_path)
|
|
197
|
-
end
|
|
198
|
-
|
|
199
100
|
def uid
|
|
200
|
-
@uid ||=
|
|
201
|
-
@project_info["uid"] || options[:source].split("/").last
|
|
202
|
-
elsif existing_directory?
|
|
203
|
-
name.to_s
|
|
204
|
-
else
|
|
205
|
-
Utils.url_safe_str(name)
|
|
206
|
-
end
|
|
207
|
-
end
|
|
208
|
-
|
|
209
|
-
def name
|
|
210
|
-
# Use name from server if available (when using --source), otherwise use argument
|
|
211
|
-
return @project_info["name"] if options.key?(:source) && @project_info
|
|
212
|
-
|
|
213
|
-
@name
|
|
101
|
+
@uid ||= existing_directory? ? name.to_s : Utils.url_safe_str(name)
|
|
214
102
|
end
|
|
215
103
|
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
nil
|
|
220
|
-
end
|
|
221
|
-
|
|
222
|
-
def production_branch
|
|
223
|
-
if options.key?(:source) && @project_info && @project_info["production_branch"]
|
|
224
|
-
return @project_info["production_branch"]
|
|
225
|
-
end
|
|
226
|
-
|
|
227
|
-
"main"
|
|
104
|
+
# Written into project.yml so deploy/branch/cache just read it.
|
|
105
|
+
def server
|
|
106
|
+
@server ||= Configuration.resolve_server(options[:server])
|
|
228
107
|
end
|
|
229
|
-
|
|
230
|
-
attr_reader :project_id
|
|
231
108
|
end
|
|
232
109
|
end
|
|
233
110
|
end
|
|
@@ -312,7 +312,7 @@ If referenced measures are not joinable in **one universe** from that host table
|
|
|
312
312
|
|
|
313
313
|
**Use calculations (compound measures), not shared measure names** across unrelated facts.
|
|
314
314
|
|
|
315
|
-
**Automatic data blending** merges results on shared **dimensions** (often via `extended_blend_group`) — not by reusing the same measure name on multiple fact tables. See [extended blending](https://strata.do/
|
|
315
|
+
**Automatic data blending** merges results on shared **dimensions** (often via `extended_blend_group`) — not by reusing the same measure name on multiple fact tables. See [extended blending](https://strata.do/docs/semantic-model/extended-blending-groups) and the [glossary](https://strata.do/docs/getting-started/glossary).
|
|
316
316
|
|
|
317
317
|
---
|
|
318
318
|
|
|
@@ -348,7 +348,7 @@ fields:
|
|
|
348
348
|
|
|
349
349
|
When users group by a time period, the engine picks the last (`ending`) or first (`beginning`) snapshot in each bucket — not a sum of every row in the period.
|
|
350
350
|
|
|
351
|
-
More detail: [snapshot measures](https://strata.do/
|
|
351
|
+
More detail: [snapshot measures](https://strata.do/docs/semantic-model/fields/measures/snapshot)
|
|
352
352
|
|
|
353
353
|
---
|
|
354
354
|
|
|
@@ -375,7 +375,7 @@ More detail: [snapshot measures](https://strata.do/developer-docs/developer-guid
|
|
|
375
375
|
|
|
376
376
|
Inner: `sum(hours)` per `(day, account)`. Outer: median of those account totals per `day`.
|
|
377
377
|
|
|
378
|
-
More detail: [inclusions](https://strata.do/
|
|
378
|
+
More detail: [inclusions](https://strata.do/docs/advanced/inclusions)
|
|
379
379
|
|
|
380
380
|
---
|
|
381
381
|
|
|
@@ -404,7 +404,7 @@ More detail: [inclusions](https://strata.do/developer-docs/developer-guide/advan
|
|
|
404
404
|
|
|
405
405
|
`Return Status` will not appear in the grouping set for this measure even if the user adds it to the query; other dimensions still group normally.
|
|
406
406
|
|
|
407
|
-
More detail: [exclusions](https://strata.do/
|
|
407
|
+
More detail: [exclusions](https://strata.do/docs/advanced/exclusions)
|
|
408
408
|
|
|
409
409
|
---
|
|
410
410
|
|
|
@@ -425,14 +425,14 @@ More detail: [exclusions](https://strata.do/developer-docs/developer-guide/advan
|
|
|
425
425
|
|
|
426
426
|
| Feature | Use when |
|
|
427
427
|
|---------|----------|
|
|
428
|
-
| [Compound measures](https://strata.do/
|
|
429
|
-
| [Snapshot measures](https://strata.do/
|
|
430
|
-
| [Inclusions](https://strata.do/
|
|
431
|
-
| [Exclusions](https://strata.do/
|
|
428
|
+
| [Compound measures](https://strata.do/docs/semantic-model/fields/measures/compound) | Formula across named measures; host table sets dimensionality |
|
|
429
|
+
| [Snapshot measures](https://strata.do/docs/semantic-model/fields/measures/snapshot) | Period-boundary state (inventory, balances) — see [Snapshot measures](#snapshot-measures) |
|
|
430
|
+
| [Inclusions](https://strata.do/docs/advanced/inclusions) | Median/percentile/avg needing intermediate grain — see [Inclusion measures](#inclusion-measures) |
|
|
431
|
+
| [Exclusions](https://strata.do/docs/advanced/exclusions) | Control grouping and filters per dimension — see [Exclusion measures](#exclusion-measures) |
|
|
432
432
|
| Table `cost` | Lower cost = preferred table when multiple tables can answer; use for rollup vs detail routing |
|
|
433
|
-
| [Partitions](https://strata.do/
|
|
434
|
-
| [Extended blending](https://strata.do/
|
|
435
|
-
| [Imports](https://strata.do/
|
|
433
|
+
| [Partitions](https://strata.do/docs/semantic-model/partitions) | Table only has subset of data — `between` (date range) or `in_list`; planner routes when partition matches |
|
|
434
|
+
| [Extended blending](https://strata.do/docs/semantic-model/extended-blending-groups) | Blend dimensions across tables in same datasource — not duplicate measure names |
|
|
435
|
+
| [Imports](https://strata.do/docs/semantic-model/imports) | Reuse field definitions across `tbl` files |
|
|
436
436
|
|
|
437
437
|
---
|
|
438
438
|
|
|
@@ -474,7 +474,7 @@ Deploy **migration files and updated YAML in the same deployment**. Update `tbl.
|
|
|
474
474
|
|
|
475
475
|
**Agents:** propose migration YAML under `migrations/` and matching model edits; do **not** run interactive `strata create migration`. Flag renames for human review and confirm branch strategy.
|
|
476
476
|
|
|
477
|
-
More detail: [migrations](https://strata.do/
|
|
477
|
+
More detail: [migrations](https://strata.do/docs/cli/migrations)
|
|
478
478
|
|
|
479
479
|
---
|
|
480
480
|
|
|
@@ -482,30 +482,30 @@ More detail: [migrations](https://strata.do/developer-docs/developer-guide/cli/m
|
|
|
482
482
|
|
|
483
483
|
**Primary:** this `AGENTS.md` — follow inline rules first.
|
|
484
484
|
|
|
485
|
-
**Optional bulk load:** fetch https://strata.do/
|
|
485
|
+
**Optional bulk load:** fetch https://strata.do/docs/llms.txt when implementing a topic not fully covered here (complete Strata doc export for AI agents).
|
|
486
486
|
|
|
487
487
|
**Curated links** (use for examples and edge cases after reading the matching section above):
|
|
488
488
|
|
|
489
489
|
| Topic | URL |
|
|
490
490
|
|-------|-----|
|
|
491
|
-
| Semantic model overview | https://strata.do/
|
|
492
|
-
| Core concepts | https://strata.do/
|
|
493
|
-
| Glossary | https://strata.do/
|
|
494
|
-
| Tables | https://strata.do/
|
|
495
|
-
| Fields | https://strata.do/
|
|
496
|
-
| Expressions (SQL) | https://strata.do/
|
|
497
|
-
| Relationships | https://strata.do/
|
|
498
|
-
| Imports | https://strata.do/
|
|
499
|
-
| Compound measures | https://strata.do/
|
|
500
|
-
| Snapshot measures | https://strata.do/
|
|
501
|
-
| Inclusions | https://strata.do/
|
|
502
|
-
| Exclusions | https://strata.do/
|
|
503
|
-
| Partitions | https://strata.do/
|
|
504
|
-
| Cost optimization | https://strata.do/
|
|
505
|
-
| Extended blending | https://strata.do/
|
|
506
|
-
| CLI audit | https://strata.do/
|
|
507
|
-
| CLI deployment | https://strata.do/
|
|
508
|
-
| CLI migrations | https://strata.do/
|
|
509
|
-
| Star schema example | https://strata.do/
|
|
510
|
-
| TPC-DS tutorial | https://strata.do/
|
|
511
|
-
| AI agents and Strata | https://strata.do/
|
|
491
|
+
| Semantic model overview | https://strata.do/docs/semantic-model |
|
|
492
|
+
| Core concepts | https://strata.do/docs/getting-started/concepts |
|
|
493
|
+
| Glossary | https://strata.do/docs/getting-started/glossary |
|
|
494
|
+
| Tables | https://strata.do/docs/semantic-model/tables |
|
|
495
|
+
| Fields | https://strata.do/docs/semantic-model/fields |
|
|
496
|
+
| Expressions (SQL) | https://strata.do/docs/semantic-model/expressions/sql |
|
|
497
|
+
| Relationships | https://strata.do/docs/semantic-model/relationships/cardinality |
|
|
498
|
+
| Imports | https://strata.do/docs/semantic-model/imports |
|
|
499
|
+
| Compound measures | https://strata.do/docs/semantic-model/fields/measures/compound |
|
|
500
|
+
| Snapshot measures | https://strata.do/docs/semantic-model/fields/measures/snapshot |
|
|
501
|
+
| Inclusions | https://strata.do/docs/advanced/inclusions |
|
|
502
|
+
| Exclusions | https://strata.do/docs/advanced/exclusions |
|
|
503
|
+
| Partitions | https://strata.do/docs/semantic-model/partitions |
|
|
504
|
+
| Cost optimization | https://strata.do/docs/advanced/cost-optimization |
|
|
505
|
+
| Extended blending | https://strata.do/docs/semantic-model/extended-blending-groups |
|
|
506
|
+
| CLI audit | https://strata.do/docs/cli/audit |
|
|
507
|
+
| CLI deployment | https://strata.do/docs/cli/deployment |
|
|
508
|
+
| CLI migrations | https://strata.do/docs/cli/migrations |
|
|
509
|
+
| Star schema example | https://strata.do/docs/examples/patterns/star-schema |
|
|
510
|
+
| TPC-DS tutorial | https://strata.do/docs/examples/tpcds-tutorial |
|
|
511
|
+
| AI agents and Strata | https://strata.do/docs/api/ai-agents |
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# Datasource key should be unique in the Project
|
|
2
|
+
<%= @ds_key %>:
|
|
3
|
+
# This is the display name of this datasource
|
|
4
|
+
name: MYDATASOURCENAME
|
|
5
|
+
# Optional description to provide more information in the UI
|
|
6
|
+
description: This DS is the warm tier of our Product Domain
|
|
7
|
+
|
|
8
|
+
# Required. Should be one of hot, warm, or cold. Helps the Query engine
|
|
9
|
+
# prioritize a datasource when multiple can handle the request.
|
|
10
|
+
tier: warm
|
|
11
|
+
|
|
12
|
+
# A valid supported adapter
|
|
13
|
+
adapter: bigquery
|
|
14
|
+
|
|
15
|
+
# default client name sent to the db
|
|
16
|
+
client_name: Strata
|
|
17
|
+
|
|
18
|
+
# Max allowed time a query can run on this Datasource before manual time out
|
|
19
|
+
query_timeout: 3600
|
|
20
|
+
|
|
21
|
+
# Required: GCP project id
|
|
22
|
+
project_id: my-gcp-project
|
|
23
|
+
|
|
24
|
+
# Required: Default dataset for unqualified table names
|
|
25
|
+
dataset: analytics
|
|
26
|
+
|
|
27
|
+
# Authenticate with a service account keyfile by running:
|
|
28
|
+
# `strata ds auth <%= @ds_key %>`
|
|
29
|
+
# keyfile:
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# Datasource key should be unique in the Project
|
|
2
|
+
<%= @ds_key %>:
|
|
3
|
+
# This is the display name of this datasource
|
|
4
|
+
name: MYDATASOURCENAME
|
|
5
|
+
# Optional description to provide more information in the UI
|
|
6
|
+
description: This DS is the hot tier of our Product Domain
|
|
7
|
+
|
|
8
|
+
# Required. Should be one of hot, warm, or cold. Helps the Query engine
|
|
9
|
+
# prioritize a datasource when multiple can handle the request.
|
|
10
|
+
tier: hot
|
|
11
|
+
|
|
12
|
+
# A valid supported adapter
|
|
13
|
+
adapter: clickhouse
|
|
14
|
+
|
|
15
|
+
# default client name sent to the db
|
|
16
|
+
client_name: Strata
|
|
17
|
+
|
|
18
|
+
# Max allowed time a query can run on this Datasource before manual time out
|
|
19
|
+
query_timeout: 3600
|
|
20
|
+
|
|
21
|
+
# Required: http or https
|
|
22
|
+
protocol: http
|
|
23
|
+
|
|
24
|
+
# Required: ClickHouse server host
|
|
25
|
+
host: localhost
|
|
26
|
+
|
|
27
|
+
# Optional: HTTP port (default: 8123)
|
|
28
|
+
port: 8123
|
|
29
|
+
|
|
30
|
+
# Required: Database name
|
|
31
|
+
database: test_db
|
|
32
|
+
|
|
33
|
+
# Required: Username for connection
|
|
34
|
+
username: default
|
|
35
|
+
|
|
36
|
+
# To set the password securely please run:
|
|
37
|
+
# `strata ds auth <%= @ds_key %>`
|
|
38
|
+
# This command will save your credentials to the .strata global or local file.
|
|
39
|
+
# password:
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# Datasource key should be unique in the Project
|
|
2
|
+
<%= @ds_key %>:
|
|
3
|
+
# This is the display name of this datasource
|
|
4
|
+
name: MYDATASOURCENAME
|
|
5
|
+
# Optional description to provide more information in the UI
|
|
6
|
+
description: This DS is the warm tier of our Product Domain
|
|
7
|
+
|
|
8
|
+
# Required. Should be one of hot, warm, or cold. Helps the Query engine
|
|
9
|
+
# prioritize a datasource when multiple can handle the request.
|
|
10
|
+
tier: warm
|
|
11
|
+
|
|
12
|
+
# A valid supported adapter
|
|
13
|
+
adapter: redshift
|
|
14
|
+
|
|
15
|
+
# default client name sent to the db
|
|
16
|
+
client_name: Strata
|
|
17
|
+
|
|
18
|
+
# Max allowed time a query can run on this Datasource before manual time out
|
|
19
|
+
query_timeout: 3600
|
|
20
|
+
|
|
21
|
+
# Required: Redshift cluster endpoint
|
|
22
|
+
host: mycluster.abc123.us-east-1.redshift.amazonaws.com
|
|
23
|
+
|
|
24
|
+
# Optional: Port number (default: 5439)
|
|
25
|
+
port: 5439
|
|
26
|
+
|
|
27
|
+
# Required: Database name
|
|
28
|
+
database: dev
|
|
29
|
+
|
|
30
|
+
# Required: Username for connection
|
|
31
|
+
username: awsuser
|
|
32
|
+
|
|
33
|
+
# Optional: Schema name (default: public)
|
|
34
|
+
schema: public
|
|
35
|
+
|
|
36
|
+
# To set the password securely please run:
|
|
37
|
+
# `strata ds auth <%= @ds_key %>`
|
|
38
|
+
# This command will save your credentials to the .strata global or local file.
|
|
39
|
+
# password:
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# Datasource key should be unique in the Project
|
|
2
|
+
<%= @ds_key %>:
|
|
3
|
+
# This is the display name of this datasource
|
|
4
|
+
name: MYDATASOURCENAME
|
|
5
|
+
# Optional description to provide more information in the UI
|
|
6
|
+
description: This DS is the hot tier of our Product Domain
|
|
7
|
+
|
|
8
|
+
# Required. Should be one of hot, warm, or cold. Helps the Query engine
|
|
9
|
+
# prioritize a datasource when multiple can handle the request.
|
|
10
|
+
tier: hot
|
|
11
|
+
|
|
12
|
+
# A valid supported adapter
|
|
13
|
+
adapter: sqlite
|
|
14
|
+
|
|
15
|
+
# default client name sent to the db
|
|
16
|
+
client_name: Strata
|
|
17
|
+
|
|
18
|
+
# Max allowed time a query can run on this Datasource before manual time out
|
|
19
|
+
query_timeout: 600
|
|
20
|
+
|
|
21
|
+
# Required: Path to the SQLite database file
|
|
22
|
+
file: ./data/warehouse.db
|
|
@@ -10,7 +10,7 @@ uid: <%= uid %>
|
|
|
10
10
|
description: Your projects description
|
|
11
11
|
|
|
12
12
|
# Target Strata server (default environment, typically dev)
|
|
13
|
-
server: <%=
|
|
13
|
+
server: <%= server %>
|
|
14
14
|
|
|
15
15
|
# Alternatively, you can configure multiple environments (different Strata server instances).
|
|
16
16
|
# Each environment represents a separate Strata cluster (dev, staging, production).
|
|
@@ -19,7 +19,7 @@ server: <%= options[:source] %>
|
|
|
19
19
|
# dev:
|
|
20
20
|
# server: http://localhost:8080
|
|
21
21
|
# prod:
|
|
22
|
-
# server: https://app.strata.
|
|
22
|
+
# server: https://app.strata.do
|
|
23
23
|
|
|
24
24
|
# Project ID will be automatically set after first deployment.
|
|
25
25
|
# WARNING: Do not change this project_id. It links this project to the Strata server.
|
|
@@ -79,6 +79,22 @@ module Strata
|
|
|
79
79
|
# don't need special handling - no file lock issues
|
|
80
80
|
end
|
|
81
81
|
|
|
82
|
+
# A file based datasource can name a path that only exists where the server runs:
|
|
83
|
+
# the sample project's /data/tpcds.duckdb ships inside the Strata image. There is
|
|
84
|
+
# nothing to open from here, so the CLI cannot verify the connection - the server
|
|
85
|
+
# does, when the model deploys. A remote URL is not this case; httpfs opens one
|
|
86
|
+
# from anywhere, so a broken URL should still be reported.
|
|
87
|
+
def local_file_missing?(config)
|
|
88
|
+
return false unless config.is_a?(Hash)
|
|
89
|
+
|
|
90
|
+
file = config["file"].to_s.strip
|
|
91
|
+
# No file key at all belongs to the adapter's own required param check, and
|
|
92
|
+
# :memory: connects without touching the disk.
|
|
93
|
+
return false if file.empty? || file.casecmp(":memory:").zero? || file.include?("://")
|
|
94
|
+
|
|
95
|
+
!File.exist?(file)
|
|
96
|
+
end
|
|
97
|
+
|
|
82
98
|
def ensure_adapter_driver_gems!(adapter_sym)
|
|
83
99
|
required = ADAPTER_DRIVER_GEMS.fetch(adapter_sym, [])
|
|
84
100
|
return if required.empty?
|
|
@@ -199,7 +215,7 @@ module Strata
|
|
|
199
215
|
when :trino
|
|
200
216
|
"Notes:\n- 'trino-client' is pure Ruby; no system libraries required."
|
|
201
217
|
when :bigquery
|
|
202
|
-
"Notes:\n- 'google-cloud-bigquery' is pure Ruby; no system libraries required.\n- Authenticate with a service account keyfile
|
|
218
|
+
"Notes:\n- 'google-cloud-bigquery' is pure Ruby; no system libraries required.\n- Authenticate with a service account keyfile (strata ds auth)."
|
|
203
219
|
when :duckdb
|
|
204
220
|
"Notes:\n- See https://github.com/suketa/ruby-duckdb"
|
|
205
221
|
else
|
|
@@ -54,8 +54,9 @@ module Strata
|
|
|
54
54
|
project_yml_content = File.read(project_yml_path)
|
|
55
55
|
|
|
56
56
|
begin
|
|
57
|
-
|
|
58
|
-
|
|
57
|
+
# Root key only -- an indented `server:` belongs to a per-environment block.
|
|
58
|
+
if /^server:/.match?(project_yml_content)
|
|
59
|
+
updated_content = project_yml_content.sub(/^server:.*$/, "server: #{server}")
|
|
59
60
|
File.write(project_yml_path, updated_content)
|
|
60
61
|
else
|
|
61
62
|
File.open(project_yml_path, "a") do |f|
|
|
@@ -5,7 +5,7 @@ module Strata
|
|
|
5
5
|
module Prompts
|
|
6
6
|
# Docs. This URL is printed by the CLI and baked into every shipped gem —
|
|
7
7
|
# strata-docs has no redirects plugin, so renaming that page 404s old binaries.
|
|
8
|
-
DOCS_BUILD_MODEL_URL = "https://strata.do/
|
|
8
|
+
DOCS_BUILD_MODEL_URL = "https://strata.do/docs/cli/building-your-model"
|
|
9
9
|
|
|
10
10
|
# Shown after 'strata init' and 'strata datasource add'. %s is the agent hint.
|
|
11
11
|
MSG_AGENTS_MD_PRESENT = "AGENTS.md in this project tells it Strata's rules."
|
|
@@ -76,7 +76,7 @@ module Strata
|
|
|
76
76
|
(operation == "swap") ? "post (after deployment)" : "pre (before deployment)"
|
|
77
77
|
end
|
|
78
78
|
|
|
79
|
-
# dir is the project root — 'strata
|
|
79
|
+
# dir is the project root — 'strata project sample' clones a repo that may lack an AGENTS.md.
|
|
80
80
|
def build_model_next_steps(dir = ".")
|
|
81
81
|
hint = File.exist?(File.join(dir, "AGENTS.md")) ? MSG_AGENTS_MD_PRESENT : MSG_AGENTS_MD_MISSING
|
|
82
82
|
format(MSG_BUILD_MODEL_NEXT_STEPS, hint)
|
data/lib/strata/cli/main.rb
CHANGED
|
@@ -30,21 +30,19 @@ module Strata
|
|
|
30
30
|
say VERSION
|
|
31
31
|
end
|
|
32
32
|
|
|
33
|
-
desc "init
|
|
33
|
+
desc "init PROJECT_NAME", "Initializes a new Strata project."
|
|
34
34
|
long_desc_from_file "init"
|
|
35
35
|
option :datasource, aliases: ["d"], type: :string, desc: "One of the supported data warehouse adapters.",
|
|
36
36
|
repeatable: true
|
|
37
|
-
option :
|
|
38
|
-
option :api_key, aliases: ["a"], type: :string, desc: "Api Key.
|
|
37
|
+
option :server, aliases: ["s"], type: :string, desc: "Strata server URL. Defaults to #{Configuration::DEFAULT_SERVER}."
|
|
38
|
+
option :api_key, aliases: ["a"], type: :string, desc: "Api Key. Optional — the first deploy signs you in via browser."
|
|
39
39
|
option :verbose, aliases: ["v"], type: :boolean, default: false,
|
|
40
40
|
desc: "Show detailed init output (file-by-file actions)."
|
|
41
41
|
|
|
42
42
|
def init(project_name = nil)
|
|
43
|
-
unless project_name
|
|
44
|
-
raise Strata::CommandError, "PROJECT_NAME is required when not using --source option."
|
|
45
|
-
end
|
|
43
|
+
raise Strata::CommandError, "PROJECT_NAME is required." unless project_name
|
|
46
44
|
|
|
47
|
-
print_status(:created,
|
|
45
|
+
print_status(:created, project_name.to_s, type: :info)
|
|
48
46
|
invoke Generators::Project, [project_name], options
|
|
49
47
|
end
|
|
50
48
|
|
|
@@ -48,6 +48,7 @@ module Strata
|
|
|
48
48
|
results[:models] = run_check("Checking model definitions") { audit_models }
|
|
49
49
|
results[:connections] = run_check("Checking data source connections") { audit_connections }
|
|
50
50
|
|
|
51
|
+
warn_unverifiable_datasources
|
|
51
52
|
warn_stray_files
|
|
52
53
|
report_results(results)
|
|
53
54
|
end
|
|
@@ -67,6 +68,7 @@ module Strata
|
|
|
67
68
|
desc "connections", "Check data source connections"
|
|
68
69
|
def connections
|
|
69
70
|
results = {connections: run_check("Checking data source connections") { audit_connections }}
|
|
71
|
+
warn_unverifiable_datasources
|
|
70
72
|
report_results(results)
|
|
71
73
|
end
|
|
72
74
|
|
|
@@ -92,6 +94,17 @@ module Strata
|
|
|
92
94
|
strays.sort.each { print_warning(" #{it}") }
|
|
93
95
|
end
|
|
94
96
|
|
|
97
|
+
# Pointing a datasource at a path the server owns is a normal setup — the sample
|
|
98
|
+
# project's /data/tpcds.duckdb ships inside the Strata image — so say which
|
|
99
|
+
# connections went unchecked instead of failing a correct project.
|
|
100
|
+
def warn_unverifiable_datasources
|
|
101
|
+
skipped = datasources.select { |_key, config| local_file_missing?(config) }
|
|
102
|
+
return if skipped.empty?
|
|
103
|
+
|
|
104
|
+
print_warning("\n Connection not checked — file not on this machine (the server opens it at deploy):")
|
|
105
|
+
skipped.each { |key, config| print_warning(" #{key}: #{config["file"]}") }
|
|
106
|
+
end
|
|
107
|
+
|
|
95
108
|
def run_check(message)
|
|
96
109
|
failures = []
|
|
97
110
|
with_spinner(message) do
|
|
@@ -347,8 +360,6 @@ module Strata
|
|
|
347
360
|
end
|
|
348
361
|
|
|
349
362
|
def audit_connections
|
|
350
|
-
return [] if CLI.config.get("git").to_s.include?("tpc-ds-retail-sample")
|
|
351
|
-
|
|
352
363
|
failures = []
|
|
353
364
|
|
|
354
365
|
unless File.exist?("datasources.yml")
|
|
@@ -362,7 +373,11 @@ module Strata
|
|
|
362
373
|
return failures
|
|
363
374
|
end
|
|
364
375
|
|
|
365
|
-
datasources.
|
|
376
|
+
datasources.each do |ds_key, config|
|
|
377
|
+
# Skip before create_adapter: it would install the adapter's driver gem to
|
|
378
|
+
# open a file that isn't here. warn_unverifiable_datasources names these.
|
|
379
|
+
next if local_file_missing?(config)
|
|
380
|
+
|
|
366
381
|
adapter = create_adapter(ds_key)
|
|
367
382
|
adapter.test_connection(raise_exception: true)
|
|
368
383
|
rescue => e
|
|
@@ -115,7 +115,10 @@ module Strata
|
|
|
115
115
|
say " Databricks uses OAuth M2M with service principal credentials (client ID + client secret).", :cyan
|
|
116
116
|
end
|
|
117
117
|
if adapter == "bigquery"
|
|
118
|
-
say " BigQuery
|
|
118
|
+
say " BigQuery: give the service account JSON keyfile path; only its client_email and private_key are stored.", :cyan
|
|
119
|
+
end
|
|
120
|
+
if adapter == "snowflake"
|
|
121
|
+
say " Snowflake key-pair: give the private key path; the PEM contents are stored, not the path.", :cyan
|
|
119
122
|
end
|
|
120
123
|
say ""
|
|
121
124
|
creds.collect
|
|
@@ -143,7 +146,6 @@ module Strata
|
|
|
143
146
|
|
|
144
147
|
desc "auth DS_KEY", "Set credentials for the given datasource key (DS_KEY)."
|
|
145
148
|
long_desc_from_file "datasource/auth"
|
|
146
|
-
method_option :remote, aliases: ["r"], type: :boolean, desc: "Set credentials on remote server"
|
|
147
149
|
def auth(ds_key)
|
|
148
150
|
unless datasources[ds_key]
|
|
149
151
|
say "Error: Datasource '#{ds_key}' not found in datasources.yml", :red
|
|
@@ -165,7 +167,10 @@ module Strata
|
|
|
165
167
|
say " Databricks uses OAuth M2M with service principal credentials (client ID + client secret).", :cyan
|
|
166
168
|
end
|
|
167
169
|
if adapter == "bigquery"
|
|
168
|
-
say " BigQuery
|
|
170
|
+
say " BigQuery: give the service account JSON keyfile path; only its client_email and private_key are stored.", :cyan
|
|
171
|
+
end
|
|
172
|
+
if adapter == "snowflake"
|
|
173
|
+
say " Snowflake key-pair: give the private key path; the PEM contents are stored, not the path.", :cyan
|
|
169
174
|
end
|
|
170
175
|
say ""
|
|
171
176
|
creds.collect
|
|
@@ -184,6 +189,12 @@ module Strata
|
|
|
184
189
|
end
|
|
185
190
|
rescue => e
|
|
186
191
|
say "\t!! Failed to connect: \n\t#{e.message}", :red
|
|
192
|
+
# datasources[] rather than ds_config: an unknown key is one of the ways we
|
|
193
|
+
# get here, and ds_config raises on it.
|
|
194
|
+
if local_file_missing?(datasources[ds_key])
|
|
195
|
+
say "\tThat path is not on this machine. If it lives inside the Strata container " \
|
|
196
|
+
"(like the sample's /data/tpcds.duckdb), the server opens it at deploy and this is expected.", :yellow
|
|
197
|
+
end
|
|
187
198
|
end
|
|
188
199
|
|
|
189
200
|
desc "tables DS_KEY", "List tables from DS_KEY datasource"
|
|
@@ -149,36 +149,7 @@ module Strata
|
|
|
149
149
|
end
|
|
150
150
|
|
|
151
151
|
def ensure_server(config)
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
config["server"] = collect_server_interactively
|
|
155
|
-
end
|
|
156
|
-
|
|
157
|
-
def collect_server_interactively
|
|
158
|
-
say "\nServer URL not found (not in project.yml)", :yellow
|
|
159
|
-
say "You can add it to project.yml or enter it now here\n", :cyan
|
|
160
|
-
|
|
161
|
-
server = prompt.ask("Enter Strata server URL:") do |q|
|
|
162
|
-
q.required true
|
|
163
|
-
q.validate(/\S+/, "Server URL cannot be empty")
|
|
164
|
-
end
|
|
165
|
-
|
|
166
|
-
server = server.strip.delete_suffix("/") # normalize: remove trailing slash
|
|
167
|
-
|
|
168
|
-
with_spinner("Saving server URL to project.yml") do
|
|
169
|
-
save_server_to_project_yml(server)
|
|
170
|
-
end
|
|
171
|
-
|
|
172
|
-
server
|
|
173
|
-
end
|
|
174
|
-
|
|
175
|
-
def save_server_to_project_yml(server)
|
|
176
|
-
Helpers::ProjectHelper.persist_server_to_project_yml(server, project_yml_path: File.join(project_path, Configuration::PROJECT_CONFIG_FILE))
|
|
177
|
-
CLI.config.reload!
|
|
178
|
-
return unless Utils::Git.git_repo?
|
|
179
|
-
|
|
180
|
-
Utils::Git.commit_file(Configuration::PROJECT_CONFIG_FILE, "[Strata-CLI] Add server URL to project.yml",
|
|
181
|
-
project_path)
|
|
152
|
+
config["server"] = Configuration.resolve_server(config["server"])
|
|
182
153
|
end
|
|
183
154
|
|
|
184
155
|
def ensure_git_url_populated
|
|
@@ -24,19 +24,15 @@ module Strata
|
|
|
24
24
|
SAMPLE_REPO = "https://github.com/stratasite/tpc-ds-retail-sample"
|
|
25
25
|
SAMPLE_DIR = "tpc-ds-retail-sample"
|
|
26
26
|
SAMPLE_PROJECT_UID = "tpc-ds-retail-sample"
|
|
27
|
-
DEFAULT_SERVER = "http://localhost:8080"
|
|
28
27
|
|
|
29
28
|
desc "sample", "Clone the Strata sample project and set it up interactively"
|
|
30
29
|
option :api_key, aliases: ["a"], type: :string, desc: "Strata API key (optional — skip browser auth)"
|
|
31
30
|
option :server, aliases: ["s"], type: :string, desc: "Strata server URL (optional — prompted if omitted)"
|
|
32
31
|
def sample
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
server = (options[:server] || prompt.ask("Strata server URL:", default: DEFAULT_SERVER) { |q| q.modify :strip }).delete_suffix("/")
|
|
38
|
-
api_key = fetch_api_key_via_browser(server)
|
|
39
|
-
end
|
|
32
|
+
server = Configuration.resolve_server(
|
|
33
|
+
options[:server] || prompt.ask("Strata server URL:", default: Configuration::DEFAULT_SERVER) { |q| q.modify :strip }
|
|
34
|
+
)
|
|
35
|
+
api_key = options[:api_key] || fetch_api_key_via_browser(server)
|
|
40
36
|
|
|
41
37
|
if File.directory?(SAMPLE_DIR)
|
|
42
38
|
print_info("Directory '#{SAMPLE_DIR}' already exists — re-using it")
|
|
@@ -51,9 +47,10 @@ module Strata
|
|
|
51
47
|
File.write(".gitignore", ".strata\n.strata_cli/\n")
|
|
52
48
|
|
|
53
49
|
strata_path = ".strata"
|
|
54
|
-
File.write(strata_path, "api_key: #{api_key}\
|
|
50
|
+
File.write(strata_path, "api_key: #{api_key}\n")
|
|
55
51
|
File.chmod(0o600, strata_path)
|
|
56
52
|
|
|
53
|
+
Helpers::ProjectHelper.persist_server_to_project_yml(server)
|
|
57
54
|
Helpers::ProjectHelper.persist_project_id_to_yml(SAMPLE_PROJECT_UID)
|
|
58
55
|
|
|
59
56
|
system("git add -A", exception: true)
|
data/lib/strata/cli/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: strata-cli
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.21
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Ajo Abraham
|
|
@@ -16,14 +16,14 @@ dependencies:
|
|
|
16
16
|
requirements:
|
|
17
17
|
- - "~>"
|
|
18
18
|
- !ruby/object:Gem::Version
|
|
19
|
-
version: 0.6.
|
|
19
|
+
version: 0.6.2
|
|
20
20
|
type: :runtime
|
|
21
21
|
prerelease: false
|
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
|
23
23
|
requirements:
|
|
24
24
|
- - "~>"
|
|
25
25
|
- !ruby/object:Gem::Version
|
|
26
|
-
version: 0.6.
|
|
26
|
+
version: 0.6.2
|
|
27
27
|
- !ruby/object:Gem::Dependency
|
|
28
28
|
name: aws-sdk-athena
|
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -274,12 +274,16 @@ files:
|
|
|
274
274
|
- lib/strata/cli/generators/table.rb
|
|
275
275
|
- lib/strata/cli/generators/templates/AGENTS.md
|
|
276
276
|
- lib/strata/cli/generators/templates/adapters/athena.yml
|
|
277
|
+
- lib/strata/cli/generators/templates/adapters/bigquery.yml
|
|
278
|
+
- lib/strata/cli/generators/templates/adapters/clickhouse.yml
|
|
277
279
|
- lib/strata/cli/generators/templates/adapters/databricks.yml
|
|
278
280
|
- lib/strata/cli/generators/templates/adapters/druid.yml
|
|
279
281
|
- lib/strata/cli/generators/templates/adapters/duckdb.yml
|
|
280
282
|
- lib/strata/cli/generators/templates/adapters/mysql.yml
|
|
281
283
|
- lib/strata/cli/generators/templates/adapters/postgres.yml
|
|
284
|
+
- lib/strata/cli/generators/templates/adapters/redshift.yml
|
|
282
285
|
- lib/strata/cli/generators/templates/adapters/snowflake.yml
|
|
286
|
+
- lib/strata/cli/generators/templates/adapters/sqlite.yml
|
|
283
287
|
- lib/strata/cli/generators/templates/adapters/sqlserver.yml
|
|
284
288
|
- lib/strata/cli/generators/templates/adapters/trino.yml
|
|
285
289
|
- lib/strata/cli/generators/templates/datasources.yml
|