strata-cli 0.1.18 → 0.1.20

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e13d7f619d5b03f2c0ab353c8064fe5f1c17e97d8a3bc63d0d32abb11da628de
-  data.tar.gz: '095c483ea8913658de1d52dbe33bd71491b04b88f91f6e5c957417760bd1db1b'
+  metadata.gz: 28b84442e2989e729c2f9241c4af084eed8b346f62267d00c7bc27f3a4cb7d5d
+  data.tar.gz: b3e436e5055ea145a030bd88e27fb7b11ec1f6965886a65cb94aeee26ba34f5f
 SHA512:
-  metadata.gz: 1444ff06c0d6ad24303c6fec60c925fda1fffa17513cb2c8c8e7a2071af0f070181c882d58f832e1fca088c6601168140b0557a20106a0e23513da47173a0302
-  data.tar.gz: e952da251f4c0562fcc1f26be54f84245e046ed3041374e5efb17faf4b293a903db3c573142ff43c6efd290306252f2ad37cfce27ec27f0a824b6f6485379bc6
+  metadata.gz: 022cb33cdb574fd26548b9239ea39520e196f019a5f5d1808e84acf19e15f22a92d7acf550b70d729a799d4a389e48cd66950fa616809afbb96f569f87c00720
+  data.tar.gz: f3bd0b00b86ddae87a55262f15d49665606fde89211d3bd2fc5ec9ca33233b998253e5f949d170052a3c8d7ee210c5f6187ed38ed9603487cd5c218fdd9ed89b

data/CHANGELOG.md

@@ -1,5 +1,22 @@
 ## [Unreleased]
 
+## [0.1.20] - 2026-09-30
+
+### Changed
+
+- **File-based secrets stored as contents**: `strata ds auth` now reads a BigQuery service account keyfile and saves only its `client_email` and `private_key`, and reads a Snowflake key-pair private key file and saves the PEM text, so a remote Strata server never needs the file. `.strata` blocks are written with YAML serialisation to hold multi-line values. Requires dwh 0.6.2 and a server with encrypted `password`/`private_key` columns.
+- **`strata ds auth --remote` removed**: the flag was documented but never implemented; credentials reach the server through `strata deploy`.
+
+### Fixed
+
+- `strata init -d ADAPTER` wrote a bare `ADAPTER:` node with no config, which then broke `ds list`, `ds auth`, and `audit`. It now renders the commented adapter template again, and templates were added for sqlite, redshift, clickhouse, and bigquery.
+
+## [0.1.19] - 2026-09-30
+
+### Added
+
+- **BigQuery datasource**: `strata datasource add bigquery` prompts for `project_id` and `dataset`; `strata ds auth` stores a service account `keyfile` path (Application Default Credentials when left blank). Requires dwh 0.6.0.
+
 ## [0.1.18] - 2026-09-26
 
 ### Changed

data/README.md

@@ -364,6 +364,8 @@ ai_api_key: your-ai-key
 
 **Security:** The `.strata` file automatically has restrictive permissions (0600) for security.
 
+File-based secrets are stored as contents, not paths: a BigQuery keyfile becomes `client_email` + `private_key`, and a Snowflake key-pair `private_key` holds the PEM text. Credentials travel to the server with each deploy and are stored encrypted there.
+
 ### Authentication
 
 You don't need to create an API key manually. When no key is configured, `strata deploy` (and `strata project sample`) asks how you'd like to authenticate:
@@ -434,6 +436,15 @@ my_databricks:
 
 Use service principal credentials and run `strata ds auth my_databricks` to save `oauth_client_id` and `oauth_client_secret` in `.strata`.
 
+```yaml
+my_bigquery:
+  adapter: bigquery
+  project_id: my-gcp-project
+  dataset: analytics
+```
+
+Run `strata ds auth my_bigquery` and give the service account JSON keyfile path: only its `client_email` and `private_key` are saved in `.strata`, so the server never needs the file.
+
 ## Supported Data Warehouse Adapters
 
 - **PostgreSQL** - Full support
@@ -448,6 +459,7 @@ Use service principal credentials and run `strata ds auth my_databricks` to save
 - **Databricks** - Databricks SQL warehouse support
 - **ClickHouse** - ClickHouse HTTP/native SQL support
 - **SQLite** - SQLite database support
+- **BigQuery** - Google BigQuery support (service account key)
 
 ## Security
 

data/lib/strata/cli/credentials.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require "yaml"
+require "json"
 require "tty-prompt"
 
 module Strata
@@ -39,7 +40,7 @@ module Strata
             credentials["personal_access_token"] = @prompt.ask("Enter Personal Access Token:")
           when "kp"
             credentials["username"] = @prompt.ask("Enter Username:")
-            credentials["private_key"] = @prompt.ask("Enter Private Key Absolute Path:")
+            credentials["private_key"] = read_secret_file("Private key path (the PEM contents are stored):")
           end
         when "athena"
           credentials["access_key_id"] = @prompt.ask("AWS Access Key ID:")
@@ -50,6 +51,8 @@ module Strata
           credentials["oauth_client_secret"] = @prompt.ask("OAuth Client Secret:")
         when "clickhouse"
           credentials["password"] = @prompt.mask("Enter Password (leave blank if none):")
+        when "bigquery"
+          credentials.merge!(bigquery_service_account)
         else
           if required?
             unless %w[postgres mysql trino sqlserver].include?(adapter)
@@ -68,12 +71,11 @@ module Strata
 
       def write_local(ds_key, context)
         context.gsub_file Configuration::STRATA_CONFIG_FILE, Utils.yaml_block_matcher(ds_key), ""
-        creds = "#{ds_key}:"
-        credentials.each do |key, val|
-          creds << "\n  #{key}: #{val}" unless val.nil? || val.empty?
-        end
+        values = credentials.reject { |_, val| val.nil? || val.to_s.empty? }
 
-        context.append_to_file Configuration::STRATA_CONFIG_FILE, creds
+        # YAML.dump so multi-line values (PEM keys) become indented block scalars
+        # that yaml_block_matcher can still find and replace on the next auth.
+        context.append_to_file Configuration::STRATA_CONFIG_FILE, YAML.dump({ds_key => values}).delete_prefix("---\n")
 
         # Set restrictive permissions (read/write for owner only)
         strata_file = Configuration::STRATA_CONFIG_FILE
@@ -81,6 +83,43 @@ module Strata
       rescue Errno::EACCES => e
         raise Strata::CommandError, "Permission denied setting permissions on #{strata_file}: #{e.message}"
       end
+
+      private
+
+      # Asks for a file path and returns the file's contents so the server never
+      # needs the file itself. Blank returns nil; an answer that is not a path is
+      # kept as typed, which allows pasting the value directly.
+      def read_secret_file(question)
+        answer = @prompt.ask(question).to_s.strip
+        return nil if answer.empty?
+
+        File.exist?(answer) ? File.read(answer) : answer
+      end
+
+      # Reads the Google service account JSON once and keeps only the two fields
+      # needed to authenticate. The server has no other way to reach BigQuery,
+      # so without the file nothing is saved and the user is told to come back.
+      def bigquery_service_account
+        path = @prompt.ask("Service account keyfile path:").to_s.strip
+        if path.empty?
+          say "  No keyfile given, so no BigQuery credentials were saved. Run `strata ds auth <key>` with the service account keyfile before deploying.", :yellow
+          return {}
+        end
+
+        key = JSON.parse(File.read(path))
+        unless key["type"] == "service_account" && key["client_email"] && key["private_key"]
+          raise Strata::CommandError, "#{path} is not a service account keyfile (needs type, client_email and private_key)"
+        end
+
+        expected = @datasource_config["project_id"]
+        if expected && key["project_id"] && expected != key["project_id"]
+          say "  Warning: keyfile project '#{key["project_id"]}' differs from datasources.yml project_id '#{expected}'", :yellow
+        end
+
+        key.slice("client_email", "private_key")
+      rescue Errno::ENOENT, JSON::ParserError => e
+        raise Strata::CommandError, "Could not read service account keyfile #{path}: #{e.message}"
+      end
     end
   end
 end

data/lib/strata/cli/descriptions/datasource/add.txt

@@ -18,4 +18,4 @@ Examples:
 
   strata datasource add snowflake        # Directly add Snowflake datasource
 
-Supported adapters: postgres, mysql, sqlserver, snowflake, athena, trino, duckdb, druid, clickhouse, databricks, redshift, sqlite
+Supported adapters: postgres, mysql, sqlserver, snowflake, athena, trino, duckdb, druid, clickhouse, databricks, redshift, sqlite, bigquery

data/lib/strata/cli/descriptions/datasource/auth.txt

@@ -2,13 +2,12 @@ Example to set local credentials for a datasource with key games_dwh:
 
   strata ds auth games_dwh
 
-Example to set remote credentials for the same datasource:
-
-  strata ds auth games_dwh -r
-
-Local credentials are saved in the projects .strata file. This will should not
-be checked into git.
-
-Remote credentials will securely send the credentials to the Strata server. This is
-not requires for some modes like OAuth. In that case each user will be prompted to 
-go through the OAuth flow. Not all adapters support OAuth.
+Credentials are saved in the project's .strata file, which must not be
+checked into git. They are sent to the Strata server with each deploy and
+stored encrypted there.
+
+File-based secrets are stored as contents, not paths, so the server never
+needs the file: for BigQuery, give the service account JSON keyfile path and
+only its client_email and private_key are kept; for Snowflake key-pair auth, give the
+private key path and the PEM contents are kept. OAuth modes need no local
+credentials; each user goes through the OAuth flow instead.

data/lib/strata/cli/generators/datasource.rb

@@ -27,8 +27,12 @@ module Strata
           @ds_key = get_unique_ds_key
           Output.print_status(:adapter, "adding #{adapter} config to datasources", type: :warning, context: self)
 
-          # Interactive mode: write config directly from prompts
-          config_yaml = {@ds_key => options[:config]}.to_yaml.sub(/^---\n/, "\n")
+          # `ds add` passes prompted config; `init -d` renders the commented adapter template.
+          config_yaml = if options[:config]
+            {@ds_key => options[:config]}.to_yaml.sub(/^---\n/, "\n")
+          else
+            "\n#{render_template("adapters/#{adapter}.yml")}"
+          end
           append_to_file pathify("datasources.yml"), config_yaml
         end
 

data/lib/strata/cli/generators/templates/adapters/bigquery.yml

@@ -0,0 +1,29 @@
+# Datasource key should be unique in the Project
+<%= @ds_key %>:
+  # This is the display name of this datasource
+  name: MYDATASOURCENAME
+  # Optional description to provide more information in the UI
+  description: This DS is the warm tier of our Product Domain
+
+  # Required. Should be one of hot, warm, or cold. Helps the Query engine
+  # prioritize a datasource when multiple can handle the request.
+  tier: warm
+
+  # A valid supported adapter
+  adapter: bigquery
+
+  # default client name sent to the db
+  client_name: Strata
+
+  # Max allowed time a query can run on this Datasource before manual time out
+  query_timeout: 3600
+
+  # Required: GCP project id
+  project_id: my-gcp-project
+
+  # Required: Default dataset for unqualified table names
+  dataset: analytics
+
+  # Authenticate with a service account keyfile by running:
+  # `strata ds auth <%= @ds_key %>`
+  # keyfile:

data/lib/strata/cli/generators/templates/adapters/clickhouse.yml

@@ -0,0 +1,39 @@
+# Datasource key should be unique in the Project
+<%= @ds_key %>:
+  # This is the display name of this datasource
+  name: MYDATASOURCENAME
+  # Optional description to provide more information in the UI
+  description: This DS is the hot tier of our Product Domain
+
+  # Required. Should be one of hot, warm, or cold. Helps the Query engine
+  # prioritize a datasource when multiple can handle the request.
+  tier: hot
+
+  # A valid supported adapter
+  adapter: clickhouse
+
+  # default client name sent to the db
+  client_name: Strata
+
+  # Max allowed time a query can run on this Datasource before manual time out
+  query_timeout: 3600
+
+  # Required: http or https
+  protocol: http
+
+  # Required: ClickHouse server host
+  host: localhost
+
+  # Optional: HTTP port (default: 8123)
+  port: 8123
+
+  # Required: Database name
+  database: test_db
+
+  # Required: Username for connection
+  username: default
+
+  # To set the password securely please run:
+  # `strata ds auth <%= @ds_key %>`
+  # This command will save your credentials to the .strata global or local file.
+  # password:

data/lib/strata/cli/generators/templates/adapters/redshift.yml

@@ -0,0 +1,39 @@
+# Datasource key should be unique in the Project
+<%= @ds_key %>:
+  # This is the display name of this datasource
+  name: MYDATASOURCENAME
+  # Optional description to provide more information in the UI
+  description: This DS is the warm tier of our Product Domain
+
+  # Required. Should be one of hot, warm, or cold. Helps the Query engine
+  # prioritize a datasource when multiple can handle the request.
+  tier: warm
+
+  # A valid supported adapter
+  adapter: redshift
+
+  # default client name sent to the db
+  client_name: Strata
+
+  # Max allowed time a query can run on this Datasource before manual time out
+  query_timeout: 3600
+
+  # Required: Redshift cluster endpoint
+  host: mycluster.abc123.us-east-1.redshift.amazonaws.com
+
+  # Optional: Port number (default: 5439)
+  port: 5439
+
+  # Required: Database name
+  database: dev
+
+  # Required: Username for connection
+  username: awsuser
+
+  # Optional: Schema name (default: public)
+  schema: public
+
+  # To set the password securely please run:
+  # `strata ds auth <%= @ds_key %>`
+  # This command will save your credentials to the .strata global or local file.
+  # password:

data/lib/strata/cli/generators/templates/adapters/sqlite.yml

@@ -0,0 +1,22 @@
+# Datasource key should be unique in the Project
+<%= @ds_key %>:
+  # This is the display name of this datasource
+  name: MYDATASOURCENAME
+  # Optional description to provide more information in the UI
+  description: This DS is the hot tier of our Product Domain
+
+  # Required. Should be one of hot, warm, or cold. Helps the Query engine
+  # prioritize a datasource when multiple can handle the request.
+  tier: hot
+
+  # A valid supported adapter
+  adapter: sqlite
+
+  # default client name sent to the db
+  client_name: Strata
+
+  # Max allowed time a query can run on this Datasource before manual time out
+  query_timeout: 600
+
+  # Required: Path to the SQLite database file
+  file: ./data/warehouse.db

data/lib/strata/cli/helpers/datasource_helper.rb

@@ -13,7 +13,8 @@ module Strata
         athena: %w[aws-sdk-athena aws-sdk-s3 rexml],
         trino: %w[trino-client],
         sqlite: %w[sqlite3],
-        duckdb: %w[duckdb]
+        duckdb: %w[duckdb],
+        bigquery: %w[google-cloud-bigquery]
       }.freeze
 
       def resolve_datasource(ds_key_arg = nil)
@@ -197,6 +198,8 @@ module Strata
           "Notes:\n- AWS SDK gems are pure Ruby; no system libraries required."
         when :trino
           "Notes:\n- 'trino-client' is pure Ruby; no system libraries required."
+        when :bigquery
+          "Notes:\n- 'google-cloud-bigquery' is pure Ruby; no system libraries required.\n- Authenticate with a service account keyfile (strata ds auth)."
         when :duckdb
           "Notes:\n- See https://github.com/suketa/ruby-duckdb"
         else

data/lib/strata/cli/sub_commands/datasource.rb

@@ -114,6 +114,12 @@ module Strata
             if adapter == "databricks"
               say "  Databricks uses OAuth M2M with service principal credentials (client ID + client secret).", :cyan
             end
+            if adapter == "bigquery"
+              say "  BigQuery: give the service account JSON keyfile path; only its client_email and private_key are stored.", :cyan
+            end
+            if adapter == "snowflake"
+              say "  Snowflake key-pair: give the private key path; the PEM contents are stored, not the path.", :cyan
+            end
             say ""
             creds.collect
             creds.write_local(ds_key, self)
@@ -140,7 +146,6 @@ module Strata
 
         desc "auth DS_KEY", "Set credentials for the given datasource key (DS_KEY)."
         long_desc_from_file "datasource/auth"
-        method_option :remote, aliases: ["r"], type: :boolean, desc: "Set credentials on remote server"
         def auth(ds_key)
           unless datasources[ds_key]
             say "Error: Datasource '#{ds_key}' not found in datasources.yml", :red
@@ -161,6 +166,12 @@ module Strata
           if adapter == "databricks"
             say "  Databricks uses OAuth M2M with service principal credentials (client ID + client secret).", :cyan
           end
+          if adapter == "bigquery"
+            say "  BigQuery: give the service account JSON keyfile path; only its client_email and private_key are stored.", :cyan
+          end
+          if adapter == "snowflake"
+            say "  Snowflake key-pair: give the private key path; the PEM contents are stored, not the path.", :cyan
+          end
           say ""
           creds.collect
           creds.write_local(ds_key, self)
@@ -396,6 +407,11 @@ module Strata
               "database" => "dev",
               "username" => "awsuser"
             }
+          when "bigquery"
+            {
+              "project_id" => "my-gcp-project",
+              "dataset" => "analytics"
+            }
           else
             {}
           end
@@ -404,6 +420,7 @@ module Strata
         def humanize_field(field)
           {
             "account_identifier" => "Account identifier",
+            "project_id" => "Project ID",
             "s3_output_location" => "S3 output location",
             "auth_mode" => "Auth mode"
           }.fetch(field, field.split("_").map(&:capitalize).join(" "))

data/lib/strata/cli/version.rb

@@ -2,6 +2,6 @@
 
 module Strata
   module CLI
-    VERSION = "0.1.18"
+    VERSION = "0.1.20"
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: strata-cli
 version: !ruby/object:Gem::Version
-  version: 0.1.18
+  version: 0.1.20
 platform: ruby
 authors:
 - Ajo Abraham
@@ -16,14 +16,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.5.0
+        version: 0.6.2
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.5.0
+        version: 0.6.2
 - !ruby/object:Gem::Dependency
   name: aws-sdk-athena
   requirement: !ruby/object:Gem::Requirement
@@ -224,7 +224,7 @@ description: |
   Comprehensive CLI tool for managing Strata Semantic Analytics projects. Create and initialize
   projects, manage datasource connections, build semantic table models with AI assistance, run
   audits and validations, and deploy projects to Strata servers. Supports multiple data warehouse
-  adapters including PostgreSQL, MySQL, SQL Server, Snowflake, Athena, Trino, ClickHouse, and more.
+  adapters including PostgreSQL, MySQL, SQL Server, Snowflake, BigQuery, Athena, Trino, ClickHouse, and more.
 email:
 - ajo@strata.site
 - allen@strata.site
@@ -274,12 +274,16 @@ files:
 - lib/strata/cli/generators/table.rb
 - lib/strata/cli/generators/templates/AGENTS.md
 - lib/strata/cli/generators/templates/adapters/athena.yml
+- lib/strata/cli/generators/templates/adapters/bigquery.yml
+- lib/strata/cli/generators/templates/adapters/clickhouse.yml
 - lib/strata/cli/generators/templates/adapters/databricks.yml
 - lib/strata/cli/generators/templates/adapters/druid.yml
 - lib/strata/cli/generators/templates/adapters/duckdb.yml
 - lib/strata/cli/generators/templates/adapters/mysql.yml
 - lib/strata/cli/generators/templates/adapters/postgres.yml
+- lib/strata/cli/generators/templates/adapters/redshift.yml
 - lib/strata/cli/generators/templates/adapters/snowflake.yml
+- lib/strata/cli/generators/templates/adapters/sqlite.yml
 - lib/strata/cli/generators/templates/adapters/sqlserver.yml
 - lib/strata/cli/generators/templates/adapters/trino.yml
 - lib/strata/cli/generators/templates/datasources.yml
@@ -344,7 +348,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.6.7
+rubygems_version: 3.6.9
 specification_version: 4
 summary: Command-line interface for the Strata Semantic Analytics System
 test_files: []