analytics_ops 0.2.0 → 0.3.0

data/docs/reports.md

@@ -38,9 +38,137 @@ analytics-ops report calculator_completions --csv
 analytics-ops realtime
 ```
 
-CSV is accepted only for `report` and `realtime`. The CSV renderer protects
-headers and cells whose first meaningful character is `=`, `+`, `-`, or `@`,
-including formula text hidden behind whitespace or control characters.
+List every built-in and saved recipe:
+
+```bash
+analytics-ops reports
+```
+
+Change dates without writing Ruby:
+
+```bash
+analytics-ops report traffic --last 7
+analytics-ops report traffic --from 2026-07-01 --to 2026-07-07
+analytics-ops report traffic --last 30 --compare
+```
+
+`--last` ends at yesterday so it uses complete days. `--from` and `--to` are
+inclusive and must be used together. `--compare` adds the equally long period
+immediately before the requested range. Comparison rows contain Google's
+automatic `dateRange` dimension with `current` or `previous`.
+
+CSV is accepted for individual reports, realtime, portfolio reports, and
+optional BigQuery/funnel results. The CSV renderer protects headers and cells
+whose first meaningful character is `=`, `+`, `-`, or `@`, including formula
+text hidden behind whitespace or control characters.
+
+## Find fields and build a custom report
+
+Google's available fields can differ by property. Search the live metadata
+catalog instead of guessing:
+
+```bash
+analytics-ops fields purchase
+analytics-ops fields event --kind dimension
+analytics-ops fields revenue --kind metric
+```
+
+Then run a small one-off report:
+
+```bash
+analytics-ops report custom \
+  --dimension eventName \
+  --metric eventCount \
+  --where eventName=purchase \
+  --last 7 \
+  --limit 100
+```
+
+Repeat `--dimension`, `--metric`, or `--where` when needed. CLI filters are
+exact dimension matches and use the easy `FIELD=VALUE` form. Analytics Ops
+checks live metadata and Google's compatibility endpoint before running a
+custom or saved standard report. Saved realtime definitions are checked
+against live field metadata; Google then performs the realtime-specific field
+pairing validation because its compatibility endpoint covers standard
+reports.
+
+Check a saved recipe without running it:
+
+```bash
+analytics-ops compatibility purchases_by_country
+```
+
+Metadata is cached only for the current process. Analytics Ops does not create
+a local field database.
+
+## Save reusable recipes
+
+Store a recipe under the property profile:
+
+```yaml
+profiles:
+  production:
+    property_id: "123456789"
+
+    reports:
+      purchases_by_country:
+        dimensions:
+          - country
+          - eventName
+        metrics:
+          - eventCount
+          - totalUsers
+        dimension_filter:
+          and:
+            - field: eventName
+              match_type: exact
+              value: purchase
+            - field: country
+              in_list: [US, CA]
+        metric_filter:
+          field: eventCount
+          operation: greater_than
+          value: 0
+        order_bys:
+          - metric: eventCount
+            desc: true
+        limit: 500
+```
+
+Run it like a built-in:
+
+```bash
+analytics-ops report purchases_by_country
+```
+
+Filters can use bounded `and`, `or`, and `not` groups. Dimension leaves support
+string matching, `in_list`, and null checks. Metric leaves support numeric
+comparisons, inclusive `between`, and null checks. Definitions are limited in
+depth, child count, field count, and row count.
+
+## Export more rows safely
+
+One normal report request is intentionally bounded. For a larger CSV, make the
+limit explicit:
+
+```bash
+analytics-ops report traffic \
+  --all \
+  --page-size 100000 \
+  --max-rows 500000 \
+  --output tmp/traffic.csv
+```
+
+The export:
+
+- pages through the Data API up to the hard maximum
+- writes as it receives each page instead of keeping all rows in memory
+- keeps headers consistent across pages
+- protects spreadsheet formula cells
+- replaces the destination atomically with mode `0600`
+
+The absolute product maximum is 1,000,000 exported rows. Use BigQuery for
+larger event-level analysis.
 
 ## Overview
 
@@ -65,6 +193,36 @@ The realtime recipe intentionally requests only `eventCount` with
 `eventName`. Google does not allow `activeUsers` with that dimension. Realtime
 is a short-lived event check rather than a complete user report.
 
+## Multiple-property portfolio
+
+```bash
+analytics-ops portfolio
+analytics-ops portfolio --last 7 --compare
+```
+
+Portfolio runs one small dimensionless totals report for each configured
+profile and shows active users, sessions, and key events together. Each
+profile uses its own saved named connection. This is read-only, but each
+property still consumes normal Google Data API quota.
+
+Run any built-in or profile-saved report across all properties:
+
+```bash
+analytics-ops portfolio report traffic
+analytics-ops portfolio report traffic --csv --concurrency 4
+```
+
+One property failing does not discard successful results from the others.
+Concurrency is bounded from 1 through 8 and output order stays stable.
+Profile `labels` are included in JSON, human output, and CSV. Human output
+shows each successful property's table. JSON preserves both successful
+results and isolated errors. CSV is produced only when every property
+succeeds, so an automation cannot mistake a partial portfolio for a complete
+one.
+
+For deterministic multi-property checks, see
+[Analytics health](health.md).
+
 ## Ruby
 
 ```ruby
@@ -81,6 +239,10 @@ puts result.metadata
 overview = workspace.overview
 puts overview.report("overview_totals").rows
 puts overview.property_quota
+
+ranges = AnalyticsOps::Reports::Period.resolve(last_days: 7, compare: true)
+comparison = workspace.report("traffic", date_ranges: ranges)
+puts comparison.rows
 ```
 
 Custom immutable definition:

data/docs/safety.md

@@ -5,12 +5,19 @@ Analytics Ops is read-only unless you explicitly apply a saved plan.
 ## What cannot mutate
 
 `properties`, `doctor`, `discover`, `snapshot`, `audit`, `plan`, `verify`,
-`overview`, `report`, and `realtime` only read Google Analytics APIs. Setup
+`overview`, `portfolio`, `health`, `report`, `realtime`, `fields`,
+`compatibility`, `governance`, `changes`, `access`, and every `mcp` tool only
+read Google Analytics APIs. Setup
 validates a service-account key, verifies access, remembers only its path, and
 creates local configuration; it never changes Analytics. Loading the gem,
 constructing a connection, parsing YAML, and booting Rails make no network
 request.
 
+Optional `raw` and `funnel` commands submit read-only BigQuery `SELECT` query
+jobs. They cannot change the GA4 export tables, but query jobs can incur cost.
+Every query is fixed, parameterized, dry-run first, and limited by a mandatory
+`maximum_bytes_billed` value.
+
 ## What apply requires
 
 An apply needs all of these:
@@ -89,11 +96,17 @@ The public schema is [plan-schema-v1.json](plan-schema-v1.json).
 ## Credential and data boundaries
 
 - Credentials never belong in YAML, plans, fixtures, logs, or command output.
-- `~/.config/analytics_ops/connection.json` contains only a key path, is
-  atomic, and uses mode `0600`; the key is never copied.
+- `~/.config/analytics_ops/connection.json` contains only named key paths and
+  local profile selections, is atomic, and uses mode `0600`; keys are never
+  copied.
+- Setup and doctor warn when a key is broadly readable or stored inside a Git
+  repository.
 - Ambient Application Default Credentials and API keys are never used.
-- Read-only commands request only `analytics.readonly`; guarded apply also
-  requests `analytics.edit`.
+- Normal read-only commands request only `analytics.readonly`. Google requires
+  `analytics.edit` to read change history, so `changes` uses that scope without
+  exposing any mutation operation. Guarded apply also requests edit scope.
+- Optional BigQuery reports request only `bigquery.readonly` and still require
+  narrow Cloud IAM permissions to create a query job and read the dataset.
 - Report rows are never logged automatically.
 - The gem has no telemetry and stores no report results.
 - The gem does not inject browser analytics or manage consent.
@@ -103,3 +116,22 @@ The public schema is [plan-schema-v1.json](plan-schema-v1.json).
 Use a Viewer service account for reports-only installations. If one service
 account has the Editor role, Analytics Ops still requests edit scope only
 inside the reviewed apply path.
+
+## AI safety
+
+The MCP server is secure by capability, not merely by instructions:
+
+- every advertised tool declares `readOnlyHint: true` and
+  `destructiveHint: false`
+- no plan, apply, create, update, delete, archive, or credential tool exists
+- starting the MCP process does not load credentials or contact Google
+- tool inputs use strict schemas and bounded report/date choices
+- custom reports are checked against property metadata and compatibility
+- optional BigQuery tools accept fixed names, not arbitrary SQL, and keep the
+  configured byte cap
+- expected errors are typed and redacted; unexpected errors are generic
+
+An AI provider receives the analytics rows or configuration returned by tools
+you let it call. Connect only an AI account and workspace whose data policy is
+appropriate for that information. The credential file and saved credential
+path are never returned.

data/docs/troubleshooting.md

@@ -56,13 +56,21 @@ Check that:
 - the Analytics Admin and Data APIs are enabled in its Cloud project
 - the service-account email is added in GA4 Access Management
 
-The saved pointer is `~/.config/analytics_ops/connection.json`. It contains
-only the key path. Rerunning setup with `--service-account` safely replaces
-that pointer after API verification succeeds.
+The saved connection file is `~/.config/analytics_ops/connection.json`. It
+contains only named key paths and local profile choices. Rerunning setup with
+`--service-account` updates the selected named connection only after API
+verification succeeds.
 
 Analytics Ops does not fall back to `gcloud`, browser login, Application
 Default Credentials, `GOOGLE_APPLICATION_CREDENTIALS`, or API keys.
 
+If several connections exist and a profile is not associated yet, choose one:
+
+```bash
+analytics-ops connections
+analytics-ops use client_b --connection client_b
+```
+
 ## Permission failure (exit 77)
 
 The service-account identity needs access inside the GA4 property. Cloud IAM
@@ -91,7 +99,8 @@ snapshot and replan so remote state is reconciled first.
 ## Drift status 2
 
 This is expected automation behavior, not a crash. `audit` and `verify`
-return 2 when managed state differs or a drift finding remains.
+return 2 when managed state differs or a drift finding remains. `health` and
+`governance` also return 2 when they find a configured issue.
 
 ## Stale plan (exit 79)
 
@@ -107,6 +116,13 @@ generate a new plan. Do not rerun the old plan.
 
 ## A custom report fails
 
+Find the exact property-aware names and check compatibility:
+
+```bash
+analytics-ops fields event
+analytics-ops compatibility REPORT_NAME
+```
+
 Confirm every requested dimension and metric is compatible and registered.
 Event-scoped custom fields use names such as
 `customEvent:calculator_slug`. Standard reports require a date range;
@@ -116,15 +132,73 @@ Google thresholding, sampling, consent coverage, cardinality, and processing
 delay can explain an empty or lower-than-expected result. Inspect
 `result.metadata` in JSON output.
 
-## Rails task cannot find a profile
+Date shortcuts are strict:
+
+- use `--last DAYS`, or use `--from YYYY-MM-DD --to YYYY-MM-DD`
+- do not combine the two forms
+- use them only with date-aware read commands
+- `--compare` uses the immediately preceding period of the same length
+
+## Governance or access is unavailable
+
+Governance uses Alpha Admin methods. Confirm the installed client with
+`analytics-ops doctor` and remember that Google can change Alpha behavior.
+
+The aggregate `access` command requires the service-account identity to be a
+GA4 Administrator. Normal Viewer/Editor reporting still works when this
+special command is denied. The `changes` command is read-only but Google
+requires the Analytics edit scope for it.
+
+## A BigQuery report or funnel fails
+
+Normal Analytics reports do not need BigQuery. For the optional feature,
+confirm:
 
-Rails tasks default to `Rails.env`. The generator creates a
-`development` profile. Add the intended profile or override it:
+- `google-cloud-bigquery ~> 1.64` is in the application's bundle
+- the GA4 BigQuery export has created `events_YYYYMMDD` tables
+- the configured project, dataset, and location are correct
+- the service account can create query jobs and read the dataset
+- the BigQuery API is enabled
+- `maximum_bytes_billed` is high enough for the selected period
+
+A quota error before execution can mean the dry-run estimate exceeded your
+configured cap. Shorten the date range or deliberately review the cap; do not
+remove it. See [BigQuery reports and funnels](bigquery.md).
+
+## Rails setup or task cannot find a profile
+
+The generator and CLI both default to `production`. It is safe to run the
+generator before setup: setup fills its untouched property placeholder. It is
+also safe to run setup first: the generator preserves that file.
+
+Rails tasks use the locally selected profile. Inspect or change it:
+
+```bash
+bundle exec analytics-ops profiles
+bundle exec analytics-ops use production
+```
+
+For one explicit task override:
 
 ```bash
 ANALYTICS_OPS_PROFILE=production bin/rake analytics:doctor
 ```
 
+## ChatGPT or Claude cannot see Analytics Ops
+
+First prove the local command is healthy:
+
+```bash
+analytics-ops doctor
+analytics-ops reports
+analytics-ops mcp --config /absolute/path/to/config/analytics_ops.yml
+```
+
+The second command waits silently for MCP protocol input; press Ctrl-C when
+testing it directly. Use an absolute config path and, for Bundler applications,
+the absolute generated `bin/analytics-ops` path. Restart the AI client after
+adding the server. See [ChatGPT, Codex, and Claude](ai-connections.md).
+
 ## Report a problem safely
 
 Include the gem version, Ruby version, command name, exit status, and a minimal

data/lib/analytics_ops/bigquery/definition.rb

@@ -0,0 +1,173 @@
+# frozen_string_literal: true
+
+require "date"
+
+module AnalyticsOps
+  # Optional, read-only GA4 BigQuery export support.
+  module BigQuery
+    RECIPE_NAMES = %w[event_counts daily_events page_views event_sample].freeze
+
+    # Immutable, bounded request for one fixed BigQuery report recipe.
+    class Definition
+      attr_reader :name, :start_suffix, :end_suffix, :limit
+
+      def initialize(name:, start_date:, end_date:, limit: 1_000)
+        @name = validate_name(name)
+        @start_suffix = suffix(start_date, "start_date")
+        @end_suffix = suffix(end_date, "end_date")
+        raise InvalidRequestError, "BigQuery start_date must not be after end_date" if @start_suffix > @end_suffix
+
+        @limit = bounded_integer(limit, "limit", 1, 10_000)
+        freeze
+      end
+
+      def self.from_date_ranges(name, date_ranges, limit: 1_000, today: Date.today)
+        unless date_ranges.is_a?(Array) && date_ranges.length == 1
+          raise InvalidRequestError, "BigQuery reports require exactly one date range"
+        end
+
+        range = date_ranges.first
+        new(
+          name:,
+          start_date: resolve_date(range.fetch("start_date"), today:),
+          end_date: resolve_date(range.fetch("end_date"), today:),
+          limit:
+        )
+      rescue KeyError
+        raise InvalidRequestError, "BigQuery report date range is invalid"
+      end
+
+      def to_h
+        { "name" => name, "start_suffix" => start_suffix, "end_suffix" => end_suffix, "limit" => limit }
+      end
+
+      def self.resolve_date(value, today:)
+        return today if value == "today"
+        return today - 1 if value == "yesterday"
+
+        relative = /\A(\d{1,4})daysAgo\z/.match(value.to_s)
+        return today - relative[1].to_i if relative
+
+        Date.iso8601(value)
+      rescue Date::Error
+        raise InvalidRequestError, "BigQuery report date is invalid"
+      end
+      private_class_method :resolve_date
+
+      private
+
+      def validate_name(value)
+        name = value.to_s
+        return name.freeze if RECIPE_NAMES.include?(name)
+
+        raise InvalidRequestError, "Unknown BigQuery report #{value.inspect}; available: #{RECIPE_NAMES.join(", ")}"
+      end
+
+      def suffix(value, label)
+        date = value.is_a?(Date) ? value : Date.iso8601(value.to_s)
+        date.strftime("%Y%m%d").freeze
+      rescue Date::Error
+        raise InvalidRequestError, "BigQuery #{label} is invalid"
+      end
+
+      def bounded_integer(value, label, minimum, maximum)
+        return value if value.is_a?(Integer) && value.between?(minimum, maximum)
+
+        raise InvalidRequestError, "BigQuery #{label} must be between #{minimum} and #{maximum}"
+      end
+    end
+
+    # Immutable, bounded result returned by the optional BigQuery adapter.
+    class Result
+      NAME = /\A[A-Za-z][A-Za-z0-9_]{0,71}\z/
+      HEADER = /\A[A-Za-z_][A-Za-z0-9_]{0,127}\z/
+      MAX_ROWS = 10_000
+      MAX_CELL_BYTES = 1_048_576
+
+      attr_reader :name, :headers, :rows, :bytes_processed, :maximum_bytes_billed
+
+      def initialize(name:, headers:, rows:, bytes_processed:, maximum_bytes_billed:)
+        validate_name!(name)
+        validate_headers!(headers)
+        normalized_rows = validate_rows(rows, headers)
+        validate_byte_counts!(bytes_processed, maximum_bytes_billed)
+
+        @name = name.dup.freeze
+        @headers = Canonical.immutable(headers)
+        @rows = Canonical.immutable(normalized_rows)
+        @bytes_processed = bytes_processed
+        @maximum_bytes_billed = maximum_bytes_billed
+        freeze
+      end
+
+      def to_h
+        {
+          "experimental" => true,
+          "name" => name,
+          "headers" => headers,
+          "rows" => rows,
+          "row_count" => rows.length,
+          "bytes_processed" => bytes_processed,
+          "maximum_bytes_billed" => maximum_bytes_billed
+        }
+      end
+
+      private
+
+      def validate_name!(value)
+        return if value.is_a?(String) && NAME.match?(value)
+
+        raise RemoteError, "BigQuery result name is invalid"
+      end
+
+      def validate_headers!(values)
+        valid = values.is_a?(Array) &&
+                values.length <= 64 &&
+                values.all? { |header| header.is_a?(String) && HEADER.match?(header) } &&
+                values.uniq.length == values.length
+        raise RemoteError, "BigQuery result headers are invalid" unless valid
+      end
+
+      def validate_rows(values, headers)
+        raise RemoteError, "BigQuery result rows are invalid" unless values.is_a?(Array) && values.length <= MAX_ROWS
+
+        values.map { |row| validate_row(row, headers) }
+      end
+
+      def validate_byte_counts!(processed, maximum)
+        valid = processed.is_a?(Integer) &&
+                !processed.negative? &&
+                maximum.is_a?(Integer) &&
+                maximum.positive? &&
+                processed <= maximum
+        raise RemoteError, "BigQuery result byte counts are invalid" unless valid
+      end
+
+      def validate_row(row, headers)
+        unless row.is_a?(Hash) && row.keys.all?(String) && row.keys.sort == headers.sort
+          raise RemoteError, "BigQuery result row does not match its headers"
+        end
+
+        row.to_h do |header, value|
+          [header, validate_cell(value)]
+        end
+      end
+
+      def validate_cell(value)
+        case value
+        when String
+          unless value.valid_encoding? && value.bytesize <= MAX_CELL_BYTES
+            raise RemoteError, "BigQuery result contains an invalid string"
+          end
+        when Float
+          raise RemoteError, "BigQuery result contains a non-finite number" unless value.finite?
+        when Integer, TrueClass, FalseClass, NilClass
+          nil
+        else
+          raise RemoteError, "BigQuery result contains an unsupported value"
+        end
+        value
+      end
+    end
+  end
+end

data/lib/analytics_ops/bigquery.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+require_relative "bigquery/definition"
+require_relative "clients/bigquery"

data/lib/analytics_ops/cli/local_commands.rb

@@ -0,0 +1,80 @@
+# frozen_string_literal: true
+
+module AnalyticsOps
+  # Command-line consumer of the public Analytics Ops API.
+  class CLI
+    # Implements local setup, selection, portfolio, and AI commands.
+    module LocalCommands
+      private
+
+      def dispatch_local(command)
+        case command
+        when "connections"
+          presenter.render_connections(service_account_store.summaries)
+        when "profiles"
+          render_profiles
+        when "use"
+          use_profile
+        when "mcp"
+          start_mcp
+        end
+      end
+
+      def render_profiles
+        config = @options.fetch(:config)
+        document = Configuration.load(config)
+        selected = service_account_store.selection(config:)
+        selected_profile = selected&.fetch("profile") || @options.fetch(:profile)
+        profiles = document.profiles.sort.map do |name, desired_state|
+          {
+            "name" => name,
+            "property_id" => desired_state.property_id,
+            "connection" => service_account_store.profile_connection(config:, profile: name),
+            "selected" => selected_profile == name
+          }
+        end
+        presenter.render_profiles(profiles)
+      end
+
+      def use_profile
+        profile = required_use_profile!
+        config = @options.fetch(:config)
+        Configuration.load(config).profile(profile)
+        selection = service_account_store.select(
+          config:,
+          profile:,
+          connection: @options[:connection]
+        )
+        presenter.render_selection(selection)
+      end
+
+      def start_mcp
+        @mcp_server_loader.call(
+          config: @options.fetch(:config),
+          profile: resolved_profile,
+          connection: @options[:connection],
+          store: service_account_store,
+          workspace_loader: @workspace_loader,
+          connection_loader: @connection_loader,
+          service_account_loader: @service_account_loader,
+          portfolio_loader: @portfolio_loader,
+          transport: @options.fetch(:transport),
+          timeout: @options[:timeout],
+          logger: operation_logger
+        ).start
+        SUCCESS
+      end
+
+      def required_use_profile!
+        raise OptionParser::MissingArgument, "use requires exactly one PROFILE" unless @arguments.length == 1
+
+        profile = @arguments.first
+        raise OptionParser::InvalidArgument, "use requires a valid profile name" unless Options::PROFILE.match?(profile)
+
+        profile
+      end
+    end
+
+    include LocalCommands
+  end
+end