analytics_ops 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +108 -0
  3. data/CONTRIBUTING.md +1 -0
  4. data/README.md +141 -13
  5. data/SECURITY.md +11 -2
  6. data/docs/ai-connections.md +192 -0
  7. data/docs/api-support-matrix.md +25 -8
  8. data/docs/architecture.md +78 -17
  9. data/docs/authentication.md +53 -3
  10. data/docs/bigquery.md +128 -0
  11. data/docs/commands.md +133 -12
  12. data/docs/configuration-schema-v1.json +608 -50
  13. data/docs/configuration.md +118 -5
  14. data/docs/google-client-compatibility.md +21 -1
  15. data/docs/governance.md +93 -0
  16. data/docs/health.md +82 -0
  17. data/docs/live-smoke-test.md +13 -10
  18. data/docs/rails.md +25 -9
  19. data/docs/reports.md +165 -3
  20. data/docs/safety.md +37 -5
  21. data/docs/troubleshooting.md +81 -7
  22. data/lib/analytics_ops/bigquery/definition.rb +173 -0
  23. data/lib/analytics_ops/bigquery.rb +4 -0
  24. data/lib/analytics_ops/cli/local_commands.rb +80 -0
  25. data/lib/analytics_ops/cli/options.rb +282 -9
  26. data/lib/analytics_ops/cli/presenter.rb +263 -28
  27. data/lib/analytics_ops/cli/presenter_csv.rb +92 -0
  28. data/lib/analytics_ops/cli/reporting_commands.rb +129 -0
  29. data/lib/analytics_ops/cli/setup_commands.rb +91 -0
  30. data/lib/analytics_ops/cli.rb +121 -44
  31. data/lib/analytics_ops/clients/admin.rb +28 -2
  32. data/lib/analytics_ops/clients/admin_governance.rb +95 -0
  33. data/lib/analytics_ops/clients/admin_governance_normalization.rb +90 -0
  34. data/lib/analytics_ops/clients/admin_resource_normalization.rb +61 -0
  35. data/lib/analytics_ops/clients/bigquery.rb +313 -0
  36. data/lib/analytics_ops/clients/data.rb +147 -54
  37. data/lib/analytics_ops/clients/data_result_normalization.rb +55 -0
  38. data/lib/analytics_ops/clients/error_translation.rb +13 -1
  39. data/lib/analytics_ops/configuration/schema.rb +175 -1
  40. data/lib/analytics_ops/configuration/validator.rb +45 -7
  41. data/lib/analytics_ops/configuration/validator_experimental.rb +136 -0
  42. data/lib/analytics_ops/configuration/validator_reporting.rb +93 -0
  43. data/lib/analytics_ops/configuration/writer.rb +112 -7
  44. data/lib/analytics_ops/desired_state.rb +28 -3
  45. data/lib/analytics_ops/errors.rb +33 -1
  46. data/lib/analytics_ops/funnels.rb +58 -0
  47. data/lib/analytics_ops/governance.rb +363 -0
  48. data/lib/analytics_ops/health.rb +310 -0
  49. data/lib/analytics_ops/mcp_server/custom_report_tool.rb +101 -0
  50. data/lib/analytics_ops/mcp_server/discovery_tools.rb +79 -0
  51. data/lib/analytics_ops/mcp_server/experimental_tools.rb +89 -0
  52. data/lib/analytics_ops/mcp_server/health_tools.rb +80 -0
  53. data/lib/analytics_ops/mcp_server/overview_tools.rb +49 -0
  54. data/lib/analytics_ops/mcp_server/standard_report_tool.rb +53 -0
  55. data/lib/analytics_ops/mcp_server.rb +402 -0
  56. data/lib/analytics_ops/portfolio.rb +216 -0
  57. data/lib/analytics_ops/rails/railtie.rb +19 -8
  58. data/lib/analytics_ops/redaction.rb +10 -6
  59. data/lib/analytics_ops/reports/csv_export.rb +91 -0
  60. data/lib/analytics_ops/reports/definition.rb +152 -14
  61. data/lib/analytics_ops/reports/metadata.rb +158 -0
  62. data/lib/analytics_ops/reports/period.rb +104 -0
  63. data/lib/analytics_ops/reports/result.rb +8 -3
  64. data/lib/analytics_ops/reports.rb +3 -0
  65. data/lib/analytics_ops/service_account.rb +321 -29
  66. data/lib/analytics_ops/setup.rb +31 -7
  67. data/lib/analytics_ops/version.rb +1 -1
  68. data/lib/analytics_ops/workspace.rb +222 -10
  69. data/lib/analytics_ops.rb +7 -2
  70. data/lib/generators/analytics_ops/templates/analytics_ops.yml +1 -1
  71. data/sig/analytics_ops.rbs +494 -9
  72. metadata +60 -1
@@ -1,6 +1,6 @@
1
1
  # API support matrix
2
2
 
3
- This matrix describes Analytics Ops 0.2.0—not every method Google exposes.
3
+ This matrix describes Analytics Ops 0.3.0—not every method Google exposes.
4
4
 
5
5
  | State | Meaning |
6
6
  | --- | --- |
@@ -10,7 +10,7 @@ This matrix describes Analytics Ops 0.2.0—not every method Google exposes.
10
10
  | Manual | Reported as an operator checklist |
11
11
  | Unsupported | Deliberately outside the public contract |
12
12
 
13
- | Capability | State | Exact 0.2.0 behavior |
13
+ | Capability | State | Exact 0.3.0 behavior |
14
14
  | --- | --- | --- |
15
15
  | Account/property discovery | Audited | Lists accessible summaries without requiring configuration |
16
16
  | Data-stream discovery | Audited | Lists web, Android, and iOS streams without requiring configuration |
@@ -22,19 +22,34 @@ This matrix describes Analytics Ops 0.2.0—not every method Google exposes.
22
22
  | Custom metrics | Managed | Reads and creates with explicit currency-data classification; updates display name/description |
23
23
  | Standard Data API reports | Audited | Immutable definitions and normalized results |
24
24
  | Realtime Data API reports | Audited | Immutable definitions and normalized results |
25
+ | Saved/custom report recipes | Audited | Strict YAML or bounded CLI definitions; property metadata and compatibility checked before execution |
26
+ | Report metadata/compatibility | Audited | Property-aware field search and Google's `checkCompatibility` result |
27
+ | Paginated CSV export | Audited | Streaming, atomic mode-0600 export with spreadsheet protection and a 1,000,000-row hard cap |
25
28
  | Batched overview | Audited | Five bounded standard reports through one `batchRunReports` call |
26
29
  | Friendly report aliases | Audited | `traffic` and `landing-pages` resolve to stable canonical recipes |
27
- | Enhanced Measurement | Experimental | Validated declaration and finding only; no mutation |
30
+ | Date overrides and comparison | Audited | Bounded complete-day or absolute ranges, with an equal preceding period |
31
+ | Analytics health | Audited | Deterministic traffic, event, acquisition, quota, and configuration-drift checks |
32
+ | Multi-property portfolio | Audited | Totals, arbitrary reports, and health across named profiles/connections; bounded concurrency and isolated failures |
33
+ | Local MCP tools | Audited | 21 read-only discovery, audit, reporting, health, governance, portfolio, BigQuery, and funnel tools |
34
+ | MCP mutation | Unsupported | No plan, apply, create, update, delete, or archive tool exists |
35
+ | Enhanced Measurement | Experimental | Read-only per-web-stream governance audit plus validated declaration; no mutation |
28
36
  | Google Signals | Experimental | Validated declaration and finding only; no mutation |
29
37
  | Consent coverage | Manual | Must be verified in tagging/consent systems |
30
38
  | Ads-personalization regions | Manual | Not inferred from similarly named API fields |
31
- | Stream data redaction | Manual | UI/API policy must be verified separately |
39
+ | Stream data redaction | Experimental | Reads email/query redaction settings and audits explicit expectations; no mutation |
40
+ | Reporting identity/attribution | Experimental | Read-only normalized snapshot; identity can be audited |
41
+ | Channel groups/calculated metrics | Experimental | Lists resources and audits expected display names; no mutation |
42
+ | Event create/edit rules | Experimental | Lists rules under each web stream and audits expected names; no mutation |
43
+ | BigQuery links | Experimental | Lists existing links and can require at least one; no create/update/delete |
44
+ | Change history | Experimental | Bounded account/property request; omits actor email and full before/after resources |
45
+ | Data Access report | Experimental | Aggregate `accessDateHour` and Data API token usage only; requires GA4 Administrator |
46
+ | GA4 BigQuery export queries | Experimental | Optional official client, fixed SELECT recipes, dry run, mandatory byte cap, bounded results |
47
+ | Ordered funnels | Experimental | Optional aggregate BigQuery sequence over 2–10 configured event steps; no user identifiers returned |
32
48
  | User-provided data setting | Unsupported | Not read or changed |
33
- | Event create/edit rules | Unsupported | Alpha, outside 0.2.0 |
34
- | Audiences and expanded datasets | Unsupported | Alpha, outside 0.2.0 |
49
+ | Audiences and expanded datasets | Unsupported | Alpha, outside 0.3.0 |
35
50
  | Google Ads, AdSense, DV360, SA360 links | Unsupported | Not read or changed |
36
- | BigQuery links | Unsupported | Not read or changed |
37
51
  | Access bindings | Unsupported | Not read or changed |
52
+ | BigQuery arbitrary SQL | Unsupported | No public Ruby, CLI, or MCP arbitrary-SQL input |
38
53
  | Measurement Protocol secrets/events | Unsupported | Never managed |
39
54
  | Account/property/stream deletion | Unsupported | No destructive path |
40
55
 
@@ -42,7 +57,9 @@ Google marks some Admin API capabilities as Alpha and warns that Alpha
42
57
  contracts may break. Analytics Ops never labels a capability managed merely
43
58
  because a generated class exists. The supported operations above have
44
59
  executable protobuf-coercion contract specs against the pinned official Ruby
45
- clients.
60
+ clients. Optional BigQuery support is labeled experimental for a different
61
+ reason: it has a separate dependency, permissions, export prerequisite, and
62
+ query cost even though every template is read-only and capped.
46
63
 
47
64
  References:
48
65
 
data/docs/architecture.md CHANGED
@@ -1,7 +1,7 @@
1
1
  # Architecture
2
2
 
3
3
  Analytics Ops is a plain-Ruby library and CLI with optional Rails hooks. It has
4
- no database and keeps Google's generated classes behind two narrow adapters.
4
+ no database and keeps Google's generated classes behind narrow adapters.
5
5
 
6
6
  ```text
7
7
  strict YAML
@@ -14,13 +14,36 @@ strict YAML
14
14
 
15
15
  immutable report definition
16
16
  → Data API adapter
17
+ → metadata + compatibility validation for custom definitions
17
18
  → immutable normalized result
19
+ → optional bounded streaming CSV
20
+
21
+ health policy
22
+ → batched current/previous reports + configuration plan
23
+ → deterministic findings
24
+
25
+ experimental governance policy
26
+ → read-only Alpha Admin resources
27
+ → normalized snapshot + findings
28
+
29
+ optional BigQuery recipe or funnel
30
+ → fixed parameterized SELECT
31
+ → dry run + mandatory byte cap
32
+ → bounded aggregate result
18
33
 
19
34
  configuration-free connection
20
35
  → explicit service-account identity
21
36
  → account/property discovery
22
37
  → API access verification
23
38
  → atomic minimal configuration
39
+
40
+ named local connections + profiles
41
+ → selected property
42
+ → single-property workspace or read-only portfolio
43
+
44
+ local MCP stdio
45
+ → read-only tool allowlist
46
+ → Workspace and generalized Portfolio
24
47
  ```
25
48
 
26
49
  ## Core objects
@@ -28,7 +51,7 @@ configuration-free connection
28
51
  | Object | Responsibility |
29
52
  | --- | --- |
30
53
  | `Configuration` | Bounded safe YAML, environment interpolation, schema validation |
31
- | `Configuration::Writer` | Non-destructive atomic creation of the smallest valid setup file |
54
+ | `Configuration::Writer` | Non-destructive atomic creation or additive profile update |
32
55
  | `DesiredState` | Immutable configuration for one profile |
33
56
  | `Resources` | Gem-owned account/property/stream/settings values |
34
57
  | `Snapshot` | Canonical managed remote state and SHA-256 fingerprint |
@@ -38,25 +61,42 @@ configuration-free connection
38
61
  | `Clients::Admin` | Official Admin request translation and response normalization |
39
62
  | `Reports::Definition` | Strict immutable Data API query |
40
63
  | `Clients::Data` | Standard/realtime request translation and result normalization |
64
+ | `Reports::Metadata` | Property-aware fields and compatibility results |
65
+ | `Reports::CSVExport` | Atomic mode-0600 streaming pagination output |
66
+ | `Health` | Deterministic traffic, event, quota, and drift findings |
67
+ | `Governance` | Experimental normalized Alpha settings and policy findings |
68
+ | `BigQuery::Definition` | Fixed optional raw-data recipe and hard result bounds |
69
+ | `Clients::BigQuery` | Optional official client, dry run, byte cap, and normalized result |
70
+ | `Funnels::Definition` | Strict aggregate ordered-event steps |
41
71
  | `ServiceAccount` | Strict JSON-key validation, scopes, and user-level path storage |
42
72
  | `Connection` | Configuration-free discovery and selected-property access verification |
43
73
  | `Setup` | Property selection, verification, and safe configuration creation |
44
74
  | `Workspace` | Public orchestration API with independently injectable clients |
75
+ | `Portfolio` | Read-only totals, reports, and health across configured profiles and connections |
76
+ | `Reports::Period` | Bounded simple dates and equal preceding-period comparisons |
77
+ | `MCPServer` | Local strictly read-only AI tools over Workspace and Portfolio |
45
78
  | `CLI` | Option validation, formats, stable statuses, and explicit confirmation |
46
79
  | `Railtie` | Optional generator and operator Rake tasks |
47
80
 
48
81
  ## Dependency direction
49
82
 
50
83
  Domain and planning code know nothing about protobufs, gRPC, Rails, Active
51
- Support, authentication flows, or HTTP transports. Admin and Data adapters translate
52
- official generated values immediately into immutable Analytics Ops values.
53
- The adapters can be injected independently, so tests use deterministic fakes.
54
-
55
- The core requires Google's wrapper gems lazily. Requiring `analytics_ops`,
84
+ Support, authentication flows, or HTTP transports. Admin, Data, and optional
85
+ BigQuery adapters translate official generated values immediately into
86
+ immutable Analytics Ops values. The adapters can be injected independently,
87
+ so tests use deterministic fakes.
88
+
89
+ The core requires Google's Analytics wrapper gems lazily. The optional
90
+ `google-cloud-bigquery` client is not a runtime dependency and is loaded only
91
+ when a configured raw report or funnel runs. Requiring `analytics_ops`,
56
92
  constructing a `Connection`, loading configuration, and booting Rails define
57
93
  or validate local objects only; they do not instantiate a generated client,
58
94
  read a service-account key, or contact Google.
59
95
 
96
+ The MCP protocol implementation uses the official Ruby MCP SDK. Starting the
97
+ server lists local tools only; credentials and Google clients remain lazy
98
+ until a tool is called.
99
+
60
100
  ## State model
61
101
 
62
102
  Google Analytics is the remote source of truth. There is no local state
@@ -65,10 +105,11 @@ relevant remote state. Canonical key ordering and normalized arrays make
65
105
  snapshot fingerprints and plan bytes deterministic.
66
106
 
67
107
  `setup` discovers properties through a configuration-free `Connection`,
68
- verifies the selected property, and creates the existing versioned YAML
69
- format. Existing conflicting configuration is never rewritten. A separate
70
- user-level `connection.json` stores only the absolute service-account key
71
- path. It is not desired state and contains no key material.
108
+ verifies the selected property, and creates or additively updates the existing
109
+ versioned YAML format. Existing conflicting profiles are never rewritten. A
110
+ separate user-level `connection.json` stores named absolute service-account
111
+ key paths and per-configuration profile selections. It is not desired state
112
+ and contains no key material.
72
113
 
73
114
  Apply accepts an already validated plan, refreshes the snapshot, compares its
74
115
  fingerprint, then sends only the saved operations. A partial response is
@@ -85,8 +126,9 @@ Expected failures become typed Analytics Ops errors:
85
126
  - quota, timeout, invalid request, and remote failure
86
127
  - partial apply with a structured result
87
128
 
88
- Messages are redacted before user-visible output. Callers do not need to parse
89
- Google exception text.
129
+ Messages are redacted before user-visible output. Structured Google reason,
130
+ metadata, and status values are preserved in bounded Analytics Ops errors so
131
+ setup does not need to recognize English wording.
90
132
 
91
133
  An injected logger receives small JSON request events containing only the
92
134
  service method and target resource. Request bodies, authorization data, and
@@ -99,9 +141,28 @@ report results are never logged automatically. CLI logging defaults to
99
141
  Engine, model, migration, route, controller, view, asset, or boot-time network
100
142
  call. Browser analytics and consent remain application concerns.
101
143
 
144
+ ## AI boundary
145
+
146
+ `analytics-ops mcp` is a local stdio server. Its tool registry contains only
147
+ property discovery, field/recipe discovery, health checks, snapshots, audits,
148
+ overviews, generalized portfolios, bounded reports, governance, optional
149
+ fixed BigQuery/funnel reads, and realtime reads. Every tool is marked
150
+ read-only and non-destructive. Plan creation, plan-file writing, apply,
151
+ create, update, delete, arbitrary SQL, and change history are absent rather
152
+ than hidden behind model instructions.
153
+
154
+ Tool outputs can contain analytics configuration and report rows. The chosen
155
+ AI product receives those requested outputs, but never the service-account
156
+ key, token, authorization header, or saved key path.
157
+
102
158
  ## Experimental boundary
103
159
 
104
- Google identifies certain Admin capabilities as Alpha. Version 0.2.0 accepts
105
- explicit Enhanced Measurement and Google Signals declarations only so they can
106
- appear as experimental findings. It does not apply them. Stable/beta
107
- operations do not depend on an experimental public Analytics Ops contract.
160
+ Google identifies several governance capabilities as Alpha. Version 0.3.0
161
+ reads them only after an explicit experimental policy is configured, or when
162
+ the operator directly runs an experimental command. They never enter an
163
+ ordinary plan or apply.
164
+
165
+ Optional BigQuery support is a separate experimental boundary. The public API
166
+ accepts only fixed recipes and configured funnel steps, never SQL. Every query
167
+ is dry-run first, is rejected above a configured byte maximum, and returns a
168
+ bounded result without user identifiers.
@@ -53,9 +53,10 @@ Analytics Ops writes this user-level file:
53
53
  ~/.config/analytics_ops/connection.json
54
54
  ```
55
55
 
56
- It contains only the absolute path to the service-account key. The connection
56
+ It contains named connections, each holding only an absolute key path, plus
57
+ the selected profile/connection for each configuration file. The connection
57
58
  file is created atomically with mode `0600`, and its directory uses mode
58
- `0700`. Analytics Ops does not copy the key or place its contents in project
59
+ `0700`. Analytics Ops does not copy a key or place its contents in project
59
60
  configuration.
60
61
 
61
62
  Keep the key in a stable location. Moving or deleting it requires setup again:
@@ -66,7 +67,37 @@ analytics-ops setup \
66
67
  ```
67
68
 
68
69
  One service account can access several GA4 accounts. Add the same
69
- service-account email at account level in each GA4 account.
70
+ service-account email at account level in each GA4 account. If different
71
+ clients provide different service accounts, save each one under a clear name:
72
+
73
+ ```bash
74
+ analytics-ops setup \
75
+ --profile client_a \
76
+ --connection client_a \
77
+ --service-account /secure/client-a/service-account.json
78
+
79
+ analytics-ops setup \
80
+ --profile client_b \
81
+ --connection client_b \
82
+ --service-account /secure/client-b/service-account.json
83
+ ```
84
+
85
+ Then switch without re-entering a key path:
86
+
87
+ ```bash
88
+ analytics-ops connections
89
+ analytics-ops profiles
90
+ analytics-ops use client_b
91
+ analytics-ops overview
92
+ ```
93
+
94
+ `connections` never prints credential paths. `profiles` shows only profile
95
+ names, property IDs, and connection names. An older version-1 single-key
96
+ connection file is read automatically and upgraded on the next successful
97
+ write. If two separate apps both use the default `production` profile with
98
+ different keys, Analytics Ops automatically chooses a collision-free
99
+ connection name instead of repointing the first app. Use `--connection NAME`
100
+ when you want a clearer client name.
70
101
 
71
102
  ## Read and edit access
72
103
 
@@ -80,6 +111,22 @@ Analytics Ops still requires a saved plan, explicit confirmation, and a fresh
80
111
  matching snapshot before any apply. Merely configuring an Editor identity
81
112
  does not cause writes.
82
113
 
114
+ Google also requires the edit scope to read configuration change history.
115
+ `analytics-ops changes` uses an edit-scoped Admin client but contains no
116
+ mutation call. Its result omits actor email and full before/after resource
117
+ bodies.
118
+
119
+ ## Optional BigQuery access
120
+
121
+ Normal Analytics commands never request a BigQuery scope. When a configured
122
+ `raw` report or funnel runs, the optional adapter requests only
123
+ `bigquery.readonly`.
124
+
125
+ OAuth scope and Cloud IAM are separate. The service account also needs narrow
126
+ permission to create query jobs in the configured project and read the GA4
127
+ export dataset. It does not need Cloud Owner or Editor. See
128
+ [BigQuery reports and funnels](bigquery.md).
129
+
83
130
  ## Automation
84
131
 
85
132
  Use the same service-account setup non-interactively:
@@ -136,6 +183,9 @@ GA4 data or settings.
136
183
 
137
184
  - Never place the JSON key or its path in `analytics_ops.yml`.
138
185
  - Never commit a service-account key.
186
+ - Protect the key with `chmod 600` on macOS and Linux.
187
+ - Keep the key outside every Git repository. Setup and `doctor` warn when
188
+ permissions or placement look unsafe.
139
189
  - Never paste key contents into chat, issues, logs, screenshots, or reports.
140
190
  - Never place production mutation credentials in a Rails web container.
141
191
  - Revoke and replace a key immediately if it may have been exposed.
data/docs/bigquery.md ADDED
@@ -0,0 +1,128 @@
1
+ # Optional BigQuery reports and funnels
2
+
3
+ You do not need BigQuery for normal Analytics Ops reports, overview, health,
4
+ audit, or AI access.
5
+
6
+ Use this optional feature only when you need GA4's exported event-level data,
7
+ such as a true ordered funnel. The output remains aggregate or deliberately
8
+ limited; Analytics Ops does not expose user identifiers.
9
+
10
+ ## What you need
11
+
12
+ 1. Link the GA4 property to BigQuery and wait for export tables named
13
+ `events_YYYYMMDD`.
14
+ 2. Enable the BigQuery API in the Google Cloud project that will run queries.
15
+ 3. Give the service account:
16
+ - permission to create query jobs in the configured billing project
17
+ - read access to the GA4 export dataset
18
+ 4. Add the optional official Ruby client to your application:
19
+
20
+ ```ruby
21
+ gem "google-cloud-bigquery", "~> 1.64", group: :development
22
+ ```
23
+
24
+ ```bash
25
+ bundle install
26
+ ```
27
+
28
+ The service account does not need Cloud Owner or Editor. Use narrow BigQuery
29
+ job-user and dataset-viewer permissions.
30
+
31
+ ## Cost-capped configuration
32
+
33
+ All identifiers here are fake:
34
+
35
+ ```yaml
36
+ profiles:
37
+ production:
38
+ property_id: "123456789"
39
+
40
+ bigquery:
41
+ experimental: true
42
+ project_id: example-analytics-project
43
+ dataset: analytics_123456789
44
+ location: US
45
+ maximum_bytes_billed: 1000000000
46
+ ```
47
+
48
+ `maximum_bytes_billed` is mandatory. The value above is one billion bytes.
49
+ Choose a limit that matches your cost policy.
50
+
51
+ ## Fixed raw-data recipes
52
+
53
+ Analytics Ops accepts names, not arbitrary SQL:
54
+
55
+ ```bash
56
+ analytics-ops raw event_counts --last 7
57
+ analytics-ops raw daily_events --last 30
58
+ analytics-ops raw page_views --last 7 --csv
59
+ analytics-ops raw event_sample --last 1 --limit 100 --json
60
+ ```
61
+
62
+ Available recipes:
63
+
64
+ | Recipe | Result |
65
+ | --- | --- |
66
+ | `event_counts` | Event names and counts |
67
+ | `daily_events` | Total events by export date |
68
+ | `page_views` | Page URLs with query strings removed and view counts |
69
+ | `event_sample` | A bounded sample of event time, name, platform, and device category |
70
+
71
+ The maximum result limit is 10,000 rows. Comparison ranges are not accepted.
72
+
73
+ ## Ordered funnels
74
+
75
+ Define a funnel with two through ten event steps:
76
+
77
+ ```yaml
78
+ profiles:
79
+ production:
80
+ property_id: "123456789"
81
+
82
+ bigquery:
83
+ experimental: true
84
+ project_id: example-analytics-project
85
+ dataset: analytics_123456789
86
+ location: US
87
+ maximum_bytes_billed: 1000000000
88
+
89
+ funnels:
90
+ checkout:
91
+ experimental: true
92
+ steps:
93
+ - name: Viewed product
94
+ event_name: view_item
95
+ - name: Began checkout
96
+ event_name: begin_checkout
97
+ - name: Purchased
98
+ event_name: purchase
99
+ ```
100
+
101
+ ```bash
102
+ analytics-ops funnel checkout --last 30
103
+ ```
104
+
105
+ The query uses `user_pseudo_id` and event timestamps only inside BigQuery to
106
+ prove step order. The returned result contains only step name, aggregate user
107
+ count, and completion rate.
108
+
109
+ ## Cost and mutation safety
110
+
111
+ Every query:
112
+
113
+ 1. uses a fixed `SELECT` template
114
+ 2. passes dates and limits as parameters
115
+ 3. performs a BigQuery dry run
116
+ 4. refuses the query if estimated bytes exceed your configured limit
117
+ 5. sends the same `maximum_bytes_billed` limit on the real query job
118
+ 6. returns a bounded, normalized result
119
+
120
+ Running a query creates an ephemeral BigQuery job and can incur query charges.
121
+ It does not update or delete the GA4 export tables. Analytics Ops has no
122
+ arbitrary-SQL public method or CLI option.
123
+
124
+ Official references:
125
+
126
+ - [GA4 BigQuery Export](https://support.google.com/analytics/answer/9358801)
127
+ - [BigQuery query jobs](https://docs.cloud.google.com/ruby/docs/reference/google-cloud-bigquery/latest/Google-Cloud-Bigquery-Project#query_job-instance_method)
128
+ - [Maximum bytes billed](https://docs.cloud.google.com/bigquery/docs/best-practices-costs#restrict-query-costs)
data/docs/commands.md CHANGED
@@ -13,16 +13,24 @@ analytics-ops COMMAND [options]
13
13
  ```bash
14
14
  analytics-ops setup --service-account /absolute/path/to/service-account.json
15
15
  analytics-ops overview
16
+ analytics-ops overview --last 7 --compare
17
+ analytics-ops portfolio
18
+ analytics-ops portfolio report traffic
19
+ analytics-ops health
16
20
  analytics-ops properties
17
21
  analytics-ops doctor
18
22
  analytics-ops audit --json
19
23
  analytics-ops plan --output tmp/ga4-plan.json
20
24
  analytics-ops apply tmp/ga4-plan.json
21
25
  analytics-ops report landing-pages --csv
26
+ analytics-ops fields event
22
27
  analytics-ops realtime --json
28
+ analytics-ops profiles
29
+ analytics-ops mcp --config /absolute/path/to/config/analytics_ops.yml
23
30
  ```
24
31
 
25
- Only `apply` can write to Google.
32
+ Only `apply` changes GA4 configuration. Optional `raw` and `funnel` commands
33
+ create read-only BigQuery query jobs and can incur a bounded query cost.
26
34
 
27
35
  ## Setup
28
36
 
@@ -39,13 +47,15 @@ pass the downloaded JSON key explicitly. Setup then:
39
47
  4. Proves Admin and Data API read access.
40
48
  5. Creates the smallest valid `config/analytics_ops.yml` using the
41
49
  `production` profile.
42
- 6. Stores only the key's absolute path in
50
+ 6. Stores a named connection containing only the key's absolute path in
43
51
  `~/.config/analytics_ops/connection.json` with mode `0600`.
44
52
  7. Prints `analytics-ops overview` as the next command.
45
53
 
46
54
  Setup never overwrites an existing profile that targets another property. A
47
- matching file is a successful no-op. The key is never copied into a project or
48
- printed.
55
+ matching file is a successful no-op. A missing profile is appended safely
56
+ without removing existing settings or comments. The untouched Rails generator
57
+ placeholder is filled automatically. The key is never copied into a project
58
+ or printed.
49
59
 
50
60
  Later setup runs use the remembered path:
51
61
 
@@ -66,6 +76,26 @@ analytics-ops setup \
66
76
  The CLI never consults `gcloud`, browser login, Application Default
67
77
  Credentials, `GOOGLE_APPLICATION_CREDENTIALS`, or API keys.
68
78
 
79
+ ## Profiles and connections
80
+
81
+ Use a profile for each GA4 property and a named connection for each Google
82
+ service account:
83
+
84
+ ```bash
85
+ analytics-ops setup \
86
+ --profile client_b \
87
+ --connection client_b \
88
+ --service-account /secure/client-b/service-account.json
89
+
90
+ analytics-ops profiles
91
+ analytics-ops connections
92
+ analytics-ops use client_b
93
+ ```
94
+
95
+ `use` remembers the selected profile and its connection for this configuration
96
+ file. Later commands use it automatically. Add `--profile NAME` or
97
+ `--connection NAME` for a one-command override.
98
+
69
99
  ## Commands that do not change GA4
70
100
 
71
101
  | Command | Result |
@@ -75,17 +105,38 @@ Credentials, `GOOGLE_APPLICATION_CREDENTIALS`, or API keys.
75
105
  | `doctor` | Checks the local file, credentials, Admin API, Data API, property access, client versions, edit visibility, and clock |
76
106
  | `discover` | Lists accessible account IDs, property IDs, and stream IDs without configuration |
77
107
  | `overview` | Returns five bounded reports for the previous 28 complete days in one batch request |
108
+ | `portfolio` | Returns users, sessions, and key events across every configured property |
109
+ | `portfolio report NAME` | Runs one built-in or saved report across all profiles with isolated failures |
110
+ | `portfolio health` | Runs deterministic health checks across all profiles with isolated failures |
111
+ | `health` | Checks traffic, expected events, attribution quality, quota, and configuration drift |
78
112
  | `snapshot` | Prints normalized managed remote state and its fingerprint |
79
113
  | `audit` | Compares desired state with a fresh snapshot; exits 2 for drift |
80
114
  | `plan` | Generates the same comparison; `--output FILE` saves its exact JSON |
81
115
  | `verify` | Replans and exits 0 only when managed state converges |
82
- | `report NAME` | Runs one built-in standard Data API report |
116
+ | `report NAME` | Runs one built-in or saved standard Data API report |
117
+ | `report custom` | Builds a property-validated one-off report from CLI field options |
118
+ | `reports` | Lists built-in and saved report recipes |
119
+ | `fields [SEARCH]` | Searches property-aware Data API dimensions and metrics |
120
+ | `compatibility NAME` | Checks whether one report's fields work together |
83
121
  | `realtime [NAME]` | Runs `realtime_events` by default |
122
+ | `governance` | Audits broader read-only GA4 Admin settings; experimental |
123
+ | `changes` | Reads bounded, de-identified change history; experimental and requires edit scope |
124
+ | `access` | Reads aggregate Data API quota usage without user-email/IP fields; experimental |
125
+ | `raw NAME` | Runs one fixed, dry-run-checked BigQuery recipe; experimental and optional |
126
+ | `funnel NAME` | Runs one configured aggregate ordered funnel in BigQuery; experimental and optional |
127
+ | `profiles` | Lists local profiles, property IDs, and connection names without contacting Google |
128
+ | `connections` | Lists connection names and key availability without printing paths |
129
+ | `use NAME` | Selects one local profile/connection without contacting Google |
130
+ | `mcp` | Starts the strictly read-only local AI tool server |
84
131
  | `schema` | Prints the version-1 configuration schema |
85
132
 
86
133
  `doctor` uses the read-only Analytics scope and proves effective access with
87
134
  small real calls to both APIs.
88
135
 
136
+ Google requires `analytics.edit` to read change history. The `changes`
137
+ implementation uses that scope but exposes no mutation method and removes
138
+ actor email and full before/after resources from its result.
139
+
89
140
  ## Apply
90
141
 
91
142
  ```bash
@@ -110,16 +161,76 @@ analytics-ops apply tmp/ga4-plan.json --non-interactive --yes --format json
110
161
  `--non-interactive` without `--yes` is an error. There is no command that
111
162
  plans and applies in one step.
112
163
 
164
+ JSON apply is deliberately non-interactive. It requires both flags so a prompt
165
+ can never corrupt the JSON stream:
166
+
167
+ ```bash
168
+ analytics-ops apply tmp/ga4-plan.json --json --non-interactive --yes
169
+ ```
170
+
171
+ ## Report dates and comparisons
172
+
173
+ The default remains the previous 28 complete days. Override it with one easy
174
+ form:
175
+
176
+ ```bash
177
+ analytics-ops report traffic --last 7
178
+ analytics-ops report traffic --from 2026-07-01 --to 2026-07-07
179
+ analytics-ops overview --last 30 --compare
180
+ analytics-ops portfolio --last 30 --compare
181
+ ```
182
+
183
+ `--compare` adds an equally long preceding period. Google returns a
184
+ `dateRange` column labeled `current` or `previous`. Date options work with
185
+ `report`, `overview`, `portfolio`, `health`, `raw`, and `funnel`. Health
186
+ always adds the preceding comparison period. Raw and funnel
187
+ reports accept only one period. Realtime reports have no date range. `--last`
188
+ accepts 1 through 1,825 complete days.
189
+
190
+ ## Custom reports and exports
191
+
192
+ ```bash
193
+ analytics-ops report custom \
194
+ --dimension eventName \
195
+ --metric eventCount \
196
+ --where eventName=purchase \
197
+ --limit 100
198
+
199
+ analytics-ops report traffic \
200
+ --all \
201
+ --page-size 100000 \
202
+ --max-rows 500000 \
203
+ --output tmp/traffic.csv
204
+ ```
205
+
206
+ `--where` is a repeatable exact dimension filter. Saved YAML recipes support
207
+ the complete bounded filter model. `--all` is a streaming CSV export, requires
208
+ an output path, and never exceeds `--max-rows`.
209
+
113
210
  ## Options
114
211
 
115
212
  | Option | Meaning |
116
213
  | --- | --- |
117
214
  | `-c, --config PATH` | Configuration file |
118
215
  | `-p, --profile NAME` | Profile inside the file |
119
- | `-f, --format FORMAT` | `human`, `json`, or report-only `csv` |
216
+ | `--connection NAME` | Named saved Google connection |
217
+ | `-f, --format FORMAT` | `human`, `json`, or result-only `csv` |
120
218
  | `--json` | Shortcut for `--format json` |
121
219
  | `--csv` | Shortcut for `--format csv` |
122
- | `-o, --output PATH` | Save generated JSON; valid only with `plan` |
220
+ | `--last DAYS` | Previous complete days for date-aware read commands |
221
+ | `--from DATE` | Inclusive `YYYY-MM-DD` start; requires `--to` |
222
+ | `--to DATE` | Inclusive `YYYY-MM-DD` end; requires `--from` |
223
+ | `--compare` | Include the equally long preceding period |
224
+ | `-o, --output PATH` | Save a plan, or the CSV created by `report --all` |
225
+ | `--dimension NAME` | Repeatable dimension for `report custom` |
226
+ | `--metric NAME` | Repeatable metric for `report custom`; at least one required |
227
+ | `--where FIELD=VALUE` | Repeatable exact dimension filter for `report custom` |
228
+ | `--limit ROWS` | Row limit for `report custom` or `raw` |
229
+ | `--all` | Page and stream a report to an explicit CSV output path |
230
+ | `--max-rows ROWS` | Hard export limit; maximum 1,000,000 |
231
+ | `--page-size ROWS` | Export rows per Data API request; maximum 250,000 |
232
+ | `--kind dimension\|metric` | Restrict `fields` results |
233
+ | `--concurrency COUNT` | Portfolio report/health workers; 1 through 8 |
123
234
  | `--property ID` | Select an accessible property without prompting; setup only |
124
235
  | `--service-account PATH` | Connect a Google service-account JSON key; setup only |
125
236
  | `--transport grpc|rest` | Official Google-client transport |
@@ -129,21 +240,31 @@ plans and applies in one step.
129
240
  | `--non-interactive` | Never prompt; apply requires `--yes`, setup requires `--property` |
130
241
 
131
242
  Unknown arguments, conflicting format flags, and non-finite timeouts fail
132
- instead of being ignored. CSV is rejected for anything except report results.
133
- CSV cells whose first meaningful character is a spreadsheet formula marker
134
- are prefixed safely, including cells hidden behind whitespace or controls.
243
+ instead of being ignored. CSV is rejected for anything except report,
244
+ portfolio-report, BigQuery, or funnel results. CSV cells whose first meaningful
245
+ character is a spreadsheet formula marker are prefixed safely, including cells
246
+ hidden behind whitespace or controls.
135
247
 
136
248
  ## Output
137
249
 
138
250
  - Human output removes terminal control characters from remote text.
251
+ - Long human table cells keep both their beginning and ending, with an
252
+ ellipsis in the middle so similar URLs remain distinguishable.
253
+ - Interactive apply prints complete redacted before/after values without the
254
+ ordinary message-length cap.
139
255
  - JSON output uses stable gem-owned fields and structured errors.
140
- - CSV contains one report's headers and rows only; it is rejected for batched
141
- overviews, plans, snapshots, and errors.
256
+ - CSV contains only result headers and rows. Portfolio CSV adds profile,
257
+ property, and label columns, and refuses partial portfolio results. CSV is
258
+ rejected for batched overviews, plans, snapshots, and errors.
142
259
 
143
260
  No command prints credentials or generated Google protobuf objects. Report
144
261
  rows are emitted only because the user requested a report; they are never
145
262
  logged automatically.
146
263
 
264
+ The MCP command writes protocol messages to standard output and rejects human,
265
+ JSON, or CSV formatting flags. Its advertised tools are all annotated
266
+ read-only and include no plan or apply operation.
267
+
147
268
  ## Exit statuses
148
269
 
149
270
  | Status | Meaning |