analytics_ops 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +108 -0
- data/CONTRIBUTING.md +1 -0
- data/README.md +141 -13
- data/SECURITY.md +11 -2
- data/docs/ai-connections.md +192 -0
- data/docs/api-support-matrix.md +25 -8
- data/docs/architecture.md +78 -17
- data/docs/authentication.md +53 -3
- data/docs/bigquery.md +128 -0
- data/docs/commands.md +133 -12
- data/docs/configuration-schema-v1.json +608 -50
- data/docs/configuration.md +118 -5
- data/docs/google-client-compatibility.md +21 -1
- data/docs/governance.md +93 -0
- data/docs/health.md +82 -0
- data/docs/live-smoke-test.md +13 -10
- data/docs/rails.md +25 -9
- data/docs/reports.md +165 -3
- data/docs/safety.md +37 -5
- data/docs/troubleshooting.md +81 -7
- data/lib/analytics_ops/bigquery/definition.rb +173 -0
- data/lib/analytics_ops/bigquery.rb +4 -0
- data/lib/analytics_ops/cli/local_commands.rb +80 -0
- data/lib/analytics_ops/cli/options.rb +282 -9
- data/lib/analytics_ops/cli/presenter.rb +263 -28
- data/lib/analytics_ops/cli/presenter_csv.rb +92 -0
- data/lib/analytics_ops/cli/reporting_commands.rb +129 -0
- data/lib/analytics_ops/cli/setup_commands.rb +91 -0
- data/lib/analytics_ops/cli.rb +121 -44
- data/lib/analytics_ops/clients/admin.rb +28 -2
- data/lib/analytics_ops/clients/admin_governance.rb +95 -0
- data/lib/analytics_ops/clients/admin_governance_normalization.rb +90 -0
- data/lib/analytics_ops/clients/admin_resource_normalization.rb +61 -0
- data/lib/analytics_ops/clients/bigquery.rb +313 -0
- data/lib/analytics_ops/clients/data.rb +147 -54
- data/lib/analytics_ops/clients/data_result_normalization.rb +55 -0
- data/lib/analytics_ops/clients/error_translation.rb +13 -1
- data/lib/analytics_ops/configuration/schema.rb +175 -1
- data/lib/analytics_ops/configuration/validator.rb +45 -7
- data/lib/analytics_ops/configuration/validator_experimental.rb +136 -0
- data/lib/analytics_ops/configuration/validator_reporting.rb +93 -0
- data/lib/analytics_ops/configuration/writer.rb +112 -7
- data/lib/analytics_ops/desired_state.rb +28 -3
- data/lib/analytics_ops/errors.rb +33 -1
- data/lib/analytics_ops/funnels.rb +58 -0
- data/lib/analytics_ops/governance.rb +363 -0
- data/lib/analytics_ops/health.rb +310 -0
- data/lib/analytics_ops/mcp_server/custom_report_tool.rb +101 -0
- data/lib/analytics_ops/mcp_server/discovery_tools.rb +79 -0
- data/lib/analytics_ops/mcp_server/experimental_tools.rb +89 -0
- data/lib/analytics_ops/mcp_server/health_tools.rb +80 -0
- data/lib/analytics_ops/mcp_server/overview_tools.rb +49 -0
- data/lib/analytics_ops/mcp_server/standard_report_tool.rb +53 -0
- data/lib/analytics_ops/mcp_server.rb +402 -0
- data/lib/analytics_ops/portfolio.rb +216 -0
- data/lib/analytics_ops/rails/railtie.rb +19 -8
- data/lib/analytics_ops/redaction.rb +10 -6
- data/lib/analytics_ops/reports/csv_export.rb +91 -0
- data/lib/analytics_ops/reports/definition.rb +152 -14
- data/lib/analytics_ops/reports/metadata.rb +158 -0
- data/lib/analytics_ops/reports/period.rb +104 -0
- data/lib/analytics_ops/reports/result.rb +8 -3
- data/lib/analytics_ops/reports.rb +3 -0
- data/lib/analytics_ops/service_account.rb +321 -29
- data/lib/analytics_ops/setup.rb +31 -7
- data/lib/analytics_ops/version.rb +1 -1
- data/lib/analytics_ops/workspace.rb +222 -10
- data/lib/analytics_ops.rb +7 -2
- data/lib/generators/analytics_ops/templates/analytics_ops.yml +1 -1
- data/sig/analytics_ops.rbs +494 -9
- metadata +60 -1
data/docs/api-support-matrix.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# API support matrix
|
|
2
2
|
|
|
3
|
-
This matrix describes Analytics Ops 0.
|
|
3
|
+
This matrix describes Analytics Ops 0.3.0—not every method Google exposes.
|
|
4
4
|
|
|
5
5
|
| State | Meaning |
|
|
6
6
|
| --- | --- |
|
|
@@ -10,7 +10,7 @@ This matrix describes Analytics Ops 0.2.0—not every method Google exposes.
|
|
|
10
10
|
| Manual | Reported as an operator checklist |
|
|
11
11
|
| Unsupported | Deliberately outside the public contract |
|
|
12
12
|
|
|
13
|
-
| Capability | State | Exact 0.
|
|
13
|
+
| Capability | State | Exact 0.3.0 behavior |
|
|
14
14
|
| --- | --- | --- |
|
|
15
15
|
| Account/property discovery | Audited | Lists accessible summaries without requiring configuration |
|
|
16
16
|
| Data-stream discovery | Audited | Lists web, Android, and iOS streams without requiring configuration |
|
|
@@ -22,19 +22,34 @@ This matrix describes Analytics Ops 0.2.0—not every method Google exposes.
|
|
|
22
22
|
| Custom metrics | Managed | Reads and creates with explicit currency-data classification; updates display name/description |
|
|
23
23
|
| Standard Data API reports | Audited | Immutable definitions and normalized results |
|
|
24
24
|
| Realtime Data API reports | Audited | Immutable definitions and normalized results |
|
|
25
|
+
| Saved/custom report recipes | Audited | Strict YAML or bounded CLI definitions; property metadata and compatibility checked before execution |
|
|
26
|
+
| Report metadata/compatibility | Audited | Property-aware field search and Google's `checkCompatibility` result |
|
|
27
|
+
| Paginated CSV export | Audited | Streaming, atomic mode-0600 export with spreadsheet protection and a 1,000,000-row hard cap |
|
|
25
28
|
| Batched overview | Audited | Five bounded standard reports through one `batchRunReports` call |
|
|
26
29
|
| Friendly report aliases | Audited | `traffic` and `landing-pages` resolve to stable canonical recipes |
|
|
27
|
-
|
|
|
30
|
+
| Date overrides and comparison | Audited | Bounded complete-day or absolute ranges, with an equal preceding period |
|
|
31
|
+
| Analytics health | Audited | Deterministic traffic, event, acquisition, quota, and configuration-drift checks |
|
|
32
|
+
| Multi-property portfolio | Audited | Totals, arbitrary reports, and health across named profiles/connections; bounded concurrency and isolated failures |
|
|
33
|
+
| Local MCP tools | Audited | 21 read-only discovery, audit, reporting, health, governance, portfolio, BigQuery, and funnel tools |
|
|
34
|
+
| MCP mutation | Unsupported | No plan, apply, create, update, delete, or archive tool exists |
|
|
35
|
+
| Enhanced Measurement | Experimental | Read-only per-web-stream governance audit plus validated declaration; no mutation |
|
|
28
36
|
| Google Signals | Experimental | Validated declaration and finding only; no mutation |
|
|
29
37
|
| Consent coverage | Manual | Must be verified in tagging/consent systems |
|
|
30
38
|
| Ads-personalization regions | Manual | Not inferred from similarly named API fields |
|
|
31
|
-
| Stream data redaction |
|
|
39
|
+
| Stream data redaction | Experimental | Reads email/query redaction settings and audits explicit expectations; no mutation |
|
|
40
|
+
| Reporting identity/attribution | Experimental | Read-only normalized snapshot; identity can be audited |
|
|
41
|
+
| Channel groups/calculated metrics | Experimental | Lists resources and audits expected display names; no mutation |
|
|
42
|
+
| Event create/edit rules | Experimental | Lists rules under each web stream and audits expected names; no mutation |
|
|
43
|
+
| BigQuery links | Experimental | Lists existing links and can require at least one; no create/update/delete |
|
|
44
|
+
| Change history | Experimental | Bounded account/property request; omits actor email and full before/after resources |
|
|
45
|
+
| Data Access report | Experimental | Aggregate `accessDateHour` and Data API token usage only; requires GA4 Administrator |
|
|
46
|
+
| GA4 BigQuery export queries | Experimental | Optional official client, fixed SELECT recipes, dry run, mandatory byte cap, bounded results |
|
|
47
|
+
| Ordered funnels | Experimental | Optional aggregate BigQuery sequence over 2–10 configured event steps; no user identifiers returned |
|
|
32
48
|
| User-provided data setting | Unsupported | Not read or changed |
|
|
33
|
-
|
|
|
34
|
-
| Audiences and expanded datasets | Unsupported | Alpha, outside 0.2.0 |
|
|
49
|
+
| Audiences and expanded datasets | Unsupported | Alpha, outside 0.3.0 |
|
|
35
50
|
| Google Ads, AdSense, DV360, SA360 links | Unsupported | Not read or changed |
|
|
36
|
-
| BigQuery links | Unsupported | Not read or changed |
|
|
37
51
|
| Access bindings | Unsupported | Not read or changed |
|
|
52
|
+
| BigQuery arbitrary SQL | Unsupported | No public Ruby, CLI, or MCP arbitrary-SQL input |
|
|
38
53
|
| Measurement Protocol secrets/events | Unsupported | Never managed |
|
|
39
54
|
| Account/property/stream deletion | Unsupported | No destructive path |
|
|
40
55
|
|
|
@@ -42,7 +57,9 @@ Google marks some Admin API capabilities as Alpha and warns that Alpha
|
|
|
42
57
|
contracts may break. Analytics Ops never labels a capability managed merely
|
|
43
58
|
because a generated class exists. The supported operations above have
|
|
44
59
|
executable protobuf-coercion contract specs against the pinned official Ruby
|
|
45
|
-
clients.
|
|
60
|
+
clients. Optional BigQuery support is labeled experimental for a different
|
|
61
|
+
reason: it has a separate dependency, permissions, export prerequisite, and
|
|
62
|
+
query cost even though every template is read-only and capped.
|
|
46
63
|
|
|
47
64
|
References:
|
|
48
65
|
|
data/docs/architecture.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Architecture
|
|
2
2
|
|
|
3
3
|
Analytics Ops is a plain-Ruby library and CLI with optional Rails hooks. It has
|
|
4
|
-
no database and keeps Google's generated classes behind
|
|
4
|
+
no database and keeps Google's generated classes behind narrow adapters.
|
|
5
5
|
|
|
6
6
|
```text
|
|
7
7
|
strict YAML
|
|
@@ -14,13 +14,36 @@ strict YAML
|
|
|
14
14
|
|
|
15
15
|
immutable report definition
|
|
16
16
|
→ Data API adapter
|
|
17
|
+
→ metadata + compatibility validation for custom definitions
|
|
17
18
|
→ immutable normalized result
|
|
19
|
+
→ optional bounded streaming CSV
|
|
20
|
+
|
|
21
|
+
health policy
|
|
22
|
+
→ batched current/previous reports + configuration plan
|
|
23
|
+
→ deterministic findings
|
|
24
|
+
|
|
25
|
+
experimental governance policy
|
|
26
|
+
→ read-only Alpha Admin resources
|
|
27
|
+
→ normalized snapshot + findings
|
|
28
|
+
|
|
29
|
+
optional BigQuery recipe or funnel
|
|
30
|
+
→ fixed parameterized SELECT
|
|
31
|
+
→ dry run + mandatory byte cap
|
|
32
|
+
→ bounded aggregate result
|
|
18
33
|
|
|
19
34
|
configuration-free connection
|
|
20
35
|
→ explicit service-account identity
|
|
21
36
|
→ account/property discovery
|
|
22
37
|
→ API access verification
|
|
23
38
|
→ atomic minimal configuration
|
|
39
|
+
|
|
40
|
+
named local connections + profiles
|
|
41
|
+
→ selected property
|
|
42
|
+
→ single-property workspace or read-only portfolio
|
|
43
|
+
|
|
44
|
+
local MCP stdio
|
|
45
|
+
→ read-only tool allowlist
|
|
46
|
+
→ Workspace and generalized Portfolio
|
|
24
47
|
```
|
|
25
48
|
|
|
26
49
|
## Core objects
|
|
@@ -28,7 +51,7 @@ configuration-free connection
|
|
|
28
51
|
| Object | Responsibility |
|
|
29
52
|
| --- | --- |
|
|
30
53
|
| `Configuration` | Bounded safe YAML, environment interpolation, schema validation |
|
|
31
|
-
| `Configuration::Writer` | Non-destructive atomic creation
|
|
54
|
+
| `Configuration::Writer` | Non-destructive atomic creation or additive profile update |
|
|
32
55
|
| `DesiredState` | Immutable configuration for one profile |
|
|
33
56
|
| `Resources` | Gem-owned account/property/stream/settings values |
|
|
34
57
|
| `Snapshot` | Canonical managed remote state and SHA-256 fingerprint |
|
|
@@ -38,25 +61,42 @@ configuration-free connection
|
|
|
38
61
|
| `Clients::Admin` | Official Admin request translation and response normalization |
|
|
39
62
|
| `Reports::Definition` | Strict immutable Data API query |
|
|
40
63
|
| `Clients::Data` | Standard/realtime request translation and result normalization |
|
|
64
|
+
| `Reports::Metadata` | Property-aware fields and compatibility results |
|
|
65
|
+
| `Reports::CSVExport` | Atomic mode-0600 streaming pagination output |
|
|
66
|
+
| `Health` | Deterministic traffic, event, quota, and drift findings |
|
|
67
|
+
| `Governance` | Experimental normalized Alpha settings and policy findings |
|
|
68
|
+
| `BigQuery::Definition` | Fixed optional raw-data recipe and hard result bounds |
|
|
69
|
+
| `Clients::BigQuery` | Optional official client, dry run, byte cap, and normalized result |
|
|
70
|
+
| `Funnels::Definition` | Strict aggregate ordered-event steps |
|
|
41
71
|
| `ServiceAccount` | Strict JSON-key validation, scopes, and user-level path storage |
|
|
42
72
|
| `Connection` | Configuration-free discovery and selected-property access verification |
|
|
43
73
|
| `Setup` | Property selection, verification, and safe configuration creation |
|
|
44
74
|
| `Workspace` | Public orchestration API with independently injectable clients |
|
|
75
|
+
| `Portfolio` | Read-only totals, reports, and health across configured profiles and connections |
|
|
76
|
+
| `Reports::Period` | Bounded simple dates and equal preceding-period comparisons |
|
|
77
|
+
| `MCPServer` | Local strictly read-only AI tools over Workspace and Portfolio |
|
|
45
78
|
| `CLI` | Option validation, formats, stable statuses, and explicit confirmation |
|
|
46
79
|
| `Railtie` | Optional generator and operator Rake tasks |
|
|
47
80
|
|
|
48
81
|
## Dependency direction
|
|
49
82
|
|
|
50
83
|
Domain and planning code know nothing about protobufs, gRPC, Rails, Active
|
|
51
|
-
Support, authentication flows, or HTTP transports. Admin
|
|
52
|
-
official generated values immediately into
|
|
53
|
-
The adapters can be injected independently,
|
|
54
|
-
|
|
55
|
-
|
|
84
|
+
Support, authentication flows, or HTTP transports. Admin, Data, and optional
|
|
85
|
+
BigQuery adapters translate official generated values immediately into
|
|
86
|
+
immutable Analytics Ops values. The adapters can be injected independently,
|
|
87
|
+
so tests use deterministic fakes.
|
|
88
|
+
|
|
89
|
+
The core requires Google's Analytics wrapper gems lazily. The optional
|
|
90
|
+
`google-cloud-bigquery` client is not a runtime dependency and is loaded only
|
|
91
|
+
when a configured raw report or funnel runs. Requiring `analytics_ops`,
|
|
56
92
|
constructing a `Connection`, loading configuration, and booting Rails define
|
|
57
93
|
or validate local objects only; they do not instantiate a generated client,
|
|
58
94
|
read a service-account key, or contact Google.
|
|
59
95
|
|
|
96
|
+
The MCP protocol implementation uses the official Ruby MCP SDK. Starting the
|
|
97
|
+
server lists local tools only; credentials and Google clients remain lazy
|
|
98
|
+
until a tool is called.
|
|
99
|
+
|
|
60
100
|
## State model
|
|
61
101
|
|
|
62
102
|
Google Analytics is the remote source of truth. There is no local state
|
|
@@ -65,10 +105,11 @@ relevant remote state. Canonical key ordering and normalized arrays make
|
|
|
65
105
|
snapshot fingerprints and plan bytes deterministic.
|
|
66
106
|
|
|
67
107
|
`setup` discovers properties through a configuration-free `Connection`,
|
|
68
|
-
verifies the selected property, and creates the existing
|
|
69
|
-
format. Existing conflicting
|
|
70
|
-
user-level `connection.json` stores
|
|
71
|
-
|
|
108
|
+
verifies the selected property, and creates or additively updates the existing
|
|
109
|
+
versioned YAML format. Existing conflicting profiles are never rewritten. A
|
|
110
|
+
separate user-level `connection.json` stores named absolute service-account
|
|
111
|
+
key paths and per-configuration profile selections. It is not desired state
|
|
112
|
+
and contains no key material.
|
|
72
113
|
|
|
73
114
|
Apply accepts an already validated plan, refreshes the snapshot, compares its
|
|
74
115
|
fingerprint, then sends only the saved operations. A partial response is
|
|
@@ -85,8 +126,9 @@ Expected failures become typed Analytics Ops errors:
|
|
|
85
126
|
- quota, timeout, invalid request, and remote failure
|
|
86
127
|
- partial apply with a structured result
|
|
87
128
|
|
|
88
|
-
Messages are redacted before user-visible output.
|
|
89
|
-
|
|
129
|
+
Messages are redacted before user-visible output. Structured Google reason,
|
|
130
|
+
metadata, and status values are preserved in bounded Analytics Ops errors so
|
|
131
|
+
setup does not need to recognize English wording.
|
|
90
132
|
|
|
91
133
|
An injected logger receives small JSON request events containing only the
|
|
92
134
|
service method and target resource. Request bodies, authorization data, and
|
|
@@ -99,9 +141,28 @@ report results are never logged automatically. CLI logging defaults to
|
|
|
99
141
|
Engine, model, migration, route, controller, view, asset, or boot-time network
|
|
100
142
|
call. Browser analytics and consent remain application concerns.
|
|
101
143
|
|
|
144
|
+
## AI boundary
|
|
145
|
+
|
|
146
|
+
`analytics-ops mcp` is a local stdio server. Its tool registry contains only
|
|
147
|
+
property discovery, field/recipe discovery, health checks, snapshots, audits,
|
|
148
|
+
overviews, generalized portfolios, bounded reports, governance, optional
|
|
149
|
+
fixed BigQuery/funnel reads, and realtime reads. Every tool is marked
|
|
150
|
+
read-only and non-destructive. Plan creation, plan-file writing, apply,
|
|
151
|
+
create, update, delete, arbitrary SQL, and change history are absent rather
|
|
152
|
+
than hidden behind model instructions.
|
|
153
|
+
|
|
154
|
+
Tool outputs can contain analytics configuration and report rows. The chosen
|
|
155
|
+
AI product receives those requested outputs, but never the service-account
|
|
156
|
+
key, token, authorization header, or saved key path.
|
|
157
|
+
|
|
102
158
|
## Experimental boundary
|
|
103
159
|
|
|
104
|
-
Google identifies
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
160
|
+
Google identifies several governance capabilities as Alpha. Version 0.3.0
|
|
161
|
+
reads them only after an explicit experimental policy is configured, or when
|
|
162
|
+
the operator directly runs an experimental command. They never enter an
|
|
163
|
+
ordinary plan or apply.
|
|
164
|
+
|
|
165
|
+
Optional BigQuery support is a separate experimental boundary. The public API
|
|
166
|
+
accepts only fixed recipes and configured funnel steps, never SQL. Every query
|
|
167
|
+
is dry-run first, is rejected above a configured byte maximum, and returns a
|
|
168
|
+
bounded result without user identifiers.
|
data/docs/authentication.md
CHANGED
|
@@ -53,9 +53,10 @@ Analytics Ops writes this user-level file:
|
|
|
53
53
|
~/.config/analytics_ops/connection.json
|
|
54
54
|
```
|
|
55
55
|
|
|
56
|
-
It contains
|
|
56
|
+
It contains named connections, each holding only an absolute key path, plus
|
|
57
|
+
the selected profile/connection for each configuration file. The connection
|
|
57
58
|
file is created atomically with mode `0600`, and its directory uses mode
|
|
58
|
-
`0700`. Analytics Ops does not copy
|
|
59
|
+
`0700`. Analytics Ops does not copy a key or place its contents in project
|
|
59
60
|
configuration.
|
|
60
61
|
|
|
61
62
|
Keep the key in a stable location. Moving or deleting it requires setup again:
|
|
@@ -66,7 +67,37 @@ analytics-ops setup \
|
|
|
66
67
|
```
|
|
67
68
|
|
|
68
69
|
One service account can access several GA4 accounts. Add the same
|
|
69
|
-
service-account email at account level in each GA4 account.
|
|
70
|
+
service-account email at account level in each GA4 account. If different
|
|
71
|
+
clients provide different service accounts, save each one under a clear name:
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
analytics-ops setup \
|
|
75
|
+
--profile client_a \
|
|
76
|
+
--connection client_a \
|
|
77
|
+
--service-account /secure/client-a/service-account.json
|
|
78
|
+
|
|
79
|
+
analytics-ops setup \
|
|
80
|
+
--profile client_b \
|
|
81
|
+
--connection client_b \
|
|
82
|
+
--service-account /secure/client-b/service-account.json
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Then switch without re-entering a key path:
|
|
86
|
+
|
|
87
|
+
```bash
|
|
88
|
+
analytics-ops connections
|
|
89
|
+
analytics-ops profiles
|
|
90
|
+
analytics-ops use client_b
|
|
91
|
+
analytics-ops overview
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
`connections` never prints credential paths. `profiles` shows only profile
|
|
95
|
+
names, property IDs, and connection names. An older version-1 single-key
|
|
96
|
+
connection file is read automatically and upgraded on the next successful
|
|
97
|
+
write. If two separate apps both use the default `production` profile with
|
|
98
|
+
different keys, Analytics Ops automatically chooses a collision-free
|
|
99
|
+
connection name instead of repointing the first app. Use `--connection NAME`
|
|
100
|
+
when you want a clearer client name.
|
|
70
101
|
|
|
71
102
|
## Read and edit access
|
|
72
103
|
|
|
@@ -80,6 +111,22 @@ Analytics Ops still requires a saved plan, explicit confirmation, and a fresh
|
|
|
80
111
|
matching snapshot before any apply. Merely configuring an Editor identity
|
|
81
112
|
does not cause writes.
|
|
82
113
|
|
|
114
|
+
Google also requires the edit scope to read configuration change history.
|
|
115
|
+
`analytics-ops changes` uses an edit-scoped Admin client but contains no
|
|
116
|
+
mutation call. Its result omits actor email and full before/after resource
|
|
117
|
+
bodies.
|
|
118
|
+
|
|
119
|
+
## Optional BigQuery access
|
|
120
|
+
|
|
121
|
+
Normal Analytics commands never request a BigQuery scope. When a configured
|
|
122
|
+
`raw` report or funnel runs, the optional adapter requests only
|
|
123
|
+
`bigquery.readonly`.
|
|
124
|
+
|
|
125
|
+
OAuth scope and Cloud IAM are separate. The service account also needs narrow
|
|
126
|
+
permission to create query jobs in the configured project and read the GA4
|
|
127
|
+
export dataset. It does not need Cloud Owner or Editor. See
|
|
128
|
+
[BigQuery reports and funnels](bigquery.md).
|
|
129
|
+
|
|
83
130
|
## Automation
|
|
84
131
|
|
|
85
132
|
Use the same service-account setup non-interactively:
|
|
@@ -136,6 +183,9 @@ GA4 data or settings.
|
|
|
136
183
|
|
|
137
184
|
- Never place the JSON key or its path in `analytics_ops.yml`.
|
|
138
185
|
- Never commit a service-account key.
|
|
186
|
+
- Protect the key with `chmod 600` on macOS and Linux.
|
|
187
|
+
- Keep the key outside every Git repository. Setup and `doctor` warn when
|
|
188
|
+
permissions or placement look unsafe.
|
|
139
189
|
- Never paste key contents into chat, issues, logs, screenshots, or reports.
|
|
140
190
|
- Never place production mutation credentials in a Rails web container.
|
|
141
191
|
- Revoke and replace a key immediately if it may have been exposed.
|
data/docs/bigquery.md
ADDED
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
# Optional BigQuery reports and funnels
|
|
2
|
+
|
|
3
|
+
You do not need BigQuery for normal Analytics Ops reports, overview, health,
|
|
4
|
+
audit, or AI access.
|
|
5
|
+
|
|
6
|
+
Use this optional feature only when you need GA4's exported event-level data,
|
|
7
|
+
such as a true ordered funnel. The output remains aggregate or deliberately
|
|
8
|
+
limited; Analytics Ops does not expose user identifiers.
|
|
9
|
+
|
|
10
|
+
## What you need
|
|
11
|
+
|
|
12
|
+
1. Link the GA4 property to BigQuery and wait for export tables named
|
|
13
|
+
`events_YYYYMMDD`.
|
|
14
|
+
2. Enable the BigQuery API in the Google Cloud project that will run queries.
|
|
15
|
+
3. Give the service account:
|
|
16
|
+
- permission to create query jobs in the configured billing project
|
|
17
|
+
- read access to the GA4 export dataset
|
|
18
|
+
4. Add the optional official Ruby client to your application:
|
|
19
|
+
|
|
20
|
+
```ruby
|
|
21
|
+
gem "google-cloud-bigquery", "~> 1.64", group: :development
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
bundle install
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
The service account does not need Cloud Owner or Editor. Use narrow BigQuery
|
|
29
|
+
job-user and dataset-viewer permissions.
|
|
30
|
+
|
|
31
|
+
## Cost-capped configuration
|
|
32
|
+
|
|
33
|
+
All identifiers here are fake:
|
|
34
|
+
|
|
35
|
+
```yaml
|
|
36
|
+
profiles:
|
|
37
|
+
production:
|
|
38
|
+
property_id: "123456789"
|
|
39
|
+
|
|
40
|
+
bigquery:
|
|
41
|
+
experimental: true
|
|
42
|
+
project_id: example-analytics-project
|
|
43
|
+
dataset: analytics_123456789
|
|
44
|
+
location: US
|
|
45
|
+
maximum_bytes_billed: 1000000000
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
`maximum_bytes_billed` is mandatory. The value above is one billion bytes.
|
|
49
|
+
Choose a limit that matches your cost policy.
|
|
50
|
+
|
|
51
|
+
## Fixed raw-data recipes
|
|
52
|
+
|
|
53
|
+
Analytics Ops accepts names, not arbitrary SQL:
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
analytics-ops raw event_counts --last 7
|
|
57
|
+
analytics-ops raw daily_events --last 30
|
|
58
|
+
analytics-ops raw page_views --last 7 --csv
|
|
59
|
+
analytics-ops raw event_sample --last 1 --limit 100 --json
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Available recipes:
|
|
63
|
+
|
|
64
|
+
| Recipe | Result |
|
|
65
|
+
| --- | --- |
|
|
66
|
+
| `event_counts` | Event names and counts |
|
|
67
|
+
| `daily_events` | Total events by export date |
|
|
68
|
+
| `page_views` | Page URLs with query strings removed and view counts |
|
|
69
|
+
| `event_sample` | A bounded sample of event time, name, platform, and device category |
|
|
70
|
+
|
|
71
|
+
The maximum result limit is 10,000 rows. Comparison ranges are not accepted.
|
|
72
|
+
|
|
73
|
+
## Ordered funnels
|
|
74
|
+
|
|
75
|
+
Define a funnel with two through ten event steps:
|
|
76
|
+
|
|
77
|
+
```yaml
|
|
78
|
+
profiles:
|
|
79
|
+
production:
|
|
80
|
+
property_id: "123456789"
|
|
81
|
+
|
|
82
|
+
bigquery:
|
|
83
|
+
experimental: true
|
|
84
|
+
project_id: example-analytics-project
|
|
85
|
+
dataset: analytics_123456789
|
|
86
|
+
location: US
|
|
87
|
+
maximum_bytes_billed: 1000000000
|
|
88
|
+
|
|
89
|
+
funnels:
|
|
90
|
+
checkout:
|
|
91
|
+
experimental: true
|
|
92
|
+
steps:
|
|
93
|
+
- name: Viewed product
|
|
94
|
+
event_name: view_item
|
|
95
|
+
- name: Began checkout
|
|
96
|
+
event_name: begin_checkout
|
|
97
|
+
- name: Purchased
|
|
98
|
+
event_name: purchase
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
analytics-ops funnel checkout --last 30
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
The query uses `user_pseudo_id` and event timestamps only inside BigQuery to
|
|
106
|
+
prove step order. The returned result contains only step name, aggregate user
|
|
107
|
+
count, and completion rate.
|
|
108
|
+
|
|
109
|
+
## Cost and mutation safety
|
|
110
|
+
|
|
111
|
+
Every query:
|
|
112
|
+
|
|
113
|
+
1. uses a fixed `SELECT` template
|
|
114
|
+
2. passes dates and limits as parameters
|
|
115
|
+
3. performs a BigQuery dry run
|
|
116
|
+
4. refuses the query if estimated bytes exceed your configured limit
|
|
117
|
+
5. sends the same `maximum_bytes_billed` limit on the real query job
|
|
118
|
+
6. returns a bounded, normalized result
|
|
119
|
+
|
|
120
|
+
Running a query creates an ephemeral BigQuery job and can incur query charges.
|
|
121
|
+
It does not update or delete the GA4 export tables. Analytics Ops has no
|
|
122
|
+
arbitrary-SQL public method or CLI option.
|
|
123
|
+
|
|
124
|
+
Official references:
|
|
125
|
+
|
|
126
|
+
- [GA4 BigQuery Export](https://support.google.com/analytics/answer/9358801)
|
|
127
|
+
- [BigQuery query jobs](https://docs.cloud.google.com/ruby/docs/reference/google-cloud-bigquery/latest/Google-Cloud-Bigquery-Project#query_job-instance_method)
|
|
128
|
+
- [Maximum bytes billed](https://docs.cloud.google.com/bigquery/docs/best-practices-costs#restrict-query-costs)
|
data/docs/commands.md
CHANGED
|
@@ -13,16 +13,24 @@ analytics-ops COMMAND [options]
|
|
|
13
13
|
```bash
|
|
14
14
|
analytics-ops setup --service-account /absolute/path/to/service-account.json
|
|
15
15
|
analytics-ops overview
|
|
16
|
+
analytics-ops overview --last 7 --compare
|
|
17
|
+
analytics-ops portfolio
|
|
18
|
+
analytics-ops portfolio report traffic
|
|
19
|
+
analytics-ops health
|
|
16
20
|
analytics-ops properties
|
|
17
21
|
analytics-ops doctor
|
|
18
22
|
analytics-ops audit --json
|
|
19
23
|
analytics-ops plan --output tmp/ga4-plan.json
|
|
20
24
|
analytics-ops apply tmp/ga4-plan.json
|
|
21
25
|
analytics-ops report landing-pages --csv
|
|
26
|
+
analytics-ops fields event
|
|
22
27
|
analytics-ops realtime --json
|
|
28
|
+
analytics-ops profiles
|
|
29
|
+
analytics-ops mcp --config /absolute/path/to/config/analytics_ops.yml
|
|
23
30
|
```
|
|
24
31
|
|
|
25
|
-
Only `apply`
|
|
32
|
+
Only `apply` changes GA4 configuration. Optional `raw` and `funnel` commands
|
|
33
|
+
create read-only BigQuery query jobs and can incur a bounded query cost.
|
|
26
34
|
|
|
27
35
|
## Setup
|
|
28
36
|
|
|
@@ -39,13 +47,15 @@ pass the downloaded JSON key explicitly. Setup then:
|
|
|
39
47
|
4. Proves Admin and Data API read access.
|
|
40
48
|
5. Creates the smallest valid `config/analytics_ops.yml` using the
|
|
41
49
|
`production` profile.
|
|
42
|
-
6. Stores only the key's absolute path in
|
|
50
|
+
6. Stores a named connection containing only the key's absolute path in
|
|
43
51
|
`~/.config/analytics_ops/connection.json` with mode `0600`.
|
|
44
52
|
7. Prints `analytics-ops overview` as the next command.
|
|
45
53
|
|
|
46
54
|
Setup never overwrites an existing profile that targets another property. A
|
|
47
|
-
matching file is a successful no-op.
|
|
48
|
-
|
|
55
|
+
matching file is a successful no-op. A missing profile is appended safely
|
|
56
|
+
without removing existing settings or comments. The untouched Rails generator
|
|
57
|
+
placeholder is filled automatically. The key is never copied into a project
|
|
58
|
+
or printed.
|
|
49
59
|
|
|
50
60
|
Later setup runs use the remembered path:
|
|
51
61
|
|
|
@@ -66,6 +76,26 @@ analytics-ops setup \
|
|
|
66
76
|
The CLI never consults `gcloud`, browser login, Application Default
|
|
67
77
|
Credentials, `GOOGLE_APPLICATION_CREDENTIALS`, or API keys.
|
|
68
78
|
|
|
79
|
+
## Profiles and connections
|
|
80
|
+
|
|
81
|
+
Use a profile for each GA4 property and a named connection for each Google
|
|
82
|
+
service account:
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
analytics-ops setup \
|
|
86
|
+
--profile client_b \
|
|
87
|
+
--connection client_b \
|
|
88
|
+
--service-account /secure/client-b/service-account.json
|
|
89
|
+
|
|
90
|
+
analytics-ops profiles
|
|
91
|
+
analytics-ops connections
|
|
92
|
+
analytics-ops use client_b
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
`use` remembers the selected profile and its connection for this configuration
|
|
96
|
+
file. Later commands use it automatically. Add `--profile NAME` or
|
|
97
|
+
`--connection NAME` for a one-command override.
|
|
98
|
+
|
|
69
99
|
## Commands that do not change GA4
|
|
70
100
|
|
|
71
101
|
| Command | Result |
|
|
@@ -75,17 +105,38 @@ Credentials, `GOOGLE_APPLICATION_CREDENTIALS`, or API keys.
|
|
|
75
105
|
| `doctor` | Checks the local file, credentials, Admin API, Data API, property access, client versions, edit visibility, and clock |
|
|
76
106
|
| `discover` | Lists accessible account IDs, property IDs, and stream IDs without configuration |
|
|
77
107
|
| `overview` | Returns five bounded reports for the previous 28 complete days in one batch request |
|
|
108
|
+
| `portfolio` | Returns users, sessions, and key events across every configured property |
|
|
109
|
+
| `portfolio report NAME` | Runs one built-in or saved report across all profiles with isolated failures |
|
|
110
|
+
| `portfolio health` | Runs deterministic health checks across all profiles with isolated failures |
|
|
111
|
+
| `health` | Checks traffic, expected events, attribution quality, quota, and configuration drift |
|
|
78
112
|
| `snapshot` | Prints normalized managed remote state and its fingerprint |
|
|
79
113
|
| `audit` | Compares desired state with a fresh snapshot; exits 2 for drift |
|
|
80
114
|
| `plan` | Generates the same comparison; `--output FILE` saves its exact JSON |
|
|
81
115
|
| `verify` | Replans and exits 0 only when managed state converges |
|
|
82
|
-
| `report NAME` | Runs one built-in standard Data API report |
|
|
116
|
+
| `report NAME` | Runs one built-in or saved standard Data API report |
|
|
117
|
+
| `report custom` | Builds a property-validated one-off report from CLI field options |
|
|
118
|
+
| `reports` | Lists built-in and saved report recipes |
|
|
119
|
+
| `fields [SEARCH]` | Searches property-aware Data API dimensions and metrics |
|
|
120
|
+
| `compatibility NAME` | Checks whether one report's fields work together |
|
|
83
121
|
| `realtime [NAME]` | Runs `realtime_events` by default |
|
|
122
|
+
| `governance` | Audits broader read-only GA4 Admin settings; experimental |
|
|
123
|
+
| `changes` | Reads bounded, de-identified change history; experimental and requires edit scope |
|
|
124
|
+
| `access` | Reads aggregate Data API quota usage without user-email/IP fields; experimental |
|
|
125
|
+
| `raw NAME` | Runs one fixed, dry-run-checked BigQuery recipe; experimental and optional |
|
|
126
|
+
| `funnel NAME` | Runs one configured aggregate ordered funnel in BigQuery; experimental and optional |
|
|
127
|
+
| `profiles` | Lists local profiles, property IDs, and connection names without contacting Google |
|
|
128
|
+
| `connections` | Lists connection names and key availability without printing paths |
|
|
129
|
+
| `use NAME` | Selects one local profile/connection without contacting Google |
|
|
130
|
+
| `mcp` | Starts the strictly read-only local AI tool server |
|
|
84
131
|
| `schema` | Prints the version-1 configuration schema |
|
|
85
132
|
|
|
86
133
|
`doctor` uses the read-only Analytics scope and proves effective access with
|
|
87
134
|
small real calls to both APIs.
|
|
88
135
|
|
|
136
|
+
Google requires `analytics.edit` to read change history. The `changes`
|
|
137
|
+
implementation uses that scope but exposes no mutation method and removes
|
|
138
|
+
actor email and full before/after resources from its result.
|
|
139
|
+
|
|
89
140
|
## Apply
|
|
90
141
|
|
|
91
142
|
```bash
|
|
@@ -110,16 +161,76 @@ analytics-ops apply tmp/ga4-plan.json --non-interactive --yes --format json
|
|
|
110
161
|
`--non-interactive` without `--yes` is an error. There is no command that
|
|
111
162
|
plans and applies in one step.
|
|
112
163
|
|
|
164
|
+
JSON apply is deliberately non-interactive. It requires both flags so a prompt
|
|
165
|
+
can never corrupt the JSON stream:
|
|
166
|
+
|
|
167
|
+
```bash
|
|
168
|
+
analytics-ops apply tmp/ga4-plan.json --json --non-interactive --yes
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
## Report dates and comparisons
|
|
172
|
+
|
|
173
|
+
The default remains the previous 28 complete days. Override it with one easy
|
|
174
|
+
form:
|
|
175
|
+
|
|
176
|
+
```bash
|
|
177
|
+
analytics-ops report traffic --last 7
|
|
178
|
+
analytics-ops report traffic --from 2026-07-01 --to 2026-07-07
|
|
179
|
+
analytics-ops overview --last 30 --compare
|
|
180
|
+
analytics-ops portfolio --last 30 --compare
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
`--compare` adds an equally long preceding period. Google returns a
|
|
184
|
+
`dateRange` column labeled `current` or `previous`. Date options work with
|
|
185
|
+
`report`, `overview`, `portfolio`, `health`, `raw`, and `funnel`. Health
|
|
186
|
+
always adds the preceding comparison period. Raw and funnel
|
|
187
|
+
reports accept only one period. Realtime reports have no date range. `--last`
|
|
188
|
+
accepts 1 through 1,825 complete days.
|
|
189
|
+
|
|
190
|
+
## Custom reports and exports
|
|
191
|
+
|
|
192
|
+
```bash
|
|
193
|
+
analytics-ops report custom \
|
|
194
|
+
--dimension eventName \
|
|
195
|
+
--metric eventCount \
|
|
196
|
+
--where eventName=purchase \
|
|
197
|
+
--limit 100
|
|
198
|
+
|
|
199
|
+
analytics-ops report traffic \
|
|
200
|
+
--all \
|
|
201
|
+
--page-size 100000 \
|
|
202
|
+
--max-rows 500000 \
|
|
203
|
+
--output tmp/traffic.csv
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
`--where` is a repeatable exact dimension filter. Saved YAML recipes support
|
|
207
|
+
the complete bounded filter model. `--all` is a streaming CSV export, requires
|
|
208
|
+
an output path, and never exceeds `--max-rows`.
|
|
209
|
+
|
|
113
210
|
## Options
|
|
114
211
|
|
|
115
212
|
| Option | Meaning |
|
|
116
213
|
| --- | --- |
|
|
117
214
|
| `-c, --config PATH` | Configuration file |
|
|
118
215
|
| `-p, --profile NAME` | Profile inside the file |
|
|
119
|
-
|
|
|
216
|
+
| `--connection NAME` | Named saved Google connection |
|
|
217
|
+
| `-f, --format FORMAT` | `human`, `json`, or result-only `csv` |
|
|
120
218
|
| `--json` | Shortcut for `--format json` |
|
|
121
219
|
| `--csv` | Shortcut for `--format csv` |
|
|
122
|
-
|
|
|
220
|
+
| `--last DAYS` | Previous complete days for date-aware read commands |
|
|
221
|
+
| `--from DATE` | Inclusive `YYYY-MM-DD` start; requires `--to` |
|
|
222
|
+
| `--to DATE` | Inclusive `YYYY-MM-DD` end; requires `--from` |
|
|
223
|
+
| `--compare` | Include the equally long preceding period |
|
|
224
|
+
| `-o, --output PATH` | Save a plan, or the CSV created by `report --all` |
|
|
225
|
+
| `--dimension NAME` | Repeatable dimension for `report custom` |
|
|
226
|
+
| `--metric NAME` | Repeatable metric for `report custom`; at least one required |
|
|
227
|
+
| `--where FIELD=VALUE` | Repeatable exact dimension filter for `report custom` |
|
|
228
|
+
| `--limit ROWS` | Row limit for `report custom` or `raw` |
|
|
229
|
+
| `--all` | Page and stream a report to an explicit CSV output path |
|
|
230
|
+
| `--max-rows ROWS` | Hard export limit; maximum 1,000,000 |
|
|
231
|
+
| `--page-size ROWS` | Export rows per Data API request; maximum 250,000 |
|
|
232
|
+
| `--kind dimension\|metric` | Restrict `fields` results |
|
|
233
|
+
| `--concurrency COUNT` | Portfolio report/health workers; 1 through 8 |
|
|
123
234
|
| `--property ID` | Select an accessible property without prompting; setup only |
|
|
124
235
|
| `--service-account PATH` | Connect a Google service-account JSON key; setup only |
|
|
125
236
|
| `--transport grpc|rest` | Official Google-client transport |
|
|
@@ -129,21 +240,31 @@ plans and applies in one step.
|
|
|
129
240
|
| `--non-interactive` | Never prompt; apply requires `--yes`, setup requires `--property` |
|
|
130
241
|
|
|
131
242
|
Unknown arguments, conflicting format flags, and non-finite timeouts fail
|
|
132
|
-
instead of being ignored. CSV is rejected for anything except report
|
|
133
|
-
CSV cells whose first meaningful
|
|
134
|
-
are prefixed safely, including cells
|
|
243
|
+
instead of being ignored. CSV is rejected for anything except report,
|
|
244
|
+
portfolio-report, BigQuery, or funnel results. CSV cells whose first meaningful
|
|
245
|
+
character is a spreadsheet formula marker are prefixed safely, including cells
|
|
246
|
+
hidden behind whitespace or controls.
|
|
135
247
|
|
|
136
248
|
## Output
|
|
137
249
|
|
|
138
250
|
- Human output removes terminal control characters from remote text.
|
|
251
|
+
- Long human table cells keep both their beginning and ending, with an
|
|
252
|
+
ellipsis in the middle so similar URLs remain distinguishable.
|
|
253
|
+
- Interactive apply prints complete redacted before/after values without the
|
|
254
|
+
ordinary message-length cap.
|
|
139
255
|
- JSON output uses stable gem-owned fields and structured errors.
|
|
140
|
-
- CSV contains
|
|
141
|
-
|
|
256
|
+
- CSV contains only result headers and rows. Portfolio CSV adds profile,
|
|
257
|
+
property, and label columns, and refuses partial portfolio results. CSV is
|
|
258
|
+
rejected for batched overviews, plans, snapshots, and errors.
|
|
142
259
|
|
|
143
260
|
No command prints credentials or generated Google protobuf objects. Report
|
|
144
261
|
rows are emitted only because the user requested a report; they are never
|
|
145
262
|
logged automatically.
|
|
146
263
|
|
|
264
|
+
The MCP command writes protocol messages to standard output and rejects human,
|
|
265
|
+
JSON, or CSV formatting flags. Its advertised tools are all annotated
|
|
266
|
+
read-only and include no plan or apply operation.
|
|
267
|
+
|
|
147
268
|
## Exit statuses
|
|
148
269
|
|
|
149
270
|
| Status | Meaning |
|