gaapi 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ab86e8204d003348a950268a6ae436f4a8864b05ee265e9bf86e6ee97537792d
-  data.tar.gz: 9946f8a7d7677b48db6df4757a8a72989695559c98369965aeb0d79779d74bd0
+  metadata.gz: 297b653a2ab27e34a552b37d270d675473eb3a282205bcffcd6b41a1ad9eb541
+  data.tar.gz: 9dfa54aae6e7d654000de0d7ba6aa6dd40619114dabb6269e35d689f7a243283
 SHA512:
-  metadata.gz: bd642771496a8728d814240b65ab52e8f07cae029b77bed5fea4cff6cbe4f5c95c7b8ad842e86b248d18e771d706b720a06c3c5615ac8ababf374421303d4be9
-  data.tar.gz: edf4b83b5f60e1b48557b3c6506bb8cf5a758599bc691ba4d7663616c6aa0534e1893297d20fac812f0a4c25070dddc4aa9d10d225a827d08c4cd2e6e0f62b70
+  metadata.gz: c93a38a5beae05b5e1583f6ddf2df29758b588ff68e196c617c06e74c723e7a3f5daefd38a9a839ae04ef8bcaba9f8a72e79aff1f1179666e93d6be4ca8de6bd
+  data.tar.gz: eaf0ca31bc981f5a0789fcf4dd03ffb294961d5a4354c9fa5923761cfc41dfe774fba7fe45268b38f9d0a47ab0fea5d9dcb53d58f5a0746dd43152e6c1bebe9a

data/CHANGELOG.md

@@ -0,0 +1,70 @@
+## [Pending Release][0.5.0]
+
+### Breaking changes
+
+* Your contribution here!
+
+### New features
+
+* Your contribution here!
+
+### Bugfixes
+
+* Your contribution here!
+
+## [0.4.0][](2018-10-31)
+
+### Breaking changes
+
+* The CSV output from the command-line tool may have changed in certain edge cases, like when no dimensions are specified in the query.
+
+### New features
+
+* Report and Row classes for the result make writing Ruby code a little easier.
+* [#5] Dimensions and metrics in rows can be accessed by name. See the README.
+
+## [0.3.0][](2018-10-23)
+
+### New features
+
+* [#4] The query can now be specified as a string, a JSON hash, or a regular Ruby hash with symbolic keys.
+* [#3] `gaapi` automatically retrieves the Google Analytics-defined maximum of 10,000 rows, unless the user specifies a lower limit.
+
+## [0.2.1][](2018-09-07)
+
+### Breaking changes
+
+### New features
+
+* Documentation changes.
+
+### Bugfixes
+
+* `gaapi` returns appropriate exit status to operating system.
+
+## [0.2.0][0.2.0](2018-09-06)
+
+### Breaking changes
+
+* Refactor code so it's more useful to incorporate in other programs.
+
+### New features
+
+### Bugfixes
+
+## [0.0.1.alpha1][] (2018-08-27)
+
+### Breaking changes
+
+* N/A.
+
+### New features
+
+* N/A.
+
+### Bugfixes
+
+* N/A.
+
+[Pending Release]: https://github.com/weenhanceit/gaapi/compare/v0.4.0...HEAD
+[0.4.0]: https://github.com/weenhanceit/gaapi/compare/v0.4.0...v0.3.0

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2018 weenhanceit
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,260 @@
+# Google Analytics Reporting for Ruby
+`gaapi` provides:
+
+- A command line executable program to retrieve reporting data from Google Analytics (GA). It takes the user's GA request, specified in JSON format, and sends it to GA. It outputs the result of the request in JSON or comma-separated values (CSV) format
+- A library of classes that can be used in other programs to retrieve reporting data from GA
+
+`gaapi` supports two ways of providing credentials. One way is more useful while testing scripts or doing ad-hoc queries. The other is more appropriate for unattended script usage. See the [Authentication](#Authentication) section for more details.
+
+Google provides a [Ruby client library](https://developers.google.com/api-client-library/ruby/apis/analyticsreporting/v4) that builds queries by constructing them from Ruby objects. `gaapi` allows you to express queries as JSON. If you prefer the JSON format, you may prefer to use `gaapi`. If you want to deal with Ruby objects (which are likely more verbose than JSON), use the Google gem.
+
+## Installation
+For stand-alone use:
+
+```bash
+gem install gaapi --no-doc
+```
+
+In a Gemfile:
+
+```
+gem 'gaapi'
+```
+
+## Usage
+### Command Line
+```bash
+gaapi [options] VIEW_ID
+```
+If no query is specified on the command line, `gaapi` tries to read the query from standard input.
+
+The `VIEW_ID` is what identifies the GA data (a view of a property). To find the view ID, log in to GA, select the account of interest, select Admin (the gear near the bottom left of the page), and select "View Settings" (on the right of the page).
+
+#### Options
+```
+    -a, --access-token TOKEN         An access token obtained from https://developers.google.com/oauthplayground.
+        --csv                        Output result as a csv file.
+    -c, --credentials CREDENTIALS    Location of the credentials file. Default: `.gaapi/ga-api-key`.
+    -d, --debug                      Print debugging information.
+    -e, --end-date END_DATE          Report including END_DATE (yyyy-mm-dd).
+    -n, --dry-run                    Don't actually send the query to Google.
+    -q, --query-file QUERYFILE       File containing the query. Default STDIN.
+    -s, --start-date START_DATE      Report including START_DATE (yyyy-mm-dd).
+```
+
+If you specify both the `-a` and `-c` options, `gaapi` will use the `-a` option.
+
+#### Example
+Get the number of visitors to a site for January, 2018, with credentials previously obtained and stored in `./credentials.json`:
+```bash
+gaapi -s "2018-01-01" -e "2018-01-31" -c ./credentials.json 000000
+{
+  "reportRequests": [{
+      "viewId": "VIEW_ID",
+      "dimensions": [{"name": "ga:date"}],
+      "dateRanges": [{
+        "startDate": "START_DATE",
+        "endDate": "END_DATE"
+      }],
+      "metrics": [{
+          "expression": "ga:users"
+      }],
+      "includeEmptyRows": true,
+      "hideTotals": false,
+      "hideValueRanges": true
+    }]
+}
+```
+
+### In a Program
+Make sure the program can find GAAPI. Without Rails:
+```ruby
+require "gaapi"
+```
+With Rails, simply include `gaapi` in the `Gemfile`:
+```ruby
+gem "gaapi"
+```
+
+Next, get an access token. To run the program unattended, the best way is to use the approach described [here](#Unattended Running), which translates to the following code:
+```ruby
+access_token = GAAPI::AccessToken.new("path/to/credential_file")
+```
+
+Set up the query. This may raise exceptions:
+```ruby
+begin
+  query = GAAPI::Query.new(query_string, 00000000, access_token, "2018-01-01", "2018-06-30")
+rescue StandardError => e
+  # Handle the error
+end
+```
+A typical exception would be from a `query_string` that isn't valid JSON. The `query_string` has to be a valid GA reporting query. See the [Queries](#Queries) section. Because the access token is lazy-evaluated, you may also get an exception here if the credential file doesn't exist or is malformed.
+
+Execute the request:
+```ruby
+result = query.execute
+if result.success?
+  ...
+end
+```
+
+If the query was successful, you have access to a few interesting methods:
+```ruby
+puts result.body  # raw response body
+puts result.pp    # a string formatted into more readable JSON
+puts result.csv   # comma-separated values format, ready to be written to a file
+```
+
+There is also some support now for a more structured use of the resulting query. If the query was successful (`result.success?`), you can use the following:
+```
+result.reports    # An array of GAAPI::Report objects
+report.dimensions # An array of the dimension names
+report.headers    # An array of the dimension names and metric names
+report.metrics    # An array of the metric names
+report.rows       # An array of GAAPI::Row objects
+```
+
+If you have a `Row` object, you can access the dimensions and metrics using method names. For example, to get the `ga:sessionDuration` metric for a row:
+```
+row.session_duration
+```
+
+The `ga:` is stripped from the front of the dimension or metric name, and then the rest is converted to snake case.
+
+You can also get all the dimensions or all the metrics for a row:
+```
+row.dimensions
+row.metrics
+```
+
+These return arrays with the values in the order corresponding to the `report.dimensions` and `report.metrics` arrays.
+
+Putting it all together, to get all the `ga:avgSessionDuration` from all the rows in all the reports:
+```
+result.reports.flat_map do |report|
+  report.rows.map do |row|
+    row.avg_session_duration
+  end
+end
+```
+
+## Queries
+`gaapi` uses the Google Analytics Reporting API v4 (https://developers.google.com/analytics/devguides/reporting/core/v4/). An introduction to querying for GA data is here: https://developers.google.com/analytics/devguides/reporting/core/v4/basics.
+A very useful reference of the dimensions and metrics available is at: https://developers.google.com/analytics/devguides/reporting/core/dimsmets.
+
+A query to find basic visit data for a web site is:
+```json
+{
+  "reportRequests": [{
+      "viewId": "VIEW_ID",
+      "dimensions": [{"name": "ga:date"}],
+      "dateRanges": [{
+        "startDate": "2017-10-01",
+        "endDate": "2017-10-31"
+      }],
+      "metrics": [{
+          "expression": "ga:avgSessionDuration"
+        },
+        {
+          "expression": "ga:pageviewsPerSession"
+        },
+        {
+          "expression": "ga:sessions"
+        },
+        {
+          "expression": "ga:users"
+        }
+      ],
+      "includeEmptyRows": true,
+      "hideTotals": false,
+      "hideValueRanges": true
+    },
+    {
+      "viewId": "VIEW_ID",
+      "dimensions": [{"name": "ga:date"}],
+      "dateRanges": [{
+        "startDate": "2017-10-01",
+        "endDate": "2017-10-31"
+      }],
+      "metrics": [{
+          "expression": "ga:goal1Completions"
+        },
+        {
+          "expression": "ga:goal2Completions"
+        },
+        {
+          "expression": "ga:goal6Completions"
+        },
+        {
+          "expression": "ga:goal8Completions"
+        },
+        {
+          "expression": "ga:goal9Completions"
+        },
+        {
+          "expression": "ga:goal11Completions"
+        },
+        {
+          "expression": "ga:goal13Completions"
+        },
+        {
+          "expression": "ga:goal14Completions"
+        },
+        {
+          "expression": "ga:goal16Completions"
+        },
+        {
+          "expression": "ga:goalCompletionsAll"
+        }
+      ],
+      "includeEmptyRows": true,
+      "hideTotals": false,
+      "hideValueRanges": true
+    },
+    {
+      "viewId": "VIEW_ID",
+      "dimensions": [{"name": "ga:date"}],
+      "dateRanges": [{
+        "startDate": "2017-10-01",
+        "endDate": "2017-10-31"
+      }],
+      "metrics": [{
+          "expression": "ga:avgSessionDuration"
+        },
+        {
+          "expression": "ga:pageviewsPerSession"
+        },
+        {
+          "expression": "ga:sessions"
+        },
+        {
+          "expression": "ga:users"
+        }
+      ],
+      "includeEmptyRows": true,
+      "hideTotals": false,
+      "hideValueRanges": true
+    }
+  ]
+}
+```
+
+By default, Google Analytics will return a maximum of 1,000 rows. `gaapi` automatically adds a `pageSize: 10000` to your query, if no `pageSize` is specified. This causes Google Analytics to return 10,000 rows, the maximum that Google Analytics will return.
+
+If `gaapi` returns 10,000 rows, it's your responsibility to use the `nextPageToken` in the returned result, to query additional rows.
+
+## Authentication
+[The introduction to authentication for Google products is here: https://developers.google.com/analytics/devguides/reporting/core/v4/authorization.]
+
+### Testing and Ad-Hoc Usage
+This method involves cutting and pasting an access token obtained from https://developers.google.com/oauthplayground onto the command line. The access token is simply a long string of characters generated by Google. The access token expires after an hour, so the user has to return to the Google URL to get a new token.
+
+### Unattended Running
+This method obtains a file of secure credentials from Google. It's very important that these credentials be kept secure, as whoever has a copy of the file, has access to the Google Analytics data for the account.
+
+To use this type of credential with `gaapi`:
+
+1. Follow the instructions at: https://developers.google.com/identity/protocols/OAuth2ServiceAccount, choose a JSON format file, and when you're prompted to save a file, save it
+2. Immediately change the permissions of the file to make it readable only by you. On Linux, Unix, OSX that's `chmod 600 filename`
+3. Give the file name in the `--credentials` option when you run `gaapi`, or pass it to `AccessToken.new`

data/lib/gaapi/report.rb

@@ -0,0 +1,57 @@
+# frozen_string_literal: true
+
+module GAAPI
+  # A single report from a query to Google Analytics
+  class Report
+    # An array of the dimensions, in the order that they appear in the response.
+    def dimensions
+      report["columnHeader"]["dimensions"] || []
+    end
+
+    # @return [Boolean] True if dimensions were returned.
+    def dimensions?
+      !report["columnHeader"]["dimensions"].nil?
+    end
+
+    # An array of the dimensions first and then the metrics, in the order that
+    # they appear in the response.
+    def headers
+      dimensions + metrics
+    end
+
+    def initialize(response, report)
+      @response = response
+      @report = report
+    end
+    attr_reader :report
+
+    # An array of the metric names, in the order that they appear in the response.
+    def metric_type(i)
+      report["columnHeader"]["metricHeader"]["metricHeaderEntries"][i]["type"]
+    end
+
+    # An array of the metric names, in the order that they appear in the response.
+    def metrics
+      report["columnHeader"]["metricHeader"]["metricHeaderEntries"].map { |metric| metric["name"] }
+    end
+
+    # The data rows in the report.
+    def rows
+      report["data"]["rows"].map { |row| Row.new(self, row) }
+    end
+
+    # The totals, if there were any.
+    def totals
+      Array.new(dimensions.size) + report["data"]["totals"][0]["values"]
+    end
+
+    # @return [Boolean] True if there totals were returned from the query.
+    def totals?
+      report["data"]["totals"]
+    end
+
+    private
+
+    attr_reader :response
+  end
+end

data/lib/gaapi/response.rb

@@ -23,16 +23,20 @@ module GAAPI
     # @return [String] The result of the query formatted as a comma-separated
     #   values string.
     def csv
-      result = to_json
       @csv ||= CSV.generate do |csv|
-        result["reports"].each do |report|
-          # puts report.column_header.dimensions.inspect
-          # puts report.column_header.metric_header.metric_header_entries.map(&:name).inspect
-          csv << csv_header_row(report)
-          report["data"]["rows"].each do |row|
-            csv << csv_data_row(row["dimensions"], row["metrics"])
-          end
-          csv << csv_data_row("Totals", report["data"]["totals"]) if report["data"]["totals"]
+        reports.each(&:report).each do |report|
+          # If there are no dimensions, but totals, we need to put an extra
+          # column at the start for the word "Total".
+          # I don't see how totals will be different than the metrics if you
+          # don't specify dimensions, but whatever.
+          totals_column = report.totals? && !report.dimensions? ? [nil] : []
+          csv << totals_column + report.headers
+          report.rows.each { |row| csv << totals_column + row.to_a }
+          csv << ["Totals"] + if report.totals? && !report.dimensions?
+                                report.totals
+                              else
+                                report.totals[1..-1]
+                              end
         end
       end
     end
@@ -47,6 +51,18 @@ module GAAPI
       @pp ||= JSON.pretty_generate(to_json)
     end
 
+    # The array of reports returned by the query.
+    # @return [Array] An array of reports.
+    def reports
+      @reports ||= if success?
+                     to_json["reports"].map do |report|
+                       Report.new(self, report)
+                     end
+                   else
+                     []
+                   end
+    end
+
     # Return true if the request was successful.
     # @return [Boolean] True if the request was successful (response code 200).
     def success?
@@ -64,17 +80,5 @@ module GAAPI
     def to_s
       response.body
     end
-
-    private
-
-    def csv_data_row(row_headers, metrics)
-      (Array(row_headers) || []) + metrics[0]["values"]
-    end
-
-    def csv_header_row(report)
-      (report["columnHeader"]["dimensions"] || []) + report["columnHeader"]["metricHeader"]["metricHeaderEntries"].map do |entry|
-        entry["name"]
-      end
-    end
   end
 end

data/lib/gaapi/row.rb

@@ -0,0 +1,111 @@
+# frozen_string_literal: true
+
+module GAAPI
+  # A single row from a query to Google Analytics
+  class Row
+    # An array of the dimension values, in the order that they appear in the
+    # dimension headers.
+    def dimensions
+      row["dimensions"] || []
+    end
+
+    def initialize(report, row)
+      @report = report
+      @row = row
+    end
+
+    # Define and call methods for the columns in the report.
+    def method_missing(method, *args)
+      if (i = dimension_method_names.find_index(method))
+        define_singleton_method(method) do
+          dimensions[i]
+        end
+        send(method)
+      elsif (i = metric_method_names.find_index(method))
+        define_singleton_method(method) do
+          convert_metric(i)
+        end
+        send(method)
+      else
+        super
+      end
+    end
+
+    # An array of the metric values, in the order that they appear in the
+    # metric headers.
+    def metrics
+      # NOTE: There is one entry in the `row["metrics"]` array for each data range.
+      # Since currently we only support one date range, this following index is
+      # always 0.
+      row["metrics"][0]["values"]
+    end
+
+    def respond_to_missing?
+      true
+    end
+
+    # Return the data from the row as an Array, ordered by:
+    #   Headers first, in the order that they appear in the Report#headers array.
+    #   Metrics next, in the order that they appear in the Rport#metrics array.
+    def to_a
+      dimensions + metrics
+    end
+
+    private
+
+    attr_reader :report, :row
+
+    # Convert metric to the right type.
+    def convert_metric(i)
+      case report.metric_type(i)
+      when "INTEGER"
+        # INTEGER  Integer metric.
+        metrics[i].to_i
+      when "FLOAT", "PERCENT"
+        # FLOAT  Float metric.
+        # PERCENT  Percentage metric.
+        metrics[i].to_f
+      when "CURRENCY"
+        # CURRENCY  Currency metric.
+        # TODO: Do this better.
+        metrics[i].to_f
+      when "TIME"
+        # TIME  Time metric in HH:MM:SS format.
+        # Time in fraction of a day.
+        (metrics[i][0..1].to_i +
+          metrics[i][3..4].to_i * 60 +
+          metrics[i][6..7].to_i * 24 * 60) / 86_400
+      else
+        # METRIC_TYPE_UNSPECIFIED  Metric type is unspecified.
+        metric[i]
+      end
+    end
+
+    # Return the dimensions in the response, as method names.
+    def dimension_method_names
+      report.dimensions.map { |d| ga_to_method_name(d) }
+    end
+
+    # Change a dimension or metric name to a method name.
+    # Strip off the "ga:" at the beginning, change it to snake case, and
+    # convert it to a symbol.
+    def ga_to_method_name(name)
+      snakecase(name.gsub(/\Aga:/, "")).to_sym
+    end
+
+    # Return the metrics in the response, as method names.
+    def metric_method_names
+      report.metrics.map { |d| ga_to_method_name(d) }
+    end
+
+    # From: https://stackoverflow.com/a/37381260/3109926
+    def snakecase(string)
+      string.to_s.gsub(/([A-Z]+)([A-Z][a-z])/, '\1_\2')
+            .gsub(/([a-z\d])([A-Z])/, '\1_\2')
+            .tr("-", "_")
+            .gsub(/\s/, "_")
+            .gsub(/__+/, "_")
+            .downcase
+    end
+  end
+end

data/lib/gaapi/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module GAAPI
-  VERSION = "0.3.0"
+  VERSION = "0.4.0"
 end

data/lib/gaapi.rb

@@ -3,7 +3,8 @@
 # http://www.rubydoc.info/github/google/google-api-ruby-client/
 require "google/apis/analyticsreporting_v4"
 require "googleauth"
-require "gaapi/access_token.rb"
-require "gaapi/main.rb"
-require "gaapi/query.rb"
-require "gaapi/response.rb"
+# require "gaapi/access_token.rb"
+# require "gaapi/main.rb"
+# require "gaapi/query.rb"
+# require "gaapi/response.rb"
+Dir.glob("lib/gaapi/**/*.rb").each { |f| require f.gsub(%r{lib/}, "") }

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: gaapi
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.4.0
 platform: ruby
 authors:
 - Larry Reid
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-09-06 00:00:00.000000000 Z
+date: 2018-11-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: chandler
@@ -90,12 +90,17 @@ executables:
 extensions: []
 extra_rdoc_files: []
 files:
+- CHANGELOG.md
+- LICENSE
+- README.md
 - bin/gaapi
 - lib/gaapi.rb
 - lib/gaapi/access_token.rb
 - lib/gaapi/main.rb
 - lib/gaapi/query.rb
+- lib/gaapi/report.rb
 - lib/gaapi/response.rb
+- lib/gaapi/row.rb
 - lib/gaapi/version.rb
 homepage: https://github.com/weenhanceit/gaapi
 licenses: