gattica 0.4.0

data/.gitignore

@@ -0,0 +1,4 @@
+*.gem
+github-test.rb
+examples/active_example.rb
+examples/.DS_Store

data/History.txt

@@ -0,0 +1,42 @@
+== 0.4.0
+  * er1c added start_index and max_results
+  * er1c added paging for all results
+  * er1c added get_to_csv to "stream" saving results to a file IO
+  * thieso2 fix DataPoint when GA-Path includec colons
+  * jeremyf Added ability to parse addition query strings
+  * nedski Added support for proxy via env var
+
+== 0.3.4
+  * rumble updated the regex used to pull apart the filters so it didn't get confused when there a filename, for example, started with a /
+
+== 0.3.2
+  * er1c updated to use standard Ruby CSV library
+
+== 0.3.0
+  * Support for filters (filters are all AND'ed together, no OR yet)
+
+== 0.2.1 
+  * More robust error checking on HTTP calls
+  * Added to_xml to get raw XML output from Google
+  
+== 0.2.0 / 2009-04-27
+  * Changed initialization format: pass a hash of options rather than individual email, password and profile_id
+  * Can initialize with a valid token and use that instead of requiring email/password each time
+  * Can initialize with your own logger object instead of having to use the default (useful if you're using with Rails, initialize with RAILS_DEFAULT_LOGGER)
+  * Show error if token is invalid or expired (Google returns a 401 on any HTTP call)
+  * Started tests
+
+== 0.1.4 / 2009-04-22
+  * Another attempt at getting the gem to build on github
+
+== 0.1.3 / 2009-04-22
+  * Getting gem to build on github
+
+== 0.1.2 / 2009-04-22
+  * Updated readme and examples, better documentation throughout
+
+== 0.1.1 / 2009-04-22
+  * When outputting as CSV, surround each piece of data with double quotes (appears pretty common for various properties (like Browser name) to contain commas
+
+== 0.1.0 / 2009-03-26
+  * Basic functionality working good. Can't use filters yet.

data/LICENSE

@@ -0,0 +1,22 @@
+(The MIT License)
+ 
+Copyright (c) 2009 Rob Cameron
+ 
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+ 
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+ 
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,190 @@
+Gattica is a Ruby library for talking to the Google Analytics API.
+
+= Installation
+Install the gattica gem using github as the source:
+
+  gem install cannikin-gattica -s http://gems.github.com
+
+When you want to require, you just use 'gattica' as the gem name:
+
+  require 'rubygems'
+  require 'gattica'
+
+= Introduction
+It's a good idea to familiarize yourself with the Google API docs: http://code.google.com/apis/analytics/docs/gdata/gdataDeveloperGuide.html
+  
+In particular there are some very specific combinations of Metrics and Dimensions that
+you are restricted to and those are explained in this document: http://code.google.com/apis/analytics/docs/gdata/gdataReferenceDimensionsMetrics.html
+
+See examples/example.rb for some code that should work automatically if you replace the email/password with your own
+
+There are generally three steps to getting info from the GA API:
+
+1. Authenticate
+2. Get a profile id
+3. Get the data you really want
+
+= Usage
+This library does all three. A typical transaction will look like this:
+
+ gs = Gattica.new({:email => 'johndoe@google.com', :password => 'password', :profile_id => 123456})
+ results = gs.get({ :start_date => '2008-01-01', 
+                    :end_date => '2008-02-01', 
+                    :dimensions => 'browser', 
+                    :metrics => 'pageviews', 
+                    :sort => '-pageviews'})
+
+So we instantiate a copy of Gattica and pass it a Google Account email address and password.
+The third parameter is the profile_id that we want to access data for.
+
+Then we call +get+ with the parameters we want to shape our data with. In this case we want
+total page views, broken down by browser, from Jan 1 2008 to Feb 1 2008, sorted by descending
+page views. If you wanted to sort pageviews ascending, just leave off the minus.
+
+If you don't know the profile_id you want to get data for, call +accounts+
+
+ gs = Gattica.new({:email => 'johndoe@google.com', :password => 'password'})
+ accounts = gs.accounts
+
+This returns all of the accounts and profiles that the user has access to. Note that if you
+use this method to get profiles, you need to manually set the profile before you can call +get+
+
+ gs.profile_id = 123456
+ results = gs.get({ :start_date => '2008-01-01', 
+                    :end_date => '2008-02-01', 
+                    :dimensions => 'browser', 
+                    :metrics => 'pageviews', 
+                    :sort => '-pageviews'})
+                    
+When you put in the names for the dimensions and metrics you want, refer to this doc for the 
+available names: http://code.google.com/apis/analytics/docs/gdata/gdataReferenceDimensionsMetrics.html
+  
+Note that you do *not* use the 'ga:' prefix when you tell Gattica which ones you want. Gattica
+adds that for you automatically.
+
+If you want to search on more than one dimension or metric, pass them in as an array (you can
+also pass in single values as arrays too, if you wish):
+                    
+  results = gs.get({ :start_date => '2008-01-01', 
+                     :end_date => '2008-02-01', 
+                     :dimensions => ['browser','browserVersion'], 
+                     :metrics => ['pageviews','visits'], 
+                     :sort => ['-pageviews']})
+                     
+== Filters
+Filters can be pretty complex as far as GA is concerned. You can filter on either dimensions or metrics
+or both. And then your filters can be ANDed together or they can ORed together. There are also rules,
+which are not immediately apparent, about what you can filter and how.
+
+By default filters passed to a +get+ are ANDed together. This means that all filters need to match for
+the result to be returned.
+
+  results = gs.get({:start_date => '2008-01-01', 
+                    :end_date => '2008-02-01', 
+                    :dimensions => ['browser','browserVersion'], 
+                    :metrics => ['pageviews','visits'], 
+                    :sort => ['-pageviews'],
+                    :filters => ['browser == Firefox','pageviews >= 10000']})
+  
+This says "return only results where the 'browser' dimension contains the word 'Firefox' and the
+'pageviews' metric is greater than or equal to 10,000.
+
+Filters can contain spaces around the operators, or not. These two lines are equivalent (I think
+the spaces make the filter more readable):
+
+  :filters => ['browser == Firefox','pageviews >= 10000']
+  
+  :filters => ['browser==Firefox','pageviews>=10000']
+  
+Once again, do _not_ include the +ga:+ prefix before the dimension/metric you're filtering against.
+Gattica will add this automatically.
+
+You will probably find that as you try different filters, GA will report that some of them aren't
+valid. I haven't found any documentation that says which filter combinations are valid in what 
+circumstances, so I suppose it's just trial and error at this point.
+
+For more on filtering syntax, see the Analytics API docs: http://code.google.com/apis/analytics/docs/gdata/gdataReference.html#filtering
+  
+= Output
+When Gattica was originally created it was intended to take the data returned and put it into
+Excel for someone else to crunch through the numbers. Thus, Gattica has great built-in support
+for CSV output. Once you have your data simply:
+
+  results.to_csv
+  
+A couple example rows of what that looks like:
+
+  "id","updated","title","browser","pageviews"
+  "http://www.google.com/analytics/feeds/data?ids=ga:12345&ga:browser=Internet%20Explorer&start-date=2009-01-01&end-date=2009-01-31","2009-01-30T16:00:00-08:00","ga:browser=Internet Explorer","Internet Explorer","53303"
+  "http://www.google.com/analytics/feeds/data?ids=ga:12345&ga:browser=Firefox&start-date=2009-01-01&end-date=2009-01-31","2009-01-30T16:00:00-08:00","ga:browser=Firefox","Firefox","20323"
+  
+Data is comma-separated and double-quote delimited. In most cases, people don't care
+about the id, updated, or title attributes of this data. They just want the dimensions and
+metrics. In that case, pass the symbol +:short+ to +to_csv+ and receive get back only the
+the good stuff:
+
+  results.to_csv(:short)
+  
+Which returns: 
+
+  "browser","pageviews"
+  "Internet Explorer","53303"
+  "Firefox","20323"
+
+You can also just output the results as a string and you'll get the standard inspect syntax:
+
+  results.to_s
+  
+Gives you:
+
+  { "end_date"=>#<Date: 4909725/2,0,2299161>, 
+    "start_date"=>#<Date: 4909665/2,0,2299161>, 
+    "points"=>[
+      { "title"=>"ga:browser=Internet Explorer", 
+        "dimensions"=>[{:browser=>"Internet Explorer"}],
+        "id"=>"http://www.google.com/analytics/feeds/data?ids=ga:12345&amp;ga:browser=Internet%20Explorer&amp;start-date=2009-01-01&amp;end-date=2009-01-31", 
+        "metrics"=>[{:pageviews=>53303}], 
+        "updated"=>#<DateTime: 212100120000001/86400000,-1/3,2299161>}]}
+
+== Notes on Authentication
+=== Authentication Token
+Google recommends not re-authenticating each time you do a request against the API. To accomplish
+this you should save the authorization token you receive from Google and use that for future 
+requests:
+
+  ga.token => 'DSasdf94...' (some huge long string)
+
+You can now initialize Gattica with this token for future requests:
+
+  ga = Gattica.new({:token => 'DSasdf94...'})
+
+(You enter the full token, of course). I'm not sure how long a token from the Google's ClientLogin
+system remains active, but if/when I do I'll add that to the docs here.
+
+=== Headers
+Google expects a special header in all HTTP requests called 'Authorization'. This contains your
+token:
+
+  Authorization = GoogleLogin auth=DSasdf94...
+
+This header is generated automatically. If you have your own headers you'd like to add, you can 
+pass them in when you initialize:
+
+  ga = Gattica.new({:token => 'DSasdf94...', :headers => {'My-Special-Header':'my_custom_value'}})
+
+And they'll be sent with every request you make.
+        
+= Limitations
+The GA API limits each call to 1000 results per "page." If you want more, you need to tell
+the API what number to begin at and it will return the next 1000. Gattica does not currently
+support this, but it's in the plan for the very next version.
+
+Currently all filters you supply are ANDed together before being sent to GA. Support for ORing
+is coming soon.
+
+= The Future
+A couple of things I have planned:
+
+1. Tests!
+2. The option to use a custom delimiter for output
+

data/Rakefile

@@ -0,0 +1,24 @@
+require 'rake'
+require 'rake/testtask'
+require 'rake/rdoctask'
+
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gemspec|
+    gemspec.name = "gattica"
+    gemspec.summary = "Gattica is a Ruby library for extracting data from the Google Analytics API."
+    gemspec.email = "cannikinn@gmail.com"
+    gemspec.homepage = "http://github.com/cannikin/gattica"
+    gemspec.description = "Gattica is a Ruby library for extracting data from the Google Analytics API."
+    gemspec.authors = ["Rob Cameron"]
+    gemspec.add_dependency('hpricot','>=0.6.164')
+  end
+rescue LoadError
+  puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
+end
+
+Rake::TestTask.new do |t|
+  t.libs << 'lib'
+  t.pattern = 'test/**/test_*.rb'
+  t.verbose = false
+end

data/VERSION.yml

@@ -0,0 +1,4 @@
+--- 
+:major: 0
+:minor: 4
+:patch: 0

data/examples/example.rb

@@ -0,0 +1,42 @@
+require '../lib/gattica'
+
+# authenticate with the API via email/password
+ga = Gattica.new({:email => 'username@gmail.com', :password => 'password'})
+
+# or, initialize via a pre-existing token. This initialization does not authenticate immediately, 
+# but will throw an error on subsequent calls (like ga.accounts) if the token is invalid
+# ga = Gattica.new({:token => 'DQAAAJYAAACN-JMelka5I0Fs-T6lF53eUSfUooeHgcKc1iEdc0wkDS3w8GaXY7LjuUB_4vmzDB94HpScrULiweW_xQsU8yyUgdInDIX7ZnHm8_o0knf6FWSR90IoAZGsphpqteOjZ3O0NlNt603GgG7ylvGWRSeHl1ybD38nysMsKJR-dj0LYgIyPMvtgXLrqr_20oTTEExYbrDSg5_q84PkoLHUcODZ' })
+
+# get the list of accounts you have access to with that username and password
+accounts = ga.accounts
+
+# for this example we just use the first account's profile_id, but you'll probably want to look
+# at this list and choose the profile_id of the account you want (the web_property_id is the
+# property you're most used to seeing in GA, looks like UA123456-1)
+ga.profile_id = accounts.first.profile_id
+
+# If you're using Gattica with a web app you'll want to save the authorization token
+# and use that on subsequent requests (Google recommends not re-authenticating each time)
+# ga.token
+
+# now get the number of page views by browser for Janurary 2009
+data = ga.get({ :start_date => '2009-01-01', 
+                :end_date => '2009-01-31',
+                :dimensions => ['browser'],
+                :metrics => ['pageviews'],
+                :sort => ['-pageviews'] })
+           
+# output the data as CSV     
+puts data.to_csv
+
+# a little more complex example with filtering. Show all pageviews by Firefox browsers
+# (any version) where the number of page views is greater than 100
+data = ga.get({ :start_date => '2009-01-01', 
+                :end_date => '2009-01-31',
+                :dimensions => ['browser','browserVersion'],
+                :metrics => ['pageviews'],
+                :sort => ['-pageviews'],
+                :filters => ['browser == Firefox', 'pageviews > 100'] })
+
+# write the data out as CSV in "short" format (doesn't include id, updated or title parameters)
+puts data.to_csv(:short)

data/gattica.gemspec

@@ -0,0 +1,72 @@
+# Generated by jeweler
+# DO NOT EDIT THIS FILE
+# Instead, edit Jeweler::Tasks in Rakefile, and run `rake gemspec`
+# -*- encoding: utf-8 -*-
+
+Gem::Specification.new do |s|
+  s.name = %q{gattica}
+  s.version = "0.4.0"
+
+  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+  s.authors = ["Rob Cameron"]
+  s.date = %q{2009-09-04}
+  s.description = %q{Gattica is a Ruby library for extracting data from the Google Analytics API.}
+  s.email = %q{cannikinn@gmail.com}
+  s.extra_rdoc_files = [
+    "LICENSE",
+     "README.rdoc"
+  ]
+  s.files = [
+    ".gitignore",
+     "History.txt",
+     "LICENSE",
+     "README.rdoc",
+     "Rakefile",
+     "VERSION.yml",
+     "examples/example.rb",
+     "gattica.gemspec",
+     "lib/gattica.rb",
+     "lib/gattica/account.rb",
+     "lib/gattica/auth.rb",
+     "lib/gattica/convertible.rb",
+     "lib/gattica/core_extensions.rb",
+     "lib/gattica/data_point.rb",
+     "lib/gattica/data_set.rb",
+     "lib/gattica/exceptions.rb",
+     "lib/gattica/user.rb",
+     "test/helper.rb",
+     "test/suite.rb",
+     "test/test_auth.rb",
+     "test/test_engine.rb",
+     "test/test_gattica.rb",
+     "test/test_user.rb"
+  ]
+  s.has_rdoc = true
+  s.homepage = %q{http://github.com/cannikin/gattica}
+  s.rdoc_options = ["--charset=UTF-8"]
+  s.require_paths = ["lib"]
+  s.rubygems_version = %q{1.3.1}
+  s.summary = %q{Gattica is a Ruby library for extracting data from the Google Analytics API.}
+  s.test_files = [
+    "test/helper.rb",
+     "test/suite.rb",
+     "test/test_auth.rb",
+     "test/test_engine.rb",
+     "test/test_gattica.rb",
+     "test/test_user.rb",
+     "examples/example.rb"
+  ]
+
+  if s.respond_to? :specification_version then
+    current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
+    s.specification_version = 2
+
+    if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
+      s.add_runtime_dependency(%q<hpricot>, [">= 0.6.164"])
+    else
+      s.add_dependency(%q<hpricot>, [">= 0.6.164"])
+    end
+  else
+    s.add_dependency(%q<hpricot>, [">= 0.6.164"])
+  end
+end

data/lib/gattica.rb

@@ -0,0 +1,348 @@
+$:.unshift File.dirname(__FILE__) # for use/testing when no gem is installed
+
+# external
+require 'net/http'
+require 'net/https'
+require 'uri'
+require 'cgi'
+require 'logger'
+require 'rubygems'
+require 'hpricot'
+require 'yaml'
+
+# internal
+require 'gattica/core_extensions'
+require 'gattica/convertible'
+require 'gattica/exceptions'
+require 'gattica/user'
+require 'gattica/auth'
+require 'gattica/account'
+require 'gattica/data_set'
+require 'gattica/data_point'
+
+# Gattica is a Ruby library for talking to the Google Analytics API.
+#
+# Please see the README for usage docs.
+
+module Gattica
+  
+  VERSION = '0.4.0'
+  
+  # Creates a new instance of Gattica::Engine and gets us going. Please see the README for usage docs.
+  #
+  #   ga = Gattica.new({:email => 'anonymous@anon.com', :password => 'password, :profile_id => 123456 })
+  
+  def self.new(*args)
+    Engine.new(*args)
+  end
+  
+  # The real meat of Gattica, deals with talking to GA, returning and parsing results. You actually get 
+  # an instance of this when you go Gattica.new()
+  
+  class Engine
+    
+    SERVER = 'www.google.com'
+    PORT = 443
+    SECURE = true
+    DEFAULT_ARGS = { :start_date => nil, :end_date => nil, :dimensions => [], :metrics => [], :filters => [], :sort => [], :start_index => 1, :max_results => 10000, :page => false }
+    DEFAULT_OPTIONS = { :email => nil, :password => nil, :token => nil, :profile_id => nil, :debug => false, :headers => {}, :logger => Logger.new(STDOUT) }
+    FILTER_METRIC_OPERATORS = %w{ == != > < >= <= }
+    FILTER_DIMENSION_OPERATORS = %w{ == != =~ !~ =@ ~@ }
+    
+    attr_reader :user
+    attr_accessor :profile_id, :token
+    
+    # Create a user, and get them authorized.
+    # If you're making a web app you're going to want to save the token that's retrieved by Gattica
+    # so that you can use it later (Google recommends not re-authenticating the user for each and every request)
+    #
+    #   ga = Gattica.new({:email => 'johndoe@google.com', :password => 'password', :profile_id => 123456})
+    #   ga.token => 'DW9N00wenl23R0...' (really long string)
+    #
+    # Or if you already have the token (because you authenticated previously and now want to reuse that session):
+    #
+    #   ga = Gattica.new({:token => '23ohda09hw...', :profile_id => 123456})
+    
+    def initialize(options={})
+      @options = DEFAULT_OPTIONS.merge(options)
+      @logger = @options[:logger]
+      
+      @profile_id = @options[:profile_id]     # if you don't include the profile_id now, you'll have to set it manually later via Gattica::Engine#profile_id=
+      @user_accounts = nil                    # filled in later if the user ever calls Gattica::Engine#accounts
+      @headers = {}.merge(@options[:headers]) # headers used for any HTTP requests (Google requires a special 'Authorization' header which is set any time @token is set)
+      
+      # save a proxy-aware http connection for everyone to use
+      proxy_host = nil
+      proxy_port = nil
+      proxy_var  = SECURE ? 'https_proxy' : 'http_proxy'
+      [proxy_var, proxy_var.upcase].each do |pxy|
+        if ENV[pxy]
+          uri = URI::parse(ENV[pxy])
+          proxy_host = uri.host
+          proxy_port = uri.port
+        end
+      end
+      @http = Net::HTTP::Proxy(proxy_host,proxy_port).new(SERVER, PORT)
+      @http.use_ssl = SECURE
+      @http.set_debug_output $stdout if @options[:debug]
+      
+      # authenticate
+      if @options[:email] && @options[:password]      # email and password: authenticate, get a token from Google's ClientLogin, save it for later
+        @user = User.new(@options[:email], @options[:password])
+        @auth = Auth.new(@http, user)
+        self.token = @auth.tokens[:auth]
+      elsif @options[:token]                          # use an existing token
+        self.token = @options[:token]
+      else                                            # no login or token, you can't do anything
+        raise GatticaError::NoLoginOrToken, 'You must provide an email and password, or authentication token'
+      end
+      
+      # TODO: check that the user has access to the specified profile and show an error here rather than wait for Google to respond with a message
+    end
+    
+    
+    # Returns the list of accounts the user has access to. A user may have multiple accounts on Google Analytics
+    # and each account may have multiple profiles. You need the profile_id in order to get info from GA. If you
+    # don't know the profile_id then use this method to get a list of all them. Then set the profile_id of your
+    # instance and you can make regular calls from then on.
+    #
+    #   ga = Gattica.new({:email => 'johndoe@google.com', :password => 'password'})
+    #   ga.get_accounts
+    #   # you parse through the accounts to find the profile_id you need
+    #   ga.profile_id = 12345678
+    #   # now you can perform a regular search, see Gattica::Engine#get
+    #
+    # If you pass in a profile id when you instantiate Gattica::Search then you won't need to
+    # get the accounts and find a profile_id - you apparently already know it!
+    #
+    # See Gattica::Engine#get to see how to get some data.
+    
+    def accounts
+      # if we haven't retrieved the user's accounts yet, get them now and save them
+      if @user_accounts.nil?
+        data = do_http_get('/analytics/feeds/accounts/default')
+        xml = Hpricot(data)
+        @user_accounts = xml.search(:entry).collect { |entry| Account.new(entry) }
+      end
+      return @user_accounts
+    end
+
+    # Performs a Gattica::Engine#get but instead of returning the dataset streams it to the file handle in a CSV format
+    #
+    # == Usage
+    #
+    #   gs = Gattica.new({:email => 'johndoe@google.com', :password => 'password', :profile_id => 123456})
+    #   fh = File.new("file.csv", "w")
+    #   gs.get_to_csv({ :start_date => '2008-01-01', 
+    #            :end_date => '2008-02-01', 
+    #            :dimensions => 'browser', 
+    #            :metrics => 'pageviews', 
+    #            :sort => 'pageviews',
+    #            :filters => ['browser == Firefox']}, fh, :short)
+    #
+    # See Gattica::Engine#get to see details of arguments
+
+    def get_to_csv(args={}, fh = nil, format = :long)   
+      raise GatticaError::InvalidFileType, "Invalid file handle" unless !fh.nil?
+      results(args, fh, :csv, format)
+    end
+    
+    # This is the method that performs the actual request to get data.
+    #
+    # == Usage
+    #
+    #   gs = Gattica.new({:email => 'johndoe@google.com', :password => 'password', :profile_id => 123456})
+    #   gs.get({ :start_date => '2008-01-01', 
+    #            :end_date => '2008-02-01', 
+    #            :dimensions => 'browser', 
+    #            :metrics => 'pageviews', 
+    #            :sort => 'pageviews',
+    #            :filters => ['browser == Firefox']})
+    #
+    # == Input
+    #
+    # When calling +get+ you'll pass in a hash of options. For a description of what these mean to 
+    # Google Analytics, see http://code.google.com/apis/analytics/docs
+    #
+    # Required values are:
+    #
+    # * +start_date+ => Beginning of the date range to search within
+    # * +end_date+ => End of the date range to search within
+    #
+    # Optional values are:
+    #
+    # * +dimensions+ => an array of GA dimensions (without the ga: prefix)
+    # * +metrics+ => an array of GA metrics (without the ga: prefix)
+    # * +filter+ => an array of GA dimensions/metrics you want to filter by (without the ga: prefix)
+    # * +sort+ => an array of GA dimensions/metrics you want to sort by (without the ga: prefix)
+    # * +page+ => true|false Does the paging to create a single set of all of the data
+    # * +start_index+ => Beginning offset of the query (default 1)
+    # * +max_results+ => How many results to grab (maximum 10,000)
+    #
+    # == Exceptions
+    #
+    # If a user doesn't have access to the +profile_id+ you specified, you'll receive an error.
+    # Likewise, if you attempt to access a dimension or metric that doesn't exist, you'll get an
+    # error back from Google Analytics telling you so.
+    
+    def get(args={})
+      return results(args)
+    end
+    
+    private
+    
+    def results(args={}, fh=nil, type=nil, format=nil)
+      raise GatticaError::InvalidFileType, "Invalid file type" unless type.nil? ||[:csv,:xml].include?(type)
+      args = validate_and_clean(DEFAULT_ARGS.merge(args))
+
+      header = 0
+      results = nil
+      total_results = args[:max_results]
+      while(args[:start_index] < total_results)
+        query_string = build_query_string(args,@profile_id)
+        @logger.debug("Query String: " + query_string) if @debug
+
+        data = do_http_get("/analytics/feeds/data?#{query_string}")
+        result = DataSet.new(Hpricot.XML(data))
+        
+        #handle returning results
+        results.points.concat(result.points) if !results.nil? && fh.nil?
+        #handle csv
+        
+        if(!fh.nil? && type == :csv && header == 0)
+          fh.write result.to_csv_header(format)
+          header = 1 
+        end
+        
+        fh.write result.to_csv(:noheader) if !fh.nil? && type == :csv
+        fh.flush if !fh.nil?
+        
+        results = result if results.nil?
+        total_results = result.total_results
+        args[:start_index] += args[:max_results]
+        break if !args[:page] # only continue while if we are suppose to page
+      end 
+      return results if fh.nil?
+    end
+    
+    # Since google wants the token to appear in any HTTP call's header, we have to set that header
+    # again any time @token is changed so we override the default writer (note that you need to set
+    # @token with self.token= instead of @token=)
+    
+    def token=(token)
+      @token = token
+      set_http_headers
+    end
+    
+    
+    # Does the work of making HTTP calls and then going through a suite of tests on the response to make
+    # sure it's valid and not an error
+    
+    def do_http_get(query_string)
+      response, data = @http.get(query_string, @headers)
+      
+      # error checking
+      if response.code != '200'
+        case response.code
+        when '400'
+          raise GatticaError::AnalyticsError, response.body + " (status code: #{response.code})"
+        when '401'
+          raise GatticaError::InvalidToken, "Your authorization token is invalid or has expired (status code: #{response.code})"
+        else  # some other unknown error
+          raise GatticaError::UnknownAnalyticsError, response.body + " (status code: #{response.code})"
+        end
+      end
+      
+      return data
+    end
+    
+    private
+    
+    # Sets up the HTTP headers that Google expects (this is called any time @token is set either by Gattica
+    # or manually by the user since the header must include the token)
+    def set_http_headers
+      @headers['Authorization'] = "GoogleLogin auth=#{@token}"
+    end
+    
+    
+    # Creates a valid query string for GA
+    def build_query_string(args,profile)
+      query_params  = args.clone
+      ga_start_date = query_params.delete(:start_date)
+      ga_end_date   = query_params.delete(:end_date)
+      ga_dimensions = query_params.delete(:dimensions)
+      ga_metrics    = query_params.delete(:metrics)
+      ga_sort       = query_params.delete(:sort)
+      ga_filters    = query_params.delete(:filters)
+      
+      output = "ids=ga:#{profile}&start-date=#{ga_start_date}&end-date=#{ga_end_date}"
+      unless ga_dimensions.nil? || ga_dimensions.empty?
+        output += '&dimensions=' + ga_dimensions.collect do |dimension|
+          "ga:#{dimension}"
+        end.join(',')
+      end
+      unless ga_metrics.nil? || ga_metrics.empty?
+        output += '&metrics=' + ga_metrics.collect do |metric|
+          "ga:#{metric}"
+        end.join(',')
+      end
+      unless ga_sort.nil? || ga_sort.empty?
+        output += '&sort=' + Array(ga_sort).collect do |sort|
+          sort[0..0] == '-' ? "-ga:#{sort[1..-1]}" : "ga:#{sort}"  # if the first character is a dash, move it before the ga:
+        end.join(',')
+      end
+      
+      # TODO: update so that in regular expression filters (=~ and !~), any initial special characters in the regular expression aren't also picked up as part of the operator (doesn't cause a problem, but just feels dirty)
+      unless args[:filters].empty?    # filters are a little more complicated because they can have all kinds of modifiers
+        output += '&filters=' + args[:filters].collect do |filter|
+          match, name, operator, expression = *filter.match(/^(\w*)\s*([=!<>~@]*)\s*(.*)$/)           # splat the resulting Match object to pull out the parts automatically
+          unless name.empty? || operator.empty? || expression.empty?                      # make sure they all contain something
+            "ga:#{name}#{CGI::escape(operator.gsub(/ /,''))}#{CGI::escape(expression)}"   # remove any whitespace from the operator before output
+          else
+            raise GatticaError::InvalidFilter, "The filter '#{filter}' is invalid. Filters should look like 'browser == Firefox' or 'browser==Firefox'"
+          end
+        end.join(';')
+      end
+ 
+      query_params.inject(output) {|m,(key,value)| m << "&#{key}=#{value}"}
+ 
+      return output
+    end
+    
+    
+    # Validates that the args passed to +get+ are valid
+    def validate_and_clean(args)
+      
+      raise GatticaError::MissingStartDate, ':start_date is required' if args[:start_date].nil? || args[:start_date].empty?
+      raise GatticaError::MissingEndDate, ':end_date is required' if args[:end_date].nil? || args[:end_date].empty?
+      raise GatticaError::TooManyDimensions, 'You can only have a maximum of 7 dimensions' if args[:dimensions] && (args[:dimensions].is_a?(Array) && args[:dimensions].length > 7)
+      raise GatticaError::TooManyMetrics, 'You can only have a maximum of 10 metrics' if args[:metrics] && (args[:metrics].is_a?(Array) && args[:metrics].length > 10)
+      
+      possible = args[:dimensions] + args[:metrics]
+      
+      # make sure that the user is only trying to sort fields that they've previously included with dimensions and metrics
+      if args[:sort]
+        missing = args[:sort].find_all do |arg|
+          !possible.include? arg.gsub(/^-/,'')    # remove possible minuses from any sort params
+        end
+        unless missing.empty?
+          raise GatticaError::InvalidSort, "You are trying to sort by fields that are not in the available dimensions or metrics: #{missing.join(', ')}"
+        end
+      end
+      
+      # make sure that the user is only trying to filter fields that are in dimensions or metrics
+      if args[:filters]
+        missing = args[:filters].find_all do |arg|
+          !possible.include? arg.match(/^\w*/).to_s    # get the name of the filter and compare
+        end
+        unless missing.empty?
+          raise GatticaError::InvalidSort, "You are trying to filter by fields that are not in the available dimensions or metrics: #{missing.join(', ')}"
+        end
+      end
+      
+      return args
+    end
+    
+    
+  end
+end

data/lib/gattica/account.rb

@@ -0,0 +1,26 @@
+require 'rubygems'
+require 'hpricot'
+
+module Gattica
+  
+  # Represents an account that an authenticated user has access to
+  
+  class Account
+    
+    include Convertible
+    
+    attr_reader :id, :updated, :title, :table_id, :account_id, :account_name, :profile_id, :web_property_id
+  
+    def initialize(xml)
+      @id = xml.at(:id).inner_html
+      @updated = DateTime.parse(xml.at(:updated).inner_html)
+      @title = xml.at(:title).inner_html
+      @table_id = xml.at('dxp:tableid').inner_html
+      @account_id = xml.at("dxp:property[@name='ga:accountId']").attributes['value'].to_i
+      @account_name = xml.at("dxp:property[@name='ga:accountName']").attributes['value']
+      @profile_id = xml.at("dxp:property[@name='ga:profileId']").attributes['value'].to_i
+      @web_property_id = xml.at("dxp:property[@name='ga:webPropertyId']").attributes['value']
+    end
+    
+  end
+end

data/lib/gattica/auth.rb

@@ -0,0 +1,52 @@
+require 'net/http'
+require 'net/https'
+
+module Gattica
+  
+  # Authenticates a user against the Google Client Login system
+  
+  class Auth
+    
+    include Convertible
+
+    SCRIPT_NAME = '/accounts/ClientLogin'
+    HEADERS = { 'Content-Type' => 'application/x-www-form-urlencoded', 'User-Agent' => 'Ruby Net::HTTP' }   # Google asks that you be nice and provide a user-agent string
+    OPTIONS = { :source => 'gattica-'+VERSION, :service => 'analytics' }                                    # Google asks that you provide the name of your app as a 'source' parameter in your POST
+
+    attr_reader :tokens
+  
+    # Try to authenticate the user
+    def initialize(http, user)
+      options = OPTIONS.merge(user.to_h)
+      
+      response, data = http.post(SCRIPT_NAME, options.to_query, HEADERS)
+      if response.code != '200'
+        case response.code
+        when '403'
+          raise GatticaError::CouldNotAuthenticate, 'Your email and/or password is not recognized by the Google ClientLogin system (status code: 403)'
+        else
+          raise GatticaError::UnknownAnalyticsError, response.body + " (status code: #{response.code})"
+        end
+      end
+      @tokens = parse_tokens(data)
+    end
+  
+  
+    private
+    
+    # Parse the authentication tokens out of the response and makes them available as a hash
+    #
+    # tokens[:auth] => Google requires this for every request (added to HTTP headers on GET requests)
+    # tokens[:sid]  => Not used
+    # tokens[:lsid] => Not used
+    
+    def parse_tokens(data)
+      tokens = {}
+      data.split("\n").each do |t|
+        tokens.merge!({ t.split('=').first.downcase.to_sym => t.split('=').last })
+      end
+      return tokens
+    end
+  
+  end
+end

data/lib/gattica/convertible.rb

@@ -0,0 +1,39 @@
+module Gattica
+  
+  # Common output methods that are sharable
+  
+  module Convertible
+    
+    # output as hash
+    def to_h
+      output = {}
+      instance_variables.each do |var|
+        output.merge!({ var[1..-1] => instance_variable_get(var) }) unless var == '@xml'    # exclude the whole XML dump
+      end
+      output
+    end
+    
+    # output nice inspect syntax
+    def to_s
+      to_h.inspect
+    end
+    
+    alias inspect to_s
+    
+    def to_query
+      to_h.to_query
+    end
+    
+    # Return the raw XML (if the object has a @xml instance variable, otherwise convert the object itself to xml)
+    def to_xml
+      if @xml
+        @xml
+      else
+        self.to_xml
+      end
+    end
+    
+    alias to_yml to_yaml
+    
+  end
+end

data/lib/gattica/core_extensions.rb

@@ -0,0 +1,25 @@
+class Hash
+  
+  def to_query
+    require 'cgi' unless defined?(CGI) && defined?(CGI::escape)
+    self.collect do |key, value|
+      "#{CGI.escape(key.to_s)}=#{CGI.escape(value.to_s)}"
+    end.sort * '&'
+  end
+  
+  def key
+    self.keys.first if self.length == 1
+  end
+
+  def value
+    self.values.first if self.length == 1
+  end
+  
+  def stringify_keys
+    inject({}) do |options, (key, value)|
+      options[key.to_s] = value
+      options
+    end
+  end
+
+end

data/lib/gattica/data_point.rb

@@ -0,0 +1,60 @@
+require "csv"
+
+module Gattica
+  
+  # Represents a single "row" of data containing any number of dimensions, metrics
+  
+  class DataPoint
+    
+    include Convertible
+    
+    attr_reader :id, :updated, :title, :dimensions, :metrics, :xml
+    
+    # Parses the XML <entry> element
+    def initialize(xml)
+      @xml = xml.to_s
+      @id = xml.at('id').inner_html
+      @updated = DateTime.parse(xml.at('updated').inner_html)
+      @title = xml.at('title').inner_html
+      @dimensions = xml.search('dxp:dimension').collect do |dimension|
+        { dimension.attributes['name'].split(':').last.to_sym => dimension.attributes['value'].split(':', 1).last }
+      end
+      @metrics = xml.search('dxp:metric').collect do |metric|
+        { metric.attributes['name'].split(':').last.to_sym => metric.attributes['value'].split(':', 1).last.to_i }
+      end
+    end
+    
+    
+    # Outputs in Comma Seperated Values format
+    def to_csv(format = :long)
+      output = ''
+      
+      columns = []
+      # only output
+      case format
+      when :long
+        columns.concat([@id, @updated, @title])
+      end
+      
+      # output all dimensions
+      columns.concat(@dimensions.map {|d| d.value})
+      
+      # output all metrics
+      columns.concat(@metrics.map {|m| m.value})
+
+      output = CSV.generate_line(columns)      
+      return output
+    end
+    
+    
+    def to_yaml
+      { 'id' => @id,
+        'updated' => @updated,
+        'title' => @title,
+        'dimensions' => @dimensions,
+        'metrics' => @metrics }.to_yaml
+    end
+    
+  end
+  
+end

data/lib/gattica/data_set.rb

@@ -0,0 +1,76 @@
+module Gattica
+  
+  # Encapsulates the data returned by the GA API
+  
+  class DataSet
+    
+    include Convertible
+    
+    attr_reader :total_results, :start_index, :items_per_page, :start_date, :end_date, :points, :xml
+      
+    def initialize(xml)
+      @xml = xml.to_s
+      @total_results = xml.at('openSearch:totalResults').inner_html.to_i
+      @start_index = xml.at('openSearch:startIndex').inner_html.to_i
+      @items_per_page = xml.at('openSearch:itemsPerPage').inner_html.to_i
+      @start_date = Date.parse(xml.at('dxp:startDate').inner_html)
+      @end_date = Date.parse(xml.at('dxp:endDate').inner_html)
+      @points = xml.search(:entry).collect { |entry| DataPoint.new(entry) }
+    end
+    
+    def to_csv_header(format = :long)
+      # build the headers
+      output = ''
+      columns = []
+
+      # only show the nitty gritty details of id, updated_at and title if requested
+      case format #it would be nice if case statements in ruby worked differently
+      when :long
+        columns.concat(["id", "updated", "title"])
+        unless @points.empty?   # if there was at least one result
+          columns.concat(@points.first.dimensions.map {|d| d.key})
+          columns.concat(@points.first.metrics.map {|m| m.key})
+        end    
+      when :short
+        unless @points.empty?   # if there was at least one result
+          columns.concat(@points.first.dimensions.map {|d| d.key})
+          columns.concat(@points.first.metrics.map {|m| m.key})
+        end    
+      when :noheader
+      end
+      
+      output = CSV.generate_line(columns) + "\n" if (columns.size > 0) 
+
+      return output    
+    end
+    
+    # output important data to CSV, ignoring all the specific data about this dataset 
+    # (total_results, start_date) and just output the data from the points
+    
+    def to_csv(format = :long)
+      output = ''
+      
+      # build the headers
+      output = to_csv_header(format)
+
+      # get the data from each point
+      @points.each do |point|
+        output += point.to_csv(format) + "\n"
+      end
+      
+      return output
+    end
+    
+    
+    def to_yaml
+      { 'total_results' => @total_results,
+        'start_index' => @start_index,
+        'items_per_page' => @items_per_page,
+        'start_date' => @start_date,
+        'end_date' => @end_date,
+        'points' => @points}.to_yaml
+    end
+    
+  end
+  
+end

data/lib/gattica/exceptions.rb

@@ -0,0 +1,23 @@
+module GatticaError
+  # usage errors
+  class InvalidFileType < StandardError; end;
+  # user errors
+  class InvalidEmail < StandardError; end;
+  class InvalidPassword < StandardError; end;
+  # authentication errors
+  class CouldNotAuthenticate < StandardError; end;
+  class NoLoginOrToken < StandardError; end;
+  class InvalidToken < StandardError; end;
+  # profile errors
+  class InvalidProfileId < StandardError; end;
+  # search errors
+  class TooManyDimensions < StandardError; end;
+  class TooManyMetrics < StandardError; end;
+  class InvalidSort < StandardError; end;
+  class InvalidFilter < StandardError; end;
+  class MissingStartDate < StandardError; end;
+  class MissingEndDate < StandardError; end;
+  # errors from Analytics
+  class AnalyticsError < StandardError; end;
+  class UnknownAnalyticsError < StandardError; end;
+end

data/lib/gattica/user.rb

@@ -0,0 +1,31 @@
+module Gattica
+  
+  # Represents a user to be authenticated by GA
+  
+  class User
+  
+    include Convertible
+  
+    attr_accessor :email, :password
+  
+    def initialize(email,password)
+      @email = email
+      @password = password
+      validate
+    end
+    
+    # User gets a special +to_h+ because Google expects +Email+ and +Passwd+ instead of our nicer internal names
+    def to_h
+      { :Email => @email,
+        :Passwd => @password }
+    end
+    
+    private
+    # Determine whether or not this is a valid user
+    def validate
+      raise GatticaError::InvalidEmail, "The email address '#{@email}' is not valid" if not @email.match(/^(?:[_a-z0-9-]+)(\.[_a-z0-9-]+)*@([a-z0-9-]+)(\.[a-zA-Z0-9\-\.]+)*(\.[a-z]{2,4})$/i)
+      raise GatticaError::InvalidPassword, "The password cannot be blank" if @password.empty? || @password.nil?
+    end
+  
+  end
+end

data/test/helper.rb

@@ -0,0 +1,15 @@
+require File.join(File.dirname(__FILE__), *%w[.. lib gattica])
+ 
+require 'rubygems'
+require 'test/unit'
+require 'mocha'
+ 
+# include Gattica
+ 
+def fixture(name)
+  File.read(File.join(File.dirname(__FILE__), 'fixtures', name))
+end
+ 
+def absolute_project_path
+  File.expand_path(File.join(File.dirname(__FILE__), '..'))
+end

data/test/suite.rb

@@ -0,0 +1,6 @@
+require 'test/unit'
+ 
+tests = Dir["#{File.dirname(__FILE__)}/test_*.rb"]
+tests.each do |file|
+  require file
+end

data/test/test_auth.rb

@@ -0,0 +1,12 @@
+require File.dirname(__FILE__) + '/helper'
+ 
+class TestAuth < Test::Unit::TestCase
+  def setup
+    
+  end
+  
+  def test_truth
+    assert true
+  end
+  
+end

data/test/test_engine.rb

@@ -0,0 +1,14 @@
+require File.dirname(__FILE__) + '/helper'
+ 
+class TestEngine < Test::Unit::TestCase
+  def setup
+    
+  end
+  
+  def test_initialization
+    # assert Gattica.new({:email => 'anonymous@anon.com', :password => 'none'}) # you can initialize with a potentially invalid email and password
+    assert Gattica.new({:token => 'test'})                                    # you can initialize with a potentially invalid token
+    assert_raise GatticaError::NoLoginOrToken do Gattica.new() end            # but, you must initialize with one or the other
+  end
+  
+end

data/test/test_gattica.rb

@@ -0,0 +1,16 @@
+require File.dirname(__FILE__) + '/helper'
+ 
+class TestUser < Test::Unit::TestCase
+  def test_build_query_string
+    @gattica = Gattica.new(:token => 'ga-token', :profile_id => 'ga-profile_id')
+    expected = "ids=ga:ga-profile_id&start-date=2008-01-02&end-date=2008-01-03&dimensions=ga:pageTitle,ga:pagePath&metrics=ga:pageviews&sort=-ga:pageviews&max-results=3"
+    result = @gattica.send(:build_query_string, {
+      :start_date => Date.civil(2008,1,2), 
+      :end_date => Date.civil(2008,1,3),
+      :dimensions => ['pageTitle','pagePath'], 
+      :metrics => ['pageviews'], 
+      :sort => '-pageviews',
+      'max-results' => '3'}, 'ga-profile_id')
+    assert_equal expected, result
+  end
+end

data/test/test_user.rb

@@ -0,0 +1,24 @@
+require File.dirname(__FILE__) + '/helper'
+ 
+class TestUser < Test::Unit::TestCase
+  def setup
+    
+  end
+  
+  def test_can_create_user
+    assert Gattica::User.new('anonymous@anon.com','none')
+  end
+  
+  def test_invalid_email
+    assert_raise GatticaError::InvalidEmail do Gattica::User.new('','') end
+    assert_raise ArgumentError do Gattica::User.new('') end
+    assert_raise GatticaError::InvalidEmail do Gattica::User.new('anonymous','none') end
+    assert_raise GatticaError::InvalidEmail do Gattica::User.new('anonymous@asdfcom','none') end
+  end
+    
+  def test_invalid_password
+    assert_raise GatticaError::InvalidPassword do Gattica::User.new('anonymous@anon.com','') end
+    assert_raise ArgumentError do Gattica::User.new('anonymous@anon.com') end
+  end
+  
+end

metadata

@@ -0,0 +1,93 @@
+--- !ruby/object:Gem::Specification 
+name: gattica
+version: !ruby/object:Gem::Version 
+  version: 0.4.0
+platform: ruby
+authors: 
+- Rob Cameron
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2009-09-04 00:00:00 -07:00
+default_executable: 
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: hpricot
+  type: :runtime
+  version_requirement: 
+  version_requirements: !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        version: 0.6.164
+    version: 
+description: Gattica is a Ruby library for extracting data from the Google Analytics API.
+email: cannikinn@gmail.com
+executables: []
+
+extensions: []
+
+extra_rdoc_files: 
+- LICENSE
+- README.rdoc
+files: 
+- .gitignore
+- History.txt
+- LICENSE
+- README.rdoc
+- Rakefile
+- VERSION.yml
+- examples/example.rb
+- gattica.gemspec
+- lib/gattica.rb
+- lib/gattica/account.rb
+- lib/gattica/auth.rb
+- lib/gattica/convertible.rb
+- lib/gattica/core_extensions.rb
+- lib/gattica/data_point.rb
+- lib/gattica/data_set.rb
+- lib/gattica/exceptions.rb
+- lib/gattica/user.rb
+- test/helper.rb
+- test/suite.rb
+- test/test_auth.rb
+- test/test_engine.rb
+- test/test_gattica.rb
+- test/test_user.rb
+has_rdoc: true
+homepage: http://github.com/cannikin/gattica
+licenses: []
+
+post_install_message: 
+rdoc_options: 
+- --charset=UTF-8
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+  version: 
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+  version: 
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.3.5
+signing_key: 
+specification_version: 2
+summary: Gattica is a Ruby library for extracting data from the Google Analytics API.
+test_files: 
+- test/helper.rb
+- test/suite.rb
+- test/test_auth.rb
+- test/test_engine.rb
+- test/test_gattica.rb
+- test/test_user.rb
+- examples/example.rb