mongoid_traffic 0.2.5 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a609625a589b09ecf9ddde37e133c90407a3824c3c52123ed323d985394136c6
-  data.tar.gz: 5f04c2ccf598f18f955865422ebc7f2d554a8cd4e4ac0b3db59bec92c7459235
+  metadata.gz: e2e697436b535a20d799099b53576a88bf7e0145ca1110010b7b27b05cab2f4d
+  data.tar.gz: d3d50e3806776e41747c2a46abe7d38da795cc8bc961794f3a273acaea0b5510
 SHA512:
-  metadata.gz: 8754362224ed919e02b548a248ec046538b4707f0e96be418a740ecd51dc24339b9221c3acba0b6dd8caa5c3e202bd953d42c4a7cd1410bd04c31ec1c508e7c9
-  data.tar.gz: ca9d9b15acb37b7897276afccf95886e06428bc43294751111e2e50169a681f58031e42926699151753f9e625a3ade8e0bab947cb98c4e3d47ce1549ceed1a05
+  metadata.gz: dddd2282c221bd0a955ed6ffd23e0c886a0a1c79119fe8b3106bc55b9714e7330f62399c07af92fea2cd88aa85dfbcbab6ceaa16224ba8cbb8ee9ab004e85a17
+  data.tar.gz: 1a86c94669b57cfa3a08b20658ec456fa3623c1d1a45c172929253a333d98a4b812a861fa66ecf8e2ee7c717194bbb6ff6ef054c78f41936b38fb7e678a53b16

data/CHANGELOG.md

@@ -1,5 +1,15 @@
 # CHANGELOG
 
+## 1.0.0
+
+* BREAKING: refactored and abstracted for more general purpose
+  * removed bot detection
+  * removed user_agent
+  * removed geo_ip
+  * removed referer
+  * removed rails controller additions
+  * removed scope
+
 ## 0.2.5
 
 * Mongoid 7 compatibility

data/README.md

@@ -30,154 +30,140 @@ Setup your class for storing the log:
 
 ```ruby
 class MyLog
-	include Mongoid::Document
-	include MongoidTraffic::Log
+  include Mongoid::Document
+  include MongoidTraffic::Log
 end
 ```
 
 Log your traffic like this:
 
 ```ruby
-MongoidTraffic::Logger.log(MyLog, *args)
+MongoidTraffic::Logger.log(MyLog)
 ```
 
 Or, if you prefer, directly on the `MyLog` class:
 
 ```ruby
-MyLog.log(*args)
+MyLog.log
 ```
 
-This will (by default) create two `MyLog` documents: first with `:df(date_from)` and `:dt(date_to)` fields specifying monthly log, second with dates specifying daily log. Each log has an `:access_count` attribute that is incremented with subsequent `.log` calls.
+This will (by default) create four `MyLog` documents: each with `:df(date_from)` and `:dt(date_to)` fields specifying yearly, monthly, weekly and daily log. Each log has an `:access_count` attribute incremented with subsequent `.log` calls.
 
 ### Optional arguments
 
 #### Time scope:
 
-By default, the `.log` method creates/updates a document with aggregations for month and a document with aggregations for a day. You can however customize this behaviour like this:
+By default, the `.log` method creates/updates a document with aggregations for year, month, week and day. Subset of those  can be specified as:
 
 ```ruby
-MyLog.log(time_scope: %i(month week day))
+MyLog.log(time_scope: %i[day])
 ```
 
-The available options are: `%(year month week day)`
+The available options are: `%i[year month week day]`
 
 #### Scope:
 
-```ruby
-MyLog.log(scope: '/pages/123')
-```
-
-Allows to create several logs for different scopes of your application (typically URLs).
-
-#### User-Agent:
+It is possible to scope the log by an arbitrary number of parameters.
 
 ```ruby
-MyLog.log(user_agent: user_agent_string)
-```
+class MyLog
+  include Mongoid::Document
+  include MongoidTraffic::Log
 
-Logs platform-browser-version access count:
+  belongs_to :page
 
-```ruby
-{ "Macintosh" => { "Safari" => { "8%2E0" => 1, "7%2E1" => 2 } } }
+  scope :for_page, -> (page) { where(page: page) }
+end
 ```
 
-Please note the keys are escaped. You might want to unescape them using for example `CGI::unescape`.
-
-#### Referer:
+For example:
 
 ```ruby
-MyLog.log(referer: http_referer_string)
+MongoidTraffic::Logger.log(MyLog.for_page(page))
 ```
 
-Logs referer access count:
+or
 
 ```ruby
-{ "http%3A%2F%2Fwww%2Egoogle%2Ecom" => 1 }
+MyLog.for_page(page).log(page: my_page)
 ```
 
-Please note the keys are escaped. You might want to unescape them using for example `CGI::unescape`.
+PS If you query by a scope often do not forget to add corresponding indexes to your log model.
 
-If the referer is included in the [bot list](http://www.user-agents.org/allagents.xml) the log will not be created.
+#### Arbitrary counter:
 
-#### Country (via IP address):
-
-```ruby
-MyLog.log(ip_address: '123.123.123.123')
-```
+It is possible to count any number of arbitrary additional values. For example, to count unique country etc.
 
-Logs access count by country code 2:
+First, specify the additional counter in the log model (prefer short aliased field names):
 
 ```ruby
-{ "CZ" => 100, "DE" => 1 }
-```
-
-Uses the [GeoIP](https://github.com/cjheath/geoip) library to infer country_code from IP address.
-
-You can use the [countries gem](https://github.com/hexorx/countries) to convert the country code to country name etc.
-
-#### Unique id:
+class MyLog
+  include Mongoid::Document
+  include MongoidTraffic::Log
 
-```ruby
-MyLog.log(unique_id: unique_id_string)
+  additional_counter :c, as: :country
+end
 ```
 
-Logs access count by id:
+Track access by country as:
 
 ```ruby
-{ "0123456789" => 100, "ABCDEFGHIJ" => 1 }
+MyLog.log(country: 'CZ')
+MyLog.log(country: 'NL')
 ```
 
-Typically you would pass it something like `session_id` to track unique visitors.
-
-## Rails
-
-In case of Rails, you can use the `.after_action` macro with the `#log_traffic` helper method in your controllers:
+Which will create Hash with counts per country:
 
 ```ruby
-class MyController < ApplicationController
-	after_action -> { log_traffic(MyLog) }, only: [:show]
-end
+{ _id: …, df(date_from): …, dt(date_to): …, ac(access_count): 2, c(country): { 'CZ' => 1, 'NL' => 1 } }
 ```
 
-The method automatically infers most of the options from the controller `request` method (User-Agent, Referer, IP address) and unique id from the Rails session.
-
-Additionally the `:log_scoped_traffic` method adds a scope by the current request path (`/pages/123`):
+You can take advantage of the fact, that the underlying queries support dot-notation, and track on deeply nested hashes. For example, should you want to track access per browser:
 
 ```ruby
-class MyController < ApplicationController
-	after_action -> { log_scoped_traffic(MyLog) }, only: [:show]
+class MyLog
+  include Mongoid::Document
+  include MongoidTraffic::Log
+
+  additional_counter :b, as: :browser
 end
 ```
 
-You can override this behavior with custom scope like this:
+Then track access by browser as:
 
 ```ruby
-class MyController < ApplicationController
-	after_action -> { log_scoped_traffic(MyLog, scope: 'my-scope-comes-here') }, only: [:show]
-end
+MyLog.log(browser: "Mac.Safari.8") # log access by Mac Safari 8
+MyLog.log(browser: "Mac.Safari.7%2E1") # log access by Mac Safari 7.1
 ```
 
-It might be good idea to use both methods in order to log access to the whole site as well as access to individual pages:
+Which will create following log document:
 
 ```ruby
-class MyController < ApplicationController
-	after_action -> { log_traffic(MyLog) }, only: [:show]
-	after_action -> { log_scoped_traffic(MyLog) }, only: [:show]
-end
+{ _id: …, df(date_from): …, dt(date_to): …, ac(access_count): 2, b(browser): { 'Mac' => { 'Safari' => { '8' => 1, '7%2E1' => 1 } } } }
 ```
 
+Please note all `.` not intended to denote nesting need to be escaped (here as `%2E`).
+
 ## Accessing the log
 
-The log is accessed with a combination of Mongoid Criteria and aggregation methods. 
+The log can be accessed using a combination of Mongoid Criteria and aggregation methods.
 
 ### Criteria
 
 The following time based criteria are predefined as Mongoid scopes:
 
-* `.yearly(year)`
-* `.monthly(month, year)`
-* `.weekly(week, year)`
-* `.daily(date)`
+* `.day(date)`
+* `.week(week, year)`
+* `.month(month, year)`
+* `.year(year)`
+* `.for_dates(date_from, date_to)`
+
+To select by log type:
+
+* `.daily`
+* `.weekly`
+* `.monthly`
+* `.yearly`
 
 To narrow down by scope:
 
@@ -186,48 +172,59 @@ To narrow down by scope:
 ### Aggregation method
 
 * `.aggregate_on(:access_count)`
-* `.aggregate_on(:browsers)`
-* `.aggregate_on(:referers)`
-* `.aggregate_on(:countries)`
-* `.aggregate_on(:unique_ids)`
+* `.aggregate_on(ARBITRARY_COUNTER)`
 
 Behind the scenes, this method will take all documents returned by your criteria and combines the values of the specified field (in case of `:access_count` it is simple sum of the values, in other cases it is sum of nested hashes).
 
-### Unique visits
-
-Lastly, to retrieve the number of unique visits:
+### Examples
 
-* `.sum(:unique_ids)`
+#### Time
 
-### Examples
+```ruby
+MyLog.day(Date.today)
+```
 
-Typically you first query by time:
+Eventually by date range (when using the `.for_dates` scope make sure to specify which log type you wish to access):
 
 ```ruby
-MyLog.daily(Date.today)
+MyLog.daily.for_dates(Date.yesterday, Date.today)
 ```
 
-And eventually by scope:
+#### Scope
 
 ```ruby
-MyLog.daily(Date.today).scoped_to('/pages/123')
+MyLog.for_page(page).day(Date.today)
 ```
 
-Followed by an aggregation. For example on access count:
+Make sure that the order of drilling down corresponds to the indexes on your model.
+
+#### Aggregations
+
+On access count:
 
 ```ruby
-MyLog.daily(Date.today).scoped_to('/pages/123').aggregate_on(:access_count)
+MyLog.day(Date.today).scoped_to('/pages/123').aggregate_on(:access_count) # => 1
 ```
 
 The scope query accepts regular expressions, which allows for aggregations on specific parts of your site. For example should you want to query for all pages that have path beginning with '/blog':
 
 ```ruby
-MyLog.monthly(8, 2014).scoped_to(/\A\/blog/).aggregate_on(:countries)
+MyLog.month(8, 2014).scoped_to(/\A\/blog/).aggregate_on(:access_count) # => 1
 ```
 
-## TODO
+On additional counter:
 
-* JavaScript + Rails helper to track client-side properties (screen/browser size)
+```ruby
+MyLog.day(Date.today).scoped_to('/pages/123').aggregate_on(:country) # => { 'CZ' => 1, 'NL' => 1 }
+```
+
+#### Pluck
+
+For plotting charts you might make use of standard `:pluck`:
+
+```ruby
+MyLog.daily.for_dates(Date.today - 1.week, Date.today).pluck(:date_from, :access_count) # => returns array of dates and counts per day
+```
 
 ## Further reading
 

data/Rakefile

@@ -1,26 +1,9 @@
 require 'bundler/gem_tasks'
 require 'rake/testtask'
- 
+
 Rake::TestTask.new do |t|
   t.libs << 'test'
   t.pattern = 'test/**/*_test.rb'
 end
  
-namespace :mongoid_traffic do
-  require File.dirname(__FILE__) + "/lib/mongoid_traffic/logger/bots"
-  require File.dirname(__FILE__) + "/lib/mongoid_traffic/logger/geo_ip"
-
-  desc "output the list of bots"
-  task :update_bots_data do
-    `cd vendor/mongoid_traffic && curl -O #{MongoidTraffic::Logger::Bots::DATA_URL}`
-  end
-
-  desc "output the geoip.dat"
-  task :update_geoip_data do
-    `cd vendor/mongoid_traffic && curl -O #{MongoidTraffic::Logger::GeoIp::DATA_URL}`
-    `cd vendor/mongoid_traffic && gunzip -f GeoIP.dat.gz`
-  end
-
-end
-
-task :default => :test
+task :default => :test

data/lib/mongoid_traffic.rb

@@ -1,5 +1,8 @@
 require 'mongoid_traffic/version'
 
-require 'mongoid_traffic/controller_additions'
 require 'mongoid_traffic/log'
 require 'mongoid_traffic/logger'
+
+module MongoidTraffic
+  TIME_SCOPE_OPTIONS = { year: 3, month: 2, week: 1, day: 0 }.freeze
+end

data/lib/mongoid_traffic/log.rb

@@ -1,83 +1,63 @@
-require 'logger'
-
 module MongoidTraffic
   module Log
     def self.included(base)
       base.extend ClassMethods
       base.class_eval do
-        field :s, as: :scope, type: String
-
-        field :ac, as: :access_count, type: Integer
-
-        field :b, as: :browsers, type: Hash, default: {}
-        field :c, as: :countries, type: Hash, default: {}
-        field :r, as: :referers, type: Hash, default: {}
-        field :u, as: :unique_ids, type: Hash, default: {}
-
-        field :uat, as: :updated_at, type: Time
-
-        # ---------------------------------------------------------------------
-
         field :df, as: :date_from, type: Date
         field :dt, as: :date_to, type: Date
+        field :ts, as: :time_scope, type: Integer
 
-        # ---------------------------------------------------------------------
+        field :ac, as: :access_count, type: Integer
+        field :uat, as: :updated_at, type: Time
 
         validates :date_from, presence: true
         validates :date_to, presence: true
 
-        # ---------------------------------------------------------------------
+        scope :yearly, -> { where(ts: TIME_SCOPE_OPTIONS[:year]) }
+        scope :monthly, -> { where(ts: TIME_SCOPE_OPTIONS[:month]) }
+        scope :weekly, -> { where(ts: TIME_SCOPE_OPTIONS[:year]) }
+        scope :daily, -> { where(ts: TIME_SCOPE_OPTIONS[:day]) }
 
-        default_scope -> { where(scope: nil) }
+        scope :for_dates, ->(date_from, date_to) { where(:date_from.lte => date_to, :date_to.gte => date_from) }
+        scope :year, ->(year) { yearly.for_dates(Date.parse("01/01/#{year}"), Date.parse("01/01/#{year}").at_end_of_year) }
+        scope :month, ->(month, year) { monthly.for_dates(Date.parse("01/#{month}/#{year}"), Date.parse("01/#{month}/#{year}").at_end_of_month) }
+        scope :week, ->(week, year) { weekly.for_dates(Date.commercial(year, week), Date.commercial(year, week).at_end_of_week) }
+        scope :day, ->(date) { daily.for_dates(date, date) }
 
-        scope :for_dates, -> (date_from, date_to) { where(date_from: date_from, date_to: date_to) }
-
-        scope :yearly, -> (year) { for_dates(Date.parse("01/01/#{year}"), Date.parse("01/01/#{year}").at_end_of_year) }
-        scope :monthly, -> (month, year) { for_dates(Date.parse("01/#{month}/#{year}"), Date.parse("01/#{month}/#{year}").at_end_of_month) }
-        scope :weekly, -> (week, year) { for_dates(Date.commercial(year, week), Date.commercial(year, week).at_end_of_week) }
-        scope :daily, -> (date) { for_dates(date, date) }
-
-        scope :scoped_to, -> (scope) { where(scope: scope) }
-
-        # ---------------------------------------------------------------------
-
-        index(scope: 1, date_from: 1, date_to: 1)
+        index time_scope: 1, date_from: 1, date_to: 1
       end
     end
 
-    # =====================================================================
-
     module ClassMethods
       def log(*args)
         MongoidTraffic::Logger.log(self, *args)
       end
 
-      def aggregate_on(att)
-        case find_field_by_name(att).type.to_s
-        when 'Integer' then sum(att)
-        when 'Hash' then sum_hash(att)
-        end
+      def additional_counter(name, as: nil)
+        field name, as: as, type: Hash, default: {}
       end
 
-      def sum(att)
-        if att.to_sym == :unique_ids
-          aggregate_on(:unique_ids).keys.count
-        else
-          super(att)
+      def aggregate_on(att)
+        field = find_field(att)
+        case field.type.to_s
+        when 'Integer' then sum(field.name)
+        when 'Hash' then sum_hash(field.name)
         end
       end
 
-      def find_field_by_name(field_name)
-        return unless f = fields.detect { |k, v| k == field_name.to_s || v.options[:as].to_s == field_name.to_s }
-        f.last
+      private
+
+      def find_field(name)
+        fields.detect do |field_name, field|
+          field_name == name.to_s || field.options[:as].to_s == name.to_s
+        end.try(:last)
       end
 
       def sum_hash(field_name)
         pluck(field_name).inject({}) do |res, h|
-          merger = proc do |_key, v1, v2|
-            case
-            when Hash === v1 && Hash === v2 then v1.merge(v2, &merger)
-            when Hash === v2 then v2
+          merger = proc do |_, v1, v2|
+            if Hash === v1 && Hash === v2 then v1.merge(v2, &merger)
+            elsif Hash === v2 then v2
             else v1.to_i + v2.to_i
             end
           end