sleek 0.0.1 → 0.0.2

data/README.md

@@ -1,10 +1,12 @@
 ![Sleek](sleek.png)
 
-[![Build Status](https://travis-ci.org/goshakkk/sleek.png)](https://travis-ci.org/goshakkk/sleek)
+[![Build Status](https://travis-ci.org/goshakkk/sleek.png)](https://travis-ci.org/goshakkk/sleek) [![Code Climate](https://codeclimate.com/github/goshakkk/sleek.png)](https://codeclimate.com/github/goshakkk/sleek)
 
 Sleek is a gem for doing analytics. It allows you to easily collect and
 analyze events that happen in your app.
 
+**Sleek is a work-in-progress development. Use with caution.**
+
 ## Installation
 
 The easiest way to install Sleek is to add it to your Gemfile:
@@ -13,15 +15,27 @@ The easiest way to install Sleek is to add it to your Gemfile:
 gem "sleek"
 ```
 
-Then, install it:
+Or, if you want the latest hotness:
 
+```ruby
+gem "sleek", github: "goshakkk/sleek"
 ```
+
+Then, install it:
+
+```bash
 $ bundle install
 ```
 
 Sleek requires MongoDB to work and assumes that you have Mongoid
 configured already.
 
+Finally, create needed indexes:
+
+```bash
+$ rake db:mongoid:create_indexes
+```
+
 ## Getting started
 
 ### Namespacing
@@ -98,8 +112,8 @@ call. Using `:interval` also requires that you specify `:timeframe`.
 ```ruby
 sleek.queries.count(:purchases, timeframe: :this_2_days, interval: :daily)
 # => [
-#      {:timeframe=>#<Sleek::Timeframe 2013-01-01 00:00:00 UTC..2013-01-02 00:00:00 UTC>, :value=>10},
-#      {:timeframe=>#<Sleek::Timeframe 2013-01-02 00:00:00 UTC..2013-01-03 00:00:00 UTC>, :value=>24}
+#      {:timeframe=>2013-01-01 00:00:00 UTC..2013-01-02 00:00:00 UTC, :value=>10},
+#      {:timeframe=>2013-01-02 00:00:00 UTC..2013-01-03 00:00:00 UTC, :value=>24}
 #    ]
 ```
 
@@ -137,8 +151,8 @@ sleek.queries.count_unique(:purchases, target_property: "customer.id")
 ### Minimum
 
 It finds the minimum numeric value for a given property. All non-numeric
-values are ignored. If none of property values are numeric, the
-exception will be raised.
+values are ignored. If none of property values are numeric, nil will
+be returned.
 
 ```ruby
 sleek.queries.minimum(:bucket, params)
@@ -154,8 +168,8 @@ sleek.queries.minimum(:purchases, target_property: "total")
 ### Maximum
 
 It finds the maximum numeric value for a given property. All non-numeric
-values are ignored. If none of property values are numeric, the
-exception will be raised.
+values are ignored. If none of property values are numeric, nill will
+be returned.
 
 ```ruby
 sleek.queries.maximum(:bucket, params)
@@ -172,7 +186,7 @@ sleek.queries.maximum(:purchases, target_property: "total")
 
 The average query finds the average value for a given property.  All
 non-numeric values are ignored. If none of property values are numeric,
-the exception will be raised.
+nil will be returned.
 
 ```ruby
 sleek.queries.average(:bucket, params)
@@ -189,7 +203,7 @@ sleek.queries.average(:purchases, target_property: "total")
 
 The sum query sums all the numeric values for a given property. All
 non-numeric values are ignored. If none of property values are numeric,
-the exception will be raised.
+nil will be returned.
 
 ```ruby
 sleek.queries.sum(:bucket, params)
@@ -202,6 +216,54 @@ sleek.queries.sum(:purchases, target_property: "total")
 # => 2_072_70
 ```
 
+## Series
+
+Series allow you to analyze trends in metrics over time. They break a
+timeframe into intervals and compute the metric for those intervals.
+
+Calculating series is simply done by adding the `:timeframe` and
+`:interval` options to the metric query.
+
+Valid intervals are:
+
+* `:hourly`
+* `:daily`
+* `:weekly`
+* `:monthly`
+
+## Group by
+
+In addition to using metrics and series, it is sometimes desired to
+group their outputs by a specific property value.
+
+For example, you might be wondering, "How much have me made from each of
+our customers?" Group by will help you answer questions like this.
+
+To group metrics or series result by value of some property, all you
+need to do is to pass the `:group_by` option to the query.
+
+```ruby
+sleek.queries.sum(:purchases, target_property: "total", group_by: "customer.email")
+# => {"first@another.com"=>214998, "first@last.com"=>64999}
+```
+
+Or, you may wonder how much did you make from each of your customers for
+every day of this week.
+
+```ruby
+sleek.queries.sum(:purchases, target_property: "total", timeframe: :this_week,
+  interval: :daily, group_by: "customer.email")
+```
+
+You can even combine it with filters. For example, how much did you make
+from each of your customers for evey day of this weeks on orders greater
+than $1000?
+
+```ruby
+sleek.queries.sum(:purchases, target_property: "total", filter: ["total", :gte, 1000_00],
+  timeframe: :this_week, interval: :daily, group_by: "customer.email")
+```
+
 ## Filters
 
 To limit the scope of events used in analysis you can use a filter. To
@@ -222,23 +284,36 @@ sleek.queries.count(:purchases, filters: [:total, :gt, 1599])
 # => 20
 ```
 
-## Series
+### Timeframe & timezone
 
-Series allow you to analyze trends in metrics over time. They break a
-timeframe into intervals and compute the metric for those intervals.
+You can pass the `:timeframe` with or without `:timezone` to any query.
 
-Calculating series is simply done by adding the `:timeframe` and
-`:interval` options to the metric query.
+Timeframe is used to limit your query by some window of time. You can
+use a range of `TimeWithRange` objects to specify absolute timeframe, or
+you can use a string that describes relative timeframe.
 
-Valid intervals are:
+Relative timeframe string (or a symbol) consists of these parts: category,
+optional number, and interval specification. Possible categories are `this`
+and `previous`, possible intervals are `minute`, `hour`, `day`, `week`,
+`month`.
 
-* `:hourly`
-* `:daily`
-* `:weekly`
-* `:monthly`
+Examples: `this_day`, `previous_3_weeks`.
+
+By default, relative times are transformed into ranges of time objects
+in UTC timezone. You can, however, pass the `:timezone` option to tell
+Sleek to construct the window of time in the given timezone.
+
+Refer to [`ActiveSupport::TimeZone` docs](http://api.rubyonrails.org/classes/ActiveSupport/TimeZone.html)
+for more details on possible timezone identifiers.
 
 ## Other
 
+### Deleting namespace
+
+```ruby
+sleek.delete!
+```
+
 ### Deleting buckets
 
 ```ruby

data/lib/sleek.rb

@@ -7,14 +7,16 @@ require 'sleek/version'
 require 'sleek/timeframe'
 require 'sleek/interval'
 require 'sleek/filter'
+require 'sleek/group_by_criteria'
 require 'sleek/event'
 require 'sleek/queries'
+require 'sleek/query_command'
 require 'sleek/query_collection'
-require 'sleek/base'
+require 'sleek/namespace'
 
 module Sleek
   def self.for_namespace(namespace)
-    Base.new namespace
+    Namespace.new namespace
   end
 
   def self.[](namespace)

data/lib/sleek/core_ext/range.rb

@@ -5,8 +5,9 @@ class Range
   end
 
   # Public: Convert both ends of range to times.
-  def to_time_range
-    Time.at(self.begin)..Time.at(self.end)
+  def to_time_range(zone = nil)
+    time = zone ? zone : Time
+    time.at(self.begin)..time.at(self.end)
   end
 
   def int_range?
@@ -33,12 +34,10 @@ class Range
   #   (1200..1300).previous
   #   # => 1100..1200
   def previous(n = 1)
-    new_begin = self.begin - difference * n
-    new_end = self.end - difference * n
-    new_begin..new_end
+    self - difference * n
   end
 
-  def -(what)
-    (self.begin - what)..(self.end - what)
+  def -(other)
+    (self.begin - other)..(self.end - other)
   end
 end

data/lib/sleek/event.rb

@@ -19,7 +19,8 @@ module Sleek
     field :ns, type: Symbol, as: :namespace
     field :b, type: String, as: :bucket
     field :d, type: Hash, as: :data
-    embeds_one :sleek, store_as: "s", class_name: 'Sleek::EventMetadata', cascade_callbacks: true
+    embeds_one :sleek, store_as: 's', class_name: 'Sleek::EventMetadata',
+      cascade_callbacks: true
     accepts_nested_attributes_for :sleek
 
     validates :namespace, presence: true
@@ -27,6 +28,8 @@ module Sleek
 
     after_initialize { build_sleek }
 
+    index ns: 1, b: 1, 's.t' => 1
+
     def self.create_with_namespace(namespace, bucket, payload)
       sleek = payload.delete(:sleek)
       event = create(namespace: namespace, bucket: bucket, data: payload)

data/lib/sleek/filter.rb

@@ -2,6 +2,12 @@ module Sleek
   class Filter
     attr_reader :property_name, :operator, :value
 
+    # Internal: Initialize a filter.
+    #
+    # property_name - the String name of target property.
+    # operator      - the Symbol operator name.
+    # value         - the value used by operator to compare with the
+    #                 value of target property.
     def initialize(property_name, operator, value)
       @property_name = "d.#{property_name}"
       @operator = operator.to_sym
@@ -12,10 +18,15 @@ module Sleek
       end
     end
 
+    # Internal: Apply the filter to a criteria.
+    #
+    # criteria - the Mongoid::Criteria object.
     def apply(criteria)
       criteria.send(operator, property_name => value)
     end
 
+    # Internal: Compare the filter with another. Filters are equal when
+    # property name, operator name, and value are equal.
     def ==(other)
       other.is_a?(Filter) && property_name == other.property_name &&
         operator == other.operator && value == other.value

data/lib/sleek/group_by_criteria.rb

@@ -0,0 +1,144 @@
+module Sleek
+  # Internal: Criteria object for group_by queries.
+  # The reason it exists is that it's not possible to group_by result of
+  # normal MongoDB queries, so MongoDB's Aggregation Framework has to be
+  # used.
+  #
+  # It provides common aggregates methods that normal criteria objects
+  # have: `count`, `distinct`, `sum`, `avg`, `min`, and `max`, but
+  # instead of just numbers, they return a hash of group value => number.
+  class GroupByCriteria
+    attr_reader :criteria, :group_by
+
+    # Internal: Initialize a group_by criteria.
+    #
+    # criteria - the Mongoid::Criteria instance, used to match events.
+    # group_by - the name of the property to group by. Should be
+    #            fully-qualified property name (not name of property
+    #            inside "d".)
+    def initialize(criteria, group_by)
+      @criteria = criteria
+      @group_by = group_by
+    end
+
+    # Internal: Compute all possible aggregates.
+    #
+    # field -        the optional name of the filed being aggregated. If
+    #                none is passed, aggregates will only count events
+    #                inside each group. If it is passed, min, max, sum,
+    #                and avg will be also included.
+    # count_unique - the boolean flag indicating whethere or not
+    #                counting distinct field values is needed. Off by
+    #                default, because calculation of distinct values
+    #                adds two additional pipeline operators and pushes
+    #                every value to the set, which might make
+    #                computation slower on large datasets when you do
+    #                NOT need to count unique values.
+    #
+    # Examples:
+    #
+    #   gc.aggregates
+    #   # => [
+    #          {"_id"=>"customer1", "count"=>2},
+    #          {"_id"=>"customer2", "count" => 1}
+    #        ]
+    #
+    # Returns an array of groups. Each group is a hash with key "_id"
+    # being the value of group_by property.
+    def aggregates(field = nil, count_unique = false)
+      pipeline = aggregates_pipeline(field, count_unique)
+      criteria.collection.aggregate(pipeline).to_a
+    end
+
+    # Internal: Run the aggregation on field and only select group value
+    # and some property.
+    #
+    # Examples:
+    #
+    #   gc.aggregates_prop(nil, "count")
+    #   # => { unique_value_1: 42, unique_value_2: 12 }
+    def aggregates_prop(field, prop, count_unique = false)
+      aggregates = aggregates(field, count_unique)
+      Hash[aggregates.map { |doc| [doc['_id'], doc[prop]] }]
+    end
+
+    def count
+      aggregates_prop(nil, 'count')
+    end
+
+    def count_unique(field)
+      aggregates_prop(field, 'count_unique', true)
+    end
+
+    def distinct(field)
+      OpenStruct.new(count: count_unique(field))
+    end
+
+    def avg(field)
+      aggregates_prop(field, 'avg')
+    end
+
+    def max(field)
+      aggregates_prop(field, 'max')
+    end
+
+    def min(field)
+      aggregates_prop(field, 'min')
+    end
+
+    def sum(field)
+      aggregates_prop(field, 'sum')
+    end
+
+    # Internal: Create aggregation pipeline.
+    #
+    # field        - the optional name of the field to aggregate.
+    # count_unique - the optional flag indicating whethere or not to
+    #                count unique values of the field or not. Off by
+    #                default. See `aggregates` doc for the rationale.
+    def aggregates_pipeline(field = nil, count_unique = false)
+      db_group = "$#{group_by}"
+      db_field = "$#{field}" if field
+
+      pipeline = []
+
+      crit = criteria
+
+      crit = crit.ne(field => nil) if field
+      pipeline << { "$match" => crit.ne(group_by => nil).selector }
+
+      group_args = { "_id" => db_group, "count" => { "$sum" => 1 } }
+
+      if field
+        group_args.merge!({
+          "max" => { "$max" => db_field },
+          "min" => { "$min" => db_field },
+          "sum" => { "$sum" => db_field },
+          "avg" => { "$avg" => db_field }
+        })
+
+        if count_unique
+          group_args.merge!({ "unique_set" => { "$addToSet" => db_field } })
+        end
+      end
+
+      pipeline << { "$group" => group_args }
+
+      if count_unique
+        pipeline << { "$unwind" => "$unique_set" }
+        pipeline << {
+          "$group" => {
+            "_id" => "$_id",
+            "count_unique" => { "$sum" => 1 },
+            "count" => { "$first" => "count" },
+            "max" => { "$first" => "max" },
+            "min" => { "$first" => "min" },
+            "avg" => { "$first" => "avg" }
+          }
+        }
+      end
+
+      pipeline
+    end
+  end
+end

data/lib/sleek/interval.rb

@@ -1,20 +1,5 @@
 module Sleek
   class Interval
-    def self.interval_value(desc)
-      case desc
-      when :hourly
-        1.hour
-      when :daily
-        1.day
-      when :weekly
-        1.week
-      when :monthly
-        1.month
-      else
-        raise ArgumentError, "invalid interval description"
-      end
-    end
-
     attr_reader :interval, :timeframe
 
     # Internal: Initialize an interval.
@@ -22,7 +7,7 @@ module Sleek
     # interval_desc - the Symbol description of the interval.
     #                 Possible values: :hourly, :daily, :weekly,
     #                 :monthly.
-    # timeframe     - the Timeframe object.
+    # timeframe     - the range of TimeWithZone objects.
     def initialize(interval_desc, timeframe)
       @interval = self.class.interval_value(interval_desc)
       @timeframe = timeframe
@@ -30,12 +15,28 @@ module Sleek
 
     # Internal: Split the timeframe into intervals.
     #
-    # Returns an Array of Timeframe objects.
+    # Returns an Array of time range objects.
     def timeframes
-      @timeframes ||= timeframe.to_time_range.to_i_range.each_slice(interval)
+      tz = timeframe.first.time_zone
+      timeframe.to_i_range.each_slice(interval)
         .to_a[0..-2]
-        .map { |tf| (tf.first..(tf.first + interval)).to_time_range }
-        .map { |tf| Timeframe.new(tf) }
+        .map { |tf, _| (tf..(tf + interval)).to_time_range(tz) }
+    end
+
+    # Internal: Convert interval description to numeric value.
+    def self.interval_value(desc)
+      case desc
+      when :hourly
+        1.hour
+      when :daily
+        1.day
+      when :weekly
+        1.week
+      when :monthly
+        1.month
+      else
+        raise ArgumentError, 'invalid interval description'
+      end
     end
   end
 end