minuteman 1.0.3 → 2.0.0.pre

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 33273982b184b97986adee038d64bc71a5ebf868
+  data.tar.gz: 1787e29f6314d78e1967140a1194692b58fc678c
+SHA512:
+  metadata.gz: 8e3d61fa6e931eceb2935d14ca8d734494e945e04d7a8676565e8c1c5e47454cfffdc336d71724e9c05ccfa2ac17e4f79fe69c4d4054bb1223988653a63c9d2a
+  data.tar.gz: 4925693e4ed12b00efe943693c0940296eb6c7d762e5a8697be551381ce0711b1feee081e778867f7a3c1d24db99c914c3a13da7cd0b9f69acb77ea9f3af9119

data/.gems

@@ -0,0 +1,3 @@
+redic:1.5.0
+ohm:2.3.0
+cutest:1.2.2

data/.travis.yml

@@ -1,7 +1,8 @@
 language: ruby
+sudo: false
+install: bash <(curl -s https://raw.githubusercontent.com/elcuervo/rpm/master/bin/rpm-install)
 rvm:
-  - "1.9.3"
-  - "jruby-19mode"
-  - "rbx-19mode"
+  - "2.0.0"
+  - "2.2.2"
 services:
   - "redis-server"

data/ORIGIN.md

@@ -0,0 +1,21 @@
+# Origin
+
+_Freenode - #cuba.rb - 2012/10/30 15:20 UYT_
+
+> **conanbatt:** anyone here knows some good web app metrics tool ?
+
+> **conanbatt:** i use google analytics for the page itself, and its good, but for the webapp its really not useful
+
+> **tizoc: conanbatt:** http://amix.dk/blog/post/19714 you can port this (if an equivalent doesn't exist already)
+
+> **conanbatt:** the metrics link is excellent but its python and released 5 days ago lol
+
+> **elcuervo: tizoc:** the idea it's awesome
+
+> **elcuervo:** interesting...
+
+## Inspiration
+
+* http://blog.getspool.com/2011/11/29/fast-easy-realtime-metrics-using-redis-bitmaps/
+* http://amix.dk/blog/post/19714
+* http://en.wikipedia.org/wiki/Bit_array

data/README.md

@@ -2,172 +2,165 @@
 
 # Minuteman
 [![Code Climate](https://codeclimate.com/github/elcuervo/minuteman.png)](https://codeclimate.com/github/elcuervo/minuteman)
-[![Build Status](https://secure.travis-ci.org/elcuervo/minuteman.png?branch=master)](https://travis-ci.org/elcuervo/minuteman)
 
-_Wikipedia_: Minutemen were members of teams from Massachusetts that were well-prepared
+[![Build Status](https://travis-ci.org/elcuervo/minuteman.svg)](https://travis-ci.org/elcuervo/minuteman)
+
+> Minutemen were members of teams from Massachusetts that were well-prepared
 militia companies of select men from the American colonial partisan militia
 during the American Revolutionary War. _They provided a highly mobile, rapidly
 deployed force that allowed the colonies to respond immediately to war threats,
 hence the name._
 
-## Origin
-Freenode - #cuba.rb - 2012/10/30 15:20 UYT
-
-**conanbatt:** anyone here knows some good web app metrics tool ?
-
-**conanbatt:** i use google analytics for the page itself, and its good, but for the webapp its really not useful
-
-**tizoc: conanbatt:** http://amix.dk/blog/post/19714 you can port this (if an equivalent doesn't exist already)
-
-**conanbatt:** the metrics link is excellent but its python and released 5 days ago lol
-
-**elcuervo: tizoc:** the idea it's awesome
-
-**elcuervo:** interesting...
-
-
-## Inspiration
-
-* http://blog.getspool.com/2011/11/29/fast-easy-realtime-metrics-using-redis-bitmaps/
-* http://amix.dk/blog/post/19714
-* http://en.wikipedia.org/wiki/Bit_array
-
 ## Installation
 
-### Important!
-
-Depends on Redis 2.6 for the `bitop` operation. You can install it using:
-
-```bash
-brew install redis
-```
-
-or upgrading your current version:
-
-```bash
-brew upgrade redis
-```
-
-And then install the gem
-
 ```bash
 gem install minuteman
 ```
 
-## Usage
-
-Currently Minutemen supports two options `:silent (default: false)` and `:redis
-(default: {})`
+### Configuration
 
-### Options
+Configuration exists within the `config` block:
 
-**silent**: when `true` the operations will not raise errors to prevent failures
+```ruby
+Minuteman.configure do |config|
+  # You need to use Redic to define a new Redis connection
+  config.redis = Redic.new("redis://127.0.0.1:6379/1")
+
+  # The prefix affects operations
+  config.prefix = "Tomato"
+
+  # The patterns is what Minuteman uses for the tracking/counting and the
+  # different analyzers
+  config.patterns = {
+    dia: -> (time) { time.strftime("%Y-%m-%d") }
+  }
+end
+```
 
-**time_spans**: an `Array` of Minuteman compatible `TimeSpan`. Eg. `%w[year month
-day]` will only track events on that timespans. Ignoring the rest
+## Tracking
 
-**redis**: can be a Hash with the options to be sent to `Redis.new` or a `Redis`
-connection already established (`Redis::Namespace` works as well).
+Tracking is the most basic scenario for Minuteman:
 
 ```ruby
-require "minuteman"
+# This will create the "landing:new" event in all the defined patterns and since
+# there is no user here it will create an annonymous one.
+# This user only exists in the Minuteman context.
+user = Minuteman.track("landing:new")
 
-# Accepts an options[:redis] hash that will be sent as is to Redis.new
-analytics = Minuteman.new
+# The id it's an internal representation, not useful for you
+user.id  # => "1"
 
-# You can also reuse your Redis or Redis::Namespace connection
-analytics = Minuteman.new(redis: Redis::Namespace.new(:mm, redis: Redis.new))
+# This is the unique id. With this you can have an identifier for the user
+user.uid # => "787c8770-0ac2-4654-9fa4-e57d152fa341"
 
-# Mark an event for a given id
-analytics.track("login:successful", user.id)
-analytics.track("login:successful", other_user.id)
+# You can use the `user` to keep tracking things:
+user.track("register:page")
 
-# Mark in bulk
-analytics.track("programming:love:ruby", User.where(favorite: "ruby").pluck(:id))
+# Or use it as an argument
+Minuteman.track("help:page", user)
 
-# Fetch events for a given time (default is Time.now.utc)
-today_events = analytics.day("login:successful", Time.now.utc)
+# Or track several users at once
+Minuteman.track("help:page", [user1, user2, user3])
 
-# This also exists
-analytics.year("login:successful")
-analytics.month("login:successful")
-analytics.week("login:successful")
-analytics.day("login:successful")
-analytics.hour("login:successful")
-analytics.minute("login:successful")
-
-# Lists all the tracked events
-analytics.events
-#=> ["login:successful", "programming:login:ruby"]
+# By default all the tracking and counting events are triggered with `Time.now.utc`
+# but you can change that as well:
+Minuteman.track("setup:account", user, Time.new(2010, 2, 10))
+```
 
-# Check event length on a given time
-today_events.length
-#=> 2
+## Analysis
 
-# Check for existance
-today_events.include?(user.id)
-#=> true
-today_events.include?(admin.id)
-#=> false
+There is a powerful engine behind all the operations which is Redis + Lua <3
 
-# Bulk check
-today_events.include?(User.all.pluck(:id))
-#=> [true, true, false, false]
+```ruby
+# The analysis of information relies on `Minuteman.patterns` and if you don't
+# change it you'll get acess to `year`, `month`, `day`, `hour`, `minute`.
+# To get information about `register:page` for today:
+Minuteman.analyze("register:page").day
+
+# You can always pass a `Time` instance to set the time you need information.
+Minuteman.analyze("register:page").day(Time.new(2004, 2, 12))
+
+# You also have a shorthand for analysis:
+register_page_month = Minuteman("register:page").month
+
+# And the power of Minuteman relies on the operations you can do with that.
+# Counting the amount:
+register_page_month.count # => 10
+
+# Or knowing if a user is included in that set:
+register_page_month.include?(User[42]) # => true
+
+# But the most important part is the ability to do bit operations on that:
+# You can intersect sets using bitwise AND(`&`), OR(`|`), NOT(`~`, `-`) and XOR(`^`).
+# Also you can use plus(`+`) and minus(`-`) operations.
+# In this example we'll get all the users that accessed our site via a promo
+# invite but didn't buy anything
+(
+  Minuteman("promo:email").day & Minuteman("promo:google").day
+) - Minuteman("buy:success").day
 ```
 
-## Bitwise operations
+## Counting
 
-You can intersect sets using bitwise AND(`&`), OR(`|`), NOT(`~`, `-`) and XOR(`^`).
-Also you can use plus(`+`) and minus(`-`) operations.
+Since Minuteman 2.0 there's the possibility to have counters.
 
 ```ruby
-set1 + set2
-set1 - set2
-set1 & set2
-set1 | set2
-set1 ^ set2
-
-~set1 \
-       |==> This are the same
--set1 /
-```
+# Counting works in a very similar way to tracking but with some important
+# differences. Trackings are idempotent unlike countings and they do not provide
+# operations between sets... you can use plain ruby for that.
+# This will add 1 to the `hits:page` counter:
+Minuteman.add("hits:page")
 
-### Intersecting with arrays
+# You can also pass a `Time` instance to define when this tracking ocurred:
+Minuteman.add("hits:page", Time.new(2012, 20, 01))
 
-Let's assume this scenario:
+# And you can also send users to also count the times that given user did that
+# event
+Minuteman.add("hits:page", Time.new(2012, 20, 01), user)
 
-You have a list of users and want to know which of them have been going throught
-some of the tracks you made.
+# You can access counting information similar to tracking:
+Minuteman.count("hits:page").day.count # => 201
 
-```ruby
-paid = analytics.month("buy:complete")
-payed_from_miami = paid & User.find_all_by_state("MIA").map(&:id)
-payed_from_miami.size
-#=> 43
-payed_users_from_miami = payed_from_miami.map { |id| User.find(id) }
+# Or with a shorthand:
+Counterman("hits:page").day.count # => 201
 ```
 
-Currently the supported commands to interact with arrays are `&` and `-`
+## Users
 
-### Example
+Minuteman 2.0 adds the concept of users which can be annonymous or have a
+relation with your own database.
 
 ```ruby
-invited = analytics.month("email:invitation")
-successful_buys = analytics.month("buy:complete")
+# This will create an annonymous user
+user = Minuteman::User.create
 
-successful_buys_after_invitation = invited & successful_buys
-successful_buys_after_invitation.include?(user.id)
+# Users are just a part of Minuteman and do not interfere with your own.
+# They do have some properties like a unique identifier you can use to find it
+# in the future
+user.uid # => "787c8770-0ac2-4654-9fa4-e57d152fa341"
 
-# Clean up all the operations cache
-analytics.reset_operations_cache
-```
+# User lookup works like this:
+# And you can use that unique identifier as a key in a cookie to see what your
+# uses do when no one is looking
+Minuteman::User['787c8770-0ac2-4654-9fa4-e57d152fa341']
 
-Also you can write more complex set operations
+# But since the point is to be able to get tied to your data you can promote a
+# user, from anonymous to "real"
+user.promote(123)
 
-```ruby
-invited = analytics.month("email:invitation")
-from_adsense = analytics.month("adsense:promo")
-successful_buys = analytics.month("buy:complete")
+# Lookups also work with promoted ids
+Minuteman::User["123"]
+
+# Having a user you can do all the same operations minus the hussle.
+# Like tracking:
+user.track("user:login")
+
+# or adding:
+user.add("failed:login")
+
+# or counting
+user.count("failed:login").month.count # => 23
 
-conversion_rate = (invited | from_adsense) & successful_buys
+# But also the counted events go to the big picture
+Counterman("failed:login").month.count # => 512
 ```

data/Rakefile

@@ -1,13 +1,13 @@
 require "rake/testtask"
 
-Rake::TestTask.new("spec") do |t|
+Rake::TestTask.new("test") do |t|
+  t.libs << "test"
   t.pattern = "test/**/*_test.rb"
 end
 
 Rake::TestTask.new("bench") do |t|
-  t.pattern = "test/bench/*_bench.rb"
+  t.pattern = "test/**/*_bench.rb"
 end
 
 task :default => [:test]
 task :all     => [:test, :bench]
-task :test    => [:spec]

data/SPEC.md

@@ -0,0 +1,21 @@
+```ruby
+# Track an event
+minuteman_user = Minuteman.track("login:successfull")
+minuteman_user.promote(user.id)
+
+# Trigger an event in the pipeline
+Minuteman.trigger("event:name")
+
+# Gets an event analyzer
+analyzer = Minuteman.analyze("event:name")
+# Counts unique users in that given time
+analyze.day(Time.now.utc)
+
+Minuteman("event:name").day | Minuteman("event2:name).day
+```
+
+Caching:
+Cache ids in events:
+
+Minuteman::Operation:50:AND:20, Minuteman::Operation:50:OR:20
+Minuteman::Operation:(50:AND:20):AND:14

data/lib/minuteman.rb

@@ -1,151 +1,117 @@
-require "redis"
-require "time"
-require "forwardable"
-require "minuteman/time_events"
+require 'redic'
 
-# Public: Minuteman core classs
-#
-class Minuteman
-  extend Forwardable
+module Minuteman
+  LUA_CACHE      = Hash.new { |h, k| h[k] = Hash.new }
+  LUA_OPERATIONS = File.expand_path("../minuteman/lua/operations.lua",   __FILE__)
 
   class << self
-    attr_accessor :redis, :options
-
-    # Public: Prevents a fatal error if the options are set to silent
-    #
-    def safe(&block)
-      yield if block
-    rescue Redis::BaseError => e
-      raise e unless options[:silent]
+    def config
+      @_configuration ||= Configuration.new
     end
-  end
-
-  PREFIX = "minuteman"
 
-  def_delegators self, :redis, :redis=, :options, :options=, :safe
-
-  # Public: Initializes Minuteman
-  #
-  #   options - An options hash to change how Minuteman behaves
-  #
-  def initialize(options = {})
-    redis_options = options.delete(:redis) || {}
+    def configure
+      yield(config)
+    end
 
-    self.options = default_options.merge!(options)
-    self.redis = define_connection(redis_options)
+    def prefix
+      config.prefix
+    end
 
-    spans = self.options.fetch(:time_spans, %w[year month week day hour minute])
-    @time_spans = generate_spans(spans)
-  end
+    def patterns
+      config.patterns
+    end
 
-  # Public: Marks an id to a given event on a given time
-  #
-  #   event_name - The event name to be searched for
-  #   ids        - The ids to be tracked
-  #
-  # Examples
-  #
-  #   analytics = Minuteman.new
-  #   analytics.track("login", 1)
-  #   analytics.track("login", [2, 3, 4])
-  #
-  def track(event_name, ids, time = Time.now.utc)
-    event_time = time.kind_of?(Time) ? time : Time.parse(time.to_s)
-    time_events = TimeEvents.start(@time_spans, event_name, event_time)
-
-    track_events(time_events, Array(ids))
-  end
+    def time_spans
+      @_time_spans = patterns.keys
+    end
 
-  # Public: List all the events given the minuteman namespace
-  #
-  def events
-    keys = safe { redis.keys([PREFIX, "*", "????"].join("_")) }
-    keys.map { |key| key.split("_")[1] }
-  end
+    def track(action, users = nil, time = Time.now.utc)
+      users = Minuteman::User.create if users.nil?
 
-  # Public: List all the operations executed in a given the minuteman namespace
-  #
-  def operations
-    safe { redis.keys([operations_cache_key_prefix, "*"].join("_")) }
-  end
+      Array(users).each do |user|
+        process do
+          time_spans.each do |time_span|
+            event = Minuteman::Event.find_or_create(
+              type: action,
+              time: patterns[time_span].call(time)
+            )
 
-  # Public: Resets the bit operation cache keys
-  #
-  def reset_operations_cache
-    keys = safe { redis.keys([operations_cache_key_prefix, "*"].join("_")) }
-    safe { redis.del(keys) } if keys.any?
-  end
+            event.setbit(user.id)
+          end
+        end
+      end
 
-  # Public: Resets all the used keys
-  #
-  def reset_all
-    keys = safe { redis.keys([PREFIX, "*"].join("_")) }
-    safe { redis.del(keys) } if keys.any?
-  end
+      users
+    end
 
-  private
+    def add(action, time = Time.now.utc, users = [])
+      time_spans.each do |time_span|
+        process do
+          counter = Minuteman::Counter.create({
+            type: action,
+            time: patterns[time_span].call(time)
+          })
 
-  # Public: Generates the methods to fech data
-  #
-  #   spans: An array of timespans corresponding to a TimeSpan class
-  #
-  def generate_spans(spans)
-    spans.map do |method_name|
-      constructor = self.class.const_get(method_name.capitalize)
+          counter.incr
+        end
+      end
 
-      define_singleton_method(method_name) do |*args|
-        event_name, date = *args
-        date ||= Time.now.utc
+      Array(users).each do |user|
+        time_spans.each do |time_span|
+          counter = Minuteman::Counter::User.create({
+            user_id: user.id,
+            type: action,
+            time: patterns[time_span].call(time)
+          })
 
-        constructor.new(event_name, date)
+          counter.incr
+        end
       end
+    end
 
-      constructor
+    def analyze(action)
+      analyzers_cache[action]
     end
-  end
 
-  # Private: Default configuration options
-  #
-  def default_options
-    { cache:  true, silent: false }
-  end
+    def count(action)
+      counters_cache[action]
+    end
 
-  # Private: Determines to use or instance a Redis connection
-  #
-  #  object: Can be the options to instance a Redis connection or a connection
-  #          itself
-  #
-  def define_connection(object)
-    case object
-    when Redis, defined?(Redis::Namespace) && Redis::Namespace
-      object
-    else
-      Redis.new(object)
+    private
+
+    def process(&block)
+      if !!config.parallel
+        Thread.current(&block)
+      else
+        block.call
+      end
     end
-  end
 
-  # Private: Marks ids for a given time events
-  #
-  #  time_events: A set of TimeEvents
-  #  ids:         The ids to be tracked
-  #
-  def track_events(time_events, ids)
-    safe_multi do
-      time_events.each do |event|
-        ids.each { |id| safe { redis.setbit(event.key, id, 1) } }
+    def analyzers_cache
+      @_analyzers_cache ||= Hash.new do |h,k|
+        h[k] = Minuteman::Analyzer.new(k)
       end
     end
-  end
 
-  # Private: Executes a block within a safe connection using redis.multi
-  #
-  def safe_multi(&block)
-    safe { redis.multi(&block) }
+    def counters_cache
+      @_counters_cache ||= Hash.new do |h,k|
+        h[k] = Minuteman::Analyzer.new(k, Minuteman::Counter)
+      end
+    end
   end
+end
 
-  # Private: The prefix key of all the operations
-  #
-  def operations_cache_key_prefix
-    [ PREFIX, Minuteman::KeysMethods::BIT_OPERATION_PREFIX ].join("_")
-  end
+def Minuteman(action)
+  Minuteman.analyze(action)
 end
+
+def Counterman(action)
+  Minuteman.count(action)
+end
+
+require 'minuteman/user'
+require 'minuteman/event'
+require 'minuteman/counter'
+require 'minuteman/result'
+require 'minuteman/analyzer'
+require 'minuteman/configuration'