btrack 1.0.1

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: b730fa2c3a34a3116f0b94591f528443855121f1
4
+ data.tar.gz: 6ce8856176cd93b9c372e07f61a5380ee0a6e67a
5
+ SHA512:
6
+ metadata.gz: da1feb3673a195c82b6e8a0cde1006688462b8866fee09587da5dc061c8ac7661a4f99bd753230f103103d309785c45c1e8aadff34a527baa83800ce2e136ad8
7
+ data.tar.gz: 4b86399ba364d3224f51189a88ec09473e88f4633785c017383f6424ad6325427fe20176a760b66ceb170ba4fc8c21cbed6c5aa8495021f0012f586328b22c27
@@ -0,0 +1,17 @@
1
+ *.gem
2
+ *.rbc
3
+ .bundle
4
+ .config
5
+ .yardoc
6
+ Gemfile.lock
7
+ InstalledFiles
8
+ _yardoc
9
+ coverage
10
+ doc/
11
+ lib/bundler/man
12
+ pkg
13
+ rdoc
14
+ spec/reports
15
+ test/tmp
16
+ test/version_tmp
17
+ tmp
data/.rspec ADDED
@@ -0,0 +1,2 @@
1
+ --color
2
+ --format documentation
@@ -0,0 +1 @@
1
+ 2.1.1
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in btrack.gemspec
4
+ gemspec
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2013 Chen Fisher
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,323 @@
1
+ # Btrack
2
+
3
+ **Btrack** is an activity tracker with extensive query mechanism, minimum memory footprint and maximum performance (thanks to redis)
4
+
5
+ With **Btrack** you can track any activity of any entity in your website or process
6
+
7
+ #### Tracking user logins (user 123 has just logged in):
8
+ ```ruby
9
+ Btrack.track :logged_in, 123
10
+ ```
11
+
12
+ #### Query for total logins:
13
+ ```ruby
14
+ Btrack.where(logged_in: :today).count
15
+ ```
16
+
17
+ #### Query if a specific user visited your website last month:
18
+ ```ruby
19
+ Btrack.where(visited: :last_month).exists? 123
20
+
21
+ # use a cool shortcut
22
+ Btrack::Query.visited? 123, :today
23
+ ```
24
+
25
+ #### You can also plot a graph!
26
+ ``` ruby
27
+ Btrack.where(clicked_a_button: 1.week.ago..Time.now).plot
28
+ #=> {2014-07-14 00:00:00 +0300=>10, 2014-07-14 00:00:00 +0300=>5, 2014-07-14 00:00:00 +0300=>30...
29
+ ```
30
+
31
+ #### Cohort analysis (for example, all users that signed in last week and visited this week )
32
+ ```ruby
33
+ Btrack.where([{signed_in: :last_week}, {visited: :this_week}]).plot
34
+ ```
35
+
36
+ # Background
37
+ **Btrack** uses Redis bitmaps to track activities over entities; you can track millions of users with a very small memory footprint and use bitwise operators to determine complex queries in realtime.
38
+
39
+ See relevant Redis commands for bitmaps here:
40
+ [http://redis.io/commands/SETBIT]
41
+
42
+ Read this to better understand how bitmaps work in redis and how you can use it for fast, realtime analytics: [http://blog.getspool.com/2011/11/29/fast-easy-realtime-metrics-using-redis-bitmaps/]
43
+
44
+
45
+ ## Installation
46
+
47
+ Add this line to your application's Gemfile:
48
+
49
+ gem 'btrack'
50
+
51
+ And then execute:
52
+
53
+ $ bundle
54
+
55
+ Or install it yourself as:
56
+
57
+ $ gem install btrack
58
+
59
+ ## Usage
60
+
61
+ ## Tracking
62
+ Basic tracking is done by specifying the event to track (can be a string or a symbol) and the id of the entity you are tracking (**must** be an interger)
63
+
64
+ User with id 123 purchased something:
65
+ ```ruby
66
+ Btrack.track "user:purchased", 123
67
+ ```
68
+ Item with id 1001 was just purchased:
69
+ ```ruby
70
+ Btrack.track "item:purchased", 1001
71
+ ```
72
+
73
+ ### Granularity
74
+ When tracking an event a default granularity is used (see **configuration** section for more details on default values); the default granularity is :hourly..:monthly, which means :hourly, :daily, :weekly and :monthly
75
+
76
+ To track with a different granualrity:
77
+ ```ruby
78
+ # track with a weekly granularity
79
+ Btrack.track :logged_in, 123, :weekly
80
+
81
+ # track with both daily and weekly granularity:
82
+ Btrack.track :logged_in, 123, [:daily, :weekly]
83
+
84
+ # track with a range of granularities:
85
+ Btrack.track :logged_in, 123, :hourly..:monthly
86
+ #=> will track with: hourly, daily, weekly and monthly
87
+ ```
88
+
89
+ Available granularities: [:minute, :hourly, :daily, :weekly, :monthly, :yearly]
90
+
91
+
92
+ #### CAVEATS with granularities
93
+ You should be aware that there is a close relation between tracking and querying in regards to granularities. see **querying/granularity** section for more details.
94
+
95
+
96
+ ```ruby
97
+ # track with daily granularity
98
+ Btrack.track :logged_in, 123, :daily
99
+
100
+ # query with weekly granualrity:
101
+ Btrack.where [{logged_in: :today, granularity: :weekly}]
102
+ #=> returns 0
103
+
104
+ # tracking with a range of granularities:
105
+ Btrack.track :logged_in, 123, :daily..:monthly
106
+
107
+ # now querying with weekly granularity is OK because it is included in the range:
108
+ Btrack.where [{logged_in: :today, granularity: :weekly}]
109
+ #=> returns 1
110
+ ```
111
+ ### Tracking with a block
112
+ You can track with a block for convenience and for specifying other tracking options:
113
+
114
+ ```ruby
115
+ Btrack::Tracker.track do |b|
116
+ b.key = "logged_in"
117
+ b.id = 123
118
+ b.granularity = :daily..:monthly
119
+ b.when = 3.days.ago # when was this event occured
120
+ b.expiration_for = { daily:6.days } # specify expiration for this event
121
+ end
122
+ ```
123
+
124
+ ### Tracking in the past tense
125
+ When tracking an event, the default time is ```Time.now``` which means: the event just happend.
126
+
127
+ You can specify a different time when tracking an event:
128
+ ```ruby
129
+ Btrack::Tracker.track do |b|
130
+ b.key = :logged_in
131
+ b.id = 123
132
+ b.when = 3.days.ago # this event happened 3 days ago
133
+ end
134
+ ```
135
+
136
+ ### Expiration time
137
+ You can specity retention for events per granularity. Use this to get rid of granularities you don't need any more and save memory (in redis)
138
+
139
+ ```ruby
140
+ Btrack::Tracker.track do |b|
141
+ b.key = :logged_in
142
+ b.id = 123
143
+ b.expiration_for = { minute: 3.days, daily: 3.months }
144
+ # after 3 days all "minute" granularities for this event will be deleted
145
+ # and 3 months later all the relevant "daily" granularities will be deleted
146
+ end
147
+ ```
148
+
149
+ ## Querying
150
+
151
+ ```ruby
152
+ # Simple querying
153
+ Btrack.where(logged_in: :today).count
154
+ Btrack.where(logged_in: :yesteday).count
155
+ Btrack.where(logged_in: :last_week).count
156
+
157
+ # Query with a time range
158
+ Btrack.where(logged_in: 3.days.ago..Time.now).count
159
+ ```
160
+
161
+ `Btrack.where` has the following form: `where(criteria, options={})`
162
+
163
+ **criteria** is an array of hashes, where each hash is the event to query and relevant optional options of that event (don't worry, there are plentty of examples in this section; just make sure to wrap the criteria with array and proper hashes when querying for more than one event or when specifying a granularity for an event)
164
+
165
+ **options={}** is preserved for feature use and is meant for specifying options that are not event specific
166
+
167
+ ### Querying for a specific user/entity
168
+ ```ruby
169
+ Btrack.where(logged_in: :today).exists? 123
170
+ #=> returns true if user 123 logged in today
171
+
172
+ Btrack.where(visited: 7.days.ago..Time.now)exists? 123
173
+ #=> returns true if user 123 visited the website in the past 7 days
174
+
175
+ # You can use a cool shortcut to query for a specific user:
176
+ Btrack::Query.visited? 123, 7.days.ago..Time.now
177
+ #=> same as above, but with a cool shortcut
178
+
179
+ Btrack::Query.logged_in? 123, :today
180
+ #=> true/false
181
+ ```
182
+ ### Lazyness
183
+ Queries are not "realized" until you perform an action:
184
+ ```ruby
185
+ a_query = Btrack.where [logged_in: 1.week.ago..Time.now, granularity: :daily]
186
+ #=> <Btrack::Query::Criteria:0x007fceb248e120 @criteria=[{:logged_in=>:today}], @options={}>
187
+
188
+ a_query.count
189
+ #=> 3
190
+
191
+ a_query.exists? 123
192
+ #=> true
193
+
194
+ a_query.plot
195
+ #=> {2014-07-01 00:00:00 +0300=>10, 2014-07-01 00:00:00 +0300=>5...
196
+
197
+ # You can use the "realize!" action to see what's under the hood
198
+ a_query.realize!
199
+ #=> [["btrack:logged_in:2014-07-11", "btrack:logged_in:2014-07-12", "btrack:logged_in:2014-07-13"...
200
+ ```
201
+
202
+ ### Intersection (querying for multiple events)
203
+ You can query for multiple events
204
+ ```ruby
205
+ # signed in AND purchased something this month
206
+ q = Btrack.where([{signed_in: :this_month}, {purchased: :this_month}])
207
+
208
+ q.count
209
+ q.exists? 123
210
+ q.plot
211
+
212
+ # logged last week AND logged in today
213
+ Btrack.where([{logged_in: :last_week}, {logged_in: :today}])
214
+
215
+ # signed in the last 30 days, logged in this week and purchased something
216
+ Btrack.where([{signed_in: 30.days.ago..Time.now}, {logged_in: :last_week}, {purchased_something: :this_month}])
217
+ ```
218
+
219
+ #### The & operator
220
+ You can use `&` for intersection
221
+ ```ruby
222
+ signed_in = Btrack.where signed_in: 30.days.ago..Time.now
223
+ visited = Btrack.where visited: 7.days.ago..Time.now
224
+
225
+ signed_in_AND_visited = signed_in & visited
226
+ signed_in_visited_and_whatever = signed_in_AND_visited & Btrack.where(whatever: :today)
227
+ ```
228
+
229
+ ### Granularity
230
+ When querying, you should make sure you are tracking in the same granularity. If you are tracking in the range of :daily..:monthly then you can only query in that range (or you will get wrong results)
231
+
232
+ To specify the granualrity when querying, add a :granualrity key to the hash (per event):
233
+
234
+ ```ruby
235
+ Btrack.where([{clicked_a_button: 3.hours.ago..Time.now, granularity: :hourly}])
236
+
237
+ # granularity is per event:
238
+ Btrack.where([{logged_in: 1.hour.ago..Time.now, granularity: :minute}, {did_something: :today, granularity: :daily}])
239
+
240
+ # see the next section (plotting) about granularity when plotting a graph
241
+ ```
242
+
243
+ Another possible error you should be aware of is when querying for a timeframe that is not correlated with the granularity:
244
+
245
+ ```ruby
246
+ # timeframe is :today, while granularity is :weekly
247
+ Btrack.where([{logged_in: :today, granularity: :weekly}])
248
+ # this will result in wrong results because :weekly granularity will refer
249
+ # to the whole week, while you probably meant to query only :today
250
+ ```
251
+
252
+
253
+ > Default granularity when querying is the highest resolution set in configuration.default_granularity (:hourly..:monthly => :hourly is the default when querying)
254
+
255
+ ## Plotting
256
+ Use `plot` to plot a graph
257
+ ```ruby
258
+ # plot a graph with a daily resolution (granularity)
259
+ Btrack.where([{logged_in: 30.days.ago..Time.now, granularity: :daily}]).plot
260
+
261
+ # plot a graph with an hourly resolution
262
+ Btrack.where([{logged_in: 30.days.ago..Time.now, granularity: :hourly}]).plot
263
+ ```
264
+
265
+ ### Cohort
266
+ You can use what you've learned so far to create a cohort analysis
267
+ ```ruby
268
+ visits = Btrack.where [visited: 30.days.ago..Time.now, granularity: :daily]
269
+ purchases = Btrack.where [purchased_something: 30.days.ago..Tome, granularity: :daily]
270
+
271
+ visits_and_purchases = visits & purchases
272
+
273
+ # now plot a cohort
274
+ visits_and_purchases.plot
275
+ #=> {2014-07-01 00:00:00 +0300=>10, 2014-07-01 00:00:00 +0300=>20, ...
276
+
277
+ # NOTE that when plotting multiple events (cohort), the returned keys for the plot are named after the first event
278
+ ```
279
+
280
+ ## Configuration
281
+ Put this in an `initializer` to configure **Btrack**
282
+ ```ruby
283
+ Btrack.config do |config|
284
+ config.namespace = 'btrack' # default namespace for redis keys
285
+ config.redis_url = nil # redis url to use; defaults to nil meaning localhost and default redis port
286
+ config.expiration_for = {minute: 1.day, hourly: 1.week, daily: 3.months, weekly: 1.year, monthly: 1.year, yearly: 1.year}
287
+ config.default_granularity: :daily..:monthly # default granularities when tracking
288
+ config.silent = false # to break or not to break on redis errors
289
+ end
290
+ ```
291
+
292
+ ### namespace
293
+ Sets the namespace to use for the keys in redis; defaults to **btrack**
294
+
295
+ > keys in redis look like this: `btrack:logged_in:2014-07-15`
296
+ >
297
+ >and in a general form: `namespace:event:datetime`
298
+
299
+ ### expiration_for
300
+ Use expiration_for to set the expiration for a specific granularity
301
+ ```ruby
302
+ Btrack.config.expiration_for = { daily: 7.months }
303
+ # will merge :daily expiration with the whole expirations hash
304
+ # {minute: 1.day, hourly: 1.week, daily: 7.months, weekly: 1.year, monthly: 1.year, yearly: 1.year}
305
+ ```
306
+ ### redis_url
307
+ Sets the connection url to the redis server; defaults to nil which means localhost and default redis port
308
+
309
+ ## Alternatives
310
+ [Minuteman](https://github.com/elcuervo/minuteman) is a nice alternative to **Btrack** but with the following caveats (and more):
311
+
312
+ 1. It does not support time frames (you cannot query for 30.days.ago..Time.now)
313
+ 2. It eagerly creates a redis key on every bitwise operator, while **Btrack** is lazy
314
+ 3. It uses redis `multi` while **Btrack** uses `lua` for better performance
315
+ 4. No plot option in Minuteman
316
+
317
+ ## Contributing
318
+
319
+ 1. Fork it
320
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
321
+ 3. Commit your changes (`git commit -am 'Add some feature'`)
322
+ 4. Push to the branch (`git push origin my-new-feature`)
323
+ 5. Create new Pull Request
@@ -0,0 +1,4 @@
1
+ require "bundler/gem_tasks"
2
+ require 'rspec/core/rake_task'
3
+
4
+ RSpec::Core::RakeTask.new('spec')
@@ -0,0 +1,28 @@
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+ require 'btrack/version'
5
+
6
+ Gem::Specification.new do |spec|
7
+ spec.name = "btrack"
8
+ spec.version = Btrack::VERSION
9
+ spec.authors = ["Chen Fisher"]
10
+ spec.email = ["chen.fisher@gmail.com"]
11
+ spec.description = %q{Enables tracking and querying of any activity in a website or process with minimum memory signature and maximum performance (thanks to redis)}
12
+ spec.summary = %q{Activity tracker with minimum memory footprint and maximum performance (thanks to redis)}
13
+ spec.homepage = ""
14
+ spec.license = "MIT"
15
+
16
+ spec.files = `git ls-files`.split($/)
17
+ spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
18
+ spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
19
+ spec.require_paths = ["lib"]
20
+
21
+ spec.add_dependency "activesupport"
22
+ spec.add_dependency "redis"
23
+
24
+ spec.add_development_dependency "bundler", "~> 1.3"
25
+ spec.add_development_dependency "rake"
26
+ spec.add_development_dependency "rspec"
27
+ spec.add_development_dependency "wrong"
28
+ end
@@ -0,0 +1,24 @@
1
+ require 'active_support/all'
2
+ require "btrack/version"
3
+ require "btrack/helper"
4
+ require "btrack/config"
5
+ require "btrack/time_frame"
6
+ require "btrack/redis"
7
+ require "btrack/tracker"
8
+ require "btrack/query"
9
+
10
+ module Btrack
11
+ extend self
12
+
13
+ delegate :where, to: Query
14
+ delegate :track, to: Tracker
15
+
16
+ def redis
17
+ @redis ||= Btrack::Redis.create
18
+ end
19
+
20
+ def config
21
+ yield Btrack::Config if block_given?
22
+ Btrack::Config
23
+ end
24
+ end
@@ -0,0 +1,28 @@
1
+ module Btrack
2
+ class Config
3
+
4
+ @config = OpenStruct.new ({
5
+ namespace: "btrack",
6
+ redis_url: nil, # nil means localhost with defailt redis port
7
+ expirations: {minute: 1.day, hourly: 1.week, daily: 3.months, weekly: 1.year, monthly: 1.year, yearly: 1.year},
8
+ default_granularity: :hourly..:monthly,
9
+ silent: false # to break or not to break (on redis errors); that is the question
10
+ })
11
+
12
+ class << self
13
+
14
+ def expiration_for(g)
15
+ @config[:expirations][g]
16
+ end
17
+
18
+ def expiration_for=(g)
19
+ @config[:expirations].merge!(g)
20
+ end
21
+
22
+ def method_missing(method, *args, &block)
23
+ @config.send(method, *args, &block)
24
+ end
25
+
26
+ end
27
+ end
28
+ end
@@ -0,0 +1,25 @@
1
+ module Btrack
2
+ class Helper
3
+
4
+ class << self
5
+ def key(k, g=Config.default_granularity, w=Time.now)
6
+ "#{Config.namespace}:#{k}:#{granularity g, w || Time.now}"
7
+ end
8
+
9
+ def keys(k, timeframe)
10
+ tf = timeframe.is_a?(TimeFrame) ? timeframe : (TimeFrame.new timeframe)
11
+ tf.splat { |t| key k, tf.granularity, t}
12
+ end
13
+
14
+ def granularity(g=:daily, w=Time.now)
15
+ return g unless [:minute, :hourly, :daily, :weekly, :monthly, :yearly].include? g
16
+ w.strftime(format(g))
17
+ end
18
+
19
+ def format(g)
20
+ { minute: "%Y-%m-%d-%H-%M", hourly: "%Y-%m-%d-%H", daily: "%Y-%m-%d", weekly: "%G-W%V", monthly: "%Y-%m", yearly: "%Y"}[g] || "%Y-%m-%d"
21
+ end
22
+ end
23
+
24
+ end
25
+ end
@@ -0,0 +1,45 @@
1
+ require 'btrack/query/criteria'
2
+ require 'btrack/query/lua'
3
+
4
+ module Btrack
5
+ class Query
6
+ attr_reader :criteria
7
+
8
+ delegate :with_sha, :with_silent, to: Btrack::Redis
9
+
10
+ class << self
11
+ delegate :where, to: Criteria
12
+ end
13
+
14
+ def initialize(criteria = nil)
15
+ @criteria = criteria.freeze
16
+ end
17
+
18
+ def count
19
+ with_silent { with_sha { [lua(:count), *@criteria.realize!] } }
20
+ end
21
+
22
+ def exists?(id = nil)
23
+ c = id ? @criteria.where([], id: id) : @criteria
24
+ with_silent { with_sha { [lua(:exists), *c.realize!] } == 1 }
25
+ end
26
+
27
+ def plot
28
+ JSON.parse(with_silent { with_sha { [plot_lua, *@criteria.realize!] } }).inject({}) do |n, (k, v)|
29
+ g = @criteria.criteria.first[:granularity] || Criteria.default_granularity
30
+ key = Time.strptime(k.rpartition(":").last, Helper.format(g))
31
+ n[key] = v
32
+ n
33
+ end
34
+ rescue
35
+ nil
36
+ end
37
+
38
+ class << self
39
+ def method_missing(method, *args, &block)
40
+ return unless method.to_s.end_with? '?'
41
+ Criteria.where({method.to_s.chomp('?') => args[1]}.merge(args.extract_options!), id: args[0]).exists?
42
+ end
43
+ end
44
+ end
45
+ end
@@ -0,0 +1,62 @@
1
+ require 'btrack/helper'
2
+ require 'btrack/time_frame'
3
+
4
+ module Btrack
5
+ class Query
6
+ class Criteria
7
+ attr_reader :options, :criteria
8
+ delegate :count, :exists?, :plot, to: :query
9
+
10
+ # initializes a new crieteria
11
+ def initialize(criteria, options={})
12
+ @criteria = parse(criteria).freeze
13
+ @options = options.freeze
14
+ end
15
+
16
+ # returns a new criteria object with the union of both criterias
17
+ def where(*args)
18
+ self & Criteria.new(*args)
19
+ end
20
+
21
+ # returns a new criteria object with the union of both criterias
22
+ def &(criteria)
23
+ Criteria.new self.criteria + criteria.criteria, self.options.merge(criteria.options)
24
+ end
25
+
26
+ # make this criteria 'real' by extracting keys and args to be passed to redis lua script
27
+ def realize!
28
+ prefix = "#{@options[:prefix]}:" if @options[:prefix]
29
+
30
+ keys = @criteria.map do |c|
31
+ cvalue = c.values.first
32
+ (Helper.keys "#{prefix}#{c.keys.first}", TimeFrame.new(cvalue, c[:granularity] || (cvalue.granularity if cvalue.is_a? TimeFrame) || self.class.default_granularity)).flatten
33
+ end
34
+
35
+ [keys.flatten << @options[:id], keys.map(&:count)]
36
+ end
37
+
38
+ # access methods from class instance
39
+ # returns a new criteria instance
40
+ class << self
41
+ def where(*args)
42
+ Criteria.new *args
43
+ end
44
+
45
+ def default_granularity
46
+ Config.default_granularity.is_a?(Range) ? Config.default_granularity.first : Array.wrap(Config.default_granularity).first
47
+ end
48
+ end
49
+
50
+ # delegate method
51
+ def query
52
+ Query.new self
53
+ end
54
+
55
+ private
56
+
57
+ def parse(criteria)
58
+ criteria.is_a?(Array) ? criteria : criteria.map { |k, v| {k => v} }
59
+ end
60
+ end
61
+ end
62
+ end
@@ -0,0 +1,74 @@
1
+ module Btrack
2
+ class Query
3
+ def lua(f)
4
+ %Q{
5
+ local index = 1
6
+
7
+ for i, count in ipairs(ARGV) do
8
+ if count == '0' then
9
+ break
10
+ end
11
+
12
+ local bitop = {}
13
+
14
+ for c = index, index * count do
15
+ table.insert(bitop, KEYS[c])
16
+ index = index + 1
17
+ end
18
+
19
+ if i == 1 then
20
+ redis.call('bitop', 'or', 'tmp', unpack(bitop))
21
+ else
22
+ redis.call('bitop', 'or', 'tmp:or', unpack(bitop))
23
+ redis.call('bitop', 'and', 'tmp', 'tmp', 'tmp:or')
24
+ redis.call('del', 'tmp:or')
25
+ end
26
+ end
27
+
28
+ #{send('lua_' + f.to_s)}
29
+
30
+ redis.call('del', 'tmp')
31
+ return results
32
+ }
33
+ end
34
+
35
+ def lua_count
36
+ "local results = redis.call('bitcount', 'tmp')"
37
+ end
38
+
39
+ def lua_exists
40
+ "local results = redis.call('getbit', 'tmp', KEYS[#KEYS])"
41
+ end
42
+
43
+ # lua script for plotting
44
+ # please note - it cannot be used with the prefixed 'lua' like count and exists
45
+ # this is a standalone script ment for plotting and allowing for cohort analysis
46
+ # all series must be of the same size
47
+ def plot_lua
48
+ %Q{
49
+ local series_count = #ARGV
50
+ local length = ARGV[1]
51
+
52
+ -- run over the first series
53
+ -- all series must be of the same size
54
+ local plot = {}
55
+ for i = 1, length do
56
+ local bitop = {}
57
+ for j = 1, series_count do
58
+ table.insert(bitop, KEYS[i*j])
59
+ end
60
+
61
+ -- make sure 'tmp' is populated with the first key (so the 'and' op would work as expected)
62
+ redis.call('bitop', 'or', 'tmp', 'tmp', bitop[1])
63
+
64
+ redis.call('bitop', 'and', 'tmp', unpack(bitop))
65
+ -- table.insert(plot, redis.call('bitcount', 'tmp'))
66
+ plot[KEYS[i]] = redis.call('bitcount', 'tmp')
67
+ redis.call('del', 'tmp')
68
+ end
69
+
70
+ return cjson.encode(plot)
71
+ }
72
+ end
73
+ end
74
+ end
@@ -0,0 +1,30 @@
1
+ require 'redis'
2
+ require 'btrack/config'
3
+
4
+ module Btrack
5
+ class Redis
6
+ class << self
7
+ def create
8
+ (::Redis.new url: Config.redis_url if Config.redis_url) || ::Redis.new
9
+ end
10
+
11
+ def with_silent(&block)
12
+ yield if block
13
+ rescue ::Redis::BaseError => e
14
+ raise e unless Config.silent
15
+ end
16
+
17
+ def with_sha(&block)
18
+ params = yield; script = params.shift
19
+ Btrack.redis.evalsha sha(script), *params
20
+ rescue ::Redis::CommandError => e
21
+ raise unless e.message.start_with?("NOSCRIPT")
22
+ Btrack.redis.eval script, *params
23
+ end
24
+
25
+ def sha(script)
26
+ Digest::SHA1.hexdigest(script)
27
+ end
28
+ end
29
+ end
30
+ end
@@ -0,0 +1,63 @@
1
+ module Btrack
2
+ class TimeFrame
3
+ attr_reader :from, :to, :granularity
4
+
5
+ def initialize(timeframe, granularity=nil)
6
+ raise ArgumentError, "TimeFrame should be initialized with Symbol, Hash, Range or Btrack::TimeFrame" unless [Symbol, Hash, Range, TimeFrame, Time].include? timeframe.class
7
+
8
+ @from, @to = self.send("init_with_#{timeframe.class.name.demodulize.underscore}", timeframe)
9
+ @granularity = granularity || (timeframe.granularity if timeframe.is_a?(TimeFrame)) || Config.default_granularity
10
+ end
11
+
12
+ def splat(granularity=self.granularity)
13
+ [].tap do |keys|
14
+ (from.to_i .. to.to_i).step(step(granularity)) do |t|
15
+ keys << (block_given? ? (yield Time.at(t)) : Time.at(t))
16
+ end
17
+ end
18
+ end
19
+
20
+ private
21
+ def init_with_time_frame(timeframe)
22
+ [timeframe.from, timeframe.to]
23
+ end
24
+
25
+ def init_with_time(time)
26
+ [time.beginning_of_day, time.end_of_day]
27
+ end
28
+
29
+ def init_with_symbol(timeframe)
30
+ case timeframe
31
+ when :hour, :day, :week, :month, :year
32
+ return 1.send(timeframe).ago, Time.now
33
+ when :today
34
+ return Time.now.beginning_of_day, Time.now
35
+ when :yesterday
36
+ return 1.day.ago.beginning_of_day, 1.day.ago.end_of_day
37
+ when :this_week
38
+ return Time.now.beginning_of_week, Time.now
39
+ when :last_week
40
+ return 1.week.ago.beginning_of_week, 1.week.ago.end_of_week
41
+ when :this_month
42
+ return Time.now.beginning_of_month, Time.now
43
+ when :last_month
44
+ return 1.month.ago.beginning_of_month, 1.month.ago.end_of_month
45
+ else
46
+ return 1.day.ago, Time.now
47
+ end
48
+ end
49
+
50
+ def init_with_hash(timeframe)
51
+ [timeframe[:from] && timeframe[:from].is_a?(String) && Time.parse(timeframe[:from]) || timeframe[:from] || 1.month.ago,
52
+ timeframe[:to] && timeframe[:to].is_a?(String) && Time.parse(timeframe[:to]) || timeframe[:to] || Time.now]
53
+ end
54
+
55
+ def init_with_range(timeframe)
56
+ init_with_hash(from: timeframe.first, to: timeframe.last)
57
+ end
58
+
59
+ def step(g)
60
+ {minute: 1.minute, hourly: 1.hour, daily: 1.day, weekly: 1.week, monthly: 1.month, yearly: 1.year}[g] || 1.day
61
+ end
62
+ end
63
+ end
@@ -0,0 +1,68 @@
1
+ module Btrack
2
+ class Tracker
3
+ class << self
4
+ #
5
+ # Tracks an activity of an entity
6
+ #
7
+ # For example: User 123 has just logged in
8
+ # Btrack::Tracker.track :logged_in, 123
9
+ #
10
+ # Can be called in 3 different ways (for convenience): with args, hash or a block
11
+ #
12
+ # Example Using a block
13
+ # Btrack::Tracker.track do |b|
14
+ # b.key = "logged_in"
15
+ # b.id = 123
16
+ # b.granularity = :daily..:monthly
17
+ # b.when = 3.days.ago
18
+ # b.expiration_for = { daily:6.days }
19
+ # end
20
+ #
21
+ # See documentation for full details on args and options
22
+ def track(*args, &block)
23
+ return track_with_block(&block) if block_given?
24
+ return track_with_hash if args.first === Hash
25
+
26
+ track_with_args(*args)
27
+ end
28
+
29
+ private
30
+
31
+ def track_with_hash(options)
32
+ granularity_range(options[:granularity] || Config.default_granularity).each do |g|
33
+ key = Helper.key options[:key], g, options[:when]
34
+
35
+ Btrack::Redis.with_silent do
36
+ Btrack.redis.pipelined do |r|
37
+ r.setbit key, options[:id].to_i, 1
38
+ r.expire key, options[:expiration_for] && options[:expiration_for][g] || Config.expiration_for(g)
39
+ end
40
+ end
41
+ end
42
+ end
43
+
44
+ def track_with_block
45
+ yield options = OpenStruct.new
46
+
47
+ track_with_hash options
48
+ end
49
+
50
+ def track_with_args(*args)
51
+ options = args.extract_options!.merge ({
52
+ key: args[0],
53
+ id: args[1],
54
+ granularity: args[2]
55
+ })
56
+
57
+ track_with_hash options
58
+ end
59
+
60
+ def granularity_range(granularities)
61
+ return [granularities].flatten unless granularities.is_a? Range
62
+ predefined[predefined.index(granularities.first)..predefined.index(granularities.last)]
63
+ end
64
+
65
+ def predefined; [:minute, :hourly, :daily, :weekly, :monthly, :yearly]; end
66
+ end
67
+ end
68
+ end
@@ -0,0 +1,3 @@
1
+ module Btrack
2
+ VERSION = "1.0.1"
3
+ end
@@ -0,0 +1,5 @@
1
+ require 'spec_helper'
2
+
3
+ describe Btrack do
4
+
5
+ end
@@ -0,0 +1,19 @@
1
+ require 'spec_helper'
2
+
3
+ describe Btrack::Config do
4
+
5
+ Btrack.config do |config|
6
+ config.namespace = "test"
7
+ end
8
+
9
+ it "sets a namespace" do
10
+ assert { Btrack::Helper.key(:logged_in).starts_with? "test" }
11
+ end
12
+
13
+ it "sets expiration time" do
14
+ expected = Btrack.config.expirations.dup
15
+ Btrack.config.expiration_for = {weekly: 2.days}
16
+
17
+ assert { Btrack.config.expirations == expected.merge(weekly: 2.days) }
18
+ end
19
+ end
@@ -0,0 +1,51 @@
1
+ require 'spec_helper'
2
+
3
+ describe Btrack::Query do
4
+
5
+ before :each do
6
+ 10.times do |i|
7
+ Btrack.track :logged_in, i, :hourly..:monthly # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
8
+ Btrack.track :logged_in, i*2, when: 1.days.ago # [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
9
+
10
+ Btrack.track :visited, i if i%3 == 0 # [0, 3, 6, 9]
11
+ end
12
+ end
13
+
14
+
15
+ it "returns count of unique logins for today" do
16
+ assert { Btrack.where(logged_in: :today).count == 10 }
17
+ end
18
+
19
+ it "returns count of unique logins yesterday" do
20
+ assert { Btrack.where(logged_in: :yesterday).count == 10 }
21
+ end
22
+
23
+ it "returns count of unique logins in a range of time" do
24
+ assert { Btrack.where(logged_in: 1.days.ago..Time.now).count == 15 }
25
+ end
26
+
27
+ it "returns count with weekly granularity" do
28
+ assert { Btrack.where([{logged_in: :this_week, granularity: :weekly}]).count == 15 }
29
+ end
30
+
31
+ it "returns the intersection of two different time frames" do
32
+ assert { Btrack.where([{logged_in: :today}, {logged_in: :yesterday}]).count == 5 }
33
+ end
34
+
35
+ it "returns the intersection of two different activities" do
36
+ assert { Btrack.where([{logged_in: :yesterday}, {visited: :today}]).count == 2 }
37
+ end
38
+
39
+ it "check if id exists in activity" do
40
+ assert { Btrack.where(logged_in: :today).exists? 5 }
41
+ end
42
+
43
+ it "checks if id did two activities (intersection)" do
44
+ assert { Btrack.where([{logged_in: :yesterday}, {visited: :today}], id: 6).exists? }
45
+ end
46
+
47
+ it "should return false for id that does not satisfies an intersection" do
48
+ assert { Btrack.where([{logged_in: :yesterday}, {visited: :today}], id: 3).exists? == false }
49
+ end
50
+
51
+ end
@@ -0,0 +1,19 @@
1
+ require 'rubygems'
2
+ require 'bundler/setup'
3
+ require "wrong/adapters/rspec"
4
+ require 'btrack'
5
+
6
+ RSpec.configure do |config|
7
+ config.before :each do
8
+ Btrack.redis.select 15
9
+ Btrack.redis.flushdb
10
+ end
11
+ end
12
+
13
+ def is_set(key, id, granularity = :daily, w = Time.now)
14
+ Btrack.redis.getbit(Btrack::Helper.key(key, granularity, w), id) == 1
15
+ end
16
+
17
+ def ttl(key, id, granularity = :daily, w = Time.now)
18
+ Btrack.redis.ttl(Btrack::Helper.key(key, granularity, w))
19
+ end
@@ -0,0 +1,15 @@
1
+ require 'spec_helper'
2
+
3
+ describe Btrack::TimeFrame do
4
+ it "accepts time as from and to" do
5
+ tf = Btrack::TimeFrame.new from: 10.days.ago, to: Time.now
6
+ tf.from.to_i.should eq 10.days.ago.to_i
7
+ tf.to.to_i.should eq Time.now.to_i
8
+ end
9
+
10
+ it "accepts range as from and to" do
11
+ tf = Btrack::TimeFrame.new 10.days.ago..Time.now
12
+ tf.from.to_i.should eq 10.days.ago.to_i
13
+ tf.to.to_i.should eq Time.now.to_i
14
+ end
15
+ end
@@ -0,0 +1,42 @@
1
+ require 'spec_helper'
2
+ require 'btrack/helper'
3
+
4
+ describe Btrack::Tracker do
5
+
6
+ it "tracks with basic args" do
7
+ Btrack::Tracker.track("logged_in", 123, :weekly)
8
+ assert { is_set "logged_in", 123, :weekly }
9
+ end
10
+
11
+ it "tracks with default granularity" do
12
+ Btrack::Tracker.track("logged_in", 123)
13
+ assert { is_set "logged_in", 123 }
14
+ end
15
+
16
+ it "accepts symbol as key" do
17
+ Btrack::Tracker.track(:logged_in, 123)
18
+ assert { is_set :logged_in, 123 }
19
+ end
20
+
21
+ it "accepts different time (past activity)" do
22
+ Btrack::Tracker.track(:logged_in, 123, :daily, when: 3.days.ago)
23
+ assert { is_set :logged_in, 123, :daily, 3.days.ago }
24
+ end
25
+
26
+ it "track with expiration time set" do
27
+ Btrack::Tracker.track(:logged_in, 123, :daily, expiration_for: { daily: 9.days })
28
+ assert { ttl(:logged_in, 123, :daily) == 9.days }
29
+ end
30
+
31
+ it "tracks with a block" do
32
+ Btrack::Tracker.track do |t|
33
+ t.key = :logged_in
34
+ t.id = 123
35
+ t.granularity = :weekly
36
+ t.when = 2.days.ago
37
+ end
38
+
39
+ assert { is_set :logged_in, 123, :weekly, 2.days.ago }
40
+ end
41
+
42
+ end
metadata ADDED
@@ -0,0 +1,161 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: btrack
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.1
5
+ platform: ruby
6
+ authors:
7
+ - Chen Fisher
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2014-07-18 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: activesupport
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: '0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: '0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: redis
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - ">="
32
+ - !ruby/object:Gem::Version
33
+ version: '0'
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ version: '0'
41
+ - !ruby/object:Gem::Dependency
42
+ name: bundler
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: '1.3'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '1.3'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rake
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - ">="
67
+ - !ruby/object:Gem::Version
68
+ version: '0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: rspec
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - ">="
74
+ - !ruby/object:Gem::Version
75
+ version: '0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: wrong
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ description: Enables tracking and querying of any activity in a website or process
98
+ with minimum memory signature and maximum performance (thanks to redis)
99
+ email:
100
+ - chen.fisher@gmail.com
101
+ executables: []
102
+ extensions: []
103
+ extra_rdoc_files: []
104
+ files:
105
+ - ".gitignore"
106
+ - ".rspec"
107
+ - ".ruby-version"
108
+ - Gemfile
109
+ - LICENSE.txt
110
+ - README.md
111
+ - Rakefile
112
+ - btrack.gemspec
113
+ - lib/btrack.rb
114
+ - lib/btrack/config.rb
115
+ - lib/btrack/helper.rb
116
+ - lib/btrack/query.rb
117
+ - lib/btrack/query/criteria.rb
118
+ - lib/btrack/query/lua.rb
119
+ - lib/btrack/redis.rb
120
+ - lib/btrack/time_frame.rb
121
+ - lib/btrack/tracker.rb
122
+ - lib/btrack/version.rb
123
+ - spec/btrack_spec.rb
124
+ - spec/config_spec.rb
125
+ - spec/query_spec.rb
126
+ - spec/spec_helper.rb
127
+ - spec/time_frame_spec.rb
128
+ - spec/tracker_spec.rb
129
+ homepage: ''
130
+ licenses:
131
+ - MIT
132
+ metadata: {}
133
+ post_install_message:
134
+ rdoc_options: []
135
+ require_paths:
136
+ - lib
137
+ required_ruby_version: !ruby/object:Gem::Requirement
138
+ requirements:
139
+ - - ">="
140
+ - !ruby/object:Gem::Version
141
+ version: '0'
142
+ required_rubygems_version: !ruby/object:Gem::Requirement
143
+ requirements:
144
+ - - ">="
145
+ - !ruby/object:Gem::Version
146
+ version: '0'
147
+ requirements: []
148
+ rubyforge_project:
149
+ rubygems_version: 2.2.2
150
+ signing_key:
151
+ specification_version: 4
152
+ summary: Activity tracker with minimum memory footprint and maximum performance (thanks
153
+ to redis)
154
+ test_files:
155
+ - spec/btrack_spec.rb
156
+ - spec/config_spec.rb
157
+ - spec/query_spec.rb
158
+ - spec/spec_helper.rb
159
+ - spec/time_frame_spec.rb
160
+ - spec/tracker_spec.rb
161
+ has_rdoc: