tally 1.0.0.beta1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (53) hide show
  1. checksums.yaml +7 -0
  2. data/README.md +217 -0
  3. data/Rakefile +32 -0
  4. data/app/assets/config/tally_manifest.js +2 -0
  5. data/app/assets/javascripts/tally/application.js +15 -0
  6. data/app/assets/stylesheets/tally/application.css +15 -0
  7. data/app/controllers/tally/application_controller.rb +17 -0
  8. data/app/controllers/tally/days_controller.rb +19 -0
  9. data/app/controllers/tally/keys_controller.rb +19 -0
  10. data/app/controllers/tally/records_controller.rb +21 -0
  11. data/app/helpers/tally/application_helper.rb +4 -0
  12. data/app/helpers/tally/increment_helper.rb +15 -0
  13. data/app/jobs/tally/application_job.rb +4 -0
  14. data/app/jobs/tally/calculator_runner_job.rb +12 -0
  15. data/app/models/tally/application_record.rb +7 -0
  16. data/app/models/tally/record.rb +30 -0
  17. data/app/presenters/tally/record_presenter.rb +29 -0
  18. data/bin/annotate +29 -0
  19. data/bin/appraisal +29 -0
  20. data/bin/bundle +114 -0
  21. data/bin/coderay +29 -0
  22. data/bin/htmldiff +29 -0
  23. data/bin/ldiff +29 -0
  24. data/bin/nokogiri +29 -0
  25. data/bin/pry +29 -0
  26. data/bin/rackup +29 -0
  27. data/bin/rails +14 -0
  28. data/bin/rake +29 -0
  29. data/bin/rspec +29 -0
  30. data/bin/rubocop +29 -0
  31. data/bin/ruby-parse +29 -0
  32. data/bin/ruby-rewrite +29 -0
  33. data/bin/sprockets +29 -0
  34. data/bin/thor +29 -0
  35. data/config/routes.rb +12 -0
  36. data/db/migrate/20181016172358_create_tally_records.rb +15 -0
  37. data/lib/tally.rb +77 -0
  38. data/lib/tally/archiver.rb +64 -0
  39. data/lib/tally/calculator.rb +30 -0
  40. data/lib/tally/calculator_runner.rb +56 -0
  41. data/lib/tally/calculators.rb +29 -0
  42. data/lib/tally/countable.rb +23 -0
  43. data/lib/tally/daily.rb +64 -0
  44. data/lib/tally/engine.rb +28 -0
  45. data/lib/tally/increment.rb +25 -0
  46. data/lib/tally/key_finder.rb +123 -0
  47. data/lib/tally/key_finder/entry.rb +66 -0
  48. data/lib/tally/keyable.rb +55 -0
  49. data/lib/tally/record_searcher.rb +110 -0
  50. data/lib/tally/sweeper.rb +46 -0
  51. data/lib/tally/version.rb +5 -0
  52. data/lib/tasks/tally_tasks.rake +16 -0
  53. metadata +256 -0
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 57c0cfe3a4ada2609c144048c1fe8ef3d6ad35ecf5a1765ae617c25b94e576b8
4
+ data.tar.gz: 36ee4d684c9b2a3a0b0c3923b2bfa38df0f4594a4451500b8c2400b248fb0d12
5
+ SHA512:
6
+ metadata.gz: 4e689ddbb2990f5a3089f3ad72480bca5581c69b774373c67a4ca4da98258568f2527b710c24ef3bc8301327580a2f709d84f62af1642be10e55abebd274839f
7
+ data.tar.gz: e5dc26b8f72fe52c247ce1bad9af2666cb07b382f30ce0571497b1be24f9b77313d734c1e179e44ba0ed932845f424d0813b7afa1221a802634e58c3146e2512
@@ -0,0 +1,217 @@
1
+ # Tally
2
+
3
+ [![CircleCI](https://circleci.com/gh/jdtornow/tally.svg?style=svg)](https://circleci.com/gh/jdtornow/tally)
4
+
5
+ Tally is a simple Rails engine for capturing counts of various activities around an app. These counts are quickly captured in Redis then are archived periodically within the app's default relational database.
6
+
7
+ Counts are captured in Redis to make them as quick as possible and not slow down your UI with unnecessary database calls.
8
+
9
+ Tally can be used to capture counts of anything in your app. It is a great local and private alternative to some basic analytics tools. Tally has been used to keep track of pageviews, image impressions, newsletter clicks, new signups, and more. All of Tally's counts are archived on a daily basis, so it makes for easy reporting and trending summaries too.
10
+
11
+ ## Requirements
12
+
13
+ * Ruby 2.2+
14
+ * Rails 5.2.x+
15
+ * Redis 4+
16
+
17
+ ## Installation
18
+
19
+ This gem is a work in process, and only available via Github currently. Installed via bundler in your `Gemfile`:
20
+
21
+ ```ruby
22
+ gem "tally", github: "jdtornow/tally"
23
+ ```
24
+
25
+ Once the gem is installed, make sure to run `rails db:migrate` to add the `tally_records` table.
26
+
27
+ ## Usage
28
+
29
+ ### Collecting counts
30
+
31
+ The basic usage of Tally is by incrementing counters. To increment a counter, just use the `Tally.increment` method.
32
+
33
+ ```ruby
34
+ # increment the "views" counter by 1
35
+ Tally.increment(:views)
36
+
37
+ # increment the "views" counter by 3
38
+ Tally.increment(:views, 3)
39
+ ```
40
+
41
+ If you're inside a Rails view, you can capture counts inline too using the `increment_tally` method:
42
+
43
+ ```rails
44
+ <div class="some-great-content">
45
+ <% increment_tally :content_views %>
46
+ </div>
47
+ ```
48
+
49
+ Typically you'd want to do this within a controller, but the view helpers are there to, um help, as needed.
50
+
51
+ In addition to basic global counters, you can also attach a counter to a specific ActiveRecord model. The `Tally::Countable` mixin can be included in a specific model, or within your `ApplicationRecord` to use globally.
52
+
53
+ For example, to increment a specific record's views:
54
+
55
+ ```ruby
56
+ # in app/models/post.rb
57
+ class Post < ApplicationRecord
58
+
59
+ include Tally::Countable
60
+
61
+ end
62
+ ```
63
+
64
+ Then, in the controller method where a post is displayed:
65
+
66
+ ```ruby
67
+ # in a controller method
68
+ class PostsController < ApplicationController
69
+
70
+ def show
71
+ @post = Post.find(params[:id])
72
+ @post.increment_tally(:views)
73
+ end
74
+
75
+ end
76
+ ```
77
+
78
+ ### Archiving counts
79
+
80
+ By default all counts are stored in a temporary location within Redis. They can be archived periodically into your primary database. An hourly archive would be reasonable.
81
+
82
+ To archive counts into the database, run one of the following Rake tasks:
83
+
84
+ ```bash
85
+ # archive the current day's records
86
+ rails tally:archive
87
+
88
+ # archive's the previous day's records, useful to run at midnight UTC to capture the previous day's counts
89
+ rails tally:archive:yesterday
90
+ ```
91
+
92
+ In addition to the rake tasks available, counts can be archived using `Tally::Archiver` with a few more options:
93
+
94
+ ```ruby
95
+ # archive current day's records
96
+ Tally::Archiver.archive(day: Date.today)
97
+
98
+ # archive current days's records for a given key
99
+ Tally::Archiver.archive(day: Date.today, key: :impressions)
100
+
101
+ # archive yesterday's records for a given model
102
+ Tally::Archiver.archive(day: 1.day.ago, record: Post.first)
103
+ ```
104
+
105
+ **Please note that the archiving step is an important one** because by default the counters will expire after a few days in Redis. This is done by design, so your Redis instance doesn't fill up with endless count data.
106
+
107
+ #### Count expiration
108
+
109
+ By default, Redis counters are kept for 4 days. To change the default time for Redis counters to be kept, adjust the `ttl` configuration:
110
+
111
+ ```ruby
112
+ # keep day counts for 30 days before they automatically expire
113
+ Tally.config.ttl = 30.days
114
+
115
+ # don't expire counts (warning: this may fill up a small redis instance over time)
116
+ Tally.config.ttl = nil
117
+ ```
118
+
119
+ ### Custom archive calculators
120
+
121
+ In addition to the default archive behavior, Tally can run additional archive classes each time the archive commands above are run. This is useful to perform aggregate calculations or pull stats from other sources to archive.
122
+
123
+ Custom archive calculators just accept a `Date` to summarize, and then have a `#call` method that returns an array of any new records to archive. Each record should be a hash with a `value` and `key`.
124
+
125
+ For example, the following calculator does a count of all blog posts as of the given date. This can be useful to show a trending chart over time of the number of posts on a blog:
126
+
127
+ ```ruby
128
+ # in config/initializers/tally.rb
129
+
130
+ # the calculator class is registered as a string so it can be dynamically loaded as needed,
131
+ # instead of on boot time
132
+ Tally.register_calculator "PostsCountCalculator"
133
+ ```
134
+
135
+ Then, somewhere in your app folder likely, would go this class. It doesn't need to go anywhere in particular, but if you have many of them, a folder to organize might be helpful.
136
+
137
+ ```
138
+ # app/calculators/posts_count_calculator.rb
139
+ class PostsCountCalculator
140
+
141
+ include Tally::Calculator
142
+
143
+ def call
144
+ posts = Post.where("created_at <= ?", day.end_of_day).count
145
+
146
+ {
147
+ key: :posts_count,
148
+ value: posts
149
+ }
150
+ end
151
+
152
+ end
153
+ ```
154
+
155
+ By default, calculators are run in the background using ActiveJob. If you'd prefer to run them inline, set the `perform_calculators` config option to `:now`:
156
+
157
+ ```ruby
158
+ Tally.config.perform_calculators = :now
159
+ ```
160
+
161
+ ### Displaying counts
162
+
163
+ After the archive commands are run, all counts are placed into the `Tally::Record` model. This is a standard ActiveRecord model that can be used as you see fit.
164
+
165
+ There are few built-in ways to explore the archived counts in your database. First, the `Tally::RecordSearcher` is a handy tool for finding counts. It just uses ActiveRecord query syntax to build a scope on top of `Tally::Record`.
166
+
167
+ ```ruby
168
+ # find all visit records in a given date range
169
+ records = Tally::RecordSearcher.search(key: "views", start_date: "2020-01-01", end_date: "2020-01-31")
170
+
171
+ # find all views for a given post by day
172
+ post = Post.first
173
+ records = Tally::RecordSearcher.search(key: "views", record: post)
174
+ views_by_day = Tally::RecordSearcher.search(key: "views", record: post).group(:day).sum(:value)
175
+
176
+ # get total views for all posts
177
+ Tally::RecordSearcher.search(key: "views", type: "Post").sum(:value)
178
+ ```
179
+
180
+ To display counts in a web service, `Tally::Engine` can be mounted to add a few endpoints. Please note that this endpoints are not protected with authentication, so you will want to handle accordingly in your routes with a constraint or something.
181
+
182
+ ```ruby
183
+ # in config/routes.rb
184
+
185
+ mount Tally::Engine, at: "/tally"
186
+ ```
187
+
188
+ This adds the following routes to your app:
189
+
190
+ ```text
191
+ recordable_days GET /days/:type/:id(.:format) tally/days#index
192
+ days GET /days(.:format) tally/days#index
193
+ recordable_keys GET /keys/:type/:id(.:format) tally/keys#index
194
+ keys GET /keys(.:format) tally/keys#index
195
+ recordable_records GET /records/:type/:id(.:format) tally/records#index
196
+ records GET /records(.:format) tally/records#index
197
+ ```
198
+
199
+ The endpoints can be used to display JSON-formatted data from `Tally::Record`. These endpoints are useful for turning stats into charts or other formatted data in your front-end. The endpoints are entirely optional, and aren't included by default.
200
+
201
+ ## Redis Connection
202
+
203
+ Tally works _really_ well with [Sidekiq](https://github.com/mperham/sidekiq/), but it isn't required. If Sidekiq is installed in your app, Tally will use its connection pooling for Redis connections. If Sidekiq isn't in use, the `Redis.current` connection is used to store stats. If you'd like to override the specific connection used for Tally's redis store, you can do so by setting `Tally.redis_connection` to another instance of `Redis`. This can be useful to use an alternate Redis store for just stats, for example.
204
+
205
+ ```ruby
206
+ # use an alternate Redis connection (for non-sidekiq integrations)
207
+ Tally.redis_connection = Redis.new(...)
208
+ ```
209
+ ## Issues
210
+
211
+ If you have any issues or find bugs running Tally, please [report them on Github](https://github.com/jdtornow/tally/issues).
212
+
213
+ ## License
214
+
215
+ Tally is released under the [MIT license](http://www.opensource.org/licenses/MIT)
216
+
217
+ Contributions and pull-requests are more than welcome.
@@ -0,0 +1,32 @@
1
+ begin
2
+ require 'bundler/setup'
3
+ rescue LoadError
4
+ puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
5
+ end
6
+
7
+ require 'rdoc/task'
8
+
9
+ RDoc::Task.new(:rdoc) do |rdoc|
10
+ rdoc.rdoc_dir = 'rdoc'
11
+ rdoc.title = 'Tally'
12
+ rdoc.options << '--line-numbers'
13
+ rdoc.rdoc_files.include('README.md')
14
+ rdoc.rdoc_files.include('lib/**/*.rb')
15
+ end
16
+
17
+ APP_RAKEFILE = File.expand_path("spec/dummy/Rakefile", __dir__)
18
+ load 'rails/tasks/engine.rake'
19
+
20
+ load 'rails/tasks/statistics.rake'
21
+
22
+ require 'bundler/gem_tasks'
23
+
24
+ require 'rake/testtask'
25
+
26
+ Rake::TestTask.new(:test) do |t|
27
+ t.libs << 'test'
28
+ t.pattern = 'test/**/*_test.rb'
29
+ t.verbose = false
30
+ end
31
+
32
+ task default: :test
@@ -0,0 +1,2 @@
1
+ //= link_directory ../javascripts/tally .js
2
+ //= link_directory ../stylesheets/tally .css
@@ -0,0 +1,15 @@
1
+ // This is a manifest file that'll be compiled into application.js, which will include all the files
2
+ // listed below.
3
+ //
4
+ // Any JavaScript/Coffee file within this directory, lib/assets/javascripts, vendor/assets/javascripts,
5
+ // or any plugin's vendor/assets/javascripts directory can be referenced here using a relative path.
6
+ //
7
+ // It's not advisable to add code directly here, but if you do, it'll appear at the bottom of the
8
+ // compiled file. JavaScript code in this file should be added after the last require_* statement.
9
+ //
10
+ // Read Sprockets README (https://github.com/rails/sprockets#sprockets-directives) for details
11
+ // about supported directives.
12
+ //
13
+ //= require rails-ujs
14
+ //= require activestorage
15
+ //= require_tree .
@@ -0,0 +1,15 @@
1
+ /*
2
+ * This is a manifest file that'll be compiled into application.css, which will include all the files
3
+ * listed below.
4
+ *
5
+ * Any CSS and SCSS file within this directory, lib/assets/stylesheets, vendor/assets/stylesheets,
6
+ * or any plugin's vendor/assets/stylesheets directory can be referenced here using a relative path.
7
+ *
8
+ * You're free to add application-wide styles to this file and they'll appear at the bottom of the
9
+ * compiled file so the styles you add here take precedence over styles defined in any other CSS/SCSS
10
+ * files in this directory. Styles in this file should be added after the last require_* statement.
11
+ * It is generally better to create a new file per style scope.
12
+ *
13
+ *= require_tree .
14
+ *= require_self
15
+ */
@@ -0,0 +1,17 @@
1
+ module Tally
2
+ class ApplicationController < ActionController::Base
3
+
4
+ protect_from_forgery with: :exception
5
+
6
+ private
7
+
8
+ def search
9
+ @search ||= RecordSearcher.new(search_params)
10
+ end
11
+
12
+ def search_params
13
+ params.permit(:type, :id, :key, :start_date, :end_date)
14
+ end
15
+
16
+ end
17
+ end
@@ -0,0 +1,19 @@
1
+ module Tally
2
+ class DaysController < Tally::ApplicationController
3
+
4
+ def index
5
+ records = search.days.page(params[:page]).per(params[:per_page] || 24).without_count
6
+
7
+ render json: {
8
+ days: records.map(&:day),
9
+ meta: {
10
+ next_page: records.next_page,
11
+ current_page: records.current_page,
12
+ previous_page: records.prev_page,
13
+ per_page: records.limit_value
14
+ }
15
+ }
16
+ end
17
+
18
+ end
19
+ end
@@ -0,0 +1,19 @@
1
+ module Tally
2
+ class KeysController < Tally::ApplicationController
3
+
4
+ def index
5
+ records = search.keys.page(params[:page]).per(params[:per_page] || 24).without_count
6
+
7
+ render json: {
8
+ keys: records.map(&:key),
9
+ meta: {
10
+ next_page: records.next_page,
11
+ current_page: records.current_page,
12
+ previous_page: records.prev_page,
13
+ per_page: records.limit_value
14
+ }
15
+ }
16
+ end
17
+
18
+ end
19
+ end
@@ -0,0 +1,21 @@
1
+ module Tally
2
+ class RecordsController < Tally::ApplicationController
3
+
4
+ def index
5
+ records = search.records.page(params[:page]).per(params[:per_page] || 24).without_count
6
+
7
+ render json: {
8
+ records: records.map { |record|
9
+ RecordPresenter.new(record).to_hash
10
+ },
11
+ meta: {
12
+ next_page: records.next_page,
13
+ current_page: records.current_page,
14
+ previous_page: records.prev_page,
15
+ per_page: records.limit_value
16
+ }
17
+ }
18
+ end
19
+
20
+ end
21
+ end
@@ -0,0 +1,4 @@
1
+ module Tally
2
+ module ApplicationHelper
3
+ end
4
+ end
@@ -0,0 +1,15 @@
1
+ module Tally
2
+ module IncrementHelper
3
+
4
+ def increment_tally(*args)
5
+ Increment.public_send(:increment, *args)
6
+
7
+ nil
8
+
9
+ # This method should never block UI
10
+ rescue
11
+ nil
12
+ end
13
+
14
+ end
15
+ end
@@ -0,0 +1,4 @@
1
+ module Tally
2
+ class ApplicationJob < ActiveJob::Base
3
+ end
4
+ end
@@ -0,0 +1,12 @@
1
+ module Tally
2
+ class CalculatorRunnerJob < ApplicationJob
3
+
4
+ def perform(class_name, date_str)
5
+ date = Date.parse(date_str)
6
+
7
+ runner = CalculatorRunner.new(class_name, date)
8
+ runner.save
9
+ end
10
+
11
+ end
12
+ end
@@ -0,0 +1,7 @@
1
+ module Tally
2
+ class ApplicationRecord < ActiveRecord::Base
3
+
4
+ self.abstract_class = true
5
+
6
+ end
7
+ end
@@ -0,0 +1,30 @@
1
+ # == Schema Information
2
+ #
3
+ # Table name: tally_records
4
+ #
5
+ # id :integer not null, primary key
6
+ # day :date
7
+ # recordable_id :integer
8
+ # recordable_type :string
9
+ # key :string
10
+ # value :integer default(0)
11
+ # created_at :datetime not null
12
+ # updated_at :datetime not null
13
+ #
14
+
15
+ module Tally
16
+ class Record < ApplicationRecord
17
+
18
+ validates :day, presence: true
19
+ validates :key, presence: true
20
+
21
+ belongs_to :recordable, polymorphic: true, optional: true
22
+
23
+ scope :today, -> { where(day: Time.current.utc.to_date) }
24
+
25
+ def self.search(*args)
26
+ RecordSearcher.search(*args)
27
+ end
28
+
29
+ end
30
+ end