active_analytics 0.2.1 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8be54d2704a6f0c7141fd6c0c4d276992d13f8cd1126868a5e71d038ba4333e5
-  data.tar.gz: 4a6eb25b26894054f9a1b669a198a6f97d2e13bdf892d8b09b5f8da5251036db
+  metadata.gz: 909ce5d6ab8a28322be42f01b2d37c752ef810a31a139b7f5e035257d1c62fda
+  data.tar.gz: 1ca79cce7bbe415afa9341e752400891d6817e780a0b679fadcfeb1df6039786
 SHA512:
-  metadata.gz: efae89347949dc906204ee4188b278a4482e46a2575a83868c4ffc73915869c46f88517fc235c4976ebec9f7474bc26b6823066156aaa2ccffa9ffaeb1128164
-  data.tar.gz: 4b94f1808a3ce5da5e8050fda39270fb7f65990aecb708f0dc6e54174139fa5d9a4f4c87431917be607fed4ad3580710af84b8449f290bbebb78308086ed6f25
+  metadata.gz: 1835fa3cb335651ef42c46767b13da30829fb86a421f06989c203814c4b9199d784f714e4bf6885a4301befb4dae9fd1bad94b47fa660f12bd2425de4c713b51
+  data.tar.gz: 19fa332fbbdbc9be99e99b9a635bc57df3e2a32c4b9c0a12157c2e44446d3417eec0d702790c2a05b803858bafc50090d31ba787d5bb5e7cd52c87a4795c7e99

data/README.md

@@ -32,38 +32,100 @@ rails active_analytics:install:migrations
 rails db:migrate
 ```
 
-Your controllers have to call `ActiveAnalytics.record_request(request)` to record page views. The Rails way to achieve is to use a `before_action` :
+Add the route to ActiveAnalytics dashboard at the desired endpoint:
+
+```ruby
+# config/routes.rb
+mount ActiveAnalytics::Engine, at: "analytics"  # http://localhost:3000/analytics
+```
+
+The next step is to collect trafic and there is 2 options.
+
+### Record requests synchronously
+
+This is the easiest way to start with.
+However it's less performant since it triggers a write into your database for each request.
+Your controllers have to call `ActiveAnalytics.record_request(request)` to record page views.
+The Rails way to achieve is to use `after_action` :
 
 ```ruby
 class ApplicationController < ActionController::Base
-  before_action :record_page_view
+  after_action :record_page_view
 
   def record_page_view
-    # Add a condition to record only your canonical domain
-    # and use a gem such as crawler_detect to skip bots.
-    ActiveAnalytics.record_request(request)
+    # This is a basic example, you might need to customize some conditions.
+    # For most sites, it makes no sense to record anything other than HTML.
+    if response.content_type && response.content_type.start_with?("text/html")
+      # Add a condition to record only your canonical domain
+      # and use a gem such as crawler_detect to skip bots.
+      ActiveAnalytics.record_request(request)
+    end
   end
 end
 ```
 
-In case you don't want to record all page views, because each application has sensitive URLs such as password reset and so on, simply define a `skip_before_action :record_page_view` in the relevant controller.
+In case you don't want to record all page views, because each application has sensitive URLs such as password reset and so on, simply define a `skip_after_action :record_page_view` in the relevant controller.
+
+### Queue requests asynchronously
 
-Finally, just add the route to ActiveAnalytics dashboard at the desired endpoint:
+It requires more work and it's relevant if your application handle a large trafic.
+The idea is to queue data into Redis because it does not require the database writing to the disk on each request.
+First you have to set the Redis URL or connection.
 
 ```ruby
-mount ActiveAnalytics::Engine, at: "analytics"  # http://localhost:3000/analytics
+# File lib/patches/active_analytics.rb or config/initializers/active_analytics.rb
+
+ActiveAnalytics.redis_url = "redis://user:password@host/1" # Default ENV["ACTIVE_ANALYTICS_REDIS_URL"] || ENV["REDIS_URL"] || "redis://localhost"
+
+# If you use special connection options you have to instantiate it yourself
+ActiveAnalytics.redis = Redis.new(
+  url: ENV["REDIS_URL"],
+  reconnect_attempts: 10,
+  ssl_params: {verify_mode: OpenSSL::SSL::VERIFY_NONE}
+)
 ```
 
+Then your controllers have to call `ActiveAnalytics.queue_request(request)` to queue page views.
+The Rails way to achieve is to use `after_action` :
+
+```ruby
+class ApplicationController < ActionController::Base
+  after_action :queue_page_view
+
+  def queue_page_view
+    # This is a basic example, you might need to customize some conditions.
+    # For most sites, it makes no sense to record anything other than HTML.
+    if response.content_type && response.content_type.start_with?("text/html")
+      # Add a condition to record only your canonical domain
+      # and use a gem such as crawler_detect to skip bots.
+      ActiveAnalytics.queue_request(request)
+    end
+  end
+end
+```
+
+Queued data need to be saved into the database in order to be viewable in the ActiveAnalytics dashboard.
+For that, call `ActiveAnalytics.flush_queue` from a cron task or a background job.
+
+It's up to you if you want to flush the queue every hour or every 10 minutes.
+I advise to execute the last flush of the day at 23:59.
+It prevents from shifting the trafic to the next day.
+In that case only the last minute will be shifted to the next day, even if the flush ends after midnight.
+This small imperfection allows a simpler implementation for now.
+Keep it simple !
+
+
 ## Authentication and permissions
 
-ActiveAnalytics cannot guess how you handle user authentication, because it is different for all Rails applications. So you have to monkey patch `ActiveAnalytics::ApplicationController` in order to inject your own mechanism. Create a file in `config/initializers/active_analytics.rb` to add a before action :
+ActiveAnalytics cannot guess how you handle user authentication, because it is different for all Rails applications.
+So you have to monkey patch `ActiveAnalytics::ApplicationController` in order to inject your own mechanism.
+The patch can be saved wherever you want.
+For example, I like to have all the patches in one place, so I put them in `lib/patches`.
 
 ```ruby
-# config/initializers/active_analytics.rb
-require_dependency "active_analytics/application_controller"
+# lib/patches/active_analytics.rb
 
-module ActiveAnalytics
-  class ApplicationController
+ActiveAnalytics::ApplicationController.class_eval do
     before_action :require_admin
 
     def require_admin
@@ -74,7 +136,18 @@ module ActiveAnalytics
 end
 ```
 
-If you have Devise, you can check the permission directly from routes.rb :
+Then you have to require the monkey patch.
+Because it's loaded via require, it won't be reloaded in development.
+Since you are not supposed to change this file often, it should not be an issue.
+
+```ruby
+# config/application.rb
+config.after_initialize do
+  require "patches/active_analytics"
+end
+```
+
+If you use Devise, you can check the permission directly from routes.rb :
 
 ```ruby
 # config/routes.rb
@@ -88,4 +161,4 @@ The gem is available as open-source under the terms of the [MIT License](https:/
 
 Made by [Base Secrète](https://basesecrete.com).
 
-Rails developer? Check out [RoRvsWild](https://rorvswild.com), our Ruby on Rails application monitoring tool.
+Rails developer? Check out [RoRvsWild](https://rorvswild.com), our Ruby on Rails application monitoring tool.

data/app/controllers/active_analytics/application_controller.rb

@@ -3,16 +3,34 @@ module ActiveAnalytics
 
     private
 
+    def from_date
+      @from_date ||= Date.parse(params[:from])
+    end
+
+    def to_date
+      @to_date ||= Date.parse(params[:to])
+    end
+
     def require_date_range
-      if Date.parse(params[:from]) > Date.parse(params[:to])
-        redirect_to(params.to_unsafe_hash.merge(from: 7.days.ago.to_date, to: Date.today))
-      end
-    rescue TypeError, ArgumentError # Raise by Date.parse when invalid format
+      redirect_to(params.to_unsafe_hash.merge(from: to_date, to: from_date)) if from_date > to_date
+    rescue TypeError, ArgumentError # Raised by Date.parse when invalid format
       redirect_to(params.to_unsafe_hash.merge(from: 7.days.ago.to_date, to: Date.today))
     end
 
     def current_views_per_days
-      ViewsPerDay.where(site: params[:site]).between_dates(params[:from], params[:to])
+      ViewsPerDay.where(site: params[:site]).between_dates(from_date, to_date)
+    end
+
+    def previous_views_per_days
+      ViewsPerDay.where(site: params[:site]).between_dates(previous_from_date, previous_to_date)
+    end
+
+    def previous_from_date
+      from_date - (to_date - from_date)
+    end
+
+    def previous_to_date
+      to_date - (to_date - from_date)
     end
   end
 end

data/app/controllers/active_analytics/assets_controller.rb

@@ -0,0 +1,29 @@
+require_dependency "active_analytics/application_controller"
+
+module ActiveAnalytics
+  class AssetsController < ApplicationController
+    protect_from_forgery except: :show
+
+    def show
+      if endpoints.include?(File.basename(request.path))
+        expires_in(1.day, public: true)
+        render(params[:id], mime_type: mime_type)
+      else
+        raise ActionController::RoutingError.new
+      end
+    end
+
+    private
+
+    def endpoints
+      return @endpoints if @endpoints
+      folder = ActiveAnalytics::Engine.root.join("app/views", controller_path)
+      files = folder.each_child.map { |path| File.basename(path).delete_suffix(".erb") }
+      @endpoints = files.delete_if { |str| str.start_with?("_") }
+    end
+
+    def mime_type
+      Mime::Type.lookup_by_extension(File.extname(request.path).delete_prefix("."))
+    end
+  end
+end

data/app/controllers/active_analytics/pages_controller.rb

@@ -7,19 +7,18 @@ module ActiveAnalytics
     before_action :require_date_range
 
     def index
-      scope = ViewsPerDay.where(site: params[:site]).between_dates(params[:from], params[:to])
-      @histogram = ViewsPerDay::Histogram.new(scope.order_by_date.group_by_date, params[:from], params[:to])
-      @pages = scope.top(100).group_by_page
+      @histogram = ViewsPerDay::Histogram.new(current_views_per_days.order_by_date.group_by_date, from_date, to_date)
+      @previous_histogram = ViewsPerDay::Histogram.new(previous_views_per_days.order_by_date.group_by_date, previous_from_date, previous_to_date)
+      @pages = current_views_per_days.top(100).group_by_page
     end
 
     def show
-      dates_scopes = ViewsPerDay.between_dates(params[:from], params[:to])
-      page_scope = dates_scopes.where(site: params[:site], page: page_from_params)
-      @histogram = ViewsPerDay::Histogram.new(page_scope.order_by_date.group_by_date, params[:from], params[:to])
-      @referrers = page_scope.top.group_by_referrer_site
-
-      @next_pages = dates_scopes.where(referrer_host: params[:site], referrer_path: page_from_params).top(100).group_by_page
-      @previous_pages = dates_scopes.where(site: params[:site], page: page_from_params).where.not(referrer_path: nil).top(100).group_by_referrer_page
+      page_scope = current_views_per_days.where(page: page_from_params)
+      previous_page_scope = previous_views_per_days.where(page: page_from_params)
+      @histogram = ViewsPerDay::Histogram.new(page_scope.order_by_date.group_by_date, from_date, to_date)
+      @previous_histogram = ViewsPerDay::Histogram.new(previous_page_scope.order_by_date.group_by_date, previous_from_date, previous_to_date)
+      @next_pages = current_views_per_days.where(referrer_host: params[:site], referrer_path: page_from_params).top(100).group_by_page
+      @previous_pages = page_scope.top(100).group_by_referrer_page
     end
   end
 end

data/app/controllers/active_analytics/referrers_controller.rb

@@ -5,14 +5,18 @@ module ActiveAnalytics
     before_action :require_date_range
 
     def index
-      scope = ViewsPerDay.where(site: params[:site]).between_dates(params[:from], params[:to])
-      @referrers = scope.top(100).group_by_referrer_site
-      @histogram = ViewsPerDay::Histogram.new(scope.order_by_date.group_by_date, params[:from], params[:to])
+      @referrers = current_views_per_days.top(100).group_by_referrer_site
+      @histogram = ViewsPerDay::Histogram.new(current_views_per_days.order_by_date.group_by_date, from_date, to_date)
+      @previous_histogram = ViewsPerDay::Histogram.new(previous_views_per_days.order_by_date.group_by_date, previous_from_date, previous_to_date)
     end
 
     def show
-      scope = ViewsPerDay.where(site: params[:site], referrer_host: params[:referrer]).between_dates(params[:from], params[:to])
-      @histogram = ViewsPerDay::Histogram.new(scope.order_by_date.group_by_date, params[:from], params[:to])
+      referrer_host, referrer_path = params[:referrer].split("/", 2)
+      scope = current_views_per_days.where(referrer_host: referrer_host)
+      scope = scope.where(referrer_path: "/" + referrer_path) if referrer_path.present?
+      previous_scope = previous_views_per_days.where(referrer_host: params[:referrer])
+      @histogram = ViewsPerDay::Histogram.new(scope.order_by_date.group_by_date, from_date, to_date)
+      @previous_histogram = ViewsPerDay::Histogram.new(previous_scope.order_by_date.group_by_date, previous_from_date, previous_to_date)
       @previous_pages = scope.top(100).group_by_referrer_page
       @next_pages = scope.top(100).group_by_page
     end

data/app/controllers/active_analytics/sites_controller.rb

@@ -10,10 +10,10 @@ module ActiveAnalytics
     end
 
     def show
-      scope = current_views_per_days
-      @histogram = ViewsPerDay::Histogram.new(scope.order_by_date.group_by_date, params[:from], params[:to])
-      @referrers = scope.top.group_by_referrer_site
-      @pages = scope.top.group_by_page
+      @histogram = ViewsPerDay::Histogram.new(current_views_per_days.order_by_date.group_by_date, from_date, to_date)
+      @previous_histogram = ViewsPerDay::Histogram.new(previous_views_per_days.order_by_date.group_by_date, previous_from_date, previous_to_date)
+      @referrers = current_views_per_days.top.group_by_referrer_site
+      @pages = current_views_per_days.top.group_by_page
     end
   end
 end

data/app/models/active_analytics/views_per_day.rb

@@ -37,8 +37,10 @@ module ActiveAnalytics
       attr_reader :bars, :from_date, :to_date
 
       def initialize(scope, from_date, to_date)
+        @scope = scope
+        @from_date, @to_date = from_date, to_date
         @bars = scope.map { |day| Bar.new(day.day, day.total, self) }
-        fill_missing_days(@bars, Date.parse(from_date.to_s), Date.parse(to_date.to_s))
+        fill_missing_days(@bars, @from_date, @to_date)
       end
 
       def fill_missing_days(bars, from, to)
@@ -68,7 +70,11 @@ module ActiveAnalytics
         end
 
         def height
-          (value.to_f / histogram.max_value).round(2)
+          if histogram.max_value > 0
+            (value.to_f / histogram.max_value).round(2)            
+          else
+            0
+          end
         end
       end
     end
@@ -108,12 +114,25 @@ module ActiveAnalytics
     end
 
     def self.append(params)
+      total = params.delete(:total) || 1
       params[:site] = params[:site].downcase if params[:site]
       params[:page] = params[:page].downcase if params[:page]
       params[:referrer_path] = nil if params[:referrer_path].blank?
       params[:referrer_path] = params[:referrer_path].downcase if params[:referrer_path]
       params[:referrer_host] = params[:referrer_host].downcase if params[:referrer_host]
-      find_or_create_by!(params) if where(params).update_all("total = total + 1") == 0
+      where(params).first.try(:increment!, :total, total) || create!(params.merge(total: total))
+    end
+
+    SLASH = "/"
+
+    def self.split_referrer(referrer)
+      return [nil, nil] if referrer.blank?
+      if (uri = URI(referrer)).host.present?
+        [uri.host, uri.path.presence]
+      else
+        strings = referrer.split(SLASH, 2)
+        [strings[0], strings[1] ? SLASH + strings[1] : nil]
+      end
     end
   end
 end

data/app/views/active_analytics/assets/_charts.css

@@ -0,0 +1,249 @@
+/*
+ * Charts.css v0.9.0 (https://ChartsCSS.org/)
+ * Copyright 2020 Rami Yushuvaev
+ * Licensed under MIT
+ */
+.active-analytics .charts-css {
+  --chart-bg-color: transparent;
+  --heading-size: 0px;
+  --primary-axis-color: rgba(var(--color-grey-100), 1);
+  --primary-axis-style: solid;
+  --primary-axis-width: 1px;
+  --secondary-axes-color: rgba(var(--color-grey-50), 1);
+  --secondary-axes-style: solid;
+  --secondary-axes-width: 1px;
+  --data-axes-color: rgba(var(--color-grey-200), 1);
+  --data-axes-style: solid;
+  --data-axes-width: 1px;
+  --legend-border-color: rgba(var(--color-grey-200), 1);
+  position: relative;
+  display: block;
+  margin: 0;
+  padding: 0;
+  border: 0;
+}
+
+/*
+ * Chart wrapper element
+ */
+
+.active-analytics .charts-css,
+.active-analytics .charts-css::after,
+.active-analytics .charts-css::before,
+.active-analytics .charts-css *,
+.active-analytics .charts-css *::after,
+.active-analytics .charts-css *::before {
+  box-sizing: border-box;
+}
+
+/*
+ * Reset table element
+ */
+.active-analytics table.charts-css {
+  border-collapse: collapse;
+  border-spacing: 0;
+  empty-cells: show;
+  overflow: initial;
+  background-color: transparent;
+}
+
+.active-analytics table.charts-css caption,
+.active-analytics table.charts-css colgroup,
+.active-analytics table.charts-css thead,
+.active-analytics table.charts-css tbody,
+.active-analytics table.charts-css tr,
+.active-analytics table.charts-css th,
+.active-analytics table.charts-css td {
+  display: block;
+  margin: 0;
+  padding: 0;
+  border: 0;
+  background-color: transparent;
+}
+
+.active-analytics table.charts-css colgroup,
+.active-analytics table.charts-css thead,
+.active-analytics table.charts-css tfoot {
+  display: none;
+}
+
+
+/*
+ * Chart colors
+ */ 
+
+.active-analytics .charts-css.column tbody tr td {
+  background: rgba(var(--color-grey-100), 1);
+  padding: 0;
+}
+
+/*
+ * Chart data
+ */
+.active-analytics .charts-css.hide-data .data {
+  opacity: 0;
+}
+
+.active-analytics .charts-css.show-data-on-hover .data {
+  transition-duration: .3s;
+  opacity: 0;
+}
+
+.active-analytics .charts-css.show-data-on-hover tr:hover .data {
+  transition-duration: .3s;
+  opacity: 1;
+}
+
+/*
+ * Chart labels
+ */
+
+.active-analytics .charts-css.column:not(.show-labels) {
+  --labels-size: 0;
+}
+
+.active-analytics .charts-css.column:not(.show-labels) tbody tr th {
+  display: none;
+}
+
+.active-analytics .charts-css.column.show-labels {
+  --labels-size: 1.5rem;
+}
+
+.active-analytics .charts-css.column.show-labels tbody tr th {
+  display: flex;
+  justify-content: var(--labels-align, center);
+  align-items: center;
+  flex-direction: column;
+}
+
+@media (max-width: 600px) {
+  .active-analytics .charts-css.column.show-labels {
+    --labels-size: 0;
+  }
+  
+  .active-analytics .charts-css.column.show-labels tbody tr th {
+    display: none;
+  }
+}
+
+/*
+ * Chart axes
+ */
+.active-analytics .charts-css.column.show-primary-axis:not(.reverse) tbody tr {
+  border-block-end: var(--primary-axis-width) var(--primary-axis-style) var(--primary-axis-color);
+}
+
+.active-analytics .charts-css.column.show-primary-axis.reverse tbody tr {
+  border-block-start: var(--primary-axis-width) var(--primary-axis-style) var(--primary-axis-color);
+}
+
+.active-analytics .charts-css.column.show-5-secondary-axes:not(.reverse) tbody tr {
+  background-size: 100% 20%;
+  background-image: linear-gradient(var(--secondary-axes-color) var(--secondary-axes-width), transparent var(--secondary-axes-width));
+}
+
+.active-analytics .charts-css.column.show-5-secondary-axes.reverse tbody tr {
+  background-size: 100% 20%;
+  background-image: linear-gradient(0deg, var(--secondary-axes-color) var(--secondary-axes-width), transparent var(--secondary-axes-width));
+}
+
+/*
+ * Chart tooltips
+ */
+.active-analytics .charts-css .tooltip {
+  position: absolute;
+  z-index: 1;
+  bottom: 50%;
+  left: 50%;
+  transform: translateX(-50%);
+  width: max-content;
+  padding: 5px 10px;
+  border-radius: 6px;
+  visibility: hidden;
+  opacity: 0;
+  transition: opacity .3s;
+  background-color: rgba(var(--color-grey-500), 1);
+  color: rgba(var(--color-grey-00), 1);
+  text-align: center;
+  font-size: .9rem;
+}
+
+.active-analytics .charts-css .tooltip::after {
+  content: "";
+  position: absolute;
+  top: 100%;
+  left: 50%;
+  margin-left: -5px;
+  border-width: 5px;
+  border-style: solid;
+  border-color: rgba(var(--color-grey-500), 1) transparent transparent;
+}
+
+.active-analytics .charts-css tr:hover .tooltip {
+  visibility: visible;
+  opacity: 1;
+}
+
+/*
+ * Column Chart
+ */
+.active-analytics .charts-css.column tbody {
+  display: flex;
+  justify-content: space-between;
+  align-items: stretch;
+  width: 100%;
+  gap: 1px;
+  height: calc(100% - var(--heading-size));
+}
+
+.active-analytics .charts-css.column tbody tr {
+  position: relative;
+  flex-grow: 1;
+  flex-shrink: 1;
+  flex-basis: 0;
+  overflow-wrap: anywhere;
+  display: flex;
+  justify-content: flex-start;
+  min-width: 0;
+}
+
+.active-analytics .charts-css.column tbody tr th {
+  position: absolute;
+  right: 0;
+  left: 0;
+}
+
+.active-analytics .charts-css.column tbody tr td {
+  display: flex;
+  justify-content: center;
+  width: 100%;
+  height: calc(100% * var(--size, 1));
+  position: relative;
+}
+
+.active-analytics .charts-css.column:not(.reverse) tbody tr {
+  align-items: flex-end;
+  margin-block-end: var(--labels-size);
+}
+
+.active-analytics .charts-css.column:not(.reverse) tbody tr th {
+  bottom: calc(-1 * var(--labels-size) - var(--primary-axis-width));
+  height: var(--labels-size);
+  color: rgba(var(--color-grey-400), 1);
+  font-weight: 400;
+}
+
+.active-analytics .charts-css.column:not(.reverse) tbody tr td {
+  align-items: flex-start;
+}
+
+.active-analytics .charts-css.column:not(.stacked) tbody tr td {
+  flex-grow: 1;
+  flex-shrink: 1;
+  flex-basis: 0;
+}
+
+.active-analytics .charts-css.column:not(.reverse-data) tbody {
+  flex-direction: row;
+}