ahoy_matey 2.0.2 → 2.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 0c48c8c5793b7e8f7c53ee1fcfed647340749cee
-  data.tar.gz: ad2e9686bebde7233be8804ac899a1672f352240
+SHA256:
+  metadata.gz: f0b1c52401a07fcd017af2e52aca841090ee47a1330c9e2c941d19a82fd2b6d7
+  data.tar.gz: 0d9ddec6c4bbe552672dc41cc444cffc2fa0ffda7b4a6a63200bd2de8d8e65e6
 SHA512:
-  metadata.gz: 5b3a16dd65efc501a56fa2a6e2574dc228dbc236d97d352153bd6012b85243450753a62f4155737a2d43610ac619f225004492ed8bcfc8a5e088ef93b67fb56a
-  data.tar.gz: 23e2ee9cc5ec7d772cce7a4af348b95b93090c5d3a7113fd1d47fcd9fe3bbca4cbbb145cbc21d3826ca159d984566e3e1c8f3b45d89d9a7319bba2ae2d8026eb
+  metadata.gz: a8c9083093116b8931861b9f180902038871d76e6a363833bd10d55bbd9e8ea89346f62a609ec3990da3232b24e151e075fa4fd3641a247850e18c40db30d912
+  data.tar.gz: a5fe8df81cf9510e8472cda07ff31e2c85d7b2af2102655303623a6677937fb922706db9f2cabac78e49aaf132936c19834be09c3e8acda4ebea3eec7ef82063

data/CHANGELOG.md

@@ -1,3 +1,11 @@
+## 2.1.0
+
+- Added option for IP masking
+- Added option to use anonymity sets instead of cookies
+- Added `user_agent_parser` option
+- Fixed `visitable` for Rails 4.2
+- Removed `search_keyword` from new installs
+
 ## 2.0.2
 
 - Fixed error on duplicate records

data/Gemfile

@@ -3,4 +3,4 @@ source "https://rubygems.org"
 # Specify your gem's dependencies in ahoy.gemspec
 gemspec
 
-gem "rails", "~> 5.1.0"
+gem "rails", "~> 5.2.0"

data/README.md

@@ -56,7 +56,11 @@ And track an event with:
 ahoy.track("My second event", {language: "JavaScript"});
 ```
 
-For native apps, see the [API spec](#api-spec).
+For Android, check out [Ahoy Android](https://github.com/instacart/ahoy-android). For other platforms, see the [API spec](#api-spec).
+
+### GDPR Compliance
+
+Ahoy provides a number of options to help with GDPR compliance. See the [GDPR section](#gdpr-compliance-1) for more info.
 
 ## How It Works
 
@@ -127,7 +131,7 @@ end
 
 #### Native Apps
 
-See the [API spec](#api-spec).
+For Android, check out [Ahoy Android](https://github.com/instacart/ahoy-android). For other platforms, see the [API spec](#api-spec).
 
 ### Associated Models
 
@@ -314,6 +318,64 @@ Exceptions are rescued so analytics do not break your app. Ahoy uses [Safely](ht
 Safely.report_exception_method = ->(e) { Rollbar.error(e) }
 ```
 
+## GDPR Compliance
+
+Ahoy provides a number of options to help with [GDPR compliance](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation).
+
+Update `config/initializers/ahoy.rb` with:
+
+```ruby
+class Ahoy::Store < Ahoy::DatabaseStore
+  def authenticate(data)
+    # disables automatic linking of visits and users
+  end
+end
+
+Ahoy.mask_ips = true
+Ahoy.cookies = false
+```
+
+This:
+
+- Masks IP addresses
+- Switches from cookies to anonymity sets
+- Disables automatic linking of visits and users
+
+If you use JavaScript tracking, also set:
+
+```javascript
+ahoy.configure({cookies: false});
+```
+
+### IP Masking
+
+Ahoy can mask IPs with the same approach [Google Analytics uses for IP anonymization](https://support.google.com/analytics/answer/2763052). This means:
+
+- For IPv4, the last octet is set to 0 (`8.8.4.4` becomes `8.8.4.0`)
+- For IPv6, the last 80 bits are set to zeros (`2001:4860:4860:0:0:0:0:8844` becomes `2001:4860:4860::`)
+
+```ruby
+Ahoy.mask_ips = true
+```
+
+To mask previously collected IPs, use:
+
+```ruby
+Ahoy::Visit.find_each do |visit|
+  visit.update_column :ip, Ahoy.mask_ip(visit.ip)
+end
+```
+
+### Anonymity Sets & Cookies
+
+Ahoy can switch from cookies to [anonymity sets](https://privacypatterns.org/patterns/Anonymity-set). Instead of cookies, visitors with the same IP mask and user agent are grouped together in an anonymity set.
+
+```ruby
+Ahoy.cookies = false
+```
+
+Previously set cookies are automatically deleted.
+
 ## Development
 
 Ahoy is built with developers in mind. You can run the following code in your browser’s console.
@@ -414,7 +476,7 @@ Ahoy::Visit.group(:country).count
 Ahoy::Visit.group(:referring_domain).count
 ```
 
-[Chartkick](http://chartkick.com/) and [Groupdate](https://github.com/ankane/groupdate) make it easy to visualize the data.
+[Chartkick](https://www.chartkick.com/) and [Groupdate](https://github.com/ankane/groupdate) make it easy to visualize the data.
 
 ```erb
 <%= line_chart Ahoy::Visit.group_by_day(:started_at).count %>
@@ -456,7 +518,7 @@ The same approach also works with visitor tokens.
 
 ### Visits
 
-Generate visit and visitor tokens as [UUIDs](http://en.wikipedia.org/wiki/Universally_unique_identifier), and include these values in the `Ahoy-Visit` and `Ahoy-Visitor` headers with all requests.
+Generate visit and visitor tokens as [UUIDs](https://en.wikipedia.org/wiki/Universally_unique_identifier), and include these values in the `Ahoy-Visit` and `Ahoy-Visitor` headers with all requests.
 
 Send a `POST` request to `/ahoy/visits` with `Content-Type: application/json` and a body like:
 
@@ -507,6 +569,42 @@ Then include it in your pack.
 import ahoy from "ahoy.js";
 ```
 
+## Upgrading
+
+### 2.1
+
+Ahoy recommends [Device Detector](https://github.com/podigee/device_detector) for user agent parsing and makes it the default for new installations. To switch, add to `config/initializers/ahoy.rb`:
+
+```ruby
+Ahoy.user_agent_parser = :device_detector
+```
+
+Backfill existing records with:
+
+```ruby
+Ahoy::Visit.find_each do |visit|
+  client = DeviceDetector.new(visit.user_agent)
+  device_type =
+    case client.device_type
+    when "smartphone"
+      "Mobile"
+    when "tv"
+      "TV"
+    else
+      client.device_type.try(:titleize)
+    end
+
+  visit.browser = client.name
+  visit.os = client.os_name
+  visit.device_type = device_type
+  visit.save(validate: false) if visit.changed?
+end
+```
+
+### 2.0
+
+See the [upgrade guide](docs/Ahoy-2-Upgrade.md)
+
 ## History
 
 View the [changelog](https://github.com/ankane/ahoy/blob/master/CHANGELOG.md)

data/ahoy_matey.gemspec

@@ -25,6 +25,7 @@ Gem::Specification.new do |spec|
   spec.add_dependency "user_agent_parser"
   spec.add_dependency "request_store"
   spec.add_dependency "safely_block", ">= 0.2.1"
+  spec.add_dependency "device_detector"
 
   spec.add_development_dependency "bundler"
   spec.add_development_dependency "rake"

data/lib/ahoy.rb

@@ -1,3 +1,5 @@
+require "ipaddr"
+
 require "active_support"
 require "active_support/core_ext"
 require "addressable/uri"
@@ -22,6 +24,9 @@ module Ahoy
   mattr_accessor :visitor_duration
   self.visitor_duration = 2.years
 
+  mattr_accessor :cookies
+  self.cookies = true
+
   mattr_accessor :cookie_domain
 
   mattr_accessor :server_side_visits
@@ -67,9 +72,26 @@ module Ahoy
   mattr_accessor :token_generator
   self.token_generator = -> { SecureRandom.uuid }
 
+  mattr_accessor :mask_ips
+  self.mask_ips = false
+
+  mattr_accessor :user_agent_parser
+  self.user_agent_parser = :legacy
+
   def self.log(message)
     Rails.logger.info { "[ahoy] #{message}" }
   end
+
+  def self.mask_ip(ip)
+    addr = IPAddr.new(ip)
+    if addr.ipv4?
+      # set last octet to 0
+      addr.mask(24).to_s
+    else
+      # set last 80 bits to zeros
+      addr.mask(48).to_s
+    end
+  end
 end
 
 ActiveSupport.on_load(:action_controller) do

data/lib/ahoy/base_store.rb

@@ -46,7 +46,21 @@ module Ahoy
     protected
 
     def bot?
-      @bot ||= request ? Browser.new(request.user_agent).bot? : false
+      unless defined?(@bot)
+        @bot = begin
+          if request
+            if Ahoy.user_agent_parser == :device_detector
+              DeviceDetector.new(request.user_agent).bot?
+            else
+              Browser.new(request.user_agent).bot?
+            end
+          else
+            false
+          end
+        end
+      end
+
+      @bot
     end
 
     def exclude_by_method?

data/lib/ahoy/controller.rb

@@ -21,8 +21,13 @@ module Ahoy
     end
 
     def set_ahoy_cookies
-      ahoy.set_visitor_cookie
-      ahoy.set_visit_cookie
+      if Ahoy.cookies
+        ahoy.set_visitor_cookie
+        ahoy.set_visit_cookie
+      else
+        # delete cookies if exist
+        ahoy.reset
+      end
     end
 
     def track_ahoy_visit

data/lib/ahoy/model.rb

@@ -2,7 +2,9 @@ module Ahoy
   module Model
     def visitable(name = :visit, **options)
       class_eval do
-        belongs_to(name, optional: true, class_name: "Ahoy::Visit", **options)
+        safe_options = options.dup
+        safe_options[:optional] = true if Rails::VERSION::MAJOR >= 5
+        belongs_to(name, class_name: "Ahoy::Visit", **safe_options)
         before_create :set_ahoy_visit
       end
       class_eval %{

data/lib/ahoy/tracker.rb

@@ -1,5 +1,9 @@
+require "active_support/core_ext/digest/uuid"
+
 module Ahoy
   class Tracker
+    UUID_NAMESPACE = "a82ae811-5011-45ab-a728-569df7499c5f"
+
     attr_reader :request, :controller
 
     def initialize(**options)
@@ -102,7 +106,7 @@ module Ahoy
     end
 
     def new_visit?
-      !existing_visit_token
+      Ahoy.cookies ? !existing_visit_token : visit.nil?
     end
 
     def new_visitor?
@@ -156,7 +160,7 @@ module Ahoy
     end
 
     def missing_params?
-      if api? && Ahoy.protect_from_forgery
+      if Ahoy.cookies && api? && Ahoy.protect_from_forgery
         !(existing_visit_token && existing_visitor_token)
       else
         false
@@ -164,6 +168,9 @@ module Ahoy
     end
 
     def set_cookie(name, value, duration = nil, use_domain = true)
+      # safety net
+      return unless Ahoy.cookies
+
       cookie = {
         value: value
       }
@@ -174,7 +181,7 @@ module Ahoy
     end
 
     def delete_cookie(name)
-      request.cookie_jar.delete(name)
+      request.cookie_jar.delete(name) if request.cookie_jar[name]
     end
 
     def trusted_time(time = nil)
@@ -200,6 +207,7 @@ module Ahoy
     def visit_token_helper
       @visit_token_helper ||= begin
         token = existing_visit_token
+        token ||= visit_anonymity_set unless Ahoy.cookies
         token ||= generate_id unless Ahoy.api_only
         token
       end
@@ -208,6 +216,7 @@ module Ahoy
     def visitor_token_helper
       @visitor_token_helper ||= begin
         token = existing_visitor_token
+        token ||= visitor_anonymity_set unless Ahoy.cookies
         token ||= generate_id unless Ahoy.api_only
         token
       end
@@ -216,7 +225,7 @@ module Ahoy
     def existing_visit_token
       @existing_visit_token ||= begin
         token = visit_header
-        token ||= visit_cookie unless api? && Ahoy.protect_from_forgery
+        token ||= visit_cookie if Ahoy.cookies && !(api? && Ahoy.protect_from_forgery)
         token ||= visit_param if api?
         token
       end
@@ -225,12 +234,20 @@ module Ahoy
     def existing_visitor_token
       @existing_visitor_token ||= begin
         token = visitor_header
-        token ||= visitor_cookie unless api? && Ahoy.protect_from_forgery
+        token ||= visitor_cookie if Ahoy.cookies && !(api? && Ahoy.protect_from_forgery)
         token ||= visitor_param if api?
         token
       end
     end
 
+    def visit_anonymity_set
+      @visit_anonymity_set ||= Digest::UUID.uuid_v5(UUID_NAMESPACE, ["visit", Ahoy.mask_ip(request.remote_ip), request.user_agent].join("/"))
+    end
+
+    def visitor_anonymity_set
+      @visitor_anonymity_set ||= Digest::UUID.uuid_v5(UUID_NAMESPACE, ["visitor", Ahoy.mask_ip(request.remote_ip), request.user_agent].join("/"))
+    end
+
     def visit_cookie
       @visit_cookie ||= request && request.cookies["ahoy_visit"]
     end

data/lib/ahoy/version.rb

@@ -1,3 +1,3 @@
 module Ahoy
-  VERSION = "2.0.2"
+  VERSION = "2.1.0"
 end

data/lib/ahoy/visit_properties.rb

@@ -1,4 +1,5 @@
 require "browser"
+require "device_detector"
 require "referer-parser"
 require "user_agent_parser"
 
@@ -36,43 +37,73 @@ module Ahoy
 
       {
         referring_domain: (Addressable::URI.parse(referrer).host.first(255) rescue nil),
-        search_keyword: (@@referrer_parser.parse(@referrer)[:term][0..255] rescue nil).presence
+        search_keyword: (@@referrer_parser.parse(@referrer)[:term].first(255) rescue nil).presence
       }
     end
 
     def tech_properties
-      # cache for performance
-      @@user_agent_parser ||= UserAgentParser::Parser.new
+      if Ahoy.user_agent_parser == :device_detector
+        client = DeviceDetector.new(request.user_agent)
+        device_type =
+          case client.device_type
+          when "smartphone"
+            "Mobile"
+          when "tv"
+            "TV"
+          else
+            client.device_type.try(:titleize)
+          end
 
-      user_agent = request.user_agent
-      agent = @@user_agent_parser.parse(user_agent)
-      browser = Browser.new(user_agent)
-      device_type =
-        if browser.bot?
-          "Bot"
-        elsif browser.device.tv?
-          "TV"
-        elsif browser.device.console?
-          "Console"
-        elsif browser.device.tablet?
-          "Tablet"
-        elsif browser.device.mobile?
-          "Mobile"
-        else
-          "Desktop"
-        end
+        {
+          browser: client.name,
+          os: client.os_name,
+          device_type: device_type
+        }
+      else
+        # cache for performance
+        @@user_agent_parser ||= UserAgentParser::Parser.new
 
-      {
-        browser: agent.name,
-        os: agent.os.name,
-        device_type: device_type,
-      }
+        user_agent = request.user_agent
+        agent = @@user_agent_parser.parse(user_agent)
+        browser = Browser.new(user_agent)
+        device_type =
+          if browser.bot?
+            "Bot"
+          elsif browser.device.tv?
+            "TV"
+          elsif browser.device.console?
+            "Console"
+          elsif browser.device.tablet?
+            "Tablet"
+          elsif browser.device.mobile?
+            "Mobile"
+          else
+            "Desktop"
+          end
+
+        {
+          browser: agent.name,
+          os: agent.os.name,
+          device_type: device_type
+        }
+      end
+    end
+
+    # masking based on Google Analytics anonymization
+    # https://support.google.com/analytics/answer/2763052
+    def ip
+      ip = request.remote_ip
+      if ip && Ahoy.mask_ips
+        Ahoy.mask_ip(ip)
+      else
+        ip
+      end
     end
 
     def request_properties
       {
-        ip: request.remote_ip,
-        user_agent: request.user_agent,
+        ip: ip,
+        user_agent: ensure_utf8(request.user_agent),
         referrer: referrer,
         landing_page: landing_page,
         platform: params["platform"],
@@ -82,5 +113,11 @@ module Ahoy
         screen_width: params["screen_width"]
       }
     end
+
+    def ensure_utf8(str)
+      if str
+        str.encode("UTF-8", "binary", invalid: :replace, undef: :replace, replace: "")
+      end
+    end
   end
 end