ahoy_matey 1.5.5 → 4.2.1

data/README.md

@@ -1,451 +1,536 @@
 # Ahoy
 
-Ahoy provides a solid foundation to track visits and events in Ruby, JavaScript, and native apps. Works with any data store so you can easily scale.
+:fire: Simple, powerful, first-party analytics for Rails
 
-:tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
+Track visits and events in Ruby, JavaScript, and native apps. Data is stored in your database by default, and you can customize it for any data store as you grow.
+
+:postbox: Check out [Ahoy Email](https://github.com/ankane/ahoy_email) for emails and [Field Test](https://github.com/ankane/field_test) for A/B testing
 
-:postbox: To track emails, check out [Ahoy Email](https://github.com/ankane/ahoy_email).
+:tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
 
-:maple_leaf: For A/B testing, check out [Field Test](https://github.com/ankane/field_test).
+[![Build Status](https://github.com/ankane/ahoy/workflows/build/badge.svg?branch=master)](https://github.com/ankane/ahoy/actions)
 
 ## Installation
 
 Add this line to your application’s Gemfile:
 
 ```ruby
-gem 'ahoy_matey'
+gem "ahoy_matey"
 ```
 
-And add the javascript file in `app/assets/javascripts/application.js` after jQuery.
+And run:
 
-```javascript
-//= require jquery
-//= require ahoy
+```sh
+bundle install
+rails generate ahoy:install
+rails db:migrate
 ```
 
-## Choose a Data Store
+Restart your web server, open a page in your browser, and a visit will be created :tada:
 
-Ahoy supports a number of data stores out of the box.  You can start with one of them and customize as needed, or create your own store from scratch.
+Track your first event from a controller with:
 
-- [PostgreSQL, MySQL, or SQLite](#postgresql-mysql-or-sqlite)
-- [MongoDB](#mongodb)
-- [Kafka](#kafka), [Fluentd](#fluentd), [RabbitMQ](#rabbitmq), [NATS](#nats), [NSQ](#nsq), or [Amazon Kinesis Firehose](#amazon-kinesis-firehose)
-- [Logs](#logs)
-- [Custom](#custom)
+```ruby
+ahoy.track "My first event", language: "Ruby"
+```
 
-### PostgreSQL, MySQL, or SQLite
+### JavaScript, Native Apps, & AMP
 
-Run:
+Enable the API in `config/initializers/ahoy.rb`:
 
-```sh
-rails generate ahoy:stores:active_record
-rake db:migrate
+```ruby
+Ahoy.api = true
 ```
 
-### MongoDB
+And restart your web server.
 
-Run:
+### JavaScript
 
-```sh
-rails generate ahoy:stores:mongoid
-```
+For Rails 7 / Importmap, add to `config/importmap.rb`:
 
-### Kafka
+```ruby
+pin "ahoy", to: "ahoy.js"
+```
 
-Add [ruby-kafka](https://github.com/zendesk/ruby-kafka) to your Gemfile.
+And add to `app/javascript/application.js`:
 
-```ruby
-gem 'ruby-kafka'
+```javascript
+import "ahoy"
 ```
 
-And run:
+For Rails 6 / Webpacker, run:
 
 ```sh
-rails generate ahoy:stores:kafka
+yarn add ahoy.js
 ```
 
-Use `ENV["KAFKA_URL"]` to configure.
+And add to `app/javascript/packs/application.js`:
 
-### Fluentd
+```javascript
+import ahoy from "ahoy.js"
+```
 
-Add [fluent-logger](https://github.com/fluent/fluent-logger-ruby) to your Gemfile.
+For Rails 5 / Sprockets, add to `app/assets/javascripts/application.js`:
 
-```ruby
-gem 'fluent-logger'
+```javascript
+//= require ahoy
 ```
 
-And run:
+Track an event with:
 
-```sh
-rails generate ahoy:stores:fluentd
+```javascript
+ahoy.track("My second event", {language: "JavaScript"});
 ```
 
-Use `ENV["FLUENTD_HOST"]` and `ENV["FLUENTD_PORT"]` to configure.
+### Native Apps
 
-### RabbitMQ
+Check out [Ahoy iOS](https://github.com/namolnad/ahoy-ios) and [Ahoy Android](https://github.com/instacart/ahoy-android).
 
-Add [bunny](https://github.com/ruby-amqp/bunny) to your Gemfile.
+### Geocoding Setup
 
-```ruby
-gem 'bunny'
-```
+To enable geocoding, see the [Geocoding section](#geocoding).
 
-And run:
+### GDPR Compliance
 
-```sh
-rails generate ahoy:stores:bunny
-```
+Ahoy provides a number of options to help with GDPR compliance. See the [GDPR section](#gdpr-compliance-1) for more info.
 
-Use `ENV["RABBITMQ_URL"]` to configure.
+## How It Works
 
-### NATS
+### Visits
 
-Add [nats-pure](https://github.com/nats-io/pure-ruby-nats) to your Gemfile.
+When someone visits your website, Ahoy creates a visit with lots of useful information.
 
-```ruby
-gem 'nats-pure'
-```
+- **traffic source** - referrer, referring domain, landing page
+- **location** - country, region, city, latitude, longitude
+- **technology** - browser, OS, device type
+- **utm parameters** - source, medium, term, content, campaign
 
-And run:
+Use the `current_visit` method to access it.
 
-```sh
-rails generate ahoy:stores:nats
+Prevent certain Rails actions from creating visits with:
+
+```ruby
+skip_before_action :track_ahoy_visit
 ```
 
-Use `ENV["NATS_URL"]` to configure.
+This is typically useful for APIs. If your entire Rails app is an API, you can use:
 
-### NSQ
+```ruby
+Ahoy.api_only = true
+```
 
-Add [nsq-ruby](https://github.com/wistia/nsq-ruby) to your Gemfile.
+You can also defer visit tracking to JavaScript. This is useful for preventing bots (that aren’t detected by their user agent) and users with cookies disabled from creating a new visit on each request. `:when_needed` will create visits server-side only when needed by events, and `false` will disable server-side creation completely, discarding events without a visit.
 
 ```ruby
-gem 'nsq-ruby'
+Ahoy.server_side_visits = :when_needed
 ```
 
-And run:
+### Events
 
-```sh
-rails generate ahoy:stores:nsq
-```
+Each event has a `name` and `properties`. There are several ways to track events.
 
-Use `ENV["NSQ_URL"]` to configure.
+#### Ruby
 
-### Amazon Kinesis Firehose
+```ruby
+ahoy.track "Viewed book", title: "Hot, Flat, and Crowded"
+```
 
-Add [aws-sdk](https://github.com/aws/aws-sdk-ruby) to your Gemfile.
+Track actions automatically with:
 
 ```ruby
-gem 'aws-sdk', '>= 2.0.0'
+class ApplicationController < ActionController::Base
+  after_action :track_action
+
+  protected
+
+  def track_action
+    ahoy.track "Ran action", request.path_parameters
+  end
+end
 ```
 
-And run:
+#### JavaScript
 
-```sh
-rails generate ahoy:stores:kinesis_firehose
+```javascript
+ahoy.track("Viewed book", {title: "The World is Flat"});
 ```
 
-Configure delivery streams and credentials in the initializer.
+See [Ahoy.js](https://github.com/ankane/ahoy.js) for a complete list of features.
 
-### Logs
+#### Native Apps
 
-```sh
-rails generate ahoy:stores:log
+See the docs for [Ahoy iOS](https://github.com/namolnad/ahoy-ios) and [Ahoy Android](https://github.com/instacart/ahoy-android).
+
+#### AMP
+
+```erb
+<head>
+  <script async custom-element="amp-analytics" src="https://cdn.ampproject.org/v0/amp-analytics-0.1.js"></script>
+</head>
+<body>
+  <%= amp_event "Viewed article", title: "Analytics with Rails" %>
+</body>
 ```
 
-This logs visits to `log/visits.log` and events to `log/events.log`.
+### Associated Models
 
-### Custom
+Say we want to associate orders with visits. Just add `visitable` to the model.
 
-```sh
-rails generate ahoy:stores:custom
+```ruby
+class Order < ApplicationRecord
+  visitable :ahoy_visit
+end
 ```
 
-This creates a class for you to fill out.
+When a visitor places an order, the `ahoy_visit_id` column is automatically set :tada:
+
+See where orders are coming from with simple joins:
 
 ```ruby
-class Ahoy::Store < Ahoy::Stores::BaseStore
-  def track_visit(options)
-  end
+Order.joins(:ahoy_visit).group("referring_domain").count
+Order.joins(:ahoy_visit).group("city").count
+Order.joins(:ahoy_visit).group("device_type").count
+```
+
+Here’s what the migration to add the `ahoy_visit_id` column should look like:
 
-  def track_event(name, properties, options)
+```ruby
+class AddAhoyVisitToOrders < ActiveRecord::Migration[7.0]
+  def change
+    add_reference :orders, :ahoy_visit
   end
 end
 ```
 
-See the [ActiveRecordTokenStore](https://github.com/ankane/ahoy/blob/master/lib/ahoy/stores/active_record_token_store.rb) for an example.
+Customize the column with:
 
-## How It Works
+```ruby
+visitable :sign_up_visit
+```
 
-### Visits
+### Users
 
-When someone visits your website, Ahoy creates a visit with lots of useful information.
+Ahoy automatically attaches the `current_user` to the visit. With [Devise](https://github.com/plataformatec/devise), it attaches the user even if they sign in after the visit starts.
 
-- **traffic source** - referrer, referring domain, landing page, search keyword
-- **location** - country, region, and city
-- **technology** - browser, OS, and device type
-- **utm parameters** - source, medium, term, content, campaign
+With other authentication frameworks, add this to the end of your sign in method:
 
-Use the `current_visit` method to access it.
+```ruby
+ahoy.authenticate(user)
+```
 
-### Events
+To see the visits for a given user, create an association:
 
-Each event has a `name` and `properties`.
+```ruby
+class User < ApplicationRecord
+  has_many :visits, class_name: "Ahoy::Visit"
+end
+```
 
-There are three ways to track events.
+And use:
 
-#### JavaScript
+```ruby
+User.find(123).visits
+```
 
-```javascript
-ahoy.track("Viewed book", {title: "The World is Flat"});
+#### Custom User Method
+
+Use a method besides `current_user`
+
+```ruby
+Ahoy.user_method = :true_user
 ```
 
-or track events automatically with:
+or use a proc
 
-```javascript
-ahoy.trackAll();
+```ruby
+Ahoy.user_method = ->(controller) { controller.true_user }
 ```
 
-See [Ahoy.js](https://github.com/ankane/ahoy.js) for a complete list of features.
+#### Doorkeeper
 
-#### Ruby
+To attach the user with [Doorkeeper](https://github.com/doorkeeper-gem/doorkeeper), be sure you have a `current_resource_owner` method in `ApplicationController`.
 
 ```ruby
-ahoy.track "Viewed book", title: "Hot, Flat, and Crowded"
+class ApplicationController < ActionController::Base
+  private
+
+  def current_resource_owner
+    User.find(doorkeeper_token.resource_owner_id) if doorkeeper_token
+  end
+end
 ```
 
-#### Native Apps
+#### Knock
 
-See the [HTTP spec](#native-apps-1) until libraries are built.
+To attach the user with [Knock](https://github.com/nsarno/knock), either include `Knock::Authenticable`in `ApplicationController`:
 
-### Users
+```ruby
+class ApplicationController < ActionController::API
+  include Knock::Authenticable
+end
+```
 
-Ahoy automatically attaches the `current_user` to the visit.
+Or include it in Ahoy:
 
-With [Devise](https://github.com/plataformatec/devise), it will attach the user even if he or she signs in after the visit starts.
+```ruby
+Ahoy::BaseController.include Knock::Authenticable
+```
 
-With other authentication frameworks, add this to the end of your sign in method:
+And use:
 
 ```ruby
-ahoy.authenticate(user)
+Ahoy.user_method = ->(controller) { controller.send(:authenticate_entity, "user") }
 ```
 
-## Customize the Store
+### Exclusions
 
-Stores are built to be highly customizable.
+Bots are excluded from tracking by default. To include them, use:
 
 ```ruby
-class Ahoy::Store < Ahoy::Stores::ActiveRecordTokenStore
-  # add methods here
-end
+Ahoy.track_bots = true
 ```
 
-### Exclude Bots and More
-
-Exclude visits and events from being tracked with:
+Add your own rules with:
 
 ```ruby
-class Ahoy::Store < Ahoy::Stores::ActiveRecordTokenStore
-  def exclude?
-    bot? || request.ip == "192.168.1.1"
-  end
+Ahoy.exclude_method = lambda do |controller, request|
+  request.ip == "192.168.1.1"
 end
 ```
 
-Bots are excluded by default.
+### Visit Duration
 
-### Track Additional Values
+By default, a new visit is created after 4 hours of inactivity. Change this with:
 
 ```ruby
-class Ahoy::Store < Ahoy::Stores::ActiveRecordTokenStore
-  def track_visit(options)
-    super do |visit|
-      visit.gclid = visit_properties.landing_params["gclid"]
-    end
-  end
-
-  def track_event(name, properties, options)
-    super do |event|
-      event.ip = request.ip
-    end
-  end
-end
+Ahoy.visit_duration = 30.minutes
 ```
 
-Some methods you can use are `request`, `controller`, `visit_properties`, and `ahoy`.
+### Cookies
 
-### Customize User
+To track visits across multiple subdomains, use:
 
-If you use a method other than `current_user`, set it here:
+```ruby
+Ahoy.cookie_domain = :all
+```
+
+Set other [cookie options](https://api.rubyonrails.org/classes/ActionDispatch/Cookies.html) with:
 
 ```ruby
-class Ahoy::Store < Ahoy::Stores::ActiveRecordTokenStore
-  def user
-    controller.true_user
-  end
-end
+Ahoy.cookie_options = {same_site: :lax}
 ```
 
-### Report Exceptions
+You can also [disable cookies](#anonymity-sets--cookies)
 
-Exceptions are rescued so analytics do not break your app. Ahoy uses [Safely](https://github.com/ankane/safely) to try to report them to a service by default.
+### Token Generation
 
-To customize this, use:
+Ahoy uses random UUIDs for visit and visitor tokens by default, but you can use your own generator like [Druuid](https://github.com/recurly/druuid).
 
 ```ruby
-Safely.report_exception_method = -> (e) { Rollbar.error(e) }
+Ahoy.token_generator = -> { Druuid.gen }
 ```
 
-### Use Different Models
+### Throttling
 
-For ActiveRecord and Mongoid stores
+You can use [Rack::Attack](https://github.com/kickstarter/rack-attack) to throttle requests to the API.
 
 ```ruby
-class Ahoy::Store < Ahoy::Stores::ActiveRecordTokenStore
-  def visit_model
-    CustomVisit
-  end
-
-  def event_model
-    CustomEvent
+class Rack::Attack
+  throttle("ahoy/ip", limit: 20, period: 1.minute) do |req|
+    if req.path.start_with?("/ahoy/")
+      req.ip
+    end
   end
 end
 ```
 
-## More Features
-
-### Automatic Tracking
+### Exceptions
 
-Page views
+Exceptions are rescued so analytics do not break your app. Ahoy uses [Safely](https://github.com/ankane/safely) to try to report them to a service by default. To customize this, use:
 
-```javascript
-ahoy.trackView();
+```ruby
+Safely.report_exception_method = ->(e) { Rollbar.error(e) }
 ```
 
-Clicks
+## Geocoding
 
-```javascript
-ahoy.trackClicks();
+Ahoy uses [Geocoder](https://github.com/alexreisner/geocoder) for geocoding. We recommend configuring [local geocoding](#local-geocoding) or [load balancer geocoding](#load-balancer-geocoding) so IP addresses are not sent to a 3rd party service. If you do use a 3rd party service and adhere to GDPR, be sure to add it to your subprocessor list. If Ahoy is configured to [mask IPs](#ip-masking), the masked IP is used (this can reduce accuracy but is better for privacy).
+
+To enable geocoding, add this line to your application’s Gemfile:
+
+```ruby
+gem "geocoder"
 ```
 
-Rails actions
+And update `config/initializers/ahoy.rb`:
 
 ```ruby
-class ApplicationController < ActionController::Base
-  after_action :track_action
+Ahoy.geocode = true
+```
 
-  protected
+Geocoding is performed in a background job so it doesn’t slow down web requests. The default job queue is `:ahoy`. Change this with:
 
-  def track_action
-    ahoy.track "Viewed #{controller_name}##{action_name}"
-  end
-end
+```ruby
+Ahoy.job_queue = :low_priority
 ```
 
-### Multiple Subdomains
+### Local Geocoding
 
-To track visits across multiple subdomains, use:
+For privacy and performance, we recommend geocoding locally. Add this line to your application’s Gemfile:
 
 ```ruby
-Ahoy.cookie_domain = :all
+gem "maxminddb"
 ```
 
-### Visit Duration
+For city-level geocoding, download the [GeoLite2 City database](https://dev.maxmind.com/geoip/geoip2/geolite2/) and create `config/initializers/geocoder.rb` with:
 
-By default, a new visit is created after 4 hours of inactivity.
+```ruby
+Geocoder.configure(
+  ip_lookup: :geoip2,
+  geoip2: {
+    file: "path/to/GeoLite2-City.mmdb"
+  }
+)
+```
+
+For country-level geocoding, install the `geoip-database` package. It’s preinstalled on Heroku. For Ubuntu, use:
+
+```sh
+sudo apt-get install geoip-database
+```
 
-Change this with:
+And create `config/initializers/geocoder.rb` with:
 
 ```ruby
-Ahoy.visit_duration = 30.minutes
+Geocoder.configure(
+  ip_lookup: :maxmind_local,
+  maxmind_local: {
+    file: "/usr/share/GeoIP/GeoIP.dat",
+    package: :country
+  }
+)
 ```
 
-### ActiveRecord
+### Load Balancer Geocoding
+
+Some load balancers can add geocoding information to request headers.
 
-Let’s associate orders with visits.
+- [nginx](https://nginx.org/en/docs/http/ngx_http_geoip_module.html)
+- [Google Cloud](https://cloud.google.com/load-balancing/docs/custom-headers)
+- [Cloudflare](https://support.cloudflare.com/hc/en-us/articles/200168236-Configuring-Cloudflare-IP-Geolocation)
 
-First, generate a migration and add a `visit_id` column.
+Update `config/initializers/ahoy.rb` with:
 
 ```ruby
-class AddVisitIdToOrders < ActiveRecord::Migration
-  def change
-    add_column :orders, :visit_id, :integer
+Ahoy.geocode = false
+
+class Ahoy::Store < Ahoy::DatabaseStore
+  def track_visit(data)
+    data[:country] = request.headers["<country-header>"]
+    data[:region] = request.headers["<region-header>"]
+    data[:city] = request.headers["<city-header>"]
+    super(data)
   end
 end
 ```
 
-**Note**: Use the `uuid` column type if the `id` column on `visits` is a `uuid`.
+## GDPR Compliance
+
+Ahoy provides a number of options to help with [GDPR compliance](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation).
 
-Then, add `visitable` to the model.
+Update `config/initializers/ahoy.rb` with:
 
 ```ruby
-class Order < ActiveRecord::Base
-  visitable
+class Ahoy::Store < Ahoy::DatabaseStore
+  def authenticate(data)
+    # disables automatic linking of visits and users
+  end
 end
+
+Ahoy.mask_ips = true
+Ahoy.cookies = false
 ```
 
-When a visitor places an order, the `visit_id` column is automatically set. :tada:
+This:
 
-Customize the column and class name with:
+- Masks IP addresses
+- Switches from cookies to anonymity sets
+- Disables automatic linking of visits and users
 
-```ruby
-visitable :sign_up_visit, class_name: "Visit"
+If you use JavaScript tracking, also set:
+
+```javascript
+ahoy.configure({cookies: false});
 ```
 
-### Doorkeeper
+### IP Masking
 
-To attach the user with [Doorkeeper](https://github.com/doorkeeper-gem/doorkeeper), be sure you have a `current_resource_owner` method in `ApplicationController`.
+Ahoy can mask IPs with the same approach [Google Analytics uses for IP anonymization](https://support.google.com/analytics/answer/2763052). This means:
 
-```ruby
-class ApplicationController < ActionController::Base
-  private
+- For IPv4, the last octet is set to 0 (`8.8.4.4` becomes `8.8.4.0`)
+- For IPv6, the last 80 bits are set to zeros (`2001:4860:4860:0:0:0:0:8844` becomes `2001:4860:4860::`)
 
-  def current_resource_owner
-    User.find(doorkeeper_token.resource_owner_id) if doorkeeper_token
-  end
-end
+```ruby
+Ahoy.mask_ips = true
 ```
 
-### Geocoding
+IPs are masked before geolocation is performed.
 
-By default, geocoding is performed inline. For performance, move it to the background with:
+To mask previously collected IPs, use:
 
 ```ruby
-Ahoy.geocode = :async
+Ahoy::Visit.find_each do |visit|
+  visit.update_column :ip, Ahoy.mask_ip(visit.ip)
+end
 ```
 
-For Rails 4.0 and 4.1, you’ll need to add [activejob_backport](https://github.com/ankane/activejob_backport).
+### Anonymity Sets & Cookies
 
-To change the queue name (`ahoy` by default), use:
+Ahoy can switch from cookies to [anonymity sets](https://privacypatterns.org/patterns/Anonymity-set). Instead of cookies, visitors with the same IP mask and user agent are grouped together in an anonymity set.
 
 ```ruby
-Ahoy.job_queue = :low_priority
+Ahoy.cookies = false
 ```
 
-Or disable geocoding with:
+Previously set cookies are automatically deleted. If you use JavaScript tracking, also set:
 
-```ruby
-Ahoy.geocode = false
+```javascript
+ahoy.configure({cookies: false});
 ```
 
-### Track Visits Immediately
+Note: With anonymity sets, visits no longer expire after 4 hours of inactivity. A new visit is only created when the IP mask or user agent changes (for instance, when a user updates their browser). There are plans to address this in the next major version.
+
+## Data Retention
 
-Visitor and visit ids are generated on the first request (so you can use them immediately), but the `track_visit` method isn’t called until the JavaScript library posts to the server.  This prevents browsers with cookies disabled from creating multiple visits and ensures visits are not created for API endpoints.  Change this with:
+Data should only be retained for as long as it’s needed. Delete older data with:
 
 ```ruby
-Ahoy.track_visits_immediately = true
+Ahoy::Visit.where("started_at < ?", 2.years.ago).find_in_batches do |visits|
+  visit_ids = visits.map(&:id)
+  Ahoy::Event.where(visit_id: visit_ids).delete_all
+  Ahoy::Visit.where(id: visit_ids).delete_all
+end
 ```
 
-**Note:** It’s highly recommended to perform geocoding in the background with this option.
+You can use [Rollup](https://github.com/ankane/rollup) to aggregate important data before you do.
 
-You can exclude API endpoints and other actions with:
+```ruby
+Ahoy::Visit.rollup("Visits", interval: "hour")
+```
+
+Delete data for a specific user with:
 
 ```ruby
-skip_before_action :track_ahoy_visit
+user_id = 123
+visit_ids = Ahoy::Visit.where(user_id: user_id).pluck(:id)
+Ahoy::Event.where(visit_id: visit_ids).delete_all
+Ahoy::Visit.where(id: visit_ids).delete_all
+Ahoy::Event.where(user_id: user_id).delete_all
 ```
 
 ## Development
 
-Ahoy is built with developers in mind.  You can run the following code in your browser’s console.
+Ahoy is built with developers in mind. You can run the following code in your browser’s console.
 
 Force a new visit
 
@@ -465,289 +550,244 @@ Turn off logging
 ahoy.debug(false);
 ```
 
-Debug endpoint requests in Ruby
+Debug API requests in Ruby
 
 ```ruby
 Ahoy.quiet = false
 ```
 
-## Explore the Data
-
-How you explore the data depends on the data store used.
-
-For SQL databases, you can use [Blazer](https://github.com/ankane/blazer) to easily generate charts and dashboards.
+## Data Stores
 
-With ActiveRecord, you can do:
+Data tracked by Ahoy is sent to your data store. Ahoy ships with a data store that uses your Rails database by default. You can find it in `config/initializers/ahoy.rb`:
 
 ```ruby
-Visit.group(:search_keyword).count
-Visit.group(:country).count
-Visit.group(:referring_domain).count
-```
-
-[Chartkick](http://chartkick.com/) and [Groupdate](https://github.com/ankane/groupdate) make it easy to visualize the data.
-
-```erb
-<%= line_chart Visit.group_by_day(:started_at).count %>
+class Ahoy::Store < Ahoy::DatabaseStore
+end
 ```
 
-See where orders are coming from with simple joins:
+There are four events data stores can subscribe to:
 
 ```ruby
-Order.joins(:visit).group("referring_domain").count
-Order.joins(:visit).group("city").count
-Order.joins(:visit).group("device_type").count
-```
+class Ahoy::Store < Ahoy::BaseStore
+  def track_visit(data)
+    # new visit
+  end
 
-To see the visits for a given user, create an association:
+  def track_event(data)
+    # new event
+  end
 
-```ruby
-class User < ActiveRecord::Base
-  has_many :visits
+  def geocode(data)
+    # visit geocoded
+  end
+
+  def authenticate(data)
+    # user authenticates
+  end
 end
 ```
 
-And use:
-
-```ruby
-user = User.first
-user.visits
-```
+Data stores are designed to be highly customizable so you can scale as you grow. Check out [examples](docs/Data-Store-Examples.md) for Kafka, RabbitMQ, Fluentd, NATS, NSQ, and Amazon Kinesis Firehose.
 
-### Create Funnels
+### Track Additional Data
 
 ```ruby
-viewed_store_ids = Ahoy::Event.where(name: "Viewed store").uniq.pluck(:user_id)
-added_item_ids = Ahoy::Event.where(user_id: viewed_store_ids, name: "Added item to cart").uniq.pluck(:user_id)
-viewed_checkout_ids = Ahoy::Event.where(user_id: added_item_ids, name: "Viewed checkout").uniq.pluck(:user_id)
+class Ahoy::Store < Ahoy::DatabaseStore
+  def track_visit(data)
+    data[:accept_language] = request.headers["Accept-Language"]
+    super(data)
+  end
+end
 ```
 
-The same approach also works with visitor tokens.
-
-### Querying Properties
+Two useful methods you can use are `request` and `controller`.
 
-With ActiveRecord, use:
+You can pass additional visit data from JavaScript with:
 
-```ruby
-Ahoy::Event.where(name: "Viewed product").where_properties(product_id: 123).count
+```javascript
+ahoy.configure({visitParams: {referral_code: 123}});
 ```
 
-**Note:** If you get a `NoMethodError`, upgrade Ahoy and add `include Ahoy::Properties` to the Ahoy::Event class:
+And use:
 
 ```ruby
-module Ahoy
-  class Event < ActiveRecord::Base
-    include Ahoy::Properties
-    ...
+class Ahoy::Store < Ahoy::DatabaseStore
+  def track_visit(data)
+    data[:referral_code] = request.parameters[:referral_code]
+    super(data)
   end
 end
 ```
 
-### Throttling
-
-By default, Ahoy uses [rack-attack](https://github.com/kickstarter/rack-attack) to throttle requests to Ahoy endpoints. Turn this off with:
+### Use Different Models
 
 ```ruby
-Ahoy.throttle = false
-```
-
-The default limit is 20 requests per minute. This can be overridden with:
+class Ahoy::Store < Ahoy::DatabaseStore
+  def visit_model
+    MyVisit
+  end
 
-```ruby
-# limit number of requests to 100 requests every 5 minutes
-Ahoy.throttle_limit = 100
-Ahoy.throttle_period = 5.minutes
+  def event_model
+    MyEvent
+  end
+end
 ```
 
-## Native Apps
-
-### Visits
-
-When a user launches the app, create a visit.
-
-Generate a `visit_token` and `visitor_token` as [UUIDs](http://en.wikipedia.org/wiki/Universally_unique_identifier).
-
-Send these values in the `Ahoy-Visit` and `Ahoy-Visitor` headers with all requests.
-
-Send a `POST` request to `/ahoy/visits` with:
+## Explore the Data
 
-- platform - `iOS`, `Android`, etc.
-- app_version - `1.0.0`
-- os_version - `7.0.6`
+[Blazer](https://github.com/ankane/blazer) is a great tool for exploring your data.
 
-After 4 hours of inactivity, create another visit and use the updated visit id.
+With ActiveRecord, you can do:
 
-### Events
+```ruby
+Ahoy::Visit.group(:search_keyword).count
+Ahoy::Visit.group(:country).count
+Ahoy::Visit.group(:referring_domain).count
+```
 
-Send a `POST` request as `Content-Type: application/json` to `/ahoy/events` with:
+[Chartkick](https://www.chartkick.com/) and [Groupdate](https://github.com/ankane/groupdate) make it easy to visualize the data.
 
-- id - `5aea7b70-182d-4070-b062-b0a09699ad5e` - UUID
-- name - `Viewed item`
-- properties - `{"item_id": 123}`
-- time - `2014-06-17T00:00:00-07:00` - [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601)
-- `Ahoy-Visit` and `Ahoy-Visitor` headers
-- user token (depends on your authentication framework)
+```erb
+<%= line_chart Ahoy::Visit.group_by_day(:started_at).count %>
+```
 
-Use an array to pass multiple events at once.
+### Querying Events
 
-## Reference
+Ahoy provides a few methods on the event model to make querying easier.
 
-By default, Ahoy create endpoints at `/ahoy/visits` and `/ahoy/events`. To disable, use:
+To query on both name and properties, you can use:
 
 ```ruby
-Ahoy.mount = false
+Ahoy::Event.where_event("Viewed product", product_id: 123).count
 ```
 
-## Upgrading
-
-### 1.4.0
-
-There’s nothing mandatory to do, but it’s worth noting the default store was changed from `ActiveRecordStore` to `ActiveRecordTokenStore` for new installations. `ActiveRecordStore` will continue to be supported.
-
-**Optional** Consider migrating `ahoy_events` table to have the following multi-column index as this *may* benefit
-query performance depending on your usage:
+Or just query properties with:
 
 ```ruby
-add_index :ahoy_events, [:name, :time]
+Ahoy::Event.where_props(product_id: 123, category: "Books").count
 ```
 
-### json -> jsonb
-
-Create a migration to add a new `jsonb` column.
+Group by properties with:
 
 ```ruby
-rename_column :ahoy_events, :properties, :properties_json
-add_column :ahoy_events, :properties, :jsonb
+Ahoy::Event.group_prop(:product_id, :category).count
 ```
 
-Restart your web server immediately afterwards, as Ahoy will rescue and report errors until then.
+Note: MySQL and MariaDB always return string keys (including `"null"` for `nil`) for `group_prop`.
 
-Sync the new column.
+### Funnels
 
-```ruby
-Ahoy::Event.where(properties: nil).select(:id).find_in_batches do |events|
-  Ahoy::Event.where(id: events.map(&:id)).update_all("properties = properties_json::jsonb")
-end
-```
-
-Then create a migration to drop the old column.
+It’s easy to create funnels.
 
 ```ruby
-remove_column :ahoy_events, :properties_json
+viewed_store_ids = Ahoy::Event.where(name: "Viewed store").distinct.pluck(:user_id)
+added_item_ids = Ahoy::Event.where(user_id: viewed_store_ids, name: "Added item to cart").distinct.pluck(:user_id)
+viewed_checkout_ids = Ahoy::Event.where(user_id: added_item_ids, name: "Viewed checkout").distinct.pluck(:user_id)
 ```
 
-### 1.0.0
+The same approach also works with visitor tokens.
+
+### Rollups
 
-Add the following code to the end of `config/intializers/ahoy.rb`.
+Improve query performance by pre-aggregating data with [Rollup](https://github.com/ankane/rollup).
 
 ```ruby
-class Ahoy::Store < Ahoy::Stores::ActiveRecordTokenStore
-  uses_deprecated_subscribers
-end
+Ahoy::Event.where(name: "Viewed store").rollup("Store views")
 ```
 
-If you use `Ahoy::Event` to track events, copy it into your project.
+This is only needed if you have a lot of data.
 
-```ruby
-module Ahoy
-  class Event < ActiveRecord::Base
-    self.table_name = "ahoy_events"
+### Forecasting
 
-    belongs_to :visit
-    belongs_to :user, polymorphic: true
+To forecast future visits and events, check out [Prophet](https://github.com/ankane/prophet).
 
-    serialize :properties, JSON
-  end
-end
+```ruby
+daily_visits = Ahoy::Visit.group_by_day(:started_at).count # uses Groupdate
+Prophet.forecast(daily_visits)
 ```
 
-That’s it!  To fix deprecations, keep reading.
+### Anomaly Detection
 
-#### Visits
-
-Remove `ahoy_visit` from your visit model and replace it with:
+To detect anomalies in visits and events, check out [AnomalyDetection.rb](https://github.com/ankane/AnomalyDetection.rb).
 
 ```ruby
-class Visit < ActiveRecord::Base
-  belongs_to :user, polymorphic: true
-end
+daily_visits = Ahoy::Visit.group_by_day(:started_at).count # uses Groupdate
+AnomalyDetection.detect(daily_visits, period: 7)
 ```
 
-#### Subscribers
-
-Remove `uses_deprecated_subscribers` from `Ahoy::Store`.
+### Breakout Detection
 
-If you have a custom subscriber, copy the `track` method to `track_event` in `Ahoy::Store`.
+To detect breakouts in visits and events, check out [Breakout](https://github.com/ankane/breakout).
 
 ```ruby
-class Ahoy::Store < Ahoy::Stores::ActiveRecordTokenStore
-  def track_event(name, properties, options)
-    # code copied from the track method in your subscriber
-  end
-end
+daily_visits = Ahoy::Visit.group_by_day(:started_at).count # uses Groupdate
+Breakout.detect(daily_visits)
 ```
 
-#### Authentication
+### Recommendations
 
-Ahoy no longer tracks the `$authenticate` event automatically.
+To make recommendations based on events, check out [Disco](https://github.com/ankane/disco#ahoy).
 
-To restore this behavior, use:
+## Tutorials
 
-```ruby
-class Ahoy::Store < Ahoy::Stores::ActiveRecordTokenStore
-  def authenticate(user)
-    super
-    ahoy.track "$authenticate"
-  end
-end
-```
+- [Tracking Metrics with Ahoy and Blazer](https://gorails.com/episodes/internal-metrics-with-ahoy-and-blazer)
 
-#### Global Options
+## API Spec
 
-Replace the `Ahoy.user_method` with `user` method, and replace `Ahoy.track_bots` and `Ahoy.exclude_method` with `exclude?` method.
+### Visits
 
-Skip this step if you do not use these options.
+Generate visit and visitor tokens as [UUIDs](https://en.wikipedia.org/wiki/Universally_unique_identifier), and include these values in the `Ahoy-Visit` and `Ahoy-Visitor` headers with all requests.
 
-```ruby
-class Ahoy::Store < Ahoy::Stores::ActiveRecordTokenStore
-  def user
-    # logic from Ahoy.user_method goes here
-    controller.true_user
-  end
+Send a `POST` request to `/ahoy/visits` with `Content-Type: application/json` and a body like:
 
-  def exclude?
-    # logic from Ahoy.track_bots and Ahoy.exclude_method goes here
-    bot? || request.ip == "192.168.1.1"
-  end
-end
+```json
+{
+  "visit_token": "<visit-token>",
+  "visitor_token": "<visitor-token>",
+  "platform": "iOS",
+  "app_version": "1.0.0",
+  "os_version": "11.2.6"
+}
 ```
 
-You made it!  Now, take advantage of Ahoy’s awesome new features, like easy customization and exception reporting.
+After 4 hours of inactivity, create another visit (use the same visitor token).
 
-### 0.3.0
+### Events
 
-Starting with `0.3.0`, visit and visitor tokens are now UUIDs.
+Send a `POST` request to `/ahoy/events` with `Content-Type: application/json` and a body like:
+
+```json
+{
+  "visit_token": "<visit-token>",
+  "visitor_token": "<visitor-token>",
+  "events": [
+    {
+      "id": "<optional-random-id>",
+      "name": "Viewed item",
+      "properties": {
+        "item_id": 123
+      },
+      "time": "2018-01-01T00:00:00-07:00"
+    }
+  ]
+}
+```
 
-### 0.1.6
+## Upgrading
 
-In `0.1.6`, a big improvement was made to `browser` and `os`. Update existing visits with:
+### 4.0
 
-```ruby
-Visit.find_each do |visit|
-  visit.set_technology
-  visit.save! if visit.changed?
-end
-```
+There are two notable changes to geocoding:
 
-## TODO
+1. Geocoding is now disabled by default (this was already the case for new installations with 3.2.0+). Check out the instructions for [how to enable it](#geocoding).
 
-- real-time dashboard of visits and events
-- more events for append only stores
-- turn off modules
+2. The `geocoder` gem is now an optional dependency. To use geocoding, add it to your Gemfile:
 
-## No Ruby?
+  ```ruby
+  gem "geocoder"
+  ```
 
-Check out [Ahoy.js](https://github.com/ankane/ahoy.js).
+Also, check out the [upgrade notes](https://github.com/ankane/ahoy.js#upgrading) for Ahoy.js.
 
 ## History
 
@@ -761,3 +801,20 @@ Everyone is encouraged to help improve this project. Here are a few ways you can
 - Fix bugs and [submit pull requests](https://github.com/ankane/ahoy/pulls)
 - Write, clarify, or fix documentation
 - Suggest or add new features
+
+To get started with development:
+
+```sh
+git clone https://github.com/ankane/ahoy.git
+cd ahoy
+bundle install
+bundle exec rake test
+```
+
+To test different adapters, use:
+
+```sh
+ADAPTER=postgresql bundle exec rake test
+ADAPTER=mysql2 bundle exec rake test
+ADAPTER=mongoid bundle exec rake test
+```