bot_challenge_page 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f388eccf957733cbaab0c019d82e9803b215bf8035b2b10dc7c656b821907e7d
-  data.tar.gz: 5b8126932d6bd901ddadab4dc68259bb1d2652992de882421fbfb0cfca67b500
+  metadata.gz: 826b15d243c27b3003ad7c37650836ee67536319177d947b4a046eb200914df7
+  data.tar.gz: 4709e5302b7c4298fc06bb490646273cce5a916e3337ed7d788c5390f345777e
 SHA512:
-  metadata.gz: e5a1b5e05aefd618aca8e14570e1e0c9a32f4d5144f6bd93dabae265d1ac69759ebe19bc4dcdaf9fc07a0b121ba81be819f4d46d7ac1b111454d1bb191131906
-  data.tar.gz: 58cf73819292626fc40a696fecb7c7737ab89f92b7093e9c707bb42fe95ff83cda7303c510b807ab41a221fd0fe5c0ed5d3251983b466726dea9f252f276bdac
+  metadata.gz: e6e260de1875ebbb96a8b809907b50f8d92abe1a9e1d6aee711fd7f3f08567ba627325a035ac08b67f45e42468ad982a85b2f4c45c6709460abe186acbb71b6a
+  data.tar.gz: 3a11694e124de65ca11df02d6fc3aa38b7ee4d93881b2eba64f89a6c30924f3bee3d98297cc8ff9340aa029823bf5cef9e19077cad3b8acfc39aa9cc57e7fe29

data/README.md

@@ -1,10 +1,10 @@
 # BotChallengePage
 
-[![CI](https://github.com/samvera-labs/bot_challenge_page/actions/workflows/ci.yml/badge.svg)](https://github.com/samvera-labs/bot_challenge_page/actions/workflows/ci.yml)
+[![CI](https://github.com/samvera-labs/bot_challenge_page/actions/workflows/ci.yml/badge.svg)](https://github.com/samvera-labs/bot_challenge_page/actions/workflows/ci.yml) [![Gem Version](https://badge.fury.io/rb/bot_challenge_page.png)](http://badge.fury.io/rb/bot_challenge_page)
 
-BotChallengePage lets you protect certain routes in your Rails app with [CloudFlare Turnstile](https://www.cloudflare.com/application-services/products/turnstile/) "CAPTHCA alternate" bot detector. Rather than the typical form submission use case for Turnstile, the user will be redirected to an interstitial challenge page, and redirected back on success.
+BotChallengePage lets you protect certain routes in your Rails app with [CloudFlare Turnstile](https://www.cloudflare.com/application-services/products/turnstile/) "CAPTHCA alternate" bot detector. Rather than the typical form submission use case for Turnstile, the user will be redirected to an interstitial challenge page, and automatically redirected back immediately on success.
 
-The motivating use case is fairly dumb (probably AI-related) crawlers, rather than targetted attacks, although we have tried to pay attention to security.  Many of our use cases were crawlers getting caught in "infinite" page variations by following every combination of voluminous facet values in search results in a near "infinite space", and causing us resource usage issues.
+The motivating use case is fairly dumb (probably AI-related) crawlers, rather than targetted attacks, although we have tried to pay attention to security.  Many of our use cases were crawlers getting caught following every combination of voluminous facet values in search results in a near "infinite space", and causing us resource usage issues.
 
 ![challenge page screenshot](docs/challenge-page-example.png)
 
@@ -18,26 +18,37 @@ The motivating use case is fairly dumb (probably AI-related) crawlers, rather th
 
 ## Installation and Configuration
 
-* Get a CloudFlare account and Turnstile widget set up, which should give you a turnstile `sitekey` and `secret_key` you will need later in configuration.
+* Get a [CloudFlare account and Turnstile widget set up](https://www.cloudflare.com/application-services/products/turnstile/), which should give you a turnstile `sitekey` and `secret_key` you will need later in configuration.
 
 * `bundle add bot_challenge_page`, `bundle install`
 
 * Run the installer
   * if you want to use rack-attack for some permissive pre-challenge rate, `rails g bot_challenge_page:install`
-  * If you do not want to use rack-attack and want challenge on FIRST request, `rails g bot_challenge_page:install --without-rack-attack`
+
+  * If you do not want to use rack-attack and want challenge on FIRST request, `rails g bot_challenge_page:install --no-rack-attack`
+
+* If you are **not using rack-attack**, you need to add a before_action to the controller(s)
+  you'd like to protect, eg:
+
+        before_action only: :index do |controller|
+          BotChallengePage::BotChallengePageController.bot_challenge_enforce_filter(controller, immediate: true)
+        end
+
 
 * Configure in the generated `./config/initializers/bot_challenge_page.rb`
   * At a minimum you need to configure your Cloudflare Turnstile keys, and some paths to protect!
     * Note that we can only protect GET paths, and also think about making sure you DON'T protect
       any path your front-end needs JS `fetch` access to, as this would block it (at least
       without custom front-end code we haven't really explored)
+
     * If you are tempted to just protect `/` that may work, but worth thinking about any hearbeat paths, front-end requestable paths, or other machine-access-desired paths.
+
   * Some other configuration options are offered -- more advanced/specialized ones are available that are not mentioned in generated config file, see [Config class](./app/models/bot_challenge_page/config.rb)
 
 
 ## Customize challenge page display
 
-Some of the default challenge page html uses bootstrap alert classes. You may want to provide custom CSS if you aren't using bootstrap. You can see the default challenge page html at [challenge.html.erb](./app/views/bot_challenge_page/bot_challenge_page/challenge.html). You may wish to CSS-style other parts too!
+Some of the default challenge page html uses bootstrap alert classes. You may want to provide custom CSS if you aren't using bootstrap. You can see the default challenge page html at [challenge.html.erb](./app/views/bot_challenge_page/bot_challenge_page/challenge.html.erb). You may wish to CSS-style other parts too!
 
 You can customize all text via I18n, see keys in [bot_challenge_page.en.yml](./config/locales/bot_challenge_page.en.yml)
 
@@ -82,7 +93,7 @@ Rails.application.config.to_prepare do
     #
     # sec-fetch-dest is set to 'empty' by browser on fetch requests, to limit us further;
     # sure an attacker could fake it, we don't mind if someone determined can avoid
-    # rate-limiting on this one action
+    # bot challenge on this one action
     ( controller.params[:action] == "facet" &&
       controller.request.headers["sec-fetch-dest"] == "empty" &&
       controller.kind_of?(CatalogController)
@@ -94,6 +105,18 @@ end
 
 ```
 
+## Development and automated testing
+
+All logic and config hangs off a controller, with the idea that you could sub-class the controller to override any functionality -- or even have multiple sub-classes in your app with different configuration or customized config. But this hasn't really been tested/fleshed out yet.
+
+Run tests with `bundle exec rspec`.
+
+We test with a checked-into-repo dummy app at `./spec/dummy`, and use [Appraisal](https://github.com/thoughtbot/appraisal) to test under different rails versions.
+
+Locally one way to test with a specific rails version appraisal is `bundle exec appraisal rails-7.2 rspec`
+
+If you make any changes to `Gemfile` you may need to run `bundle exec appraisal install` and commit changes.
+
 ## Possible future features?
 
 * allow regex in default location_matcher? Easy to do if you want it, just say so.
@@ -114,6 +137,10 @@ The gem is available as open source under the terms of the [MIT License](https:/
 
 * [Similar feature built into PHP VuFind app](https://github.com/vufind-org/vufind/pull/4079)
 
-* Wow only after I developed all this did I notice [rails-cloudflare-turnstile](https://github.com/instrumentl/rails-cloudflare-turnstile) which implements some pieces that could have been re-used here, but I feel good.
+* [My own blog post about this approach](https://bibwild.wordpress.com/2025/01/16/using-cloudflare-turnstile-to-protect-certain-pages-on-a-rails-app/).
+
+* Wow only after I developed all this did I notice [rails-cloudflare-turnstile](https://github.com/instrumentl/rails-cloudflare-turnstile) which implements some pieces that could have been re-used here, but I feel good becuase we wanted these weird features. But if you want a much simpler more straightforward Turnstile implementation for more standard use cases or your own different use cases, I'd go here.
 
 * And yet another implementation in Rails that perhaps makes more assumptions about use cases, [turnstile-captcha](https://github.com/pfeiffer/turnstile-captcha). Haven't looked at it much.
+
+

data/app/controllers/bot_challenge_page/bot_challenge_page_controller.rb

@@ -11,6 +11,9 @@ require 'http'
 #
 module BotChallengePage
   class BotChallengePageController < ::ApplicationController
+    include BotChallengePage::RackAttackInit
+    include BotChallengePage::EnforceFilter
+
     # Config for bot detection is held in class object here -- idea is
     # to support different controllers with different config protecting
     # different paths in your app if you like, is why config is with controller
@@ -25,91 +28,6 @@ module BotChallengePage
     # for allowing unsubscribe for testing
     class_attribute :_track_notification_subscription, instance_accessor: false
 
-    # perhaps in an initializer, and after changing any config, run:
-    #
-    #     Rails.application.config.to_prepare do
-    #       BotChallengePage::BotChallengePageController.rack_attack_init
-    #     end
-    #
-    # Safe to call more than once if you change config and want to call again, say in testing.
-    def self.rack_attack_init
-      self._rack_attack_uninit # make it safe for calling multiple times
-
-      ## Turnstile bot detection throttling
-      #
-      # for paths matched by `rate_limited_locations`, after over rate_limit count requests in rate_limit_period,
-      # token will be stored in rack env instructing challenge is required.
-      #
-      # For actual challenge, need before_action in controller.
-      #
-      # You could rate limit detect on wider paths than you actually challenge on, or the same. You probably
-      # don't want to rate-limit detect on narrower list of paths than you challenge on!
-      Rack::Attack.track("bot_detect/rate_exceeded/#{self.name}",
-          limit: self.bot_challenge_config.rate_limit_count,
-          period: self.bot_challenge_config.rate_limit_period) do |req|
-        if self.bot_challenge_config.enabled && self.bot_challenge_config.location_matcher.call(req, self.bot_challenge_config)
-          self.bot_challenge_config.rate_limit_discriminator.call(req, self.bot_challenge_config)
-        end
-      end
-
-      self._track_notification_subscription = ActiveSupport::Notifications.subscribe("track.rack_attack") do |_name, _start, _finish, request_id, payload|
-        rack_request = payload[:request]
-        rack_env     = rack_request.env
-        match_name = rack_env["rack.attack.matched"]  # name of rack-attack rule
-                                                      #
-        if match_name == "bot_detect/rate_exceeded/#{self.name}"
-          match_data   = rack_env["rack.attack.match_data"]
-          match_data_formatted = match_data.slice(:count, :limit, :period).map { |k, v| "#{k}=#{v}"}.join(" ")
-          discriminator = rack_env["rack.attack.match_discriminator"] # unique key for rate limit, usually includes ip
-
-          rack_env[self.bot_challenge_config.env_challenge_trigger_key] = true
-        end
-      end
-    end
-
-    def self._rack_attack_uninit
-      Rack::Attack.track("bot_detect/rate_exceeded/#{self.name}") {} # overwrite track name with empty proc
-      ActiveSupport::Notifications.unsubscribe(self._track_notification_subscription) if self._track_notification_subscription
-      self._track_notification_subscription = nil
-    end
-
-    # Usually in your ApplicationController,
-    #
-    #     before_action { |controller| BotChallengePage::BotChallengePageController.bot_challenge_enforce_filter(controller) }
-    #
-    # @param immediate [Boolean] always force bot protection, ignore any allowed pre-challenge rate limit
-    def self.bot_challenge_enforce_filter(controller, immediate: false)
-      if self.bot_challenge_config.enabled &&
-          (controller.request.env[self.bot_challenge_config.env_challenge_trigger_key] || immediate) &&
-          ! self._bot_detect_passed_good?(controller.request) &&
-          ! controller.kind_of?(self) && # don't ever guard ourself, that'd be a mess!
-          ! self.bot_challenge_config.allow_exempt.call(controller, self.bot_challenge_config)
-
-        # we can only do GET requests right now
-        if !controller.request.get?
-          Rails.logger.warn("#{self}: Asked to protect request we could not, unprotected: #{controller.request.method} #{controller.request.url}, (#{controller.request.remote_ip}, #{controller.request.user_agent})")
-          return
-        end
-
-        Rails.logger.info("#{self.name}: Cloudflare Turnstile challenge redirect: (#{controller.request.remote_ip}, #{controller.request.user_agent}): from #{controller.request.url}")
-        # status code temporary
-        controller.redirect_to controller.bot_detect_challenge_path(dest: controller.request.original_fullpath), status: 307
-      end
-    end
-
-    # Does the session already contain a bot detect pass that is good for this request
-    # Tie to IP address to prevent session replay shared among IPs
-    def self._bot_detect_passed_good?(request)
-      session_data = request.session[self.bot_challenge_config.session_passed_key]
-
-      return false unless session_data && session_data.kind_of?(Hash)
-
-      datetime = session_data[SESSION_DATETIME_KEY]
-      ip   = session_data[SESSION_IP_KEY]
-
-      (ip == request.remote_ip) && (Time.now - Time.iso8601(datetime) < self.bot_challenge_config.session_passed_good_for )
-    end
-
 
     def challenge
       # possible custom render to choose layouts or templates, but normally

data/app/controllers/concerns/bot_challenge_page/enforce_filter.rb

@@ -0,0 +1,48 @@
+module BotChallengePage
+
+  # Extracted to concern in separate file mostly for readability, not expected to be used
+  # anywehre but BotChallengePageController -- we hang all logic off controller to allow multiple
+  # controllers in an app, and over-ride in sub-classes.
+  module EnforceFilter
+    extend ActiveSupport::Concern
+
+    class_methods do
+      # Usually in your ApplicationController, unless using `immediate`.
+      #
+      #     before_action { |controller| BotChallengePage::BotChallengePageController.bot_challenge_enforce_filter(controller) }
+      #
+      # @param immediate [Boolean] always force bot protection, ignore any allowed pre-challenge rate limit
+      def bot_challenge_enforce_filter(controller, immediate: false)
+        if self.bot_challenge_config.enabled &&
+            (controller.request.env[self.bot_challenge_config.env_challenge_trigger_key] || immediate) &&
+            ! self._bot_detect_passed_good?(controller.request) &&
+            ! controller.kind_of?(self) && # don't ever guard ourself, that'd be a mess!
+            ! self.bot_challenge_config.allow_exempt.call(controller, self.bot_challenge_config)
+
+          # we can only do GET requests right now
+          if !controller.request.get?
+            Rails.logger.warn("#{self}: Asked to protect request we could not, unprotected: #{controller.request.method} #{controller.request.url}, (#{controller.request.remote_ip}, #{controller.request.user_agent})")
+            return
+          end
+
+          Rails.logger.info("#{self.name}: Cloudflare Turnstile challenge redirect: (#{controller.request.remote_ip}, #{controller.request.user_agent}): from #{controller.request.url}")
+          # status code temporary
+          controller.redirect_to controller.bot_detect_challenge_path(dest: controller.request.original_fullpath), status: 307
+        end
+      end
+
+      # Does the session already contain a bot detect pass that is good for this request
+      # Tie to IP address to prevent session replay shared among IPs
+      def _bot_detect_passed_good?(request)
+        session_data = request.session[self.bot_challenge_config.session_passed_key]
+
+        return false unless session_data && session_data.kind_of?(Hash)
+
+        datetime = session_data[BotChallengePageController::SESSION_DATETIME_KEY]
+        ip   = session_data[BotChallengePageController::SESSION_IP_KEY]
+
+        (ip == request.remote_ip) && (Time.now - Time.iso8601(datetime) < self.bot_challenge_config.session_passed_good_for )
+      end
+    end
+  end
+end

data/app/controllers/concerns/bot_challenge_page/rack_attack_init.rb

@@ -0,0 +1,60 @@
+module BotChallengePage
+
+  # Extracted to concern in separate file mostly for readability, not expected to be used
+  # anywehre but BotChallengePageController -- we hang all logic off controller to allow multiple
+  # controllers in an app, and over-ride in sub-classes.
+  module RackAttackInit
+    extend ActiveSupport::Concern
+
+
+    class_methods do
+      # perhaps in an initializer, and after changing any config, run:
+      #
+      #     Rails.application.config.to_prepare do
+      #       BotChallengePage::BotChallengePageController.rack_attack_init
+      #     end
+      #
+      # Safe to call more than once if you change config and want to call again, say in testing.
+      def rack_attack_init
+        self._rack_attack_uninit # make it safe for calling multiple times
+
+        ## Turnstile bot detection throttling
+        #
+        # for paths matched by `rate_limited_locations`, after over rate_limit count requests in rate_limit_period,
+        # token will be stored in rack env instructing challenge is required.
+        #
+        # For actual challenge, need before_action in controller.
+        #
+        # You could rate limit detect on wider paths than you actually challenge on, or the same. You probably
+        # don't want to rate-limit detect on narrower list of paths than you challenge on!
+        Rack::Attack.track("bot_detect/rate_exceeded/#{self.name}",
+            limit: self.bot_challenge_config.rate_limit_count,
+            period: self.bot_challenge_config.rate_limit_period) do |req|
+          if self.bot_challenge_config.enabled && self.bot_challenge_config.location_matcher.call(req, self.bot_challenge_config)
+            self.bot_challenge_config.rate_limit_discriminator.call(req, self.bot_challenge_config)
+          end
+        end
+
+        self._track_notification_subscription = ActiveSupport::Notifications.subscribe("track.rack_attack") do |_name, _start, _finish, request_id, payload|
+          rack_request = payload[:request]
+          rack_env     = rack_request.env
+          match_name = rack_env["rack.attack.matched"]  # name of rack-attack rule
+                                                        #
+          if match_name == "bot_detect/rate_exceeded/#{self.name}"
+            match_data   = rack_env["rack.attack.match_data"]
+            match_data_formatted = match_data.slice(:count, :limit, :period).map { |k, v| "#{k}=#{v}"}.join(" ")
+            discriminator = rack_env["rack.attack.match_discriminator"] # unique key for rate limit, usually includes ip
+
+            rack_env[self.bot_challenge_config.env_challenge_trigger_key] = true
+          end
+        end
+      end
+
+      def _rack_attack_uninit
+        Rack::Attack.track("bot_detect/rate_exceeded/#{self.name}") {} # overwrite track name with empty proc
+        ActiveSupport::Notifications.unsubscribe(self._track_notification_subscription) if self._track_notification_subscription
+        self._track_notification_subscription = nil
+      end
+    end
+  end
+end

data/lib/bot_challenge_page/version.rb

@@ -1,3 +1,3 @@
 module BotChallengePage
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 end

data/lib/generators/bot_challenge_page/install_generator.rb

@@ -10,12 +10,12 @@ module BotChallengePage
     end
 
     def add_before_filter_enforcement
+      # make the user do this themselves if they aren't using rack-attack, as it should
+      # only be on protected filters
+      return unless options[:rack_attack]
+
       inject_into_class "app/controllers/application_controller.rb", "ApplicationController" do
-        filter_code = if options[:rack_attack]
-          "BotChallengePage::BotChallengePageController.bot_challenge_enforce_filter(controller)"
-        else
-          "BotChallengePage::BotChallengePageController.bot_challenge_enforce_filter(controller, immediate: true)"
-        end
+        filter_code = "BotChallengePage::BotChallengePageController.bot_challenge_enforce_filter(controller)"
 
         <<-EOS
   # This will only protect CONFIGURED routes, but also could be put on just certain
@@ -41,5 +41,23 @@ module BotChallengePage
       template "initializer.rb.erb", "config/initializers/bot_challenge_page.rb"
     end
 
+    def suggest_filter
+      unless options[:rack_attack]
+        instructions = <<~EOS
+        You must add before_action to protect controllers
+
+        Add, eg:
+
+            before_action only: :index do |controller|
+              BotChallengePage::BotChallengePageController.bot_challenge_enforce_filter(controller, immediate: true)
+            end
+
+        To desired controllers and/or ApplicationController
+        EOS
+
+        say_status("advise", instructions, :green)
+      end
+    end
+
   end
 end

data/lib/generators/bot_challenge_page/templates/initializer.rb.erb

@@ -3,9 +3,11 @@ Rails.application.config.to_prepare do
   BotChallengePage::BotChallengePageController.bot_challenge_config.enabled = true
 
   # Get from CloudFlare Turnstile: https://www.cloudflare.com/application-services/products/turnstile/
+  # Some testing keys are also available: https://developers.cloudflare.com/turnstile/troubleshooting/testing/
   BotChallengePage::BotChallengePageController.bot_challenge_config.cf_turnstile_sitekey = "MUST GET"
   BotChallengePage::BotChallengePageController.bot_challenge_config.cf_turnstile_secret_key = "MUST GET"
 
+  <%- if options[:rack_attack] %>
   # What paths do you want to protect?
   #
   # You can use path prefixes: "/catalog" or even "/"
@@ -22,15 +24,14 @@ Rails.application.config.to_prepare do
   BotChallengePage::BotChallengePageController.bot_challenge_config.rate_limited_locations = [
   ]
 
-  # How long will a challenge success exempt a session from further challenges?
-  # BotChallengePage::BotChallengePageController.bot_challenge_config.session_passed_good_for = 36.hours
-
-  <%- if options[:rack_attack] %>
   # allow rate_limit_count requests in rate_limit_period, before issuing challenge
   BotChallengePage::BotChallengePageController.bot_challenge_config.rate_limit_period = 12.hour
   BotChallengePage::BotChallengePageController.bot_challenge_config.rate_limit_count = 2
   <% end -%>
 
+  # How long will a challenge success exempt a session from further challenges?
+  # BotChallengePage::BotChallengePageController.bot_challenge_config.session_passed_good_for = 36.hours
+
   # Exempt some requests from bot challenge protection
   # BotChallengePage::BotChallengePageController.allow_exempt = ->(controller) {
   #   # controller.params

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: bot_challenge_page
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Jonathan Rochkind
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2025-02-27 00:00:00.000000000 Z
+date: 2025-03-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: appraisal
@@ -153,6 +153,8 @@ files:
 - README.md
 - Rakefile
 - app/controllers/bot_challenge_page/bot_challenge_page_controller.rb
+- app/controllers/concerns/bot_challenge_page/enforce_filter.rb
+- app/controllers/concerns/bot_challenge_page/rack_attack_init.rb
 - app/models/bot_challenge_page/config.rb
 - app/views/bot_challenge_page/_local_turnstile_script_tag.html.erb
 - app/views/bot_challenge_page/_turnstile_widget_placeholder.html.erb