capybara_accessibility_audit 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6f3c291518b387bd0ebb540dd63aa42e7ff1740741ca5dbf1a25cc4be69cbcfc
-  data.tar.gz: bf7184f0a2ac3b7014665cb58bb9bed8d5d8ef180974b78e4d852d2759cec221
+  metadata.gz: 152ffd3d124d45c7cc7cec829bcd30c18a2f97dbc988c23cb7629cfe0e8eaa09
+  data.tar.gz: c174479db34168f55e3526e011e833d8fb6d7282392ab1f706d56be2786f1ca4
 SHA512:
-  metadata.gz: 7f6975cda1ae0f1465159dc662a171b1236eda9d8e3afaa605ce4dee7ee49cb29ed806f40b01c4394a9c37257ae689c825669371fc943aa3142d272f53c95eb2
-  data.tar.gz: 40cff319bee4767c4e1e12d37b074728d24d84dc9ab302ec481104a22c87f30dad4c851ee96aaecf170188d821d27a09e793dba4455a6495f45c1afe6416b200
+  metadata.gz: fa0be00cf21080048e5fec94d3cabb82dbe61aca8a5d04ea341d8aef776aefe5fae2936e958e66de86d0c064cea2d61a2c4f22ac764808c9f6d28c09059f385b
+  data.tar.gz: 31d70ffeab59611501a8bbf3933137526273b2e871a716d636cf157168b299b0fa5145925b87be58aeebd314bd7937801fc184cff6d257a1979003893ec1a0d9

data/README.md

@@ -60,6 +60,56 @@ end
 [aXe]: https://www.deque.com/axe/
 [axe-core-rspec]: https://github.com/dequelabs/axe-core-gems/blob/develop/packages/axe-core-rspec/README.md#matcher
 
+## Active Support instrumentation
+
+Capybara Accessibility Audit integrates with [Active Support's
+instrumentation][] through publishing [ActiveSupport::Notifications][].
+
+### `audit.capybara_accessibility_audit` notification
+
+Subscribe to `audit.capybara_accessibility_audit` notifications emitted when an
+accessibility audit is automatically conducted. In addition to the
+metadata provided by default (like `name`, `duration`, and `allocations`, etc.),
+the `payload` includes additional information:
+
+| Payload       | Type                               | Description |
+| ------------- | ---------------------------------- | ----------- |
+| method        | Symbol                             | The Capybara method that triggered the audit
+| options       | [ActiveSupport::OrderedOptions][]  | The audit's configuration
+| test          | [ActionDispatch::SystemTestCase][] | The test case that triggered the audit
+
+> [!NOTE]
+> The `audit.capybara_accessibility_audit` notifications are only published when
+> an audit is conducted automatically. No `audit.capybara_accessibility_audit`
+> notifications will be published when `assert_no_accessibility_violations` is
+> invoked directly.
+
+[Active Support's instrumentation]: https://guides.rubyonrails.org/active_support_instrumentation.html
+[ActiveSupport::Notifications]: https://api.rubyonrails.org/classes/ActiveSupport/Notifications.html
+[ActiveSupport::OrderedOptions]: https://api.rubyonrails.org/classes/ActiveSupport/OrderedOptions.html
+[ActionDispatch::SystemTestCase]: https://api.rubyonrails.org/classes/ActionDispatch/SystemTestCase.html
+
+### `report.capybara_accessibility_audit` notification
+
+Subscribe to `report.capybara_accessibility_audit` notifications emitted when an
+accessibility audit detects violations. The `payload` includes:
+
+| Payload       | Type                               | Description |
+| ------------- | ---------------------------------- | ----------- |
+| report        | [Axe::API::Results][]              | The underlying axe.js [Results][] object
+| options       | [ActiveSupport::OrderedOptions][]  | The audit's configuration
+| test          | [ActionDispatch::SystemTestCase][] | The test case that triggered the audit
+
+[ActiveSupport::OrderedOptions]: https://api.rubyonrails.org/classes/ActiveSupport/OrderedOptions.html
+[ActionDispatch::SystemTestCase]: https://api.rubyonrails.org/classes/ActionDispatch/SystemTestCase.html
+[Axe::API::Results]: https://github.com/dequelabs/axe-core-gems/blob/v4.11.0/packages/axe-core-api/lib/axe/api/results.rb
+[Results]: https://www.deque.com/axe/core-documentation/api-documentation/#results-object
+
+> [!NOTE]
+> The `report.capybara_accessibility_audit` notifications are only published when
+> `config.capybara_accessibility_audit.reporter` is configured with
+> `:notification` or `:log`.
+
 ## Frequently Asked Questions
 
 My application already exists, automated accessibility audits are uncovering violations left and right. Do I have to fix them all at once?
@@ -114,6 +164,28 @@ end
 As you resolve the violations, you can remove entries from the list of skipped
 rules.
 
+How many I conduct a preliminary audit that comprehensively exercises my system test suite without failing tests that result in violations?
+---
+
+You can configure the audit's reporting mechanism. By default, the
+`config.capybara_accessibility_audit.reporter` value is set to `:raise`, which
+will raise violation errors that will fail the test suite.
+
+To log violations, rather than raise them, you can configure
+`config.capybara_accessibility_audit.reporter` to `:log`:
+
+
+```ruby
+class MySystemTest < ApplicationSystemTestCase
+  self.accessibility_audit_reporter = :log
+
+  test "with overridden accessibility :log reporter" do
+    visit examples_path
+    # ...
+  end
+end
+```
+
 I've implemented a custom Capybara action to toggle a disclosure element. How can I automatically audit for violations after it's called?
 ---
 
@@ -204,6 +276,22 @@ class MySystemTest < ApplicationSystemTestCase
 end
 ```
 
+My pages render on the client. How long does the audit wait for them?
+---
+
+Capybara returns from an action as soon as the driver is done with it, and a
+page that renders on the client keeps changing after that point. The auditor
+therefore waits for the DOM to stay unchanged before it runs `axe`. It waits
+for 100 milliseconds of quiet, and it audits the page as it is after 2000
+milliseconds, so a page that never settles cannot block the suite.
+
+To change those values, override them on the auditor:
+
+```ruby
+CapybaraAccessibilityAudit::AxeAuditor.quiet_period_ms = 250
+CapybaraAccessibilityAudit::AxeAuditor.quiet_period_timeout_ms = 5000
+```
+
 ## Installation
 Add this line to your application's Gemfile:
 
@@ -238,5 +326,4 @@ We are [available for hire][hire].
 [community]: https://thoughtbot.com/community?utm_source=github
 [hire]: https://thoughtbot.com/hire-us?utm_source=github
 
-
 <!-- END /templates/footer.md -->

data/lib/capybara_accessibility_audit/{auditor.rb → adapter.rb}

@@ -1,5 +1,5 @@
 module CapybaraAccessibilityAudit
-  class Auditor
+  class Adapter
     delegate_missing_to :@test
 
     def initialize(test)
@@ -8,7 +8,13 @@ module CapybaraAccessibilityAudit
 
     def audit!(method)
       if accessibility_audit_enabled && method.in?(accessibility_audit_after_methods) && javascript_enabled?
-        assert_no_accessibility_violations(**accessibility_audit_options)
+        ActiveSupport::Notifications.instrument "audit.capybara_accessibility_audit" do |payload|
+          payload[:method] = method
+          payload[:options] = accessibility_audit_options
+          payload[:test] = @test
+
+          assert_no_accessibility_violations(**accessibility_audit_options)
+        end
       end
     end
 

data/lib/capybara_accessibility_audit/audit_system_test_extensions.rb

@@ -1,5 +1,3 @@
-require "axe/matchers/be_axe_clean"
-
 module CapybaraAccessibilityAudit
   module AuditSystemTestExtensions
     extend ActiveSupport::Concern
@@ -15,11 +13,14 @@ module CapybaraAccessibilityAudit
       class_attribute :accessibility_audit_after_methods, default: Set.new
       class_attribute :accessibility_audit_enabled, default: true
       class_attribute :accessibility_audit_options, default: ActiveSupport::OrderedOptions.new
+      class_attribute :accessibility_audit_reporter, default: :raise
+
+      attr_accessor :accessibility_audit_auditor
 
       MODAL_METHODS.each do |method|
         define_method method do |*arguments, **options, &block|
           result = super(*arguments, **options) { skip_accessibility_audits(&block) }
-          result.tap { Auditor.new(self).audit!(method) }
+          result.tap { Adapter.new(self).audit!(method) }
         end
       end
     end
@@ -35,7 +36,7 @@ module CapybaraAccessibilityAudit
       def accessibility_audit_after(*methods)
         (methods.flatten.to_set - accessibility_audit_after_methods).each do |method|
           define_method method do |*arguments, **options, &block|
-            super(*arguments, **options, &block).tap { Auditor.new(self).audit!(method) }
+            super(*arguments, **options, &block).tap { Adapter.new(self).audit!(method) }
           end
 
           accessibility_audit_after_methods << method
@@ -89,7 +90,7 @@ module CapybaraAccessibilityAudit
       accessibility_audit_options.skipping = skipping
     end
 
-    def assert_no_accessibility_violations(**options)
+    def assert_no_accessibility_violations(auditor: accessibility_audit_auditor, **options)
       options.assert_valid_keys(
         :according_to,
         :checking,
@@ -100,10 +101,7 @@ module CapybaraAccessibilityAudit
       )
       options.compact_blank!
 
-      axe_matcher = Axe::Matchers::BeAxeClean.new
-      axe_matcher = options.inject(axe_matcher) { |matcher, option| matcher.public_send(*option) }
-
-      assert axe_matcher.matches?(page), axe_matcher.failure_message
+      auditor.audit(**options)
     end
   end
 end

data/lib/capybara_accessibility_audit/axe_auditor.rb

@@ -0,0 +1,131 @@
+require "axe/api/context"
+require "axe/api/options"
+require "axe/api/results"
+require "axe/configuration"
+
+module CapybaraAccessibilityAudit
+  class AxeAuditor
+    Error = Class.new(StandardError)
+
+    ERROR_KEY = "capybara_accessibility_audit_error"
+
+    class_attribute :source, instance_accessor: false, default: Axe::Configuration.instance.jslib
+
+    class_attribute :quiet_period_ms, instance_accessor: false, default: 100
+    class_attribute :quiet_period_timeout_ms, instance_accessor: false, default: 2000
+
+    def initialize(test, reporter)
+      @test = test
+      @reporter = reporter
+    end
+
+    def audit(**options)
+      settle
+      install
+
+      results = run(options)
+
+      @reporter.report Axe::API::Results.new(results)
+    end
+
+    private
+
+    def page
+      @test.page
+    end
+
+    def settle
+      quiet_period_ms = self.class.quiet_period_ms
+      earliest = monotonic_time + quiet_period_ms
+      deadline = monotonic_time + self.class.quiet_period_timeout_ms
+
+      observe_mutations
+
+      until monotonic_time >= deadline
+        if (duration = unchanged_for).nil?
+          observe_mutations
+        elsif monotonic_time >= earliest && duration >= quiet_period_ms
+          break
+        end
+
+        sleep 0.01
+      end
+    end
+
+    def monotonic_time
+      Process.clock_gettime(Process::CLOCK_MONOTONIC, :millisecond)
+    end
+
+    # Records the time of the last change to the DOM. The document keeps the
+    # observer that is already there.
+    def observe_mutations
+      page.execute_script <<~JS
+        if (!window.capybaraAccessibilityAudit) {
+          window.capybaraAccessibilityAudit = {changedAt: Date.now()}
+
+          new MutationObserver(() => {
+            window.capybaraAccessibilityAudit.changedAt = Date.now()
+          }).observe(document, {attributes: true, characterData: true, childList: true, subtree: true})
+        }
+      JS
+    end
+
+    # Answers how long the DOM has been unchanged, in milliseconds. Answers
+    # nil when a page load replaced the document that has the observer. Each
+    # call is a short round trip. A single long-lived script would lose its
+    # callback when a page load discards the document it runs in.
+    def unchanged_for
+      duration = page.evaluate_script <<~JS
+        window.capybaraAccessibilityAudit ? Date.now() - window.capybaraAccessibilityAudit.changedAt : null
+      JS
+
+      duration if duration.is_a?(Numeric) # :playwright returns null as {}
+    end
+
+    def run(config)
+      context, options = split(config)
+
+      json = page.evaluate_async_script <<~JS, context.as_json, options.as_json, ERROR_KEY
+        const [ context, options, errorKey, callback ] = arguments
+
+        axe.run(context, options)
+          .then(JSON.stringify)
+          .catch(error => JSON.stringify({ [errorKey]: error.message }))
+          .then(callback)
+      JS
+
+      case JSON.parse(json, symbolize_names: true)
+      in {capybara_accessibility_audit_error:}
+        raise Error, capybara_accessibility_audit_error
+      in Hash => report
+        report
+      else
+        raise Error, "Unexpected report structure"
+      end
+    end
+
+    def split(config)
+      context = Axe::API::Context.new
+      options = Axe::API::Options.new
+
+      config.each do |name, value|
+        case name
+        when :within, :excluding then context.public_send(name, value)
+        else options.public_send(name, value)
+        end
+      end
+
+      [context, options]
+    end
+
+    def install
+      page.execute_script(self.class.source) unless installed?
+    end
+
+    def installed?
+      page.evaluate_script <<~JS
+        "axe" in window && typeof axe.run === "function"
+      JS
+    end
+  end
+end

data/lib/capybara_accessibility_audit/engine.rb

@@ -1,6 +1,7 @@
 module CapybaraAccessibilityAudit
   class Engine < ::Rails::Engine
     config.capybara_accessibility_audit = ActiveSupport::OrderedOptions.new
+    config.capybara_accessibility_audit.auditor = AxeAuditor
     config.capybara_accessibility_audit.audit_after = %i[
       visit
       click_button
@@ -9,14 +10,23 @@ module CapybaraAccessibilityAudit
       click_on
     ]
     config.capybara_accessibility_audit.audit_enabled = true
+    config.capybara_accessibility_audit.reporter = :raise
 
     initializer "capybara_accessibility_audit.minitest" do |app|
       ActiveSupport.on_load :action_dispatch_system_test_case do
         include CapybaraAccessibilityAudit::AuditSystemTestExtensions
 
         self.accessibility_audit_enabled = app.config.capybara_accessibility_audit.audit_enabled
+        self.accessibility_audit_reporter = app.config.capybara_accessibility_audit.reporter
 
         accessibility_audit_after app.config.capybara_accessibility_audit.audit_after
+
+        setup do
+          auditor_class = app.config.capybara_accessibility_audit.auditor
+          reporter_class = CapybaraAccessibilityAudit.reporter_class(accessibility_audit_reporter)
+
+          self.accessibility_audit_auditor = auditor_class.new(self, reporter_class.new(self))
+        end
       end
     end
 
@@ -32,6 +42,11 @@ module CapybaraAccessibilityAudit
             self.accessibility_audit_enabled = app.config.capybara_accessibility_audit.audit_enabled
 
             accessibility_audit_after app.config.capybara_accessibility_audit.audit_after
+
+            auditor_class = app.config.capybara_accessibility_audit.auditor
+            reporter_class = CapybaraAccessibilityAudit.reporter_class(accessibility_audit_reporter)
+
+            self.accessibility_audit_auditor = auditor_class.new(self, reporter_class.new(self))
           end
 
           config.before(type: :system, &configure)

data/lib/capybara_accessibility_audit/log_subscriber.rb

@@ -0,0 +1,17 @@
+module CapybaraAccessibilityAudit
+  class LogSubscriber < ActiveSupport::LogSubscriber
+    def report(event)
+      report = event.payload[:report]
+      test = event.payload[:test]
+
+      if test.accessibility_audit_reporter == :log
+        error color(<<~ERROR, :red)
+          [capybara_accessibility_audit] Accessibility audit detected violations in "#{test.class.name}##{test.name}":
+          #{report.failure_message}
+        ERROR
+      end
+    end
+
+    attach_to :capybara_accessibility_audit
+  end
+end

data/lib/capybara_accessibility_audit/notification_reporter.rb

@@ -0,0 +1,24 @@
+module CapybaraAccessibilityAudit
+  class NotificationReporter
+    def initialize(test)
+      @test = test
+    end
+
+    def report(results)
+      if results.violations.present?
+        publish(results)
+      end
+    end
+
+    private
+
+    def publish(report)
+      ActiveSupport::Notifications.instrument "report.capybara_accessibility_audit", {
+        auditor: @test.accessibility_audit_auditor,
+        options: @test.accessibility_audit_options,
+        report: report,
+        test: @test
+      }
+    end
+  end
+end

data/lib/capybara_accessibility_audit/raise_reporter.rb

@@ -0,0 +1,11 @@
+module CapybaraAccessibilityAudit
+  class RaiseReporter
+    def initialize(test)
+      @test = test
+    end
+
+    def report(results)
+      @test.assert results.violations.empty?, -> { results.failure_message }
+    end
+  end
+end

data/lib/capybara_accessibility_audit/version.rb

@@ -1,3 +1,3 @@
 module CapybaraAccessibilityAudit
-  VERSION = "0.2.0"
+  VERSION = "0.3.0"
 end

data/lib/capybara_accessibility_audit.rb

@@ -3,7 +3,20 @@ loader = Zeitwerk::Loader.for_gem
 loader.setup
 
 module CapybaraAccessibilityAudit
-  # Your code goes here...
+  extend self
+
+  def reporter_class(name)
+    case name
+    when :notification, :log
+      NotificationReporter
+    when :raise
+      RaiseReporter
+    when ::Class
+      name
+    else
+      raise ArgumentError.new("unsupported reporter: #{name}")
+    end
+  end
 end
 
 loader.eager_load

metadata

@@ -1,14 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: capybara_accessibility_audit
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Sean Doyle
-autorequire:
 bindir: bin
 cert_chain: []
-date: 2024-04-08 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -66,6 +65,48 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: bigdecimal
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: drb
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: mutex_m
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: Accessibility tooling for Capybara
 email:
 - sean.p.doyle24@gmail.com
@@ -77,9 +118,13 @@ files:
 - README.md
 - Rakefile
 - lib/capybara_accessibility_audit.rb
+- lib/capybara_accessibility_audit/adapter.rb
 - lib/capybara_accessibility_audit/audit_system_test_extensions.rb
-- lib/capybara_accessibility_audit/auditor.rb
+- lib/capybara_accessibility_audit/axe_auditor.rb
 - lib/capybara_accessibility_audit/engine.rb
+- lib/capybara_accessibility_audit/log_subscriber.rb
+- lib/capybara_accessibility_audit/notification_reporter.rb
+- lib/capybara_accessibility_audit/raise_reporter.rb
 - lib/capybara_accessibility_audit/version.rb
 - lib/tasks/capybara_accessibility_audit.rake
 homepage: https://github.com/thoughtbot/capybara_accessibility_audit
@@ -89,7 +134,7 @@ metadata:
   homepage_uri: https://github.com/thoughtbot/capybara_accessibility_audit
   source_code_uri: https://github.com/thoughtbot/capybara_accessibility_audit
   changelog_uri: https://github.com/thoughtbot/capybara_accessibility_audit/blob/main/CHANGELOG.md
-post_install_message:
+  rubygems_mfa_required: 'true'
 rdoc_options: []
 require_paths:
 - lib
@@ -104,8 +149,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.1
-signing_key:
+rubygems_version: 4.0.17
 specification_version: 4
 summary: Accessibility tooling for Capybara
 test_files: []