bugsnag 6.12.1 → 6.15.0

data/features/rails_features/auto_notify.feature

@@ -1,118 +1,41 @@
 Feature: Auto notify
 
-Background:
-  Given I set environment variable "BUGSNAG_API_KEY" to "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And I set environment variable "APP_PATH" to "/usr/src"
-  And I configure the bugsnag endpoint
-
-Scenario Outline: Auto_notify set to false in the initializer prevents unhandled error sending
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I set environment variable "BUGSNAG_AUTO_NOTIFY" to "false"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/auto_notify/unhandled" on port "6128<rails_version>"
-  Then I should receive 0 requests
-
-  Examples:
-    | ruby_version | rails_version |
-    | 2.0          | 3             |
-    | 2.1          | 3             |
-    | 2.2          | 3             |
-    | 2.2          | 4             |
-    | 2.2          | 5             |
-    | 2.3          | 3             |
-    | 2.3          | 4             |
-    | 2.3          | 5             |
-    | 2.4          | 3             |
-    | 2.4          | 5             |
-    | 2.5          | 3             |
-    | 2.5          | 5             |
-    | 2.5          | 6             |
-
-Scenario Outline: Auto_notify set to false in the initializer still sends handled errors
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I set environment variable "BUGSNAG_AUTO_NOTIFY" to "false"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/auto_notify/handled" on port "6128<rails_version>"
-  Then I should receive a request
-  And the request is a valid for the error reporting API
-  And the request used the "Ruby Bugsnag Notifier" notifier
-  And the request contained the api key "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And the payload field "events" is an array with 1 element
+@rails3 @rails4 @rails5 @rails6
+Scenario: Auto_notify set to false in the initializer prevents unhandled error sending
+  Given I set environment variable "BUGSNAG_AUTO_NOTIFY" to "false"
+  And I start the rails service
+  When I navigate to the route "/auto_notify/unhandled" on the rails app
+  And I wait for 3 seconds
+  Then I should receive no requests
+
+@rails3 @rails4 @rails5 @rails6
+Scenario: Auto_notify set to false in the initializer still sends handled errors
+  Given I set environment variable "BUGSNAG_AUTO_NOTIFY" to "false"
+  And I start the rails service
+  When I navigate to the route "/auto_notify/handled" on the rails app
+  And I wait to receive a request
+  Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
   And the event "unhandled" is false
   And the exception "errorClass" equals "RuntimeError"
   And the exception "message" starts with "handled string"
   And the event "app.type" equals "rails"
   And the event "metaData.request.url" ends with "/auto_notify/handled"
 
-  Examples:
-    | ruby_version | rails_version |
-    | 2.0          | 3             |
-    | 2.1          | 3             |
-    | 2.2          | 3             |
-    | 2.2          | 4             |
-    | 2.2          | 5             |
-    | 2.3          | 3             |
-    | 2.3          | 4             |
-    | 2.3          | 5             |
-    | 2.4          | 3             |
-    | 2.4          | 5             |
-    | 2.5          | 3             |
-    | 2.5          | 5             |
-    | 2.5          | 6             |
-
-Scenario Outline: Auto_notify set to false after the initializer prevents unhandled error sending
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/auto_notify/unhandled_after" on port "6128<rails_version>"
-  Then I should receive 0 requests
-
-  Examples:
-    | ruby_version | rails_version |
-    | 2.0          | 3             |
-    | 2.1          | 3             |
-    | 2.2          | 3             |
-    | 2.2          | 4             |
-    | 2.2          | 5             |
-    | 2.3          | 3             |
-    | 2.3          | 4             |
-    | 2.3          | 5             |
-    | 2.4          | 3             |
-    | 2.4          | 5             |
-    | 2.5          | 3             |
-    | 2.5          | 5             |
-    | 2.5          | 6             |
-
-Scenario Outline: Auto_notify set to false after the initializer still sends handled errors
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/auto_notify/handled_after" on port "6128<rails_version>"
-  Then I should receive a request
-  And the request is a valid for the error reporting API
-  And the request used the "Ruby Bugsnag Notifier" notifier
-  And the request contained the api key "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And the payload field "events" is an array with 1 element
+@rails3 @rails4 @rails5 @rails6
+Scenario: Auto_notify set to false after the initializer prevents unhandled error sending
+  Given I start the rails service
+  When I navigate to the route "/auto_notify/unhandled_after" on the rails app
+  And I wait for 3 seconds
+  Then I should receive no requests
+
+@rails3 @rails4 @rails5 @rails6
+Scenario: Auto_notify set to false after the initializer still sends handled errors
+  Given I start the rails service
+  When I navigate to the route "/auto_notify/handled_after" on the rails app
+  And I wait to receive a request
+  Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
   And the exception "errorClass" equals "RuntimeError"
   And the exception "message" starts with "handled string"
   And the event "unhandled" is false
   And the event "metaData.request.url" ends with "/auto_notify/handled_after"
   And the event "app.type" equals "rails"
-
-  Examples:
-    | ruby_version | rails_version |
-    | 2.0          | 3             |
-    | 2.1          | 3             |
-    | 2.2          | 3             |
-    | 2.2          | 4             |
-    | 2.2          | 5             |
-    | 2.3          | 3             |
-    | 2.3          | 4             |
-    | 2.3          | 5             |
-    | 2.4          | 3             |
-    | 2.4          | 5             |
-    | 2.5          | 3             |
-    | 2.5          | 5             |
-    | 2.5          | 6             |

data/features/rails_features/before_notify.feature

@@ -1,20 +1,11 @@
 Feature: Before notify callbacks
 
-Background:
-  Given I set environment variable "BUGSNAG_API_KEY" to "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And I set environment variable "APP_PATH" to "/usr/src"
-  And I configure the bugsnag endpoint
-
-Scenario Outline: Rails before_notify controller method works on handled errors
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/before_notify/handled" on port "6128<rails_version>"
-  Then I should receive a request
-  And the request is a valid for the error reporting API
-  And the request used the "Ruby Bugsnag Notifier" notifier
-  And the request contained the api key "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And the payload field "events" is an array with 1 element
+@rails3 @rails4 @rails5 @rails6
+Scenario: Rails before_notify controller method works on handled errors
+  Given I start the rails service
+  When I navigate to the route "/before_notify/handled" on the rails app
+  And I wait to receive a request
+  Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
   And the exception "errorClass" equals "RuntimeError"
   And the exception "message" starts with "handled string"
   And the event "unhandled" is false
@@ -23,32 +14,12 @@ Scenario Outline: Rails before_notify controller method works on handled errors
   And the event "metaData.before_notify.source" equals "rails_before_handled"
   And the event "metaData.controller.name" equals "BeforeNotifyController"
 
-  Examples:
-    | ruby_version | rails_version |
-    | 2.0          | 3             |
-    | 2.1          | 3             |
-    | 2.2          | 3             |
-    | 2.2          | 4             |
-    | 2.2          | 5             |
-    | 2.3          | 3             |
-    | 2.3          | 4             |
-    | 2.3          | 5             |
-    | 2.4          | 3             |
-    | 2.4          | 5             |
-    | 2.5          | 3             |
-    | 2.5          | 5             |
-    | 2.5          | 6             |
-
-Scenario Outline: Rails before_notify controller method works on unhandled errors
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/before_notify/unhandled" on port "6128<rails_version>"
-  Then I should receive a request
-  And the request is a valid for the error reporting API
-  And the request used the "Ruby Bugsnag Notifier" notifier
-  And the request contained the api key "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And the payload field "events" is an array with 1 element
+@rails3 @rails4 @rails5 @rails6
+Scenario: Rails before_notify controller method works on unhandled errors
+  Given I start the rails service
+  When I navigate to the route "/before_notify/unhandled" on the rails app
+  And I wait to receive a request
+  Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
   And the exception "errorClass" equals "NameError"
   And the exception "message" starts with "undefined local variable or method `generate_unhandled_error' for #<BeforeNotifyController"
   And the event "unhandled" is true
@@ -57,32 +28,12 @@ Scenario Outline: Rails before_notify controller method works on unhandled error
   And the event "metaData.before_notify.source" equals "rails_before_unhandled"
   And the event "metaData.controller.name" equals "BeforeNotifyController"
 
-  Examples:
-    | ruby_version | rails_version |
-    | 2.0          | 3             |
-    | 2.1          | 3             |
-    | 2.2          | 3             |
-    | 2.2          | 4             |
-    | 2.2          | 5             |
-    | 2.3          | 3             |
-    | 2.3          | 4             |
-    | 2.3          | 5             |
-    | 2.4          | 3             |
-    | 2.4          | 5             |
-    | 2.5          | 3             |
-    | 2.5          | 5             |
-    | 2.5          | 6             |
-
-Scenario Outline: Inline block on handled errors is called
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/before_notify/inline" on port "6128<rails_version>"
-  Then I should receive a request
-  And the request is a valid for the error reporting API
-  And the request used the "Ruby Bugsnag Notifier" notifier
-  And the request contained the api key "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And the payload field "events" is an array with 1 element
+@rails3 @rails4 @rails5 @rails6
+Scenario: Inline block on handled errors is called
+  Given I start the rails service
+  When I navigate to the route "/before_notify/inline" on the rails app
+  And I wait to receive a request
+  Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
   And the exception "errorClass" equals "RuntimeError"
   And the exception "message" starts with "handled string"
   And the event "unhandled" is false
@@ -91,19 +42,3 @@ Scenario Outline: Inline block on handled errors is called
   And the event "metaData.before_notify.source" equals "rails_inline"
   And the event "metaData.controller.name" equals "BeforeNotifyController"
 
-  Examples:
-    | ruby_version | rails_version |
-    | 2.0          | 3             |
-    | 2.1          | 3             |
-    | 2.2          | 3             |
-    | 2.2          | 4             |
-    | 2.2          | 5             |
-    | 2.3          | 3             |
-    | 2.3          | 4             |
-    | 2.3          | 5             |
-    | 2.4          | 3             |
-    | 2.4          | 5             |
-    | 2.5          | 3             |
-    | 2.5          | 5             |
-    | 2.5          | 6             |
-

data/features/rails_features/breadcrumbs.feature

@@ -1,140 +1,43 @@
 Feature: Rails automatic breadcrumbs
 
-Background:
-  Given I set environment variable "BUGSNAG_API_KEY" to "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And I set environment variable "APP_PATH" to "/usr/src"
-  And I configure the bugsnag endpoint
-
-Scenario Outline: Request breadcrumb
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/breadcrumbs/handled" on port "6128<rails_version>"
-  Then I should receive a request
-  And the request is a valid for the error reporting API
-  And the request used the "Ruby Bugsnag Notifier" notifier
-  And the request contained the api key "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And the event has a "request" breadcrumb named "Controller started processing"
-  And the event "breadcrumbs.0.timestamp" is a timestamp
-  And the event "breadcrumbs.0.metaData.controller" equals "BreadcrumbsController"
-  And the event "breadcrumbs.0.metaData.action" equals "handled"
-  And the event "breadcrumbs.0.metaData.method" equals "GET"
-  And the event "breadcrumbs.0.metaData.path" equals "/breadcrumbs/handled"
-  And the event "breadcrumbs.0.metaData.event_name" equals "start_processing.action_controller"
-  And the event "breadcrumbs.0.metaData.event_id" is not null
-
-  Examples:
-    | ruby_version | rails_version |
-    | 2.0          | 3             |
-    | 2.1          | 3             |
-    | 2.2          | 3             |
-    | 2.2          | 4             |
-    | 2.2          | 5             |
-    | 2.3          | 3             |
-    | 2.3          | 4             |
-    | 2.3          | 5             |
-    | 2.4          | 3             |
-    | 2.4          | 5             |
-    | 2.5          | 3             |
-    | 2.5          | 5             |
-    | 2.5          | 6             |
-
-Scenario Outline: SQL Breadcrumb without bindings
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I set environment variable "SQL_ONLY_BREADCRUMBS" to "true"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/breadcrumbs/sql_breadcrumb" on port "6128<rails_version>"
-  Then I should receive a request
-  And the request is a valid for the error reporting API
-  And the request used the "Ruby Bugsnag Notifier" notifier
-  And the request contained the api key "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And the event has a "process" breadcrumb named "ActiveRecord SQL query"
-  And the event "breadcrumbs.0.timestamp" is a timestamp
-  And the event "breadcrumbs.0.metaData.name" equals "User Load"
-  And the event "breadcrumbs.0.metaData.connection_id" is not null
-  And the event "breadcrumbs.0.metaData.event_name" equals "sql.active_record"
-  And the event "breadcrumbs.0.metaData.event_id" is not null
-
-  Examples:
-    | ruby_version | rails_version |
-    | 2.0          | 3             |
-    | 2.1          | 3             |
-    | 2.2          | 3             |
-    | 2.2          | 4             |
-    | 2.3          | 3             |
-    | 2.3          | 4             |
-    | 2.4          | 3             |
-    | 2.5          | 3             |
-    | 2.5          | 6             |
-
-Scenario Outline: SQL Breadcrumb with bindings
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I set environment variable "SQL_ONLY_BREADCRUMBS" to "true"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/breadcrumbs/sql_breadcrumb" on port "6128<rails_version>"
-  Then I should receive a request
-  And the request is a valid for the error reporting API
-  And the request used the "Ruby Bugsnag Notifier" notifier
-  And the request contained the api key "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And the event has a "process" breadcrumb named "ActiveRecord SQL query"
-  And the event "breadcrumbs.0.timestamp" is a timestamp
-  And the event "breadcrumbs.0.metaData.name" equals "User Load"
-  And the event "breadcrumbs.0.metaData.connection_id" is not null
-  And the event "breadcrumbs.0.metaData.event_name" equals "sql.active_record"
-  And the event "breadcrumbs.0.metaData.event_id" is not null
-  And the event "breadcrumbs.0.metaData.binds" equals "{"email":"?","LIMIT":"?"}"
-
-  Examples:
-    | ruby_version | rails_version |
-    | 2.2          | 5             |
-    | 2.3          | 5             |
-    | 2.4          | 5             |
-    | 2.5          | 5             |
-    | 2.5          | 6             |
-
-Scenario Outline: Active job breadcrumb
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/breadcrumbs/active_job" on port "6128<rails_version>"
-  Then I should receive a request
-  And the request is a valid for the error reporting API
-  And the request used the "Ruby Bugsnag Notifier" notifier
-  And the request contained the api key "a35a2a72bd230ac0aa0f52715bbdc6aa"
-  And the event has a "process" breadcrumb named "Start perform ActiveJob"
-  And the event "breadcrumbs.0.timestamp" is a timestamp
-  And the event "breadcrumbs.0.metaData.event_name" equals "perform_start.active_job"
-  And the event "breadcrumbs.0.metaData.event_id" is not null
-
-  Examples:
-    | ruby_version | rails_version |
-    | 2.2          | 4             |
-    | 2.2          | 5             |
-    | 2.3          | 4             |
-    | 2.3          | 5             |
-    | 2.4          | 5             |
-    | 2.5          | 5             |
-    | 2.5          | 6             |
-
-Scenario Outline: Cache read
-  Given I set environment variable "RUBY_VERSION" to "<ruby_version>"
-  And I start the service "rails<rails_version>"
-  And I wait for the app to respond on port "6128<rails_version>"
-  When I navigate to the route "/breadcrumbs/cache_read" on port "6128<rails_version>"
-  Then I should receive a request
-  And the request is a valid for the error reporting API
-  And the request used the "Ruby Bugsnag Notifier" notifier
-  And the request contained the api key "a35a2a72bd230ac0aa0f52715bbdc6aa"
+@rails3 @rails4 @rails5 @rails6
+Scenario: Request breadcrumb
+  Given I start the rails service
+  When I navigate to the route "/breadcrumbs/handled" on the rails app
+  And I wait to receive a request
+  Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
+  And the event contains a breadcrumb matching the JSON fixture in "features/fixtures/expected_breadcrumbs/request.json"
+
+@rails3 @rails4
+Scenario: SQL Breadcrumb without bindings
+  Given I set environment variable "SQL_ONLY_BREADCRUMBS" to "true"
+  And I start the rails service
+  When I navigate to the route "/breadcrumbs/sql_breadcrumb" on the rails app
+  And I wait to receive a request
+  Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
+  And the event contains a breadcrumb matching the JSON fixture in "features/fixtures/expected_breadcrumbs/sql_without_bindings.json"
+
+@rails5 @rails6
+Scenario: SQL Breadcrumb with bindings
+  Given I set environment variable "SQL_ONLY_BREADCRUMBS" to "true"
+  And I start the rails service
+  When I navigate to the route "/breadcrumbs/sql_breadcrumb" on the rails app
+  And I wait to receive a request
+  Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
+  And the event contains a breadcrumb matching the JSON fixture in "features/fixtures/expected_breadcrumbs/sql_with_bindings.json"
+
+@rails4 @rails5 @rails6
+Scenario: Active job breadcrumb
+  Given I start the rails service
+  When I navigate to the route "/breadcrumbs/active_job" on the rails app
+  And I wait to receive a request
+  Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
+  And the event contains a breadcrumb matching the JSON fixture in "features/fixtures/expected_breadcrumbs/active_job.json"
+
+@rails4 @rails5 @rails6
+Scenario: Cache read
+  Given I start the rails service
+  When I navigate to the route "/breadcrumbs/cache_read" on the rails app
+  And I wait to receive a request
+  Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
   And the event has a "process" breadcrumb named "Read cache"
-
-  Examples:
-    | ruby_version | rails_version |
-    | 2.2          | 4             |
-    | 2.2          | 5             |
-    | 2.3          | 4             |
-    | 2.3          | 5             |
-    | 2.4          | 5             |
-    | 2.5          | 5             |
-    | 2.5          | 6             |