safe_query 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: c32a8798dd6eb9c24383f4857c3f1d75f8bbc37119a307dfa038f0e9b121455a
+  data.tar.gz: 27bbc0fc2ae83ab9c38c7448f3a174eb430a6fe156cc9f5d9cf6e40acb8235a9
+SHA512:
+  metadata.gz: 40ce4b878b2a67c199e46128ad73aac5bcba1def0e54740cfe076421ebf31852d88bc4b0454100cd55339a5fa157e7cdf57b1c8fa3235a9b3b9ba963e1699187
+  data.tar.gz: 983393fd6fc71ec8bb1dd4ff6565eb423c43652372466be0148fb85c74c8c14b7dad82e5b0d3de8e2aa3e10a175421a1c7939945b93fb33c94f0402db917851d

data/CHANGELOG.md

@@ -0,0 +1,7 @@
+# SafeQuery changelog
+
+## Unreleased
+- Add your PR changelog line here
+
+## 0.1.0 (2023-03-22)
+- Initial release

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) Peter Cai
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,92 @@
+# SafeQuery
+
+Safely query stuff in ActiveRecord.
+
+## Why
+
+To prevent unbounded resource consumption, we always want to limit how many rows can be returned from database fetches.
+
+Calls to `ActiveRecord::Relation#each` (without a LIMIT clause) are dangerous and should be avoided, because they can accidentally trigger an 
+unpaginated database fetch for millions of rows, which can exhaust your web server or database resources. Exceptions to this rule should be carefully vetted. 
+
+Worse, it's common to hit this problem only in production, because development environments seldom contain enough database rows to highlight the issue. 
+This makes it easy to write code that seems to work well, but fails when operating on a database with more data.
+
+This gem raises an exception whenever you attempt to call `ActiveRecord::Relation#each`, giving you the opportunity to catch and fix 
+this before any unsafe code hits production.
+
+## How it works
+
+With this gem installed, Rails will throw an exception when you make an unsafe query.
+
+## Compatibility:
+
+- Rails 5+
+- Ruby 2.6+
+- Postgres, MySQL, SQLite, maybe others (untested)
+
+## Installation:
+
+Add to your gemfile:
+
+```
+gem 'safe_query', group: [:development, :test]
+```
+
+then `bundle install`.
+
+It's recommended to set `config.active_record.warn_on_records_fetched_greater_than` (available since Rails 5), so you have warnings 
+whenever a query is returning more rows than expected, even when using this gem. 
+For example, if your app is never supposed to have no more than 100 records per page, add to `config/environments/development.rb`:
+
+```ruby
+  config.active_record.warn_on_records_fetched_greater_than = 100
+```
+
+## Example fixes:
+
+### Use `find_each` instead
+
+Sometimes the fix is as easy as changing
+
+```ruby
+book.authors.each do |author|
+```
+
+to this:
+
+```ruby
+book.authors.find_each do |author|
+```
+
+Sometimes this doesn't work:
+- For some reason you don't have an autoincrementing primary key ID for Rails to paginate on
+- You have a specific sort order and you want to maintain the sort order
+
+In those cases, you may have to add some custom code to maintain your existing app behavior. But otherwise, you can 
+use `find_each` or any other solution from the [ActiveRecord::Batches](https://api.rubyonrails.org/classes/ActiveRecord/Batches.html) API.
+
+### Paginate your results
+
+Use your existing pagination solution, or look at adding [pagy](https://github.com/ddnexus/pagy), [kaminari](https://github.com/kaminari/kaminari), 
+[will_paginate](https://github.com/mislav/will_paginate), etc to your app.
+
+### Just add a `limit` clause
+
+Sometimes you are simply missing a limit clause in your query. This might be the case if you have an implied 
+upper bound on the number of results enforced by the application elsewhere. SafeQuery will find cases where this limit isn't expressed in your queries, 
+which might be a problem if your enforcement logic is flawed in some way.
+
+### Ignore this problem
+
+You can ignore this problem by converting the relation to an array with `to_a` before you operate on it:
+
+```ruby
+book.authors.find_by ...
+```
+
+to this:
+
+```ruby
+book.authors.to_a.find_by ...
+```

data/lib/safe_query.rb

@@ -0,0 +1,47 @@
+module ActiveRecord
+  class UnsafeQueryError < StandardError
+    # skip ourselves in the backtrace so it ends in the user code that generated the issue
+    def backtrace
+      return @lines if @lines
+      @lines = super
+      @lines.shift if @lines.present?
+      @lines
+    end
+  end
+  
+  class Relation
+    module SafeQuery
+      def each
+        QueryRegistry.reset
+        super
+
+        query_to_check = QueryRegistry.queries.first.to_s
+
+        unless query_to_check.blank? || query_to_check.upcase.include?("LIMIT ") || query_to_check.upcase.include?("IN ")
+          raise UnsafeQueryError, "Detected a potentially dangerous #each iterator on an unpaginated query. " +
+            "Perhaps you need to add pagination, a limit clause, or use the ActiveRecord::Batches methods. \n\n" +
+            "To ignore this problem, or if it is a false positive, convert it to an array with ActiveRecord::Relation#to_a before iterating.\n\n" ++
+            "Potentially unpaginated query: \n\n  #{query_to_check}"
+        end
+      end
+
+      ActiveSupport::Notifications.subscribe("sql.active_record") do |*, payload|
+        QueryRegistry.queries << payload[:sql]
+      end
+
+      module QueryRegistry
+        extend self
+
+        def queries
+          ActiveSupport::IsolatedExecutionState[:active_record_query_registry] ||= []
+        end
+
+        def reset
+          queries.clear
+        end
+      end
+    end
+  end
+end
+
+ActiveRecord::Relation.prepend ActiveRecord::Relation::SafeQuery

metadata

@@ -0,0 +1,115 @@
+--- !ruby/object:Gem::Specification
+name: safe_query
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Peter Cai
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2023-03-22 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '5.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '5.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.0'
+- !ruby/object:Gem::Dependency
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '5.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '5.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.12'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.12'
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.6.1
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.6.1
+description: "    Helps developers avoid unsafe queries in ActiveRecord. This gem
+  will raise an error \n    when iterating over a relation that is potentially unpaginated.\n"
+email: hello@petercai.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE
+- README.md
+- lib/safe_query.rb
+homepage: https://github.com/pcai/safe_query
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 2.6.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.3.26
+signing_key: 
+specification_version: 4
+summary: Safely query stuff in ActiveRecord
+test_files: []