filter_param 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: ff8a89676339b2fb5deed26115ac897d11e77b554ddd10156c746b8313e9626e
+  data.tar.gz: 7f1f7f22bd1f106fe4818edf3dcc454f34897a0e0f9b73a9110f026424872008
+SHA512:
+  metadata.gz: f2cc08c9fdbae1de993f41b9d4204592a1bf80fd35586e45eb684cb5cd1478977e0d55a35b53d9b7d14b49514db93268d8baa9bf5a26be59ea8b4bfe2aa41c5a
+  data.tar.gz: 1aec98d47ae4177b0061fcaec187ce894cfda9d9d288b6fd0d5c15e9327619139066767fce65489bf0dccc5e84e5b29eae069c10f8f4ae2c0a2a94e4fd08f27b

data/CHANGELOG.md

@@ -0,0 +1,5 @@
+## [Unreleased]
+
+## [0.1.0] - 2023-06-15
+
+- Initial release

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Jayson
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,138 @@
+# FilterParam
+
+### Record Filtering for apps built on Rails/ActiveRecord 
+
+Quickly implement record filtering in your APIs using a filter expression inspired by [SCIM Query](https://datatracker.ietf.org/doc/html/rfc7644#section-3.4.2.2):
+
+```ruby
+https://{some origin}/users?filter="first_name eq 'John' and last_name pr and 
+  not (active eq false and (birth_date gt '1991-01-01' or birth_date eq null))"
+```
+
+**TL;DR** See [ sample usage for Rails here ](#rails-usage). 
+
+## Features
+
+* Transpilation of the filter expression into SQL
+* Whitelisting of allowed filter attributes
+* Column name aliasing / expose a different attribute name in the API
+* Pre-processing of filter values/literals
+* Type validation of filter values (e.g., date and datetime literals should be in standard ISO 8601 format)
+* Allows custom filter operators
+* Expression grouping
+* Supports **MySQL**, **PostgreSQL**, and **SQLite**
+* Supports **Rails 4.2** and above
+
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'filter_param'
+```
+
+And then execute:
+
+```sh
+bundle install
+```
+
+Or install it yourself as:
+
+```sh
+gem install filter_param
+```
+
+## Usage
+### Basic
+
+#### 1. Whitelist/define the filter fields
+
+```ruby
+fitler_param = FilterParam.define do
+                 field :first_name, type: :string
+                 field :last_name, rename: "family_name"
+                 field :birth_date, type: :date
+                 field :member_since, type: :datetime
+                 field :active, type: :boolean
+               end
+```
+
+
+This is is equivalent to:
+
+```ruby
+filter_param = FilterParam::Definition.new
+                                      .field(:first_name, type: :string)
+                                      .field(:last_name, rename: "family_name")
+                                      .field(:birth_date, type: :date)
+                                      .field(:member_since, type: :datetime)
+                                      .field(:active, type: :boolean)
+```
+
+`field` method accepts the filter field name as the first argument. Any other configuration such as `:type` follows the name.
+
+#### 2. Filter records using a filter expression
+
+The `filter!` method accepts an `ActiveRecord_Relation` and the filter expression string from your API's request parameter. This method then transpiles the filter expression into SQL and returns a new `ActiveRecord_Relation` with the SQL conditions applied.
+
+```ruby
+rel = filter_param.filter!(User.all, "first_name eq 'John' and last_name pr and not (active eq false and (birth_date gt '1991-01-
+01' or birth_date eq null))")
+``` 
+
+To see the SQL that will be executed in the ActiveRecord relation:
+
+```ruby
+rel.to_sql
+=> "SELECT \"users\".* FROM \"users\" WHERE (first_name = 'John' AND 
+    (family_name IS NOT NULL AND TRIM(family_name) != '') AND 
+      NOT (active = 0 AND (birth_date > '1991-01-01' OR birth_date IS NULL)))"
+```
+
+### Errors
+
+| Class | Description |
+| ----------- | ----------- |
+| `FilterParam::UnknownField` | A filter field in the given filter expression is not whitelisted in the filter definition. |
+| `FilterParam::ParseError` | The given filter expression can't be parsed possibly due to malformed expression or syntax issue. |
+| `FilterParam::InvalidLiteral` | A filter field value in the given filter expression is invalie (e.g., date and datetime should be in ISO 8601 format) |
+| `FilterParam::ExpressionError` | Generic error caused by the given filter expression. |
+| `FilterParam::UnknownType` | Configured `:type` of a filter field in the definition is invalid. |
+
+## Development
+
+1. If testing/developing for MySQL or PG, create the database first:<br/>
+
+  ###### MySQL
+  ```sh
+  mysql> CREATE DATABASE filter_param;
+  ```
+
+  ###### PostgreSQL
+  ```sh
+  $ createdb filter_param
+  ```
+
+2. After checking out the repo, run `bin/setup` to install dependencies.
+3. Run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment. Use the environment variables below to target the database<br/><br/>
+  
+  By default, SQLite and the latest stable Rails version are used in tests and console. Refer to the environment variables below to change this:
+
+  | Environment Variable | Values | Example |
+  | ----------- | ----------- |----------- |
+  | `DB_ADAPTER` | **Default: :sqlite**. `sqlite`,`mysql2`, or `postgresql` | ```DB_ADAPTER=postgresql bundle exec rspec```<br/><br/> ```DB_ADAPTER=postgresql ./bin/console``` |
+  | `RAILS_VERSION` | **Default: 7-0** <br/><br/> `4-2`,`5-0`,`5-1`,`5-2`,`6-0`,`6-1`,`7-0` |```RAILS_VERSION=5-2 ./bin/setup```<br/><br/>```RAILS_VERSION=5-2 bundle exec rspec```<br/><br/> ```RAILS_VERSION=5-2 ./bin/console```|
+
+
+<br/><br/>
+To install this gem onto your local machine, run `bundle exec rake install`.
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/jsonb-uy/rotulus.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/lib/filter_param/ast/attribute.rb

@@ -0,0 +1,15 @@
+module FilterParam
+  module AST
+    class Attribute < Node
+      attr_reader :name
+
+      def initialize(name)
+        super()
+
+        @name = name
+      end
+
+      alias to_s name
+    end
+  end
+end

data/lib/filter_param/ast/expressions.rb

@@ -0,0 +1,37 @@
+module FilterParam
+  module AST
+    module Expressions
+      class Expression < Node
+        attr_reader :operator, :operands
+
+        def initialize(operator, *operands)
+          super()
+
+          @operator = operator.to_sym
+          @operands = operands
+        end
+      end
+
+      class UnaryExpression < Expression
+        attr_reader :operand
+
+        def initialize(operator, operand)
+          super(operator, operand)
+
+          @operand = operand
+        end
+      end
+
+      class BinaryExpression < Expression
+        attr_reader :left_operand, :right_operand
+
+        def initialize(operator, left_operand, right_operand)
+          super(operator, left_operand, right_operand)
+
+          @left_operand = left_operand
+          @right_operand = right_operand
+        end
+      end
+    end
+  end
+end

data/lib/filter_param/ast/group.rb

@@ -0,0 +1,13 @@
+module FilterParam
+  module AST
+    class Group < Expressions::UnaryExpression
+      attr_reader :expression
+
+      def initialize(expression)
+        super(:group, expression)
+
+        @expression = expression
+      end
+    end
+  end
+end

data/lib/filter_param/ast/literal.rb

@@ -0,0 +1,26 @@
+module FilterParam
+  module AST
+    class Literal < Node
+      attr_accessor :value
+
+      def initialize(value = nil)
+        @value = value
+      end
+
+      def type_cast(type)
+        return self if type.blank?
+
+        cast_method = "to_#{type}"
+        return send(cast_method) if respond_to?(cast_method, true)
+
+        raise InvalidLiteral.new("Cannot cast '#{value}' to #{type}")
+      end
+
+      private
+
+      def visit_method
+        :visit_literal
+      end
+    end
+  end
+end

data/lib/filter_param/ast/literals/boolean.rb

@@ -0,0 +1,38 @@
+module FilterParam
+  module AST
+    module Literals
+      class Boolean < Literal
+        def initialize(value)
+          @value = (value.to_s == "true")
+        end
+
+        def data_type
+          :boolean
+        end
+
+        private_class_method :new
+
+        TRUE = new("true")
+        FALSE = new("false")
+
+        private
+
+        def to_boolean
+          self
+        end
+
+        def to_string
+          Literals::String.new(value)
+        end
+
+        def to_integer
+          Literals::Integer.new(value ? 1 : 0)
+        end
+
+        def to_decimal
+          Literals::Decimal.new(value ? 1.0 : 0.0)
+        end
+      end
+    end
+  end
+end

data/lib/filter_param/ast/literals/date.rb

@@ -0,0 +1,33 @@
+require "date"
+
+module FilterParam
+  module AST
+    module Literals
+      class Date < Literal
+        def initialize(value)
+          @value = ::Date.iso8601(value.to_s)
+        rescue ::Date::Error
+          raise FilterParam::InvalidLiteral.new("Invalid ISO8601 Date: #{value}")
+        end
+
+        def data_type
+          :date
+        end
+
+        private
+
+        def to_string
+          Literals::String.new(value)
+        end
+
+        def to_date
+          self
+        end
+
+        def to_datetime
+          Literals::DateTime.new(value)
+        end
+      end
+    end
+  end
+end

data/lib/filter_param/ast/literals/date_time.rb

@@ -0,0 +1,36 @@
+require "date"
+
+module FilterParam
+  module AST
+    module Literals
+      class DateTime < Date
+        def initialize(value)
+          @raw_value = value
+          @value = ::DateTime.iso8601(value.to_s)
+        rescue ::Date::Error
+          raise FilterParam::InvalidLiteral.new("Invalid ISO8601 Datetime: #{value}")
+        end
+
+        def data_type
+          :datetime
+        end
+
+        private
+
+        attr_reader :raw_value
+
+        def to_string
+          Literals::String.new(raw_value)
+        end
+
+        def to_date
+          Literals::Date.new(value)
+        end
+
+        def to_datetime
+          self
+        end
+      end
+    end
+  end
+end

data/lib/filter_param/ast/literals/decimal.rb

@@ -0,0 +1,29 @@
+require "bigdecimal"
+
+module FilterParam
+  module AST
+    module Literals
+      class Decimal < Integer
+        def initialize(value)
+          @value = BigDecimal(value.to_s)
+        rescue ArgumentError
+          raise InvalidLiteral.new("Invalid Decimal: #{value}")
+        end
+
+        def data_type
+          :decimal
+        end
+
+        private
+
+        def to_integer
+          Literals::Integer.new(value.to_i)
+        end
+
+        def to_decimal
+          self
+        end
+      end
+    end
+  end
+end

data/lib/filter_param/ast/literals/integer.rb

@@ -0,0 +1,39 @@
+module FilterParam
+  module AST
+    module Literals
+      class Integer < Literal
+        def initialize(value)
+          whole_num = value.to_s.split(".").first
+
+          @value = Integer(whole_num)
+        rescue ArgumentError
+          raise InvalidLiteral.new("Invalid Integer: #{value}")
+        end
+
+        def data_type
+          :integer
+        end
+
+        private
+
+        def to_boolean
+          return Literals::Boolean::FALSE if value.zero?
+
+          Literals::Boolean::TRUE
+        end
+
+        def to_string
+          Literals::String.new(value)
+        end
+
+        def to_integer
+          self
+        end
+
+        def to_decimal
+          Literals::Decimal.new(value)
+        end
+      end
+    end
+  end
+end

data/lib/filter_param/ast/literals/null.rb

@@ -0,0 +1,19 @@
+require "singleton"
+
+module FilterParam
+  module AST
+    module Literals
+      class Null < Literal
+        include Singleton
+
+        def data_type
+          :null
+        end
+
+        def type_cast(type)
+          self
+        end
+      end
+    end
+  end
+end

data/lib/filter_param/ast/literals/string.rb

@@ -0,0 +1,43 @@
+module FilterParam
+  module AST
+    module Literals
+      class String < Literal
+        def initialize(value)
+          @value = value.to_s
+        end
+
+        def data_type
+          :string
+        end
+
+        private
+
+        def to_boolean
+          return Literals::Boolean::TRUE if value.downcase == "true"
+
+          Literals::Boolean::FALSE
+        end
+
+        def to_string
+          self
+        end
+
+        def to_integer
+          Literals::Integer.new(value)
+        end
+
+        def to_decimal
+          Literals::Decimal.new(value)
+        end
+
+        def to_date
+          Literals::Date.new(value)
+        end
+
+        def to_datetime
+          Literals::DateTime.new(value)
+        end
+      end
+    end
+  end
+end

data/lib/filter_param/ast/node.rb

@@ -0,0 +1,15 @@
+module FilterParam
+  module AST
+    class Node
+      def accept(visitor)
+        visitor.send(visit_method, self)
+      end
+
+      private
+
+      def visit_method
+        "visit_#{self.class.name.demodulize.underscore}".to_sym
+      end
+    end
+  end
+end

data/lib/filter_param/ast/scope.rb

@@ -0,0 +1,12 @@
+module FilterParam
+  module AST
+    class Scope < Node
+      attr_reader :name, :args
+
+      def initialize(name, args)
+        @name = name.to_s
+        @args = args
+      end
+    end
+  end
+end

data/lib/filter_param/definition.rb

@@ -0,0 +1,157 @@
+module FilterParam
+  # FilterParam definition that whitelists the columns that are allowed to
+  # filtered (i.e. used in SQL WHERE condition) and the allowed scopes.
+  class Definition
+    # Allows whitelisting columns using a block
+    #
+    # @param [Proc] block Field definition block
+    #
+    # @return [self] Definition instance
+    def define(&block)
+      raise ArgumentError.new("Missing block") unless block_given?
+
+      instance_eval(&block)
+
+      self
+    end
+
+    # Whitelist a column
+    #
+    # @param [String, Symbol] name column name
+    # @param [Hash] options column options:
+    #   * type [Symbol] expected field type:
+    #     :string (default), :int, :decimal :boolean, :date, :datetime
+    #   * rename [String, Proc] rename field in the formatted output.
+    #     This can be a Proc code block that receives the :name as argument and
+    #     returns a transformed field name.
+    #   * value [Proc] pre-process literal operand values. This receives the :value
+    #     argument parsed from the expression string and returns a transformed field value.
+    #
+    # @return [self] Definition instance
+    def field(name, **options)
+      name = name.to_s
+      return if name.blank?
+
+      fields_hash[name] = Field.new(name, options[:type], options)
+
+      self
+    end
+
+    # Whitelist multiple columns with the same column options.
+    #
+    # @param [Array<String, Symbol>] names list of column names
+    # @param [Hash] options column configuration options
+    #
+    # @see #field
+    #
+    # @return [self] Definition instance
+    #
+    def fields(*names, **options)
+      restrict_string_rename!(options[:rename])
+
+      names.each { |name| field(name, **options) }
+
+      self
+    end
+
+    # Whitelist a scope name
+    #
+    # @param [String, Symbol] name scope name. ar_relation passed to `#filter!` must expose this scope.
+    # @param [Hash] options column options:
+    #   * rename [String, Proc] rename to actual scope name.
+    #     This can be a Proc code block that receives the :name as argument and
+    #     returns a transformed scope name.
+    def scope(name, **options)
+      name = name.to_s
+      return if name.blank?
+
+      scopes_hash[name] = Scope.new(name, options)
+
+      self
+    end
+
+    # Whitelist multiple scope names with the same scope options.
+    #
+    # @param [Array<String, Symbol>] names list of scope names
+    # @param [Hash] options scope configuration options
+    #
+    # @see #scope
+    #
+    # @return [self] Definition instance
+    #
+    def scopes(*names, **options)
+      restrict_string_rename!(options[:rename])
+
+      names.each { |name| scope(name, **options) }
+
+      self
+    end
+
+    # Filters an :ar_relation by the filter :expression
+    #
+    # @param [ActiveRecord::Relation] ar_relation Relation to filter
+    # @param [String] expression Filter expression.
+    #
+    def filter!(ar_relation, expression)
+      transpiler = Transpiler.new(ar_relation, self)
+
+      ar_relation.where(
+        transpiler.transpile!(expression)
+      )
+    end
+
+    # Returns the declared Field instance
+    #
+    # @param [String, Symbol] field_name
+    #
+    # @return [Field]
+    def find_field!(field_name)
+      field = fields_hash[field_name.to_s].presence
+      return field if field
+
+      raise UnknownField.new("Unknown field: '#{field_name}'")
+    end
+
+    # Returns the declared Scope instance
+    #
+    # @param [String, Symbol] scope_name
+    #
+    # @return [Field]
+    def find_scope!(scope_name)
+      scope = scopes_hash[scope_name.to_s].presence
+      return scope if scope
+
+      raise UnknownScope.new("Unknown scope: '#{scope_name}'")
+    end
+
+    # Returns the declared Field names
+    #
+    # @return [Array<String>]
+    def field_names
+      fields_hash.keys
+    end
+
+    # Returns the declared Scope names
+    #
+    # @return [Array<String>]
+    def scope_names
+      scopes_hash.keys
+    end
+
+    private
+
+    def fields_hash
+      @fields_hash ||= {}
+    end
+
+    def scopes_hash
+      @scopes_hash ||= {}
+    end
+
+    def restrict_string_rename!(rename)
+      return if rename.blank? || rename.is_a?(Proc)
+
+      raise ArgumentError.new(":rename should be a Proc")
+    end
+  end
+end