dynamoid 3.9.0 → 3.10.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: aedb91a28972fd9357cbb260341e617c3d0d35ae05a620d5ac8312f5d3edbf96
-  data.tar.gz: a026d8c3d1e53f4e6800dcffe2b8350da50f70d953a19ae86dfeff0f65555c8c
+  metadata.gz: c2cc1feec6853b756fff467fa687cf2eaa3a2dc5f0a88dac948e5a92a4aa1073
+  data.tar.gz: 7165e081fa9979c45e7402265138e62cb6ee3c1f6f594ab6949c1f984621bc89
 SHA512:
-  metadata.gz: dcb483ea21eaa16246d82603d18952a70bab15556b65ea94fad3ac9dbfe6358f00730e1825275d241a18675fdc0a2e4c3c4b0553acb41f9205e5fe350337f54c
-  data.tar.gz: 4e4c96ed9b90ccab85f2f09ad848455affa681c93740dcf1f7b6283797f8e47bebd1c83a70a524eb6ea44906bdd6baec07549f67a9e6bd6561b857ea8997b692
+  metadata.gz: e18700ff74b9466e1ec166706446a1dca5248c6b8c33365e469c74e9a67b92f4afc9cacb0a06c2698d5165f4b68a93cdd214b83f5e3b749b92bcaff06fbc7628
+  data.tar.gz: 9b18fb2522f90eb3cf9b0b6014ca24beb08bcdebf403c57487222c46ec71d306f4e85e686aa89607c77ad7d994359a934537e8d22202b145bd478c0862bf497b

data/CHANGELOG.md

@@ -7,14 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Fixed
-
 ### Added
-
 ### Changed
-
 ### Removed
 
-## 3.9.0
+## 3.10.0
+### Fixed
+* [#681](https://github.com/Dynamoid/dynamoid/pull/681) Fixed saving persisted model and deleting attributes with `nil` value if `config.store_attribute_with_nil_value` is `false`
+* [#716](https://github.com/Dynamoid/dynamoid/pull/716), [#691](https://github.com/Dynamoid/dynamoid/pull/691), [#687](https://github.com/Dynamoid/dynamoid/pull/687), [#660](https://github.com/Dynamoid/dynamoid/pull/660) Numerous fixes in README.md and RDoc documentation (@ndjndj, @kiharito, @dunkOnIT)
+### Added
+* [#656](https://github.com/Dynamoid/dynamoid/pull/656) Added a `create_table_on_save` configuration flag to create table on save (@imaximix)
+* [#697](https://github.com/Dynamoid/dynamoid/pull/697) Ensure Ruby 3.3 and Rails 7.1 versions are supported and added them on CI
+### Changed
+* [#655](https://github.com/Dynamoid/dynamoid/pull/655) Support multiple `where` in the same chain with multiple conditions for the same field
+
+## 3.9.0 / 2023-04-13
 ### Fixed
 * [#610](https://github.com/Dynamoid/dynamoid/pull/610) Specs in JRuby; Support for JRuby 9.4.0.0 (@pboling)
 * [#624](https://github.com/Dynamoid/dynamoid/pull/624) Fixed `#increment!`/`#decrement!` methods and made them compatible with Rails counterparts
@@ -50,15 +57,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   * `#touch`
   * `#increment!`
   * `#decrement!`
-* [#642](https://github.com/Dynamoid/dynamoid/pull/642) Run specs on CI agains Ruby 3.2
+* [#642](https://github.com/Dynamoid/dynamoid/pull/642) Run specs on CI against Ruby 3.2
 * [#645](https://github.com/Dynamoid/dynamoid/pull/645) Added `after_find` callback
 ### Changed
 * [#610](https://github.com/Dynamoid/dynamoid/pull/610) Switch to [`rubocop-lts`](https://rubocop-lts.gitlab.io/) (@pboling)
-### Removed
 * [#633](https://github.com/Dynamoid/dynamoid/pull/633) Change `#inspect` method to return only attributes
 * [#623](https://github.com/Dynamoid/dynamoid/pull/623) Optimized performance of persisting to send only changed attributes in a request to DynamoDB
 
-## 3.8.0
+## 3.8.0 / 2022-11-09
 ### Fixed
 * [#525](https://github.com/Dynamoid/dynamoid/pull/525) Don't mark an attribute as changed if new assigned value equals the old one (@a5-stable)
 * Minor changes in the documentation:

data/README.md

@@ -67,10 +67,10 @@ For example, to configure AWS access:
 Create `config/initializers/aws.rb` as follows:
 
 ```ruby
-Aws.config.update({
-                    region: 'us-west-2',
+Aws.config.update(
+  region: 'us-west-2',
   credentials: Aws::Credentials.new('REPLACE_WITH_ACCESS_KEY_ID', 'REPLACE_WITH_SECRET_ACCESS_KEY'),
-                  })
+)
 ```
 
 Alternatively, if you don't want Aws connection settings to be
@@ -132,8 +132,8 @@ end
 Dynamoid supports Ruby >= 2.3 and Rails >= 4.2.
 
 Its compatibility is tested against following Ruby versions: 2.3, 2.4,
-2.5, 2.6, 2.7, 3.0, 3.1 and 3.2, JRuby 9.4.x and against Rails versions: 4.2, 5.0, 5.1,
-5.2, 6.0, 6.1 and 7.0.
+2.5, 2.6, 2.7, 3.0, 3.1, 3.2 and 3.3, JRuby 9.4.x and against Rails versions: 4.2, 5.0, 5.1,
+5.2, 6.0, 6.1, 7.0 and 7.1.
 
 ## Setup
 
@@ -723,18 +723,6 @@ join, but instead finds all the user's addresses and naively filters
 them in Ruby. For large associations this is a performance hit compared
 to relational database engines.
 
-**WARNING:** There is a limitation of conditions passed to `where`
-method. Only one condition for some particular field could be specified.
-The last one only will be applied and others will be ignored. E.g. in
-examples:
-
-```ruby
-User.where('age.gt': 10, 'age.lt': 20)
-User.where(name: 'Mike').where('name.begins_with': 'Ed')
-```
-
-the first one will be ignored and the last one will be used.
-
 **Warning:** There is a caveat with filtering documents by `nil` value
 attribute. By default Dynamoid ignores attributes with `nil` value and
 doesn't store them in a DynamoDB document. This behavior could be
@@ -752,7 +740,7 @@ If Dynamoid keeps `nil` value attributes `eq`/`ne` operators should be
 used instead:
 
 ```ruby
-Address.where('postcode': nil)
+Address.where(postcode: nil)
 Address.where('postcode.ne': nil)
 ```
 
@@ -926,8 +914,8 @@ If you have a range index, Dynamoid provides a number of additional
 other convenience methods to make your life a little easier:
 
 ```ruby
-User.where("created_at.gt": DateTime.now - 1.day).all
-User.where("created_at.lt": DateTime.now - 1.day).all
+User.where('created_at.gt': DateTime.now - 1.day).all
+User.where('created_at.lt': DateTime.now - 1.day).all
 ```
 
 It also supports `gte` and `lte`. Turning those into symbols and
@@ -975,7 +963,7 @@ To idempotently create-but-not-update a record, apply the `unless_exists` condit
 to its keys when you upsert.
 
 ```ruby
-Address.upsert(id, { city: 'Chicago' }, if: { unless_exists: [:id] })
+Address.upsert(id, { city: 'Chicago' }, { unless_exists: [:id] })
 ```
 
 ### Deleting
@@ -1144,12 +1132,12 @@ Listed below are all configuration options.
   in ISO 8601 string format. Default is `false`
 * `store_boolean_as_native` - if `true` Dynamoid stores boolean fields
   as native DynamoDB boolean values. Otherwise boolean fields are stored
-  as string values `'t'` and `'f'`. Default is true
+  as string values `'t'` and `'f'`. Default is `true`
 * `backoff` - is a hash: key is a backoff strategy (symbol), value is
   parameters for the strategy. Is used in batch operations. Default id
   `nil`
 * `backoff_strategies`: is a hash and contains all available strategies.
-  Default is { constant: ..., exponential: ...}
+  Default is `{ constant: ..., exponential: ...}`
 * `log_formatter`: overrides default AWS SDK formatter. There are
   several canned formatters: `Aws::Log::Formatter.default`,
   `Aws::Log::Formatter.colored` and `Aws::Log::Formatter.short`. Please
@@ -1167,6 +1155,9 @@ Listed below are all configuration options.
 * `http_read_timeout`:The number of seconds to wait for HTTP response
   data. Default option value is `nil`. If not specified effected value
   is `60`
+* `create_table_on_save`: if `true` then Dynamoid creates a
+  corresponding table in DynamoDB at model persisting if the table
+  doesn't exist yet. Default is `true`
 
 
 ## Concurrency
@@ -1305,6 +1296,12 @@ RSpec.configure do |config|
 end
 ```
 
+In addition, the first test for each model may fail if the relevant models are not included in `included_models`. This can be fixed by adding this line before the `DynamoidReset` module:
+```ruby
+Dir[File.join(Dynamoid::Config.models_dir, '**/*.rb')].sort.each { |file| require file }
+```
+Note that this will require _all_ models in your models folder - you can also explicitly require only certain models if you would prefer to.
+
 In Rails, you may also want to ensure you do not delete non-test data
 accidentally by adding the following to your test environment setup:
 

data/dynamoid.gemspec

@@ -59,8 +59,7 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency 'bundler'
   spec.add_development_dependency 'pry', '~> 0.14'
   spec.add_development_dependency 'rake', '~> 13.0'
+  spec.add_development_dependency 'rexml'
   spec.add_development_dependency 'rspec', '~> 3.12'
-  # 'rubocop-lts' is for Ruby 2.3+, see https://rubocop-lts.gitlab.io/
-  spec.add_development_dependency 'rubocop-lts', '~> 10.0'
   spec.add_development_dependency 'yard'
 end

data/lib/dynamoid/adapter.rb

@@ -171,19 +171,25 @@ module Dynamoid
     # only really useful for range queries, since it can only find by one hash key at once. Only provide
     # one range key to the hash.
     #
+    #   Dynamoid.adapter.query('users', { id: [[:eq, '1']], age: [[:between, [10, 30]]] }, { batch_size: 1000 })
+    #
     # @param [String] table_name the name of the table
-    # @param [Hash] opts the options to query the table with
-    # @option opts [String] :hash_value the value of the hash key to find
-    # @option opts [Range] :range_value find the range key within this range
-    # @option opts [Number] :range_greater_than find range keys greater than this
-    # @option opts [Number] :range_less_than find range keys less than this
-    # @option opts [Number] :range_gte find range keys greater than or equal to this
-    # @option opts [Number] :range_lte find range keys less than or equal to this
-    #
-    # @return [Array] an array of all matching items
-    #
-    def query(table_name, opts = {})
-      adapter.query(table_name, opts)
+    # @param [Array[Array]] key_conditions conditions for the primary key attributes
+    # @param [Array[Array]] non_key_conditions (optional) conditions for non-primary key attributes
+    # @param [Hash] options (optional) the options to query the table with
+    # @option options [Boolean] :consistent_read You can set the ConsistentRead parameter to true and obtain a strongly consistent result
+    # @option options [Boolean] :scan_index_forward Specifies the order for index traversal: If true (default), the traversal is performed in ascending order; if false, the traversal is performed in descending order.
+    # @option options [Symbop] :select The attributes to be returned in the result (one of ALL_ATTRIBUTES, ALL_PROJECTED_ATTRIBUTES, ...)
+    # @option options [Symbol] :index_name The name of an index to query. This index can be any local secondary index or global secondary index on the table.
+    # @option options [Hash] :exclusive_start_key The primary key of the first item that this operation will evaluate.
+    # @option options [Integer] :batch_size The number of items to lazily load one by one
+    # @option options [Integer] :record_limit The maximum number of items to return (not necessarily the number of evaluated items)
+    # @option options [Integer] :scan_limit The maximum number of items to evaluate (not necessarily the number of matching items)
+    # @option options [Array[Symbol]] :project The attributes to retrieve from the table
+    #
+    # @return [Enumerable] matching items
+    def query(table_name, key_conditions, non_key_conditions = {}, options = {})
+      adapter.query(table_name, key_conditions, non_key_conditions, options)
     end
 
     def self.adapter_plugin_class

data/lib/dynamoid/adapter_plugin/aws_sdk_v3/create_table.rb

@@ -29,7 +29,7 @@ module Dynamoid
           gs_indexes = options[:global_secondary_indexes]
 
           key_schema = {
-            hash_key_schema: { key => (options[:hash_key_type] || :string) },
+            hash_key_schema: { key => options[:hash_key_type] || :string },
             range_key_schema: options[:range_key]
           }
           attribute_definitions = build_all_attribute_definitions(
@@ -69,7 +69,7 @@ module Dynamoid
             end
           end
           resp = client.create_table(client_opts)
-          options[:sync] = true if !options.key?(:sync) && ls_indexes.present? || gs_indexes.present?
+          options[:sync] = true if (!options.key?(:sync) && ls_indexes.present?) || gs_indexes.present?
 
           if options[:sync]
             status = PARSE_TABLE_STATUS.call(resp, :table_description)

data/lib/dynamoid/adapter_plugin/aws_sdk_v3/filter_expression_convertor.rb

@@ -0,0 +1,78 @@
+# frozen_string_literal: true
+
+module Dynamoid
+  # @private
+  module AdapterPlugin
+    class AwsSdkV3
+      class FilterExpressionConvertor
+        attr_reader :expression, :name_placeholders, :value_placeholders
+
+        def initialize(conditions, name_placeholders, value_placeholders, name_placeholder_sequence, value_placeholder_sequence)
+          @conditions = conditions
+          @name_placeholders = name_placeholders.dup
+          @value_placeholders = value_placeholders.dup
+          @name_placeholder_sequence = name_placeholder_sequence
+          @value_placeholder_sequence = value_placeholder_sequence
+
+          build
+        end
+
+        private
+
+        def build
+          clauses = @conditions.map do |name, attribute_conditions|
+            attribute_conditions.map do |operator, value|
+              name_or_placeholder = name_or_placeholder_for(name)
+
+              case operator
+              when :eq
+                "#{name_or_placeholder} = #{value_placeholder_for(value)}"
+              when :ne
+                "#{name_or_placeholder} <> #{value_placeholder_for(value)}"
+              when :gt
+                "#{name_or_placeholder} > #{value_placeholder_for(value)}"
+              when :lt
+                "#{name_or_placeholder} < #{value_placeholder_for(value)}"
+              when :gte
+                "#{name_or_placeholder} >= #{value_placeholder_for(value)}"
+              when :lte
+                "#{name_or_placeholder} <= #{value_placeholder_for(value)}"
+              when :between
+                "#{name_or_placeholder} BETWEEN #{value_placeholder_for(value[0])} AND #{value_placeholder_for(value[1])}"
+              when :begins_with
+                "begins_with (#{name_or_placeholder}, #{value_placeholder_for(value)})"
+              when :in
+                list = value.map(&method(:value_placeholder_for)).join(' , ')
+                "#{name_or_placeholder} IN (#{list})"
+              when :contains
+                "contains (#{name_or_placeholder}, #{value_placeholder_for(value)})"
+              when :not_contains
+                "NOT contains (#{name_or_placeholder}, #{value_placeholder_for(value)})"
+              when :null
+                "attribute_not_exists (#{name_or_placeholder})"
+              when :not_null
+                "attribute_exists (#{name_or_placeholder})"
+              end
+            end
+          end.flatten
+
+          @expression = clauses.join(' AND ')
+        end
+
+        def name_or_placeholder_for(name)
+          return name unless name.upcase.in?(Dynamoid::AdapterPlugin::AwsSdkV3::RESERVED_WORDS)
+
+          placeholder = @name_placeholder_sequence.call
+          @name_placeholders[placeholder] = name
+          placeholder
+        end
+
+        def value_placeholder_for(value)
+          placeholder = @value_placeholder_sequence.call
+          @value_placeholders[placeholder] = value
+          placeholder
+        end
+      end
+    end
+  end
+end

data/lib/dynamoid/adapter_plugin/aws_sdk_v3/item_updater.rb

@@ -50,7 +50,19 @@ module Dynamoid
         # Replaces the values of one or more attributes
         #
         def set(values)
-          @updates.merge!(sanitize_attributes(values))
+          values_sanitized = sanitize_attributes(values)
+
+          if Dynamoid.config.store_attribute_with_nil_value
+            @updates.merge!(values_sanitized)
+          else
+            # delete explicitly attributes if assigned nil value and configured
+            # to not store nil values
+            values_to_update = values_sanitized.reject { |_, v| v.nil? }
+            values_to_delete = values_sanitized.select { |_, v| v.nil? }
+
+            @updates.merge!(values_to_update)
+            @deletions.merge!(values_to_delete)
+          end
         end
 
         #
@@ -85,7 +97,12 @@ module Dynamoid
 
         private
 
+        # Keep in sync with AwsSdkV3.sanitize_item.
+        #
+        # The only difference is that to update item we need to track whether
+        # attribute value is nil or not.
         def sanitize_attributes(attributes)
+          # rubocop:disable Lint/DuplicateBranch
           attributes.transform_values do |v|
             if v.is_a?(Hash)
               v.stringify_keys
@@ -97,6 +114,7 @@ module Dynamoid
               v
             end
           end
+          # rubocop:enable Lint/DuplicateBranch
         end
       end
     end

data/lib/dynamoid/adapter_plugin/aws_sdk_v3/projection_expression_convertor.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+module Dynamoid
+  # @private
+  module AdapterPlugin
+    class AwsSdkV3
+      class ProjectionExpressionConvertor
+        attr_reader :expression, :name_placeholders
+
+        def initialize(names, name_placeholders, name_placeholder_sequence)
+          @names = names
+          @name_placeholders = name_placeholders.dup
+          @name_placeholder_sequence = name_placeholder_sequence
+
+          build
+        end
+
+        private
+
+        def build
+          return if @names.nil? || @names.empty?
+
+          clauses = @names.map do |name|
+            if name.upcase.in?(Dynamoid::AdapterPlugin::AwsSdkV3::RESERVED_WORDS)
+              placeholder = @name_placeholder_sequence.call
+              @name_placeholders[placeholder] = name
+              placeholder
+            else
+              name.to_s
+            end
+          end
+
+          @expression = clauses.join(' , ')
+        end
+      end
+    end
+  end
+end

data/lib/dynamoid/adapter_plugin/aws_sdk_v3/query.rb

@@ -3,6 +3,8 @@
 require_relative 'middleware/backoff'
 require_relative 'middleware/limit'
 require_relative 'middleware/start_key'
+require_relative 'filter_expression_convertor'
+require_relative 'projection_expression_convertor'
 
 module Dynamoid
   # @private
@@ -10,20 +12,19 @@ module Dynamoid
     class AwsSdkV3
       class Query
         OPTIONS_KEYS = %i[
-          limit hash_key hash_value range_key consistent_read scan_index_forward
-          select index_name batch_size exclusive_start_key record_limit scan_limit
-          project
+          consistent_read scan_index_forward select index_name batch_size
+          exclusive_start_key record_limit scan_limit project
         ].freeze
 
         attr_reader :client, :table, :options, :conditions
 
-        def initialize(client, table, opts = {})
+        def initialize(client, table, key_conditions, non_key_conditions, options)
           @client = client
           @table = table
 
-          opts = opts.symbolize_keys
-          @options = opts.slice(*OPTIONS_KEYS)
-          @conditions = opts.except(*OPTIONS_KEYS)
+          @key_conditions = key_conditions
+          @non_key_conditions = non_key_conditions
+          @options = options.slice(*OPTIONS_KEYS)
         end
 
         def call
@@ -53,6 +54,37 @@ module Dynamoid
         private
 
         def build_request
+          # expressions
+          name_placeholder = +'#_a0'
+          value_placeholder = +':_a0'
+
+          name_placeholder_sequence = -> { name_placeholder.next!.dup }
+          value_placeholder_sequence = -> { value_placeholder.next!.dup }
+
+          name_placeholders = {}
+          value_placeholders = {}
+
+          # Deal with various limits and batching
+          batch_size = options[:batch_size]
+          limit = [record_limit, scan_limit, batch_size].compact.min
+
+          # key condition expression
+          convertor = FilterExpressionConvertor.new(@key_conditions, name_placeholders, value_placeholders, name_placeholder_sequence, value_placeholder_sequence)
+          key_condition_expression = convertor.expression
+          value_placeholders = convertor.value_placeholders
+          name_placeholders = convertor.name_placeholders
+
+          # filter expression
+          convertor = FilterExpressionConvertor.new(@non_key_conditions, name_placeholders, value_placeholders, name_placeholder_sequence, value_placeholder_sequence)
+          filter_expression = convertor.expression
+          value_placeholders = convertor.value_placeholders
+          name_placeholders = convertor.name_placeholders
+
+          # projection expression
+          convertor = ProjectionExpressionConvertor.new(options[:project], name_placeholders, name_placeholder_sequence)
+          projection_expression = convertor.expression
+          name_placeholders = convertor.name_placeholders
+
           request = options.slice(
             :consistent_read,
             :scan_index_forward,
@@ -61,15 +93,13 @@ module Dynamoid
             :exclusive_start_key
           ).compact
 
-          # Deal with various limits and batching
-          batch_size = options[:batch_size]
-          limit = [record_limit, scan_limit, batch_size].compact.min
-
-          request[:limit]             = limit if limit
-          request[:table_name]        = table.name
-          request[:key_conditions]    = key_conditions
-          request[:query_filter]      = query_filter
-          request[:attributes_to_get] = attributes_to_get
+          request[:table_name]                  = table.name
+          request[:limit]                       = limit                     if limit
+          request[:key_condition_expression]    = key_condition_expression  if key_condition_expression.present?
+          request[:filter_expression]           = filter_expression         if filter_expression.present?
+          request[:expression_attribute_values] = value_placeholders        if value_placeholders.present?
+          request[:expression_attribute_names]  = name_placeholders         if name_placeholders.present?
+          request[:projection_expression]       = projection_expression     if projection_expression.present?
 
           request
         end
@@ -81,51 +111,6 @@ module Dynamoid
         def scan_limit
           options[:scan_limit]
         end
-
-        def hash_key_name
-          (options[:hash_key] || table.hash_key)
-        end
-
-        def range_key_name
-          (options[:range_key] || table.range_key)
-        end
-
-        def key_conditions
-          result = {
-            hash_key_name => {
-              comparison_operator: AwsSdkV3::EQ,
-              attribute_value_list: AwsSdkV3.attribute_value_list(AwsSdkV3::EQ, options[:hash_value].freeze)
-            }
-          }
-
-          conditions.slice(*AwsSdkV3::RANGE_MAP.keys).each do |k, _v|
-            op = AwsSdkV3::RANGE_MAP[k]
-
-            result[range_key_name] = {
-              comparison_operator: op,
-              attribute_value_list: AwsSdkV3.attribute_value_list(op, conditions[k].freeze)
-            }
-          end
-
-          result
-        end
-
-        def query_filter
-          conditions.except(*AwsSdkV3::RANGE_MAP.keys).reduce({}) do |result, (attr, cond)|
-            condition = {
-              comparison_operator: AwsSdkV3::FIELD_MAP[cond.keys[0]],
-              attribute_value_list: AwsSdkV3.attribute_value_list(AwsSdkV3::FIELD_MAP[cond.keys[0]], cond.values[0].freeze)
-            }
-            result[attr] = condition
-            result
-          end
-        end
-
-        def attributes_to_get
-          return if options[:project].nil?
-
-          options[:project].map(&:to_s)
-        end
       end
     end
   end

data/lib/dynamoid/adapter_plugin/aws_sdk_v3/scan.rb

@@ -3,6 +3,8 @@
 require_relative 'middleware/backoff'
 require_relative 'middleware/limit'
 require_relative 'middleware/start_key'
+require_relative 'filter_expression_convertor'
+require_relative 'projection_expression_convertor'
 
 module Dynamoid
   # @private
@@ -45,6 +47,31 @@ module Dynamoid
         private
 
         def build_request
+          # expressions
+          name_placeholder = +'#_a0'
+          value_placeholder = +':_a0'
+
+          name_placeholder_sequence = -> { name_placeholder.next!.dup }
+          value_placeholder_sequence = -> { value_placeholder.next!.dup }
+
+          name_placeholders = {}
+          value_placeholders = {}
+
+          # Deal with various limits and batching
+          batch_size = options[:batch_size]
+          limit = [record_limit, scan_limit, batch_size].compact.min
+
+          # filter expression
+          convertor = FilterExpressionConvertor.new(conditions, name_placeholders, value_placeholders, name_placeholder_sequence, value_placeholder_sequence)
+          filter_expression = convertor.expression
+          value_placeholders = convertor.value_placeholders
+          name_placeholders = convertor.name_placeholders
+
+          # projection expression
+          convertor = ProjectionExpressionConvertor.new(options[:project], name_placeholders, name_placeholder_sequence)
+          projection_expression = convertor.expression
+          name_placeholders = convertor.name_placeholders
+
           request = options.slice(
             :consistent_read,
             :exclusive_start_key,
@@ -52,14 +79,12 @@ module Dynamoid
             :index_name
           ).compact
 
-          # Deal with various limits and batching
-          batch_size = options[:batch_size]
-          limit = [record_limit, scan_limit, batch_size].compact.min
-
-          request[:limit]             = limit if limit
-          request[:table_name]        = table.name
-          request[:scan_filter]       = scan_filter
-          request[:attributes_to_get] = attributes_to_get
+          request[:table_name]                  = table.name
+          request[:limit]                       = limit                 if limit
+          request[:filter_expression]           = filter_expression     if filter_expression.present?
+          request[:expression_attribute_values] = value_placeholders    if value_placeholders.present?
+          request[:expression_attribute_names]  = name_placeholders     if name_placeholders.present?
+          request[:projection_expression]       = projection_expression if projection_expression.present?
 
           request
         end
@@ -71,25 +96,6 @@ module Dynamoid
         def scan_limit
           options[:scan_limit]
         end
-
-        def scan_filter
-          conditions.reduce({}) do |result, (attr, cond)|
-            condition = {
-              comparison_operator: AwsSdkV3::FIELD_MAP[cond.keys[0]],
-              attribute_value_list: AwsSdkV3.attribute_value_list(AwsSdkV3::FIELD_MAP[cond.keys[0]], cond.values[0].freeze)
-            }
-            # nil means operator doesn't require attribute value list
-            conditions.delete(:attribute_value_list) if conditions[:attribute_value_list].nil?
-            result[attr] = condition
-            result
-          end
-        end
-
-        def attributes_to_get
-          return if options[:project].nil?
-
-          options[:project].map(&:to_s)
-        end
       end
     end
   end