fmrest 0.7.0 → 0.10.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ab46f70bffcfea32ef2bfd5021f03fa654bb43134e22b8e22a20d9955afe9caf
-  data.tar.gz: 32ce4378ff291d2c41d9b255b032d4562d7544a477765748c87c49a655449405
+  metadata.gz: ccf62593af664237c1cd0c67928c869ae74cdcde4b2e3c9bd82c20fa0f6b0354
+  data.tar.gz: 9f6bd645aed2551d278b0b31e194dc97726e66396a76148ee7dae3bca5c67a93
 SHA512:
-  metadata.gz: a68af2d8e53631b1d5c4d9e5e0849d224ff3918d1138d437463e30318ae81c36ca968d226f2f3631dbaa0d21b6287c1899e3b322a716729ad0d8b77aff80fd8d
-  data.tar.gz: ace50f913960e6b2edd190c8c1cbb75a0ebf15c0c3f0e8343720155f1c53b78bb9fd942be0a5e9bf2a2bc1ff729277f86360ffa4bba397ba48abe86c07db02ad
+  metadata.gz: 421a3be4119f862e31787b6027c7824e42641d85d6dbdcb6d5df2b849ef3e93556a03060a74e093a11fed962cdfbe61be19833c5d2195f6b4dad121ef759aeb6
+  data.tar.gz: 7bd1359945a7da96afdfd05c8948148bc6c8740351eb7c4dd1a31ba501fc82f5ad79dd68a166b2fee3022c636fddd187f0b27b97fc2d645922afddb813f98787

data/CHANGELOG.md

@@ -1,5 +1,36 @@
 ## Changelog
 
+### 0.10.1
+
+* Fix `URI.escape` obsolete warning messages in Ruby 2.7 by replacing it with
+  `URI.encode_www_form_component`
+  ([PR#40](https://github.com/beezwax/fmrest-ruby/pull/40))
+
+### 0.10.0
+
+* Added `FmRest::StringDateAwareness` module to correct some issues when using
+  `FmRest::StringDate`
+* Added basic timezones support
+* Deprecated `class < FmRest::Spyke::Base(config_hash)` syntax in favor of
+  using `self.fmrest_config=`
+
+### 0.9.0
+
+* Added `FmRest::Spyke::Base.set_globals`
+
+### 0.8.0
+
+* Improved metadata when using `FmRest::Spyke::Model`. Metadata now uses
+  Struct/OpenStruct, so properties are accessible through `.property`, as well
+  as `[:property]`
+* Added batch-finders `.find_in_batches` and `.find_each` for
+* `FmRest::Spyke::Base`
+
+### 0.7.1
+
+* Made sure `Model.find_one` and `Model.find_some` work without needing to call
+  `Model.all` in between
+
 ### 0.7.0
 
 * Added date coercion feature

data/README.md

@@ -3,7 +3,7 @@
 <a href="https://rubygems.org/gems/fmrest"><img src="https://badge.fury.io/rb/fmrest.svg?style=flat" alt="Gem Version"></a>
 
 A Ruby client for
-[FileMaker 18's Data API](https://fmhelp.filemaker.com/docs/18/en/dataapi/)
+[FileMaker 18 and 19's Data API](https://help.claris.com/en/data-api-guide)
 using
 [Faraday](https://github.com/lostisland/faraday) and with optional
 [Spyke](https://github.com/balvig/spyke) support (ActiveRecord-ish models).
@@ -120,6 +120,7 @@ Option              | Description                                | Format
 `:date_format`      | Date parsing format                        | String (FM date format)     | `"MM/dd/yyyy"`
 `:timestamp_format` | Timestmap parsing format                   | String (FM date format)     | `"MM/dd/yyyy HH:mm:ss"`
 `:time_format`      | Time parsing format                        | String (FM date format)     | `"HH:mm:ss"`
+`:timezone`         | The timezone for the FM server             | `:local` \| `:utc` \| `nil` | `nil`
 
 ### Default connection settings
 
@@ -195,7 +196,7 @@ FmRest.token_store = FmRest::TokenStore::Redis.new(redis: Redis.new, prefix: "my
 FmRest.token_store = FmRest::TokenStore::Redis.new(prefix: "my-fancy-prefix:", host: "10.0.1.1", port: 6380, db: 15)
 ```
 
-**NOTE:** redis-rb is not included as a gem dependency of fmrest-ruby, so you'll
+NOTE: redis-rb is not included as a gem dependency of fmrest-ruby, so you'll
 have to add it to your Gemfile.
 
 ### Moneta
@@ -232,7 +233,7 @@ FmRest.token_store = FmRest::TokenStore::Moneta.new(
 )
 ```
 
-**NOTE:** the moneta gem is not included as a dependency of fmrest-ruby, so
+NOTE: the moneta gem is not included as a dependency of fmrest-ruby, so
 you'll have to add it to your Gemfile.
 
 
@@ -246,14 +247,14 @@ to automatically "coerce" them into Ruby date objects.
 The connection option `:coerce_dates` controls this feature. Possible values
 are:
 
-* `:full`: whenever a string matches the given date/timestamp/time format,
+* `:full` - whenever a string matches the given date/timestamp/time format,
   convert them to `Date` or `DateTime` objects as appropriate
-* `:hybrid` or `true`: similar as above, but instead of converting to regular
+* `:hybrid` or `true` - similar as above, but instead of converting to regular
   `Date`/`DateTime` it converts strings to `FmRest::StringDate` and
   `FmRest::StringDateTime`, "hybrid" classes provided by fmrest-ruby that
   retain the functionality of `String` while also providing most the
   functionality of `Date`/`DateTime` (more on this below)
-* `false`: disable date coercion entirely (default), leave original string
+* `false` - disable date coercion entirely (default), leave original string
   values untouched
 
 Enabling date coercion works with both basic fmrest-ruby connections and Spyke
@@ -270,9 +271,9 @@ a DSL in model classes).
 ### Hybrid string/date objects
 
 `FmRest::StringDate` and `FmRest::StringDateTime` are special classes that
-inherit from `String`, but internally parse and store a `Date`/`DateTime`
-(respectively), and delegate any methods not provided by `String` to those
-objects. In other words, they quack like a duck *and* bark like a dog.
+inherit from `String`, but internally parse and store a `Date` or `DateTime`,
+and delegate any methods not provided by `String` to those objects. In other
+words, they quack like a duck *and* bark like a dog.
 
 You can use these when you want fmrest-ruby to provide you with date objects,
 but you don't want to worry about date coercion of false positives (i.e. a
@@ -280,7 +281,29 @@ string field that gets converted to `Date` because it just so matched the given
 date format).
 
 Be warned however that these classes come with a fair share of known gotchas
-(see GitHub wiki for more info).
+(see GitHub wiki for more info). Some of those gothas can be removed by calling
+
+```ruby
+FmRest::StringDateAwareness.enable
+```
+
+Which will extend the core `Date` and `DateTime` classes to be aware of
+`FmRest::StringDate`, especially when calling `Date.===`, `Date.parse` or
+`Date._parse`.
+
+If you're working with ActiveRecord models this will also make them accept
+`FmRest::StringDate` values for date fields.
+
+### Timezones
+
+fmrest-ruby has basic timezone support. You can set the `:timezone` option in
+your connection settings to one of the following values:
+
+* `:local` - dates will be converted to your system local time offset (as
+  defined by `ENV["TZ"]`), or the timezone set by `Time.zone` if you're using
+  ActiveSupport
+* `:utc` - dates will be converted to UTC offset
+* `nil` - (default) ignore timezones altogether
 
 
 ## Spyke support (ActiveRecord-like ORM)
@@ -322,17 +345,6 @@ class Honeybee < FmRest::Spyke::Base
 end
 ```
 
-In this case you can pass the [`fmrest_config`](#modelfmrest_config) hash as an
-argument to `Base()`:
-
-```ruby
-class Honeybee < FmRest::Spyke::Base(host: "...", database: "...", username: "...", password: "...")
-end
-
-Honeybee.fmrest_config
-# => { host: "...", database: "...", username: "...", password: "..." }
-```
-
 All of Spyke's basic ORM operations work:
 
 ```ruby
@@ -555,7 +567,7 @@ Honeybee.limit(10)
 ```
 
 NOTE: You can also set a default limit value for a model class, see
-[Other notes on querying](#other-notes-on-querying).
+[other notes on querying](#other-notes-on-querying).
 
 You can also use `.limit` to set limits on portals:
 
@@ -727,15 +739,15 @@ the scope object:
 Honeybee.limit(10).sort(:name).find_some # => [<Honeybee...>, ...]
 ```
 
-If you want just a single result you can use `.find_one` instead (this will
+If you want just a single result you can use `.first` instead (this will
 force `.limit(1)`):
 
 ```ruby
-Honeybee.query(name: "Hutch").find_one # => <Honeybee...>
+Honeybee.query(name: "Hutch").first # => <Honeybee...>
 ```
 
 If you know the id of the record you should use `.find(id)` instead of
-`.query(id: id).find_one` (so that the sent request is
+`.query(id: id).first` (so that the sent request is
 `GET ../:layout/records/:id` instead of `POST ../:layout/_find`).
 
 ```ruby
@@ -746,6 +758,52 @@ Note also that if you use `.find(id)` your `.query()` parameters (as well as
 limit, offset and sort parameters) will be discarded as they're not supported
 by the single record endpoint.
 
+
+### Finding records in batches
+
+Sometimes you want to iterate over a very large number of records to do some
+processing, but requesting them all at once would result in one huge request to
+the Data API, and loading too many records in memory all at once.
+
+To mitigate this problem you can use `.find_in_batches` and `.find_each`. If
+you've used ActiveRecord you're probably familiar with how they operate:
+
+```ruby
+# Find records in batches of 100 each
+Honeybee.query(hive: "Queensville").find_in_batches(batch_size: 100) do |batch|
+  dispatch_bees(batch)
+end
+
+# Iterate over all records using batches
+Honeybee.query(hive: "Queensville").find_each(batch_size: 100) do |bee|
+  bee.dispatch
+end
+```
+
+`.find_in_batches` yields collections of records (batches), while `.find_each`
+yields individual records, but using batches behind the scenes.
+
+Both methods accept a block-less form in which case they return an
+`Enumerator`:
+
+```ruby
+batch_enum = Honeybee.find_in_batches
+
+batch = batch_enum.next # => Spyke::Collection
+
+batch_enum.each do |batch|
+  process_batch(batch)
+end
+
+record_enum = Honeybee.find_each
+
+record_enum.next # => Honeybee
+```
+
+NOTE: By its nature, batch processing is subject to race conditions if other
+processes are modifying the database.
+
+
 ### Container fields
 
 You can define container fields on your model class with `container`:
@@ -783,6 +841,7 @@ bee.photo.upload(filename_or_io) # Upload a file to the container
 * `:content_type` - The MIME content type to use (defaults to
   `application/octet-stream`)
 
+
 ### Script execution
 
 The Data API allows running scripts as part of many types of requests.
@@ -870,7 +929,7 @@ separately, under their matching key.
 ```ruby
 bee.save(script: { presort: "My Presort Script", after: "My Script" })
 
-Honeybee.last_request_metadata[:script]
+Honeybee.last_request_metadata.script
 # => { after: { result: "oh hi", error: "0" }, presort: { result: "lo", error: "0" } }
 ```
 
@@ -884,7 +943,7 @@ is performed on that scope.
 
 ```ruby
 # Find one Honeybee record executing a presort and after script
-Honeybee.script(presort: ["My Presort Script", "parameter"], after: "My Script").find_one
+Honeybee.script(presort: ["My Presort Script", "parameter"], after: "My Script").first
 ```
 
 The model class' `.last_request_metadata` will be set in case you need to get the result.
@@ -896,6 +955,32 @@ to retrieving single records, in that case you'll have to use
 `.last_request_metadata`.
 
 
+### Setting global field values
+
+You can call `.set_globals` on any `FmRest::Spyke::Base` model to set glabal
+field values on the database that model is configured for.
+
+You can pass it either a hash of fully qualified field names
+(table_name::field_name), or 1-level-deep nested hashes, with the outer being a
+table name and the inner keys being the field names:
+
+```ruby
+Honeybee.set_globals(
+  "beeTable::myVar"      => "value",
+  "beeTable::myOtherVar" => "also a value"
+)
+
+# Equivalent to the above example
+Honeybee.set_globals(beeTable: { myVar: "value", myOtherVar: "also a value" })
+
+# Combined
+Honeybee.set_globals(
+  "beeTable::myVar" => "value",
+  beeTable: { myOtherVar: "also a value" }
+)
+```
+
+
 ## Logging
 
 If using fmrest-ruby + Spyke in a Rails app pretty log output will be set up
@@ -964,7 +1049,7 @@ FM Data API reference: https://fmhelp.filemaker.com/docs/18/en/dataapi/
 | Get container data                  | Manual*                       | Yes                              |
 | Upload container data               | Manual*                       | Yes                              |
 | Perform a find request              | Manual*                       | Yes                              |
-| Set global field values             | Manual*                       | No                               |
+| Set global field values             | Manual*                       | Yes
 | Run a script                        | Manual*                       | Yes                              |
 | Run a script with another request   | Manual*                       | Yes                              |
 

data/fmrest.gemspec

@@ -23,8 +23,8 @@ Gem::Specification.new do |spec|
   spec.add_dependency 'faraday', '>= 0.9.0', '< 2.0'
   spec.add_dependency 'faraday_middleware', '>= 0.9.1', '< 2.0'
 
-  spec.add_development_dependency "bundler", "~> 1.16"
-  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "bundler"
+  spec.add_development_dependency "rake"
   spec.add_development_dependency "rspec", "~> 3.0"
   spec.add_development_dependency "spyke"
   spec.add_development_dependency "webmock"

data/lib/fmrest/spyke/base.rb

@@ -8,6 +8,8 @@ module FmRest
 
     class << self
       def Base(config = nil)
+        warn "[DEPRECATION] Inheriting from `FmRest::Spyke::Base(config)` is deprecated and will be removed, inherit from `FmRest::Spyke::Base` (without arguments) and use `fmrest_config=` instead"
+
         if config
           return Class.new(::FmRest::Spyke::Base) do
                    self.fmrest_config = config

data/lib/fmrest/spyke/model.rb

@@ -7,6 +7,7 @@ require "fmrest/spyke/model/serialization"
 require "fmrest/spyke/model/associations"
 require "fmrest/spyke/model/orm"
 require "fmrest/spyke/model/container_fields"
+require "fmrest/spyke/model/global_fields"
 require "fmrest/spyke/model/http"
 require "fmrest/spyke/model/auth"
 
@@ -22,6 +23,7 @@ module FmRest
       include Associations
       include Orm
       include ContainerFields
+      include GlobalFields
       include Http
       include Auth
 

data/lib/fmrest/spyke/model/connection.rb

@@ -4,10 +4,19 @@ module FmRest
   module Spyke
     module Model
       module Connection
-        extend ::ActiveSupport::Concern
+        extend ActiveSupport::Concern
 
         included do
-          class_attribute :fmrest_config, instance_accessor: false, instance_predicate: false
+          class_attribute :fmrest_config, instance_writer: false, instance_predicate: false
+
+          # Overrides the fmrest_config reader created by class_attribute so we
+          # can default set the default at call time.
+          #
+          # This method gets overwriten in subclasses if self.fmrest_config= is
+          # called.
+          define_singleton_method(:fmrest_config) do
+            FmRest.default_connection_settings
+          end
 
           class_attribute :faraday_block, instance_accessor: false, instance_predicate: false
           class << self; private :faraday_block, :faraday_block=; end
@@ -40,7 +49,7 @@ module FmRest
           def fmrest_connection
             @fmrest_connection ||=
               begin
-                config = fmrest_config || FmRest.default_connection_settings
+                config = fmrest_config
 
                 FmRest::V1.build_connection(config) do |conn|
                   faraday_block.call(conn) if faraday_block

data/lib/fmrest/spyke/model/global_fields.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+module FmRest
+  module Spyke
+    module Model
+      module GlobalFields
+        extend ::ActiveSupport::Concern
+
+        FULLY_QUALIFIED_FIELD_NAME_MATCHER = /\A[^:]+::[^:]+\Z/.freeze
+
+        class_methods do
+          def set_globals(values_hash)
+            connection.patch(FmRest::V1.globals_path, {
+              globalFields: normalize_globals_hash(values_hash)
+            })
+          end
+
+          private
+
+          def normalize_globals_hash(hash)
+            hash.each_with_object({}) do |(k, v), normalized|
+              if v.kind_of?(Hash)
+                v.each do |k2, v2|
+                  normalized["#{k}::#{k2}"] = v2
+                end
+                next
+              end
+
+              unless FULLY_QUALIFIED_FIELD_NAME_MATCHER === k.to_s
+                raise ArgumentError, "global fields must be given in fully qualified format (table name::field name)"
+              end
+
+              normalized[k] = v
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/fmrest/spyke/model/orm.rb

@@ -24,7 +24,8 @@ module FmRest
           # Methods delegated to FmRest::Spyke::Relation
           delegate :limit, :offset, :sort, :order, :query, :omit, :portal,
                    :portals, :includes, :with_all_portals, :without_portals,
-                   :script, to: :all
+                   :script, :find_one, :first, :any, :find_some,
+                   :find_in_batches, :find_each, to: :all
 
           def all
             # Use FmRest's Relation instead of Spyke's vanilla one

data/lib/fmrest/spyke/model/serialization.rb

@@ -4,8 +4,8 @@ module FmRest
   module Spyke
     module Model
       module Serialization
-        FM_DATE_FORMAT = "%m/%d/%Y".freeze
-        FM_DATETIME_FORMAT = "#{FM_DATE_FORMAT} %H:%M:%S".freeze
+        FM_DATE_FORMAT = "%m/%d/%Y"
+        FM_DATETIME_FORMAT = "#{FM_DATE_FORMAT} %H:%M:%S"
 
         # Override Spyke's to_params to return FM Data API's expected JSON
         # format, and including only modified fields
@@ -63,9 +63,9 @@ module FmRest
         def serialize_values!(params)
           params.transform_values! do |value|
             case value
-            when DateTime, Time
-              value.strftime(FM_DATETIME_FORMAT)
-            when Date
+            when DateTime, Time, FmRest::StringDateTime
+              convert_datetime_timezone(value.to_datetime).strftime(FM_DATETIME_FORMAT)
+            when Date, FmRest::StringDate
               value.strftime(FM_DATE_FORMAT)
             else
               value
@@ -74,6 +74,17 @@ module FmRest
 
           params
         end
+
+        def convert_datetime_timezone(dt)
+          case fmrest_config.fetch(:timezone, nil)
+          when :utc, "utc"
+            dt.new_offset(0)
+          when :local, "local"
+            dt.new_offset(FmRest::V1.local_offset_for_datetime(dt))
+          when nil
+            dt
+          end
+        end
       end
     end
   end

data/lib/fmrest/spyke/relation.rb

@@ -190,6 +190,79 @@ module FmRest
       rescue ::Spyke::ConnectionError => error
         fallback_or_reraise(error, default: nil)
       end
+      alias_method :first, :find_one
+      alias_method :any, :find_one
+
+      # Yields each batch of records that was found by the find options.
+      #
+      # NOTE: By its nature, batch processing is subject to race conditions if
+      # other processes are modifying the database
+      #
+      # @param batch_size [Integer] Specifies the size of the batch.
+      # @return [Enumerator] if called without a block.
+      def find_in_batches(batch_size: 1000)
+        unless block_given?
+          return to_enum(:find_in_batches, batch_size: batch_size) do
+            total = limit(1).find_some.metadata.data_info.found_count
+            (total - 1).div(batch_size) + 1
+          end
+        end
+
+        offset = 1 # DAPI offset is 1-based
+
+        loop do
+          relation = offset(offset).limit(batch_size)
+
+          records = relation.find_some
+
+          yield records if records.length > 0
+
+          break if records.length < batch_size
+
+          # Save one iteration if the total is a multiple of batch_size
+          if found_count = records.metadata.data_info && records.metadata.data_info.found_count
+            break if found_count == (offset - 1) + batch_size
+          end
+
+          offset += batch_size
+        end
+      end
+
+      # Looping through a collection of records from the database (using the
+      # #all method, for example) is very inefficient since it will fetch and
+      # instantiate all the objects at once.
+      #
+      # In that case, batch processing methods allow you to work with the
+      # records in batches, thereby greatly reducing memory consumption and be
+      # lighter on the Data API server.
+      #
+      # The find_each method uses #find_in_batches with a batch size of 1000
+      # (or as specified by the :batch_size option).
+      #
+      # NOTE: By its nature, batch processing is subject to race conditions if
+      # other processes are modifying the database
+      #
+      # @param (see #find_in_batches)
+      # @example
+      #   Person.find_each do |person|
+      #     person.greet
+      #   end
+      #
+      #   Person.query(name: "==Mitch").find_each do |person|
+      #     person.say_hi
+      #   end
+      # @return (see #find_in_batches)
+      def find_each(batch_size: 1000)
+        unless block_given?
+          return to_enum(:find_each, batch_size: batch_size) do
+            limit(1).find_some.metadata.data_info.found_count
+          end
+        end
+
+        find_in_batches(batch_size: batch_size) do |records|
+          records.each { |r| yield r }
+        end
+      end
 
       protected
 

data/lib/fmrest/spyke/spyke_formatter.rb

@@ -1,9 +1,21 @@
 # frozen_string_literal: true
 
 require "json"
+require "ostruct"
 
 module FmRest
   module Spyke
+    # Metadata class to be passed to Spyke::Collection#metadata
+    class Metadata < Struct.new(:messages, :script, :data_info)
+      alias_method :scripts, :script
+    end
+
+    class DataInfo < OpenStruct
+      def total_record_count; totalRecordCount; end
+      def found_count; foundCount; end
+      def returned_count; returnedCount; end
+    end
+
     # Response Faraday middleware for converting FM API's response JSON into
     # Spyke's expected format
     class SpykeFormatter < ::Faraday::Response::Middleware
@@ -77,36 +89,61 @@ module FmRest
 
       # @param json [Hash]
       # @param include_errors [Boolean]
-      # @return [Hash] the skeleton structure for a Spyke-formatted response
+      # @return [FmRest::Spyke::Metadata] the skeleton structure for a
+      #   Spyke-formatted response
       def build_base_hash(json, include_errors = false)
         {
-          metadata: { messages: json[:messages] }.merge(script: prepare_script_results(json).presence),
-          errors:   include_errors ? prepare_errors(json) : {}
+          metadata: Metadata.new(
+            prepare_messages(json),
+            prepare_script_results(json),
+            prepare_data_info(json)
+          ).freeze,
+          errors: include_errors ? prepare_errors(json) : {}
         }
       end
 
       # @param json [Hash]
-      # @return [Hash] the script(s) execution results for Spyke metadata format
+      # @return [Array<OpenStruct>] the skeleton structure for a
+      #   Spyke-formatted response
+      def prepare_messages(json)
+        return [] unless json[:messages]
+        json[:messages].map { |m| OpenStruct.new(m).freeze }.freeze
+      end
+
+      # @param json [Hash]
+      # @return [OpenStruct] the script(s) execution results for Spyke metadata
+      #   format
       def prepare_script_results(json)
         results = {}
 
         [:prerequest, :presort].each do |s|
           if json[:response][:"scriptError.#{s}"]
-            results[s] = {
+            results[s] = OpenStruct.new(
               result: json[:response][:"scriptResult.#{s}"],
               error:  json[:response][:"scriptError.#{s}"]
-            }
+            ).freeze
           end
         end
 
         if json[:response][:scriptError]
-          results[:after] = {
+          results[:after] = OpenStruct.new(
             result: json[:response][:scriptResult],
             error:  json[:response][:scriptError]
-          }
+          ).freeze
         end
 
-        results
+        results.present? ? OpenStruct.new(results).freeze : nil
+      end
+
+      # @param json [Hash]
+      # @return [OpenStruct] the script(s) execution results for
+      #   Spyke metadata format
+      def prepare_data_info(json)
+        data_info = json[:response] && json[:response][:dataInfo]
+
+        return nil unless data_info.present?
+
+        DataInfo.new(data_info).freeze
       end
 
       # @param json [Hash]

data/lib/fmrest/string_date.rb

@@ -79,17 +79,24 @@ module FmRest
     class InvalidDate < ArgumentError; end
 
     class << self
-      alias_method :strptime, :new
+      def strptime(str, date_format, *_)
+        begin
+          date = self::DELEGATE_CLASS.strptime(str, date_format)
+        rescue ArgumentError
+          raise InvalidDate
+        end
+
+        new(str, date)
+      end
     end
 
-    def initialize(str, date_format, **str_args)
+    def initialize(str, date, **str_args)
+      raise ArgumentError, "str must be of class String" unless str.is_a?(String)
+      raise ArgumentError, "date must be of class #{self.class::DELEGATE_CLASS.name}" unless date.is_a?(self.class::DELEGATE_CLASS)
+
       super(str, **str_args)
 
-      begin
-        @delegate = self.class::DELEGATE_CLASS.strptime(str, date_format)
-      rescue ArgumentError
-        raise InvalidDate
-      end
+      @delegate = date
 
       freeze
     end
@@ -178,4 +185,36 @@ module FmRest
       @delegate
     end
   end
+
+  module StringDateAwareness
+    def _parse(v, *_)
+      if v.is_a?(StringDateTime)
+        return { year: v.year, mon: v.month, mday: v.mday, hour: v.hour, min: v.min, sec: v.sec, sec_fraction: v.sec_fraction, offset: v.offset }
+      end
+      if v.is_a?(StringDate)
+        return { year: v.year, mon: v.month, mday: v.mday }
+      end
+      super
+    end
+
+    def parse(v, *_)
+      if v.is_a?(StringDate)
+        return self == ::DateTime ? v.to_datetime : v.to_date
+      end
+      super
+    end
+
+    # Overriding case equality method so that it returns true for
+    # `FmRest::StringDate` instances
+    #
+    # Calls superclass method
+    #
+    def ===(other)
+      super || other.is_a?(StringDate)
+    end
+
+    def self.enable(classes: [Date, DateTime])
+      classes.each { |klass| klass.singleton_class.prepend(self) }
+    end
+  end
 end

data/lib/fmrest/v1.rb

@@ -4,6 +4,7 @@ require "fmrest/v1/connection"
 require "fmrest/v1/paths"
 require "fmrest/v1/container_fields"
 require "fmrest/v1/utils"
+require "fmrest/v1/dates"
 
 module FmRest
   module V1
@@ -15,5 +16,6 @@ module FmRest
     extend Paths
     extend ContainerFields
     extend Utils
+    extend Dates
   end
 end

data/lib/fmrest/v1/connection.rb

@@ -31,10 +31,6 @@ module FmRest
           conn.request :multipart
           conn.request :json
 
-          if options[:log]
-            conn.response :logger, nil, bodies: true, headers: true
-          end
-
           # Allow overriding the default response middleware
           if block_given?
             yield conn, options
@@ -43,6 +39,10 @@ module FmRest
             conn.response :json
           end
 
+          if options[:log]
+            conn.response :logger, nil, bodies: true, headers: true
+          end
+
           conn.adapter Faraday.default_adapter
         end
       end
@@ -74,8 +74,10 @@ module FmRest
         faraday_options[:ssl] = options[:ssl] if options.key?(:ssl)
         faraday_options[:proxy] = options[:proxy] if options.key?(:proxy)
 
+        database = URI.encode_www_form_component(options.fetch(:database))
+
         Faraday.new(
-          "#{scheme}://#{host}#{BASE_PATH}/#{URI.escape(options.fetch(:database))}/".freeze,
+          "#{scheme}://#{host}#{BASE_PATH}/#{database}/".freeze,
           faraday_options,
           &block
         )

data/lib/fmrest/v1/dates.rb

@@ -0,0 +1,81 @@
+# frozen_string_literal: true
+
+module FmRest
+  module V1
+    module Dates
+      FM_DATETIME_FORMAT_MATCHER = /MM|mm|dd|HH|ss|yyyy/.freeze
+
+      FM_DATE_TO_STRPTIME_SUBSTITUTIONS = {
+        "MM"   => "%m",
+        "dd"   => "%d",
+        "yyyy" => "%Y",
+        "HH"   => "%H",
+        "mm"   => "%M",
+        "ss"   => "%S"
+      }.freeze
+
+      FM_DATE_TO_REGEXP_SUBSTITUTIONS = {
+        "MM"   => '(?:0[1-9]|1[012])',
+        "dd"   => '(?:0[1-9]|[12][0-9]|3[01])',
+        "yyyy" => '\d{4}',
+        "HH"   => '(?:[01]\d|2[0123])',
+        "mm"   => '[0-5]\d',
+        "ss"   => '[0-5]\d'
+      }.freeze
+
+      def self.extended(mod)
+        mod.instance_eval do
+          @date_strptime = {}
+          @date_regexp = {}
+        end
+      end
+
+      # Converts a FM date-time format to `DateTime.strptime` format
+      #
+      # @param fm_format [String] The FileMaker date-time format
+      # @return [String] The `DateTime.strpdate` equivalent of the given FM
+      #   date-time format
+      def fm_date_to_strptime_format(fm_format)
+        @date_strptime[fm_format] ||=
+          fm_format.gsub(FM_DATETIME_FORMAT_MATCHER, FM_DATE_TO_STRPTIME_SUBSTITUTIONS).freeze
+      end
+
+      # Converts a FM date-time format to a Regexp. This is mostly used a
+      # quicker way of checking whether a FM field is a date field than
+      # Date|DateTime.strptime
+      #
+      # @param fm_format [String] The FileMaker date-time format
+      # @return [Regexp] A reegular expression matching strings in the given FM
+      #   date-time format
+      def fm_date_to_regexp(fm_format)
+        @date_regexp[fm_format] ||= 
+          Regexp.new('\A' + fm_format.gsub(FM_DATETIME_FORMAT_MATCHER, FM_DATE_TO_REGEXP_SUBSTITUTIONS) + '\Z').freeze
+      end
+
+      # Takes a DateTime dt, and returns the correct local offset for that dt,
+      # daylight savings included, in fraction of a day.
+      #
+      # By default, if ActiveSupport's Time.zone is set it will be used instead
+      # of the system timezone.
+      #
+      # @param dt [DateTime] The DateTime to get the offset for
+      # @param zone [nil, String, TimeZone] The timezone to use to calculate
+      #   the offset (defaults to system timezone, or ActiveSupport's Time.zone
+      #   if set)
+      # @return [Rational] The offset in fraction of a day
+      def local_offset_for_datetime(dt, zone = nil)
+        dt = dt.new_offset(0)
+        time = ::Time.utc(dt.year, dt.month, dt.day, dt.hour, dt.min, dt.sec)
+
+        # Do we have ActiveSupport's TimeZone?
+        time = if time.respond_to?(:in_time_zone)
+                 time.in_time_zone(zone || ::Time.zone)
+               else
+                 time.localtime
+               end
+
+        Rational(time.utc_offset, 86400) # seconds in one day (24*60*60)
+      end
+    end
+  end
+end

data/lib/fmrest/v1/type_coercer.rb

@@ -5,7 +5,8 @@ require "fmrest/string_date"
 module FmRest
   module V1
     class TypeCoercer < Faraday::Response::Middleware
-      # We use this date to represent a time for consistency with ginjo-rfm
+      # We use this date to represent a FileMaker time for consistency with
+      # ginjo-rfm
       JULIAN_ZERO_DAY = "-4712/1/1"
 
       COERCE_HYBRID = [:hybrid, "hybrid", true].freeze
@@ -30,15 +31,13 @@ module FmRest
           field_data = record["fieldData"] || record[:fieldData]
           portal_data = record["portalData"] || record[:portalData]
 
-          # Build an enumerator that iterates over hashes of fields
-          enum = Enumerator.new { |y| y << field_data }
-          if portal_data
-            portal_data.each_value do |portal_records|
-              enum += portal_records.to_enum
+          coerce_fields(field_data)
+
+          portal_data.try(:each_value) do |portal_records|
+            portal_records.each do |pr|
+              coerce_fields(pr)
             end
           end
-
-          enum.each { |hash| coerce_fields(hash) }
         end
       end
 
@@ -49,29 +48,49 @@ module FmRest
           next unless v.is_a?(String)
           next if k == "recordId" || k == :recordId || k == "modId" || k == :modId
 
-          begin
-            str_timestamp = datetime_class.strptime(v, datetime_format)
-            hash[k] = str_timestamp
-            next
-          rescue ArgumentError
+          if quick_check_timestamp(v)
+            begin
+              hash[k] = coerce_timestamp(v)
+              next
+            rescue ArgumentError
+            end
           end
 
-          begin
-            str_date = date_class.strptime(v, date_format)
-            hash[k] = str_date
-            next
-          rescue ArgumentError
+          if quick_check_date(v)
+            begin
+              hash[k] = date_class.strptime(v, date_strptime_format)
+              next
+            rescue ArgumentError
+            end
           end
 
-          begin
-            str_time = datetime_class.strptime("#{JULIAN_ZERO_DAY} #{v}", time_format)
-            hash[k] = str_time
-            next
-          rescue ArgumentError
+          if quick_check_time(v)
+            begin
+              hash[k] = datetime_class.strptime("#{JULIAN_ZERO_DAY} #{v}", time_strptime_format)
+              next
+            rescue ArgumentError
+            end
           end
         end
       end
 
+      def coerce_timestamp(str)
+        str_timestamp = DateTime.strptime(str, datetime_strptime_format)
+
+        if local_timezone?
+          # Change the DateTime to the local timezone, keeping the same
+          # time and just modifying the timezone
+          offset = FmRest::V1.local_offset_for_datetime(str_timestamp)
+          str_timestamp = str_timestamp.new_offset(offset) - offset
+        end
+
+        if datetime_class == StringDateTime
+          str_timestamp = StringDateTime.new(str, str_timestamp)
+        end
+
+        str_timestamp
+      end
+
       def date_class
         @date_class ||=
           case coerce_dates
@@ -92,19 +111,75 @@ module FmRest
           end
       end
 
-      def date_format
-        @date_format ||=
-          FmRest::V1.convert_date_time_format(@options[:date_format] || DEFAULT_DATE_FORMAT)
+      def date_fm_format
+        @options[:date_format] || DEFAULT_DATE_FORMAT
+      end
+
+      def timestamp_fm_format
+        @options[:timestamp_format] || DEFAULT_TIMESTAMP_FORMAT
+      end
+
+      def time_fm_format
+        @options[:time_format] || DEFAULT_TIME_FORMAT
       end
 
-      def datetime_format
-        @datetime_format ||=
-          FmRest::V1.convert_date_time_format(@options[:timestamp_format] || DEFAULT_TIMESTAMP_FORMAT)
+      def date_strptime_format
+        FmRest::V1.fm_date_to_strptime_format(date_fm_format)
+      end
+
+      def datetime_strptime_format
+        FmRest::V1.fm_date_to_strptime_format(timestamp_fm_format)
+      end
+
+      def time_strptime_format
+        @time_strptime_format ||=
+          "%Y/%m/%d " + FmRest::V1.fm_date_to_strptime_format(time_fm_format)
+      end
+
+      # We use a string length test, followed by regexp match test to try to
+      # identify date fields. Benchmarking shows this should be between 1 and 3
+      # orders of magnitude faster for fails (i.e. non-dates) than just using
+      # Date.strptime.
+      #
+      #                       user     system      total        real
+      # strptime:         0.268496   0.000962   0.269458 (  0.270865)
+      # re=~:             0.024872   0.000070   0.024942 (  0.025057)
+      # re.match?:        0.019745   0.000095   0.019840 (  0.020058)
+      # strptime fail:    0.141309   0.000354   0.141663 (  0.142266)
+      # re=~ fail:        0.031637   0.000095   0.031732 (  0.031872)
+      # re.match? fail:   0.011249   0.000056   0.011305 (  0.011375)
+      # length fail:      0.007177   0.000024   0.007201 (  0.007222)
+      #
+      # NOTE: The faster Regexp#match? was introduced in Ruby 2.4.0, so we
+      # can't really rely on it being available
+      if //.respond_to?(:match?)
+        def quick_check_timestamp(v)
+          v.length == timestamp_fm_format.length && FmRest::V1::fm_date_to_regexp(timestamp_fm_format).match?(v)
+        end
+
+        def quick_check_date(v)
+          v.length == date_fm_format.length && FmRest::V1::fm_date_to_regexp(date_fm_format).match?(v)
+        end
+
+        def quick_check_time(v)
+          v.length == time_fm_format.length && FmRest::V1::fm_date_to_regexp(time_fm_format).match?(v)
+        end
+      else
+        def quick_check_timestamp(v)
+          v.length == timestamp_fm_format.length && FmRest::V1::fm_date_to_regexp(timestamp_fm_format) =~ v
+        end
+
+        def quick_check_date(v)
+          v.length == date_fm_format.length && FmRest::V1::fm_date_to_regexp(date_fm_format) =~ v
+        end
+
+        def quick_check_time(v)
+          v.length == time_fm_format.length && FmRest::V1::fm_date_to_regexp(time_fm_format) =~ v
+        end
       end
 
-      def time_format
-        @time_format ||=
-          "%Y/%m/%d " + FmRest::V1.convert_date_time_format(@options[:time_format] || DEFAULT_TIME_FORMAT)
+      def local_timezone?
+        @local_timezone ||= @options.fetch(:timezone, nil).try(:to_sym) == :local
       end
 
       def coerce_dates

data/lib/fmrest/v1/utils.rb

@@ -5,17 +5,6 @@ module FmRest
     module Utils
       VALID_SCRIPT_KEYS = [:prerequest, :presort, :after].freeze
 
-      FM_DATETIME_FORMAT_MATCHER = /MM|mm|dd|HH|ss|yyyy/.freeze
-
-      FM_DATETIME_FORMAT_SUBSTITUTIONS = {
-        "MM"   => "%m",
-        "dd"   => "%d",
-        "yyyy" => "%Y",
-        "HH"   => "%H",
-        "mm"   => "%M",
-        "ss"   => "%S"
-      }.freeze
-
       # Converts custom script options to a hash with the Data API's expected
       # JSON script format.
       #
@@ -83,11 +72,6 @@ module FmRest
         params
       end
 
-      # Converts a FM date-time format to `Date.strptime` format
-      #
-      def convert_date_time_format(fm_format)
-        fm_format.gsub(FM_DATETIME_FORMAT_MATCHER, FM_DATETIME_FORMAT_SUBSTITUTIONS)
-      end
 
       private
 

data/lib/fmrest/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module FmRest
-  VERSION = "0.7.0"
+  VERSION = "0.10.1"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: fmrest
 version: !ruby/object:Gem::Version
-  version: 0.7.0
+  version: 0.10.1
 platform: ruby
 authors:
 - Pedro Carbajal
-autorequire: 
+autorequire:
 bindir: exe
 cert_chain: []
-date: 2020-05-11 00:00:00.000000000 Z
+date: 2020-08-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: faraday
@@ -54,30 +54,30 @@ dependencies:
   name: bundler
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '1.16'
+        version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '1.16'
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '10.0'
+        version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '10.0'
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: rspec
   requirement: !ruby/object:Gem::Requirement
@@ -247,6 +247,7 @@ files:
 - lib/fmrest/spyke/model/auth.rb
 - lib/fmrest/spyke/model/connection.rb
 - lib/fmrest/spyke/model/container_fields.rb
+- lib/fmrest/spyke/model/global_fields.rb
 - lib/fmrest/spyke/model/http.rb
 - lib/fmrest/spyke/model/orm.rb
 - lib/fmrest/spyke/model/serialization.rb
@@ -265,6 +266,7 @@ files:
 - lib/fmrest/v1.rb
 - lib/fmrest/v1/connection.rb
 - lib/fmrest/v1/container_fields.rb
+- lib/fmrest/v1/dates.rb
 - lib/fmrest/v1/paths.rb
 - lib/fmrest/v1/raise_errors.rb
 - lib/fmrest/v1/token_session.rb
@@ -277,7 +279,7 @@ homepage: https://github.com/beezwax/fmrest-ruby
 licenses:
 - MIT
 metadata: {}
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -293,7 +295,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubygems_version: 3.0.6
-signing_key: 
+signing_key:
 specification_version: 4
 summary: FileMaker Data API client using Faraday
 test_files: []