shopify_api_retry 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 949d65d19ed205d18a64ea886ef817bcd9d8b26895036228dddc3dcf4d91f0ca
-  data.tar.gz: 6b776cf5985be1593e791a81c6f4ce31c88ca4f029c7ba03bd8e24ca5f44f26f
+  metadata.gz: 52d7c3cac56c5a6bc3146fe32d68853bd36411718a9a24a23eecb1be5055ef2e
+  data.tar.gz: f6164b541570442bd62781338ab391f17d9d6a79cfa3d182960efc3046041b0a
 SHA512:
-  metadata.gz: 781e898074a0f3b9aaed28f1202087173e42483920375f7921384854437f9ed40ad77a0ef37be5d1221f5e76141c988a35e50368da22e83b5e8722c185558e70
-  data.tar.gz: 243547c486e88cfabed0ea4a2c4fbba9524aa44234c5852110cf4a485f28c551309e78a055222576da2bcc6240232d783706807ebeee31c41956dddd6e130495
+  metadata.gz: 4098e6e2399e42830d4cad365b047c4ad8a1d1a24027e1a800ad04b7d3aa78d3acd5e07c279cdfd802bec916542211c6ad0ca67b819187329639ec762403dcc7
+  data.tar.gz: 6dcff84ded14acbfd948fe239ddc0b1cc71d25963e064f4f01db030b3d1cc5a157e6b0216a248c4f384c7f19f73e965e5c9da428e6438a04d2ccd81110148660

data/Changes

@@ -0,0 +1,20 @@
+--------------------
+2021-05-08  v0.1.3
+--------------------
+* Add support for retrying GraphQL requests
+* Mark ShopifyAPIRetry.retry as deprecated
+
+--------------------
+2021-03-11  v0.1.2
+--------------------
+* Check proper base class for ActiveResource error handling
+
+--------------------
+2021-03-09  v0.1.1
+--------------------
+* Fix typo in deprication warning
+
+--------------------
+2021-01-31  v0.1.0
+--------------------
+* Support for retrying user-defined errors

data/README.md

@@ -1,9 +1,9 @@
-# ShopifyAPIRetry
+# Shopify API Retry
 
 ![CI](https://github.com/ScreenStaring/shopify_api_retry/workflows/CI/badge.svg)
 
-Simple Ruby module to retry a [Shopify API request](https://github.com/Shopify/shopify_api) if rate limited (HTTP 429) or other errors
-occur.
+Simple Ruby module to retry a [`ShopifyAPI` request](https://github.com/Shopify/shopify_api) if rate-limited or other errors
+occur. Works with the REST and GraphQL APIs.
 
 ## Installation
 
@@ -21,25 +21,27 @@ gem install shopify_api_retry
 
 ## Usage
 
-By default requests are retried when a Shopify rate limit error is returned. The retry happens once after waiting for
+### REST API
+
+By default requests are retried when a Shopify rate limit error (HTTP 429) is returned. The retry happens once after waiting for
 [the seconds given by the HTTP `Retry-After` header](https://shopify.dev/concepts/about-apis/rate-limits):
 ```rb
 require "shopify_api_retry" # requires "shopify_api" for you
 
-ShopifyAPIRetry.retry { customer.update_attribute(:tags, "foo") }
-customer = ShopifyAPIRetry.retry { ShopifyAPI::Customer.find(id) }
+ShopifyAPIRetry::REST.retry { customer.update_attribute(:tags, "foo") }
+customer = ShopifyAPIRetry::REST.retry { ShopifyAPI::Customer.find(id) }
 ```
 
 You can override this:
 ```rb
-ShopifyAPIRetry.retry(:wait => 3, :tries => 5) { customer.update_attribute(:tags, "foo")  }
+ShopifyAPIRetry::REST.retry(:wait => 3, :tries => 5) { customer.update_attribute(:tags, "foo")  }
 ```
 This will try the request 5 times, waiting 3 seconds between each attempt. If a retry fails after the given number
-of `:tries` the original error will be raised.
+of `:tries` the last error will be raised.
 
 You can also retry requests when other errors occur:
 ```rb
-ShopifyAPIRetry.retry "5XX" => { :wait => 10, :tries => 2 } do
+ShopifyAPIRetry::REST.retry "5XX" => { :wait => 10, :tries => 2 } do
   customer.update_attribute(:tags, "foo")
 end
 ```
@@ -47,24 +49,75 @@ This still retries rate limit requests, but also all HTTP 5XX errors.
 
 Classes can be specified too:
 ```rb
-ShopifyAPIRetry.retry SocketError => { :wait => 1, :tries => 5 } do
+ShopifyAPIRetry::REST.retry SocketError => { :wait => 1, :tries => 5 } do
   customer.update_attribute(:tags, "foo")
 end
 ```
 
-Global defaults can be set as well:
+You can also set [global defaults](#global-defaults).
+
+### GraphQL API
+
+By default a retry attempt is made when your [GraphQL request is rate-limited](https://shopify.dev/concepts/about-apis/rate-limits#graphql-admin-api-rate-limits).
+In order to calculate the proper amount of time to wait before retrying you must set the `X-GraphQL-Cost-Include-Fields` header
+(_maybe this library should do this?_):
+
+```rb
+require "shopify_api_retry" # requires "shopify_api" for you
+
+ShopifyAPI::Base.headers["X-GraphQL-Cost-Include-Fields"] = "true"
+```
+
+Once this is set run your queries and mutations and rate-limited requests will be retried.
+The retry happens once, calculating the wait time from the API's cost data:
+```rb
+result = ShopifyAPIRetry::GraphQL.retry { ShopifyAPI::GraphQL.client.query(YOUR_QUERY) }
+p result.data.whatever
+```
+
+To calculate the retry time **the query's return value must be returned by the block**.
+
+You can override the retry times:
+```rb
+result = ShopifyAPIRetry::GraphQL.retry(:wait => 4, :tries => 4) do
+  ShopifyAPI::GraphQL.client.query(YOUR_QUERY)
+end
+p result.data.whatever
+```
+
+Like retry attempts made with the REST API you can specify errors to retry but, due to the GraphQL specification, these must not
+be HTTP status codes and are only relevant to network connection-related errors:
+
+```rb
+result = ShopifyAPIRetry::GraphQL.retry SocketError => { :wait => 1, :tries => 5 } do
+  ShopifyAPI::GraphQL.client.query(YOUR_QUERY)
+end
+```
+
+To give wait and try times for specifically for rate limit errors, use the key `:graphql`:
+```rb
+result = ShopifyAPIRetry::GraphQL.retry :graphql => { :wait => 10, :tries => 5 } do
+  ShopifyAPI::GraphQL.client.query(YOUR_QUERY)
+end
+```
+
+Note that specifying a `:wait` via `:graphql` or without an error key will skip calculating the retry based on the API
+response's cost data.
+
+### Global Defaults
+
+You can configure global defaults to be used by REST and GraphQL calls. For example:
+
 ```rb
 ShopifyAPIRetry.configure do |config|
   config.default_wait  = 2.5
   config.default_tries = 10
 
-  # Use defaults for these
-  config.on ["5XX", Net::TimeoutError]
+  # Use default_* for these
+  config.on ["5XX", Net::ReadTimeout]
 
   config.on SocketError, :tries => 2, :wait => 1
 end
-
-ShopifyAPIRetry.retry { customer.update_attribute(:tags, "foo")  }
 ```
 
 ## License

data/lib/shopify_api_retry.rb

@@ -1,14 +1,15 @@
 # frozen_string_literal: true
 
-require "shopify_api"
+# We don't use this. We require it as convenience for the caller in case they do.
+begin
+  require "shopify_api"
+rescue LoadError
+end
 
 module ShopifyAPIRetry
-  VERSION = "0.1.0"
-
-  HTTP_RETRY_AFTER = "Retry-After"
-  HTTP_RETRY_STATUS = "429"
+  VERSION = "0.2.0"
 
-  class Config
+  class Config                  # :nodoc:
     attr_writer :default_wait
     attr_writer :default_tries
 
@@ -40,8 +41,38 @@ module ShopifyAPIRetry
       @default_tries = nil
     end
 
+    def merge(userconfig)
+      config = to_h
+      return config unless userconfig
+
+      if userconfig.is_a?(Integer)
+        warn "#{self.class}: using an Integer for the retry time is deprecated and will be removed, use :wait => #{userconfig} instead"
+        userconfig = { :wait => userconfig }
+      elsif !userconfig.is_a?(Hash)
+        raise ArgumentError, "config must be a Hash"
+      end
+
+      userconfig.each do |k, v|
+        if v.is_a?(Hash)
+          config[k.to_s] = v.dup
+        else
+          config["graphql"][k] = config[REST::HTTP_RETRY_STATUS][k] = v
+        end
+      end
+
+      config.values.each do |cfg|
+        raise ArgumentError, "seconds to wait must be >= 0" if cfg[:wait] && cfg[:wait] < 0
+      end
+
+      config
+    end
+
     def to_h
-      settings = { HTTP_RETRY_STATUS => { :tries => default_tries, :wait => default_wait } }
+      settings = {
+        "graphql" => { :tries => default_tries, :wait => default_wait },
+        REST::HTTP_RETRY_STATUS => { :tries => default_tries, :wait => default_wait }
+      }
+
       @settings.each_with_object(settings) { |(k, v), o| o[k.to_s] = v.dup }
     end
   end
@@ -58,75 +89,123 @@ module ShopifyAPIRetry
     @config
   end
 
-  #
-  # Execute the provided block. If an HTTP 429 response is returned try
-  # it again. If any errors are provided try them according to their wait/return spec.
-  #
-  # If no spec is provided the value of the HTTP header <code>Retry-After</code>
-  # is waited for before retrying. If it's not given (it always is) +2+ is used.
-  #
-  # If retry fails the original error is raised.
-  #
-  # Returns the value of the block.
-  #
-  def retry(cfg = nil)
-    raise ArgumentError, "block required" unless block_given?
-
-    attempts = build_config(cfg)
-
-    begin
-      result = yield
-    rescue => e
-      handler = attempts[e.class.name]
-      raise if handler.nil? && (!e.is_a?(ActiveResource::ClientError) || !e.response.respond_to?(:code))
-
-      handler ||= attempts[e.response.code] || attempts["#{e.response.code[0]}XX"]
-      handler[:wait] ||= e.response[HTTP_RETRY_AFTER] || config.default_wait if e.response.code == HTTP_RETRY_STATUS
+  class Request                 # :nodoc:
+    def self.retry(cfg = nil, &block)
+      new(cfg).retry(&block)
+    end
+
+    def initialize(cfg = nil)
+      @handlers = ShopifyAPIRetry.config.merge(cfg)
+    end
+
+    def retry(&block)
+      raise ArgumentError, "block required" unless block_given?
+
+      begin
+        result = request(&block)
+      rescue => e
+        handler = find_handler(e)
+        raise unless handler && snooze(handler)
 
+        retry
+      end
+
+      result
+    end
+
+    protected
+
+    attr_reader :handlers
+
+    def request
+      yield
+    end
+
+    def find_handler(error)
+      @handlers[error.class.name]
+    end
+
+    def snooze(handler)
       handler[:attempts] ||= 1
-      raise if handler[:attempts] == handler[:tries]
+      return false if handler[:attempts] == handler[:tries]
 
-      snooze = handler[:wait].to_i
+      snooze = handler[:wait].to_f
       waited = sleep snooze
-      snooze -= waited
-      # Worth looping?
+      snooze = snooze - waited
+      # sleep returns the rounded time slept but sometimes it's rounded up, others it's down
+      # given this, we may sleep for more than requested
       sleep snooze if snooze > 0
 
       handler[:attempts] += 1
 
-      retry
+      true
     end
-
-    result
   end
 
-  module_function :retry
+  class REST < Request
+    HTTP_RETRY_AFTER = "Retry-After"
+    HTTP_RETRY_STATUS = "429"
+
+    protected
 
-  def self.build_config(userconfig)
-    config = ShopifyAPIRetry.config.to_h
-    return config unless userconfig
+    def find_handler(error)
+      handler = super
+      return handler if handler || (!error.is_a?(ActiveResource::ConnectionError) || !error.response.respond_to?(:code))
 
-    if userconfig.is_a?(Integer)
-      userconfig = { :wait => cfg }
-      warn "passing an Integer to retry is deprecated and will be removed, use an :wait => #{cfg} instead"
-    elsif !userconfig.is_a?(Hash)
-      raise ArgumentError, "config must be a Hash"
+      handler = handlers[error.response.code] || handlers["#{error.response.code[0]}XX"]
+      handler[:wait] ||= error.response[HTTP_RETRY_AFTER] || config.default_wait if error.response.code == HTTP_RETRY_STATUS
+
+      handler
     end
+  end
 
-    userconfig.each do |k, v|
-      if v.is_a?(Hash)
-        config[k.to_s] = v.dup
-      else
-        config[HTTP_RETRY_STATUS][k] = v
+  class GraphQL < Request
+    CONVERSION_WARNING = "#{name}.retry: skipping retry, cannot convert GraphQL response to a Hash: %s. " \
+                         "To retry requests your block's return value must be a Hash or something that can be converted via #to_h"
+    protected
+
+    def request
+      loop do
+        data = og_data = yield
+
+        # Shopify's client does not return a Hash but
+        # technically we work with any Hash response
+        unless data.is_a?(Hash)
+          unless data.respond_to?(:to_h)
+            warn CONVERSION_WARNING % "respond_to?(:to_h) is false"
+            return og_data
+          end
+
+          begin
+            data = data.to_h
+          rescue TypeError, ArgumentError => e
+            warn CONVERSION_WARNING % e.message
+            return og_data
+          end
+        end
+
+        cost = data.dig("extensions", "cost")
+        # If this is nil then the X-GraphQL-Cost-Include-Fields header was not set
+        # If actualQueryCost is present then the query was not rate limited
+        return og_data if cost.nil? || cost["actualQueryCost"]
+
+        handler = handlers["graphql"]
+        handler[:wait] ||= sleep_time(cost)
+
+        return og_data unless snooze(handler)
       end
     end
 
-    config.values.each do |cfg|
-      raise ArgumentError, "seconds to wait must be >= 0" if cfg[:wait] && cfg[:wait] < 0
+    def sleep_time(cost)
+      status = cost["throttleStatus"]
+      (cost["requestedQueryCost"] - status["currentlyAvailable"]) / status["restoreRate"]# + 0.33
     end
+  end
 
-    config
+  def retry(cfg = nil, &block)
+    warn "#{name}.retry has been deprecated, use ShopifyAPIRetry::REST.retry or ShopifyAPIRetry::GraphQL.retry"
+    REST.new(cfg).retry(&block)
   end
 
-  private_class_method :build_config
+  module_function :retry
 end

data/shopify_api_retry.gemspec

@@ -1,11 +1,14 @@
+lib = File.expand_path("../lib", __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require "shopify_api_retry"
+
 Gem::Specification.new do |spec|
   spec.name          = "shopify_api_retry"
-  spec.version       = "0.1.0"
+  spec.version       = ShopifyAPIRetry::VERSION
   spec.authors       = ["Skye Shaw"]
   spec.email         = ["skye.shaw@gmail.com"]
 
-  spec.summary       = %q{Retry a ShopifyAPI request if an HTTP 429 (too many requests) is returned}
-  spec.description   = %q{Simple module to retry a ShopifyAPI request if an HTTP 429 (too many requests) is returned. No monkey patching.}
+  spec.summary       = %q{Retry a ShopifyAPI request if if rate-limited or other errors occur. Works with the REST and GraphQL APIs.}
   spec.homepage      = "https://github.com/ScreenStaring/shopify_api_retry"
   spec.license       = "MIT"
 
@@ -18,7 +21,10 @@ Gem::Specification.new do |spec|
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
 
-  spec.add_dependency "shopify_api", ">= 4.0"
+  spec.required_ruby_version = ">= 2.3"
+
+  # I don't think we _really_ need this just too lazy to remove now.
+  spec.add_development_dependency "shopify_api", ">= 4.0"
 
   spec.add_development_dependency "minitest", "~> 5.0"
   spec.add_development_dependency "bundler"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: shopify_api_retry
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Skye Shaw
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2021-02-01 00:00:00.000000000 Z
+date: 2021-12-24 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: shopify_api
@@ -17,7 +17,7 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '4.0'
-  type: :runtime
+  type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
@@ -66,8 +66,7 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '12.0'
-description: Simple module to retry a ShopifyAPI request if an HTTP 429 (too many
-  requests) is returned. No monkey patching.
+description: 
 email:
 - skye.shaw@gmail.com
 executables: []
@@ -77,6 +76,7 @@ files:
 - ".github/workflows/ci.yml"
 - ".gitignore"
 - ".travis.yml"
+- Changes
 - Gemfile
 - LICENSE.txt
 - README.md
@@ -95,7 +95,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '0'
+      version: '2.3'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
@@ -106,5 +106,6 @@ rubyforge_project:
 rubygems_version: 2.7.6
 signing_key: 
 specification_version: 4
-summary: Retry a ShopifyAPI request if an HTTP 429 (too many requests) is returned
+summary: Retry a ShopifyAPI request if if rate-limited or other errors occur. Works
+  with the REST and GraphQL APIs.
 test_files: []