oci 2.0.9 → 2.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 40f4fc8cef8b30940cc5d4357f659cd7da8a077c
-  data.tar.gz: 3442d01039aaa3c13d203e8f195a3e9372487c99
+  metadata.gz: 710c56b90571dcc168a349f896579265e90fe8d7
+  data.tar.gz: 4a77c01309de5a1b22e14b123c4038c5e714f0a1
 SHA512:
-  metadata.gz: 09d6e76897744e3911bee15dd75f154996c8654e351f6a5683139beb7012dbd1b0263c99e4a9a7de7ae7623e72a2dd656e6f49d9ceb4893491efef5ce149425d
-  data.tar.gz: ef317dec6a0135a5dde2e9e9ca82a2a9266d2cd50594fcc1ae896012b37711db27bf27816437ddb9cfda13e8ac0c8d93a3a313e4c1ce0f63954fef7b4dd1c51d
+  metadata.gz: 56cda480d05e618cfb517a1b1a13bc9137e13852e789dfdcbc78fab0eedfdabe4c9be1e2a28d342ac119003da1323c987edcee78d7ebc5b15b5b5a03a01769da
+  data.tar.gz: c47dc6cce662d968890c7b8f5c405db3eee70f70cc5a265cca84c43d35fe8a632cdf9d8757f089c26903d00697c9713c47cf667cc1efc61e4b5d40d43ce727ec

data/README.md

@@ -1,5 +1,5 @@
 # Oracle Cloud Infrastructure Ruby SDK
-**Version 2.0.9**
+**Version 2.1.0**
 
 This topic describes how to install, configure, and use the Oracle Cloud Infrastructure Ruby SDK.
 
@@ -39,7 +39,24 @@ The following table provides details about some of the attributes of the SDK.
 	</tr>
 	<tr>
 		<td>Errors</td>
-		<td>Unsuccessful requests will always raise an exception. {OCI::Errors::ServiceError ServiceError} is raised when the service returns an error, and {OCI::Errors::NetworkError NetworkError} is raised for network issues (such as failed host resolution).</td>
+		<td>
+            The main exception classes for the SDK are:
+            <ul>
+                <li>{OCI::Errors::ServiceError ServiceError} is thrown when an OCI service returns a non-2xx HTTP status code</li>
+                <li>
+                    {OCI::Errors::NetworkError NetworkError} is thrown when the issue is likely to be network-related rather than an application issue. This is defined as:
+                    <ul>
+                        <li>
+                            Requests that were sent from the SDK, but for which the outcome was not a response from an OCI service. Examples include a gateway timeout, read or connection timeouts from Net::HTTP, and any (Errno) exception that was generated by the request itself.
+                        </li>
+                        <li>Requests that result in a HTTP 408 (timeout)</li>
+                    </ul>
+                </li>
+                <li>
+                    {OCI::Errors::ResponseParsingError ResponseParsingError} is thrown when a response is received from an OCI service, but the SDK could not parse it into the appropriate model type for inserting into an {OCI::Response}
+                </li>
+            </ul>
+        </td>
 	</tr>
     <tr>
 		<td>Signing</td>
@@ -55,8 +72,7 @@ The following table provides details about some of the attributes of the SDK.
 	</tr>
 	<tr>
 		<td>HTTP Client</td>
-		<td>The Ruby SDK uses Net::HTTP for HTTP requests, if needed, options may be passed to each Net::HTTP by specifying them in {OCI::ApiClient#request_option_overrides ApiClient.request_option_overrides}.</br>
-		Please check http://ruby-doc.org/stdlib-2.4.1/libdoc/net/http/rdoc/Net/HTTP.html#method-c-start for the supported options.</td>
+		<td>The Ruby SDK uses Net::HTTP for HTTP requests. Information on how to configure the client, including configuring a proxy, can be found in the <b>Configuring the HTTP Client</b> section of this page</td>
 	</tr>
     <tr>
 		<td>Instance Principals Authentication</td>
@@ -69,7 +85,7 @@ The following table provides details about some of the attributes of the SDK.
 		<td>
             The Object Storage service supports multipart uploads to make large object uploads easier by splitting the large object into parts. The Ruby SDK supports raw multipart upload operations for advanced use cases, as well as a higher-level upload class that uses the multipart upload APIs.
             <p>
-                <a href="https://docs.us-phoenix-1.oraclecloud.com/Content/Object/Tasks/managingmultipartuploads.htm">Managing Multipart Uploads</a> provides links to the APIs used for raw multipart upload operations. Higher-level uploads can be performed using the {OCI::ObjectStorage::Transfer::UploadManager}. 
+                <a href="https://docs.us-phoenix-1.oraclecloud.com/Content/Object/Tasks/managingmultipartuploads.htm">Managing Multipart Uploads</a> provides links to the APIs used for raw multipart upload operations. Higher-level uploads can be performed using the {OCI::ObjectStorage::Transfer::UploadManager}.
             </p>
             <p>
                 The UploadManager simplifies interaction with the Object Storage service by abstracting away the method used to upload objects and can handle uploading an entire object at once, or in multiple parts if it is of sufficient size (which is configurable via a {OCI::ObjectStorage::Transfer::UploadManagerConfig} object). In the latter case, the UploadManager will split a large object into parts for you, upload the parts in parallel, and then recombine and commit the parts as a single object in Object Storage.
@@ -154,9 +170,165 @@ Or you can create a {OCI::Config Config} programmatically. Note that the global
 
 The default config file location is `~/.oci/config` (on Windows `C:\Users\{user}\.oci\config`).
 
-## Service Errors
+## Configuring the HTTP Client
+
+The underlying HTTP client used by the SDK can be configured by setting the {OCI::ApiClient#request_option_overrides ApiClient.request_option_overrides} on the client. An example on how to do this, showing all supported options, is:
+
+    require 'oci'
+
+    identity = OCI::Identity::IdentityService.new
+
+    identity.api_client.request_option_overrides = {
+      # The path to the PEM-formatted CA certificate file.
+    	# The file can contain several CA certificates
+    	ca_file: '/path/to/pem/file',
+
+    	# The path to a directory of PEM-formatted CA certificate files
+    	ca_path_cert: '/path/to/cert/directory',
+
+    	# An OpenSSL::X509::Certificate object as client certificate
+      cert: OpenSSL::X509::Certificate.new(...),
+
+    	# The X509::Store to verify peer certificate.
+    	cert_store: OpenSSL::X509::Store.new(...),
+
+    	# One of the available ciphers from OpenSSL::SSL::SSLContext#ciphers
+    	ciphers: ...,
+
+    	# Close the connection immediately if the server sent no response body
+        close_on_empty_response: true,
+
+    	# An OpenSSL::PKey::RSA or OpenSSL::PKey::DSA object
+    	key: ...,
+
+    	# The SSL timeout in seconds
+    	ssl_timeout: 60,
+
+    	# The SSL version to use. See OpenSSL::SSL::SSLContext#ssl_version
+    	ssl_version: ...,
+
+    	# The verify callback for the server certification verification
+    	verify_callback: ...,
+
+    	# The maximum depth for the certificate chain verification
+    	verify_depth: 3,
+
+    	# Either OpenSSL::SSL::VERIFY_PEER or OpenSSL::SSL::VERIFY_NONE
+    	# Using OpenSSL::SSL::VERIFY_NONE (i.e. not validating the SSL certificate) is not
+    	# recommended
+    	verify_mode: OpenSSL::SSL::VERIFY_PEER
+    }
+
+Note that these correspond to a subset of the keys that can be provided as `opts` to the `Net::HTTP` [start](https://ruby-doc.org/stdlib-2.4.3/libdoc/net/http/rdoc/Net/HTTP.html#method-c-start) method.
+
+### Setting a proxy
+If the SDK is running in an environment which requires the use of a proxy server for outgoing HTTP requests, this can be configured by providing via:
+
+* Providing an environment variable; or
+* Providing an {OCI::ApiClientProxySettings} object. Here is an example of the different ways to create this object:
+
+#### Proxy via environment variable
+As the SDK uses `Net::HTTP`, the `http_proxy` environment variable can be used to specify a proxy:
+
+    export HTTP_PROXY="http://10.10.1.10:3128"
+
+When using the environment variable approach, no code changes need to be made.
+
+Note that this does not support providing a username and password to the proxy, *except* when using Ruby 2.5+ on Linux, FreeBSD or macOS (Darwin). Otherwise, if a username and password is required then the {OCI::ApiClientProxySettings} object method can be used.
+
+#### Proxy via the ApiClientProxySettings object
+An {OCI::ApiClientProxySettings} object can be used to capture proxy settings. Here are the different ways an object can be created:
+
+
+	# This creates an ApiClientProxySettings with a nil proxy address and will make the API client
+	# bypass all proxies
+	proxy_settings = OCI::ApiClientProxySettings.new(nil)
+
+	# This creates an ApiClientProxySettings that will use the proxy at 10.10.1.10:3128 and will
+	# not provide a username and password to the proxy
+	proxy_settings = OCI::ApiClientProxySettings.new("10.10.1.10", 3128)
+
+	# This creates an ApiClientProxySettings with a username and password. In production, you should use an appropriate password/secret management mechanism to vend the username and password
+	# rather than having the credentials in code
+	proxy_settings = OCI::ApiClientProxySettings.new("10.10.1.10", 3128, "username", "pass")
+
+Once an {OCI::ApiClientProxySettings} object has been created, it can be used when creating or modifying a client:
 
-Any operation resulting in a service error will cause an exception of type OCI::Errors::ServiceError to be thrown by the SDK. For information about common service errors returned by OCI, see [API Errors](https://docs.us-phoenix-1.oraclecloud.com/Content/API/References/apierrors.htm).
+	# At client creation time
+	identity = OCI::Identity::IdentityService.new(proxy_settings: proxy_settings)
+
+	# When updating a client
+	identity.api_client.proxy_settings = proxy_settings
+
+	# The proxy_settings can also be nil'ed out. This will make the ApiClient fall back to
+	# using the http_proxy environment variable (if present)
+	identity.api_client.proxy_settings = nil
+
+## Exceptions
+
+### Service Errors
+Any operation that receives a response with a non-2xx HTTP status code from an OCI service will cause an exception of type {OCI::Errors::ServiceError ServiceError} to be thrown by the SDK. 
+
+For information about common service errors returned by OCI, see [API Errors](https://docs.us-phoenix-1.oraclecloud.com/Content/API/References/apierrors.htm).
+
+The key attributes to inspect when dealing with a {OCI::Errors::ServiceError} are:
+
+* {OCI::Errors::ServiceError#request_id request_id} if you need to contact Oracle about a particular request, please provide this request ID
+* {OCI::Errors::ServiceError#status_code status_code} contains the HTTP status code of the response
+* {OCI::Errors::ServiceError#service_code service_code} should contain a value from the *Error Code* field as described in the *API Errors* page
+* {OCI::Errors::ServiceError#request_made request_made} contains the actual `Net::HTTPRequest` which was sent by the SDK
+
+You can also call {OCI::Errors::ServiceError#to_s to_s} on the error to get a summary of the key information about the error.
+
+#### HTTP 3xx Responses
+The SDK will throw exceptions of type {OCI::Errors::ServiceError ServiceError} on HTTP 3xx responses. This impacts operations that support conditional GETs. This includes {OCI::ObjectStorage::ObjectStorageClient#get_object} and {OCI::ObjectStorage::ObjectStorageClient#head_object} methods as these can return responses with a HTTP status code of 304 if passed an `if_none_match`, which corresponds to the current etag of the object or bucket.
+
+In order to account for this, you should catch {OCI::Errors::ServiceError ServiceError} and check its `status_code` attribute for the HTTP status code. For example: 
+
+    require 'oci'
+
+    object_storage = OCI::ObjectStorage::ObjectStorageClient.new
+
+    begin
+        get_object_response = object_storage.get_object('my_namespace', 'my_bucket', 'my_object', if_none_match: 'some_etag_value')
+    rescue OCI::Errors::ServiceError => err
+        # Do nothing if the object exists but has not been modified (based on the etag value), otherwise raise the exception
+        raise if err.status_code != 304
+    end
+
+
+### Network Errors
+{OCI::Errors::NetworkError NetworkError} is thrown when the issue is likely to be network related rather than an application issue. This is defined as:
+
+* Requests which were sent from the SDK but the outcome was not a response from an OCI service. Further examples of include:
+  * Gateway timeouts
+  * Read or connection timeouts
+  * Any {Errno} exception which was generated by making the request
+* Requests which resulted in a HTTP 408 (timeout)
+
+The key attributes to inspect when dealing with a {OCI::Errors::NetworkError} are:
+
+* {OCI::Errors::NetworkError#request_made request_made} and {OCI::Errors::NetworkError#response_received response_received} contain the `Net::HTTPRequest` and `Net::HTTPResponse`, respectively
+* {OCI::Errors::NetworkError#code code} contains the HTTP status code of the response
+
+You can also call {OCI::Errors::NetworkError#to_s to_s} on the error to get a summary of the key information about the error. In addition, the {OCI::Errors::NetworkError#cause NetworkError.cause} of the {OCI::Errors::NetworkError NetworkError} can be inspected to see the original error that caused the {OCI::Errors::NetworkError NetworkError} to be thrown.
+
+### Response Parsing Errors
+{OCI::Errors::ResponseParsingError ResponseParsingError} is thrown when a response was received from an OCI service but the SDK could not parse it into the appropriate model type to put into an {OCI::Response}. 
+
+The key attributes to inspect when dealing with a {OCI::Errors::ResponseParsingError} are: 
+
+* {OCI::Errors::ResponseParsingError#request_made request_made} and {OCI::Errors::ResponseParsingError#response_received response_received} contain the `Net::HTTPRequest` and `Net::HTTPResponse`, respectively
+* {OCI::Errors::ResponseParsingError#response_body response_body} contains the response data which failed to parse
+
+You can also call {OCI::Errors::ResponseParsingError#to_s to_s} on the error to get a summary of the key information about the error. In addition, the {OCI::Errors::ResponseParsingError#cause ResponseParsingError.cause} of the {OCI::Errors::ResponseParsingError ResponseParsingError} can be inspected to see the original error that caused the {OCI::Errors::ResponseParsingError ResponseParsingError} to be thrown.
+
+### Other Errors
+The other errors you may encounter while using the SDK are:
+
+* {OCI::Errors::MaximumWaitTimeExceededError MaximumWaitTimeExceededError} when using {OCI::Response#wait_until} or when using {OCI::LoadBalancer::Util#wait_on_work_request}
+* {OCI::Errors::WorkRequestFailedError WorkRequestFailedError} when using {OCI::LoadBalancer::Util#wait_on_work_request}. The {OCI::Errors::WorkRequestFailedError#work_request work_request} attribute can be inspected to get the actual work request details
+* {OCI::Errors::MultipartUploadError MultipartUploadError} when using {OCI::ObjectStorage::Transfer::UploadManager UploadManager}. The {OCI::Errors::MultipartUploadError#errors errors} attribute can be inspected to see what errors occurred during the upload, and the {OCI::Errors::MultipartUploadError#upload_id upload_id} can be used if you wish to try and resume the upload via the UploadManager's {OCI::ObjectStorage::Transfer::UploadManager#resume resume} method
 
 ## Examples
 The example code in this section shows how various parts of the Ruby SDK work. More examples can be found in the SDK download.
@@ -349,6 +521,3 @@ Ways to get in touch:
 * [Stack Overflow](https://stackoverflow.com/): Please use the [oracle-cloud-infrastructure](https://stackoverflow.com/questions/tagged/oracle-cloud-infrastructure) and [oci-ruby-sdk](https://stackoverflow.com/questions/tagged/oci-ruby-sdk) tags in your post
 * [Developer Tools section](https://community.oracle.com/community/cloud_computing/bare-metal/content?filterID=contentstatus%5Bpublished%5D~category%5Bdeveloper-tools%5D&filterID=contentstatus%5Bpublished%5D~objecttype~objecttype%5Bthread%5D) of the Oracle Cloud forums
 * [My Oracle Support](https://support.oracle.com)
-
-
-

data/lib/oci.rb

@@ -1,6 +1,7 @@
 # Copyright (c) 2016, 2018, Oracle and/or its affiliates. All rights reserved.
 
 require 'oci/api_client'
+require 'oci/api_client_proxy_settings'
 require 'oci/config'
 require 'oci/config_file_loader'
 require 'oci/errors'

data/lib/oci/api_client.rb

@@ -45,7 +45,12 @@ module OCI
     # @return [Hash]
     attr_accessor :request_option_overrides
 
-    def initialize(config, signer)
+    # The proxy settings which this ApiClient will use
+    #
+    # @return [OCI::ApiClientProxySettings]
+    attr_accessor :proxy_settings
+
+    def initialize(config, signer, proxy_settings: nil)
       raise "Missing the required parameter 'config' when initializing ApiClient." if config.nil?
       raise "Missing the required parameter 'signer' when initializing ApiClient." if signer.nil?
 
@@ -53,6 +58,7 @@ module OCI
       @signer = signer
       @default_headers = {}
       @request_option_overrides = {}
+      @proxy_settings = proxy_settings
     end
 
     # Call an API with given options.
@@ -101,8 +107,8 @@ module OCI
       # to support IO-like object as well like StringIO
       return model if model.nil? || model.is_a?(String) || (model.respond_to?(:read) && model.respond_to?(:write))
 
-      # Supports IO-wrapping objects we can convert to an IO. An example is Rails' 
-      # ActionDispatch::Http::UploadedFile, which wraps an IO (a Tempfile) but 
+      # Supports IO-wrapping objects we can convert to an IO. An example is Rails'
+      # ActionDispatch::Http::UploadedFile, which wraps an IO (a Tempfile) but
       # doesn't expose all the IO operations directly (e.g. you can't write to it, it's not seekable)
       #
       # This should be safe to use with IO and its subclasses as well as to_io is a method on IO:
@@ -177,7 +183,7 @@ module OCI
       elsif http_method == :patch
         request = Net::HTTP::Patch.new(uri)
       else
-        fail "new http method (#{http_method}) needs to be supported!"
+        raise "new http method (#{http_method}) needs to be supported!"
       end
 
       if body.respond_to?(:read) && body.respond_to?(:write)
@@ -204,7 +210,7 @@ module OCI
       request['opc-request-id'] ||= build_request_id
       request['User-Agent'] = build_user_agent
 
-      http = Net::HTTP.new(uri.hostname, uri.port)
+      http = get_http_object(uri.hostname, uri.port)
 
       unless @request_option_overrides.empty?
         http.methods.grep(/\A(\w+)=\z/) do |meth|
@@ -219,74 +225,166 @@ module OCI
       http.open_timeout = @config.connection_timeout
       http.read_timeout = @config.timeout.zero? ? 31_536_000 : @config.timeout # 31536000 means 365 days
 
+      response_ref = nil
       begin
         http.start do
-          http.request request do |response|
-
-            # If the response is timeout already, does not make sense to parse the http body because partial
-            # response may be returned, we should skip JSON parser and raise exception immediately
-            if response.is_a? Net::HTTPRequestTimeOut
-              raise Errors::NetworkError.new(response.message, response.code.to_i)
-            end
+          http.request(request) do |response|
+            response_ref = response
 
             # process headers for opc-meta- key
-            metadata = {}
-            prefix_size = 'opc-meta-'.length.freeze
-            response.each_header do |key, value|
-              if key.start_with? 'opc-meta-'
-                metadata[key[prefix_size..-1]] = value
-              end
-            end
-
-            metadata.each do |key, value|
-              response.delete "opc-meta-#{key}"
-              response[key] = value
-            end
-
-            if block.nil? || !success?(response)
-              if @config.logger
-                @config.logger.debug "HTTP response body ~BEGIN~\n#{response.body}\n~END~\n"
-              end
-
-              # In the case of an error, deserialize to a plain hash by indicating 'Object'.
-              return_type = success?(response) ? opts[:return_type] : 'Object'
-              data = return_type ? deserialize(response, return_type) : nil
-
-              unless success? response
-                if data
-                  raise Errors::ServiceError.new(response.code.to_i,
-                                                 data[:code],
-                                                 response.header['opc-request-id'],
-                                                 data[:message])
-                else
-                  raise Errors::NetworkError.new(response.message, response.code.to_i)
-                end
-              end
-
-              if @config.logger
-                @config.logger.debug "API Response Received:\nData: #{data.inspect}\nStatus code: #{response.code}\nHeaders: #{response.header}"
-              end
-
-              return Response.new(response.code.to_i, response.header, data)
-            else
-              response.read_body do |chunk|
-                yield chunk, response
-              end
-              return Response.new(response.code.to_i, response.header, nil)
-            end
+            replace_keys_in_response_headers_with_non_prefixed_equivalents(response, 'opc-meta-')
+
+            # Either stream the body through a block (if we have one) and return, or read the body so that it can be accessed
+            # again later and jump out of the http.start block
+            #
+            # The idea is to terminate the HTTP connection as early as possible. There are two main reasons for
+            # this:
+            #
+            #   1. Net::HTTP can throw a lot of exceptions, both from itself and what bubbles up from the underlying OS
+            #      via Errno:: so we have a very broad rescue block. This isn't great because the more work we do inside the
+            #      http.start block the more gaps we leave for non-Net::HTTP related errors. This makes it hard to throw
+            #      a meaningful error back to the customer (i.e. we leave ourselves gaps where Errors::NetworkError does not apply
+            #      and could throw something disingenuous if we mask everything as a NetworkError)
+            #
+            #   2. Release resources sooner/don't hold connections open longer than necessary (e.g. we don't need the
+            #      connection open while we're deserialising)
+            return process_response_block(response, &block) if success?(response) && !block.nil?
+            response.body
           end
         end
-      rescue Errors::NetworkError, Errors::ServiceError, JSON::ParserError
-        raise
-      rescue => ex
-        raise Errors::NetworkError.new(ex.message, 0)
+      rescue StandardError => err
+        # Unfortunately, catching StandardError is the surest way to capture all the errors originating from Net::HTTP
+        code_from_response = if !response_ref.nil?
+                               response_ref.code.to_i
+                             else
+                               0
+                             end
+
+        raise Errors::NetworkError.new(
+          err.message,
+          code_from_response,
+          request_made: request,
+          response_received: response_ref
+        )
       end
+
+      # If the response is a timeout (HTTP 408), it does not make sense to parse the http body because a partial
+      # response may be returned. We should skip JSON parsing and raise an error immediately
+      handle_timeout_response(request, response_ref)
+
+      @config.logger.debug("HTTP response body ~BEGIN~\n#{response_ref.body}\n~END~\n") if @config.logger
+
+      return handle_success_response(request, response_ref, opts[:return_type]) if success?(response_ref)
+      handle_non_success_response(request, response_ref)
+    end
+
+    def get_http_object(hostname, port)
+      return Net::HTTP.new(hostname, port) if @proxy_settings.nil?
+      return Net::HTTP.new(hostname, port, @proxy_settings.proxy_address) if @proxy_settings.proxy_address.nil?
+
+      Net::HTTP.new(
+        hostname,
+        port,
+        @proxy_settings.proxy_address,
+        @proxy_settings.proxy_port,
+        @proxy_settings.proxy_user,
+        @proxy_settings.proxy_password
+      )
     end
 
     def success?(response)
       response.is_a?(Net::HTTPSuccess)
     end
 
+    # Determines whether a response "looks like" one returned from an OCI service (regardless of whether the response itself
+    # represents success or failure).
+    #
+    # This is determined by whether the response includes an opc-request-id header or some header which starts with
+    # opc-
+    def response_from_oci_service?(response)
+      response.key?('opc-request-id') || response.to_hash.keys.any? { |k| k.downcase.start_with?('opc-') }
+    end
+
+    def replace_keys_in_response_headers_with_non_prefixed_equivalents(response, prefix)
+      raise 'Response cannot be nil' if response.nil?
+      raise 'A non-blank prefix must be provided' if prefix.nil? || prefix.strip.empty?
+
+      prefix_size = prefix.length.freeze
+      processed_keys = {}
+
+      response.each_header do |key, value|
+        if key.start_with?(prefix)
+          processed_keys[key[prefix_size..-1]] = value
+        end
+      end
+
+      processed_keys.each do |key, value|
+        response.delete("#{prefix}#{key}")
+        response[key] = value
+      end
+
+      processed_keys
+    end
+
+    def process_response_block(response)
+      raise 'A block must be provided' unless block_given?
+      response.read_body do |chunk|
+        yield chunk, response
+      end
+
+      Response.new(response.code.to_i, response.header, nil)
+    end
+
+    def handle_timeout_response(request, response)
+      return unless response.is_a?(Net::HTTPRequestTimeOut)
+      raise Errors::NetworkError.new(
+        response.message,
+        response.code.to_i,
+        request_made: request,
+        response_received: response
+      )
+    end
+
+    def handle_success_response(request, response, return_type)
+      data = return_type ? deserialize(request, response, return_type) : nil
+
+      @config.logger.debug("API Response Received:\nData: #{data.inspect}\nStatus code: #{response.code}\nHeaders: #{response.header}") \
+        if @config.logger
+
+      Response.new(response.code.to_i, response.header, data)
+    end
+
+    def handle_non_success_response(request, response)
+      # For errors, we expect a plain JSON object so pass 'Object' here in order to serialize it into a hash
+      data = deserialize(request, response, 'Object')
+      if data
+        raise Errors::ServiceError.new(
+          response.code.to_i,
+          data[:code],
+          response.header['opc-request-id'],
+          data[:message],
+          request_made: request
+        )
+      elsif response_from_oci_service?(response)
+        # TODO: Once we have a standard for how service errors are transmitted when there is no body (e.g. in the header),
+        # change how the ServiceError gets constructed to get data from those fields
+        raise Errors::ServiceError.new(
+          response.code.to_i,
+          nil,
+          response.header['opc-request-id'],
+          response.message,
+          request_made: request
+        )
+      else
+        raise Errors::NetworkError.new(
+          response.message,
+          response.code.to_i,
+          request_made: request,
+          response_received: response
+        )
+      end
+    end
+
     def self.append_query_params(url, query_params)
       if query_params.empty?
         url
@@ -374,7 +472,7 @@ module OCI
     #
     # @param [Response] response HTTP response
     # @param [String] return_type some examples: "User", "Array[User]", "Hash[String,Integer]"
-    def deserialize(response, return_type)
+    def deserialize(request, response, return_type)
       body = response.body
       return if body.nil? || body.empty?
 
@@ -386,15 +484,39 @@ module OCI
       # There are some cases, the error is not returned by services but by like gateway, for example in bug
       # https://jira.aka.lgl.grungy.us/browse/DEX-564, gateway timeouts and 504 is returned and content is generated
       # by gateway, so there is no guarantee that the content-type will be application/json.
-      raise Errors::NetworkError.new(response.message, response.code.to_i) unless json_mime?(content_type)
+      if !success?(response) && !response_from_oci_service?(response)
+        raise Errors::NetworkError.new(
+          response.message,
+          response.code.to_i,
+          request_made: request,
+          response_received: response
+        )
+      end
 
       begin
         data = JSON.parse("[#{body}]", :symbolize_names => true)[0]
-      rescue JSON::ParserError => e
+      rescue JSON::ParserError
         if %w(String Date DateTime).include?(return_type)
           data = body
         else
-          raise e
+          # If we received an error here then we received a response from the OCI service that
+          # we could not parse. Instead of throwing a parsing error, throw out a service error
+          # (so the customer has all the information) and put the response body in the message
+          # so that they may also have a sense of what the error from the service was
+          unless success?(response)
+            raise OCI::Errors::ServiceError.new(
+              response.code.to_i,
+              nil,
+              response['opc-request-id'],
+              response.body,
+              request_made: request
+            )
+          end
+
+          raise OCI::Errors::ResponseParsingError.new(
+            request_made: request,
+            response_received: response
+          )
         end
       end