salesforce_bulk_api 1.1.0 → 1.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a396ff6ca0368b07ef0d575fed3345ec4c607a1598d4907424e4ad9d8a31f320
-  data.tar.gz: 951672b7488408a307cfa2f5b839db7e41e5bac355f68dd73f78b2daff640e8c
+  metadata.gz: 795266bc08c0155140c8281a50f60e45ebbfada3013b8d70032a7d1ce833097d
+  data.tar.gz: b119ea90f9cf8c4f9ef5a5af96c3d0c340a37eb7531605893be9e03738805bb9
 SHA512:
-  metadata.gz: de9dea929fb56da1d7c5d8e40e5d17069c0aa83bc124928d49f5e51a9cb0d87c09fc44613472394eeff5573347f9f9c7599623e54d88c65256a7efaaf0211d14
-  data.tar.gz: 875aa664d88b10f545fe346990619e914051dffc12141fb3e48fbaa7094b362fca4c6390abb007bb4d3be7faf255c21d6c2c08c7d1372b99da9744c0555123bb
+  metadata.gz: b5a1d165ab6c908dd13ed49eb15e5928a918e337b011375fac49a226656921e6900058975901d27270cf344053751c2cea5ea3365b435b8243283ebb12dc1eb2
+  data.tar.gz: 4be3074ac16caa9de211d34e31677e5e460c94423ee4aff1057318f3112b9f963d6a165f8966c487a31c457a1160f4a610999142217b7f33cc10844c0a2ef189

data/LICENCE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2015 Yatish Mehta
+Copyright (c) 2025 Yatish Mehta
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -1,141 +1,182 @@
 # Salesforce-Bulk-Api
+
 [![Gem Version](https://badge.fury.io/rb/salesforce_bulk_api.png)](http://badge.fury.io/rb/salesforce_bulk_api)
 
+## Table of Contents
+
+- [Overview](#overview)
+- [Installation](#installation)
+- [Authentication](#authentication)
+- [Usage](#usage)
+  - [Basic Operations](#basic-operations)
+  - [Job Management](#job-management)
+  - [Event Listening](#event-listening)
+  - [Retrieving Batch Records](#retrieving-batch-records)
+  - [API Call Throttling](#api-call-throttling)
+- [Contributing](#contributing)
+- [License](#license)
+
 ## Overview
 
-`SalesforceBulkApi` is a ruby wrapper for the Salesforce Bulk API.
-It is rewritten from [salesforce_bulk](https://github.com/jorgevaldivia/salesforce_bulk).
-It adds some missing features of `salesforce_bulk`.
+`SalesforceBulkApi` is a Ruby wrapper for the Salesforce Bulk API. It is rewritten from [salesforce_bulk](https://github.com/jorgevaldivia/salesforce_bulk) and adds several missing features, making it easier to perform bulk operations with Salesforce from Ruby applications.
 
-## How to use
+## Installation
 
-Using this gem is simple and straight forward.
+Add this line to your application's Gemfile:
 
-### Install
+```ruby
+gem 'salesforce_bulk_api'
+```
 
-   `gem install salesforce_bulk_api`
+And then execute:
 
-or add it to your Gemfile
+```
+bundle install
+```
 
-   `gem salesforce_bulk_api`
+Or install it directly:
 
-### Authenticate
+```
+gem install salesforce_bulk_api
+```
 
-You can authenticate with Salesforce using two gems, `databasedotcom` & `restforce`.
+## Authentication
 
-Please check the documentation of the respective gems to learn how to authenticate with Salesforce
+You can authenticate with Salesforce using either `databasedotcom` or `restforce` gems. Both support various authentication methods including username/password, OmniAuth, and OAuth2.
 
-[Databasedotcom](https://github.com/heroku/databasedotcom)
-[Restforce](https://github.com/ejholmes/restforce)
+Please refer to the documentation of these gems for detailed authentication options:
 
-You can use username password combo, OmniAuth, Oauth2
-You can use as many records possible in the Array. Governor limits are taken care of inside the gem.
+- [Databasedotcom](https://github.com/heroku/databasedotcom)
+- [Restforce](https://github.com/ejholmes/restforce)
+
+### Authentication Examples
+
+#### Using Databasedotcom:
 
 ```ruby
 require 'salesforce_bulk_api'
 
 client = Databasedotcom::Client.new(
-  :client_id =>  SFDC_APP_CONFIG["client_id"],
-  :client_secret => SFDC_APP_CONFIG["client_secret"]
+  client_id: SFDC_APP_CONFIG["client_id"],
+  client_secret: SFDC_APP_CONFIG["client_secret"]
 )
 client.authenticate(
-  :token => " ",
-  :instance_url => "http://na1.salesforce.com"
+  token: " ",
+  instance_url: "http://na1.salesforce.com"
 )
 
 salesforce = SalesforceBulkApi::Api.new(client)
 ```
 
-OR
+#### Using Restforce:
 
 ```ruby
 require 'salesforce_bulk_api'
+
 client = Restforce.new(
-  username:       SFDC_APP_CONFIG['SFDC_USERNAME'],
-  password:       SFDC_APP_CONFIG['SFDC_PASSWORD'],
+  username: SFDC_APP_CONFIG['SFDC_USERNAME'],
+  password: SFDC_APP_CONFIG['SFDC_PASSWORD'],
   security_token: SFDC_APP_CONFIG['SFDC_SECURITY_TOKEN'],
-  client_id:      SFDC_APP_CONFIG['SFDC_CLIENT_ID'],
-  client_secret:  SFDC_APP_CONFIG['SFDC_CLIENT_SECRET'].to_i,
-  host:           SFDC_APP_CONFIG['SFDC_HOST']
+  client_id: SFDC_APP_CONFIG['SFDC_CLIENT_ID'],
+  client_secret: SFDC_APP_CONFIG['SFDC_CLIENT_SECRET'],
+  host: SFDC_APP_CONFIG['SFDC_HOST']
 )
 client.authenticate!
 
 salesforce = SalesforceBulkApi::Api.new(client)
 ```
 
-### Sample operations:
+## Usage
+
+### Basic Operations
+
+#### Create/Insert Records
 
 ```ruby
-# Insert/Create
-# Add as many fields per record as needed.
-new_account = Hash["name" => "Test Account", "type" => "Other"]
-records_to_insert = Array.new
-# You can add as many records as you want here, just keep in mind that Salesforce has governor limits.
-records_to_insert.push(new_account)
+new_account = { "name" => "Test Account", "type" => "Other" }
+records_to_insert = [new_account]
 result = salesforce.create("Account", records_to_insert)
-puts "result is: #{result.inspect}"
+puts "Result: #{result.inspect}"
+```
 
-# Update
-updated_account = Hash["name" => "Test Account -- Updated", id => "a00A0001009zA2m"] # Nearly identical to an insert, but we need to pass the salesforce id.
-records_to_update = Array.new
-records_to_update.push(updated_account)
+#### Update Records
+
+```ruby
+updated_account = { "name" => "Test Account -- Updated", "id" => "a00A0001009zA2m" }
+records_to_update = [updated_account]
 salesforce.update("Account", records_to_update)
+```
+
+#### Upsert Records
+
+```ruby
+upserted_account = { "name" => "Test Account -- Upserted", "External_Field_Name" => "123456" }
+records_to_upsert = [upserted_account]
+salesforce.upsert("Account", records_to_upsert, "External_Field_Name")
+```
 
-# Upsert
-upserted_account = Hash["name" => "Test Account -- Upserted", "External_Field_Name" => "123456"] # Fields to be updated. External field must be included
-records_to_upsert = Array.new
-records_to_upsert.push(upserted_account)
-salesforce.upsert("Account", records_to_upsert, "External_Field_Name") # Note that upsert accepts an extra parameter for the external field name
+#### Delete Records
 
-# Delete
-deleted_account = Hash["id" => "a00A0001009zA2m"] # We only specify the id of the records to delete
-records_to_delete = Array.new
-records_to_delete.push(deleted_account)
+```ruby
+deleted_account = { "id" => "a00A0001009zA2m" }
+records_to_delete = [deleted_account]
 salesforce.delete("Account", records_to_delete)
+```
 
-# Query
-res = salesforce.query("Account", "select id, name, createddate from Account limit 3") # We just need to pass the sobject name and the query string
+#### Query Records
+
+```ruby
+res = salesforce.query("Account", "SELECT id, name, createddate FROM Account LIMIT 3")
 ```
 
-### Helpful methods:
+### Job Management
+
+You can check the status of a job using its ID:
 
 ```ruby
-# Check status of a job via #job_from_id
-job = salesforce.job_from_id('a00A0001009zA2m') # Returns a SalesforceBulkApi::Job instance
-puts "status is: #{job.check_job_status.inspect}"
+job = salesforce.job_from_id('a00A0001009zA2m')
+puts "Status: #{job.check_job_status.inspect}"
 ```
 
-### Listening to events:
+### Event Listening
+
+You can listen for job creation events:
 
 ```ruby
-# A job is created
-# Useful when you need to store the job_id before any work begins, then if you fail during a complex load scenario, you can wait for your
-# previous job(s) to finish.
 salesforce.on_job_created do |job|
   puts "Job #{job.job_id} created!"
 end
 ```
 
-### Fetching records from a batch
+### Retrieving Batch Records
+
+Fetch records from a specific batch in a job:
 
 ```ruby
 job_id = 'l02A0231009Za8m'
 batch_id = 'H24a0708089zA2J'
-salesforce.get_batch_records(job_id, batch_id)
-# => [{"Id"=>["RECORD_ID_1"], "AField__c"=>["123123"]},
-      {"Id"=>["RECORD_ID_2"], "AField__c"=>["123123"]},
-      {"Id"=>["RECORD_ID_3"], "AField__c"=>["123123"]}]
-
+records = salesforce.get_batch_records(job_id, batch_id)
 ```
 
-### Throttling API calls:
+### API Call Throttling
+
+You can control how frequently status checks are performed:
 
 ```ruby
-# By default, this gem (and maybe your app driving it) will query job/batch statuses at an unbounded rate.  We
-# can fix that, e.g.:
-salesforce.connection.set_status_throttle(30) # only check status of individual jobs/batches every 30 seconds
+# Set status check interval to 30 seconds
+salesforce.connection.set_status_throttle(30)
 ```
 
-## Contribute
+## Contributing
+
+We welcome contributions to improve this gem. Feel free to:
+
+1. Fork the repository
+2. Create your feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -am 'Add some amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Create a new Pull Request
+
+## License
 
-Feel to fork and send Pull request
+This project is licensed under the MIT License, Copyright (c) 2025 - see the [LICENCE](LICENCE) file for details.

data/Rakefile

@@ -1,3 +1,3 @@
-require 'rspec/core/rake_task'
-task :default => :spec
-RSpec::Core::RakeTask.new
+require "rspec/core/rake_task"
+task default: :spec
+RSpec::Core::RakeTask.new

data/lib/salesforce_bulk_api/concerns/throttling.rb

@@ -1,6 +1,5 @@
 module SalesforceBulkApi::Concerns
   module Throttling
-
     def throttles
       @throttles.dup
     end
@@ -20,10 +19,10 @@ module SalesforceBulkApi::Concerns
         key = extract_constraint_key_from(details, throttle_by_keys)
         last_request = limit_log[key]
 
-        if !last_request.nil? && only_if.call(details)
+        if last_request && only_if.call(details)
           seconds_since_last_request = Time.now.to_f - last_request.to_f
           need_to_wait_seconds = limit_seconds - seconds_since_last_request
-          sleep(need_to_wait_seconds) if need_to_wait_seconds > 0
+          sleep(need_to_wait_seconds) if need_to_wait_seconds.positive?
         end
 
         limit_log[key] = Time.now
@@ -33,28 +32,18 @@ module SalesforceBulkApi::Concerns
     private
 
     def extract_constraint_key_from(details, throttle_by_keys)
-      hash = {}
-      throttle_by_keys.each { |k| hash[k] = details[k] }
-      hash
+      throttle_by_keys.each_with_object({}) { |k, hash| hash[k] = details[k] }
     end
 
     def get_limit_log(prune_older_than)
-      @limits ||= Hash.new(0)
-
-      @limits.delete_if do |k, v|
-        v < prune_older_than
-      end
-
-      @limits
+      @limits ||= {}
+      @limits.delete_if { |_, v| v < prune_older_than }
     end
 
-    def throttle(details={})
+    def throttle(details = {})
       (@throttles || []).each do |callback|
-        args = [details]
-        args = args[0..callback.arity]
-        callback.call(*args)
+        callback.call(details)
       end
     end
-
   end
 end

data/lib/salesforce_bulk_api/connection.rb

@@ -1,98 +1,92 @@
-require 'timeout'
+require "timeout"
+require "net/https"
 
 module SalesforceBulkApi
   class Connection
     include Concerns::Throttling
 
-    LOGIN_HOST = 'login.salesforce.com'
+    LOGIN_HOST = "login.salesforce.com".freeze
+
+    attr_reader :session_id, :server_url, :instance, :instance_host
 
     def initialize(api_version, client)
       @client = client
       @api_version = api_version
       @path_prefix = "/services/async/#{@api_version}/"
+      @counters = Hash.new(0)
 
-      login()
-    end
-
-    def login()
-      client_type = @client.class.to_s
-      case client_type
-      when "Restforce::Data::Client"
-        @session_id = @client.options[:oauth_token]
-        @server_url = @client.options[:instance_url]
-      else
-        @session_id = @client.oauth_token
-        @server_url = @client.instance_url
-      end
-      @instance = parse_instance()
-      @instance_host = "#{@instance}.salesforce.com"
+      login
     end
 
     def post_xml(host, path, xml, headers)
-      host = host || @instance_host
-      if host != LOGIN_HOST # Not login, need to add session id to header
-        headers['X-SFDC-Session'] = @session_id
-        path = "#{@path_prefix}#{path}"
-      end
-      i = 0
-      begin
-        count :post
-        throttle(http_method: :post, path: path)
-        https(host).post(path, xml, headers).body
-      rescue
-        i += 1
-        if i < 3
-          puts "Request fail #{i}: Retrying #{path}"
-          retry
-        else
-          puts "FATAL: Request to #{path} failed three times."
-          raise
-        end
-      end
+      host ||= @instance_host
+      headers["X-SFDC-Session"] = @session_id unless host == LOGIN_HOST
+      path = "#{@path_prefix}#{path}" unless host == LOGIN_HOST
+
+      perform_request(:post, host, path, xml, headers)
     end
 
     def get_request(host, path, headers)
-      host = host || @instance_host
+      host ||= @instance_host
       path = "#{@path_prefix}#{path}"
-      if host != LOGIN_HOST # Not login, need to add session id to header
-        headers['X-SFDC-Session'] = @session_id;
-      end
-
-      count :get
-      throttle(http_method: :get, path: path)
-      https(host).get(path, headers).body
-    end
+      headers["X-SFDC-Session"] = @session_id unless host == LOGIN_HOST
 
-    def https(host)
-      req = Net::HTTP.new(host, 443)
-      req.use_ssl = true
-      req.verify_mode = OpenSSL::SSL::VERIFY_NONE
-      req
+      perform_request(:get, host, path, nil, headers)
     end
 
     def counters
       {
-        get: get_counters[:get],
-        post: get_counters[:post]
+        get: @counters[:get],
+        post: @counters[:post]
       }
     end
 
     private
 
-    def get_counters
-      @counters ||= Hash.new(0)
+    def login
+      client_type = @client.class.to_s
+      @session_id, @server_url = if client_type == "Restforce::Data::Client"
+        [@client.options[:oauth_token], @client.options[:instance_url]]
+      else
+        [@client.oauth_token, @client.instance_url]
+      end
+      @instance = parse_instance
+      @instance_host = "#{@instance}.salesforce.com"
     end
 
-    def count(http_method)
-      get_counters[http_method] += 1
+    def perform_request(method, host, path, body, headers)
+      retries = 0
+      begin
+        count(method)
+        throttle(http_method: method, path: path)
+        response = https(host).public_send(method, path, body, headers)
+        response.body
+      rescue => e
+        retries += 1
+        if retries < 3
+          puts "Request fail #{retries}: Retrying #{path}"
+          retry
+        else
+          puts "FATAL: Request to #{path} failed three times."
+          raise e
+        end
+      end
     end
 
-    def parse_instance()
-      @instance = @server_url.match(/https:\/\/[a-z]{2}[0-9]{1,2}\./).to_s.gsub("https://","").split(".")[0]
-      @instance = @server_url.split(".salesforce.com")[0].split("://")[1] if @instance.nil? || @instance.empty?
-      @instance
+    def https(host)
+      Net::HTTP.new(host, 443).tap do |http|
+        http.use_ssl = true
+        http.verify_mode = OpenSSL::SSL::VERIFY_NONE
+      end
     end
 
-  end
+    def count(http_method)
+      @counters[http_method] += 1
+    end
 
+    def parse_instance
+      instance = @server_url.match(%r{https://([a-z]{2}[0-9]{1,2})\.})&.captures&.first
+      instance || @server_url.split(".salesforce.com").first.split("://").last
+    end
+  end
 end