active_force 0.18.0 → 0.20.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6de1df55a3215360cd78f76fa3f55d05ca9e8f9f9603d5ab4257667a5fd961a5
4
- data.tar.gz: e85f2ae5bd075070adc261ec3a6d566eea2f9cf19c4d4423fa1b859cafdabcc4
3
+ metadata.gz: 1647fdc453090811c384514ae857c83b686dc966e2af5e47af3a6dc67e42059d
4
+ data.tar.gz: 515a98a5715c24359739d76730758b728a9cbf1b8c0cbf2b3c1bb5a355a21c9e
5
5
  SHA512:
6
- metadata.gz: 5268395d3fcc92a705b909026b6de6deb36b4c6a9b622d0ee63307ed08d16142e96abca2bafb902c55fca02955d75c34aa3348eb84fc14c47784d3548b492f89
7
- data.tar.gz: 3ea676afd55bb62601e211250c487a86e7902f1dee1f8b6fdc41410b903ce2052e5a4f893cc869798ba3bf5c1270aad2c84ea21c24dd137909e494e7c0564ff9
6
+ metadata.gz: 48115b7a0df0201a24f477ba2f658e692824337a78c0d288afd3c056b42fccaaf065e1e3ab296fa16bda1f2f2eaf82ff1322adcea6d9d6a9f82c2318cea15d98
7
+ data.tar.gz: a7b6dacca6775374b089cc9022462c470d8103d63792c7e7668ccd3e082446b0429a2d54d0e3392694c0e25ea25139916d11bc9f0080659faea6e7eaa0e41541
data/.gitignore CHANGED
@@ -15,3 +15,4 @@ spec/reports
15
15
  test/tmp
16
16
  test/version_tmp
17
17
  tmp
18
+ .idea
data/CHANGELOG.md CHANGED
@@ -2,6 +2,15 @@
2
2
 
3
3
  ## Not released
4
4
 
5
+ ## 0.20.0
6
+
7
+ - Change `.first` to not query the API if records have already been retrieved (https://github.com/Beyond-Finance/active_force/pull/73)
8
+ - Bugfix: Transform NULL values for SF Bulk API, which expects "#N/A" (https://github.com/Beyond-Finance/active_force/pull/74)
9
+
10
+ ## 0.19.0
11
+
12
+ - Bulk API methods. (https://github.com/Beyond-Finance/active_force/pull/65)
13
+
5
14
  ## 0.18.0
6
15
 
7
16
  - Fix eager loading of scoped associations. (https://github.com/Beyond-Finance/active_force/pull/67)
data/README.md CHANGED
@@ -194,7 +194,7 @@ Comment.includes(:post)
194
194
 
195
195
  It is possible to eager load multi level associations
196
196
 
197
- In order to utilize multi level eager loads, the API version should be set to 58.0 or higher when instantiating a Restforce client
197
+ In order to utilize multi level eager loads, the API version should be set to 58.0 or higher when instantiating a Restforce client
198
198
 
199
199
  ```ruby
200
200
  Restforce.new({api_version: '58.0'})
@@ -212,8 +212,8 @@ Comment.includes({post: {owner: :account}})
212
212
  Summing the values of a column:
213
213
  ```ruby
214
214
  Transaction.where(offer_id: 'ABD832024').sum(:amount)
215
- #=> This will query "SELECT SUM(Amount__c)
216
- # FROM Transaction__c
215
+ #=> This will query "SELECT SUM(Amount__c)
216
+ # FROM Transaction__c
217
217
  # WHERE offer_id = 'ABD832024'"
218
218
  ```
219
219
 
@@ -249,6 +249,22 @@ accounts = Account.where(web_enabled: 1).limit(2)
249
249
  with data from another API, and will only query the other API once.
250
250
  ```
251
251
 
252
+ ### Bulk Jobs
253
+
254
+ For more information about usage and limits of the Salesforce Bulk API see the [docs][4].
255
+
256
+ Convenience class methods have been added to `ActiveForce::SObject` to make it possible to utilize the Salesforce Bulk API v2.0.
257
+ The methods are: `bulk_insert_all`, `bulk_update_all`, & `bulk_delete_all`. They all expect input as an Array of attributes as a Hash:
258
+ ```ruby
259
+ [
260
+ { id: '11111111', attribute1: 'value1', attribute2: 'value2'},
261
+ { id: '22222222', attribute1: 'value3', attribute2: 'value4'},
262
+ ]
263
+ ```
264
+ The attributes will be mapped back to what's expected on the SF side automatically. The response is a `ActiveForce::Bulk::JobResult` object
265
+ which can access `successful` & `failed` results, has some `stats`, and the original `job` object that was used to create and process the
266
+ Bulk job.
267
+
252
268
  ### Model generator
253
269
 
254
270
  When using rails, you can generate a model with all the fields you have on your SFDC table by running:
@@ -268,4 +284,5 @@ When using rails, you can generate a model with all the fields you have on your
268
284
  [1]: http://www.salesforce.com
269
285
  [2]: https://github.com/ejholmes/restforce
270
286
  [3]: https://github.com/bkeepers/dotenv
287
+ [4]: https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/bulk_api_2_0.htm
271
288
 
@@ -0,0 +1,136 @@
1
+ require 'active_force/bulk/job_result'
2
+
3
+ module ActiveForce
4
+ module Bulk
5
+ class Job
6
+ attr_reader :operation, :records, :state, :object, :content_url
7
+ attr_accessor :id
8
+
9
+ STATES = {
10
+ Open: 'Open',
11
+ UploadComplete: 'UploadComplete',
12
+ InProgress: 'InProgress',
13
+ JobComplete: 'JobComplete',
14
+ Failed: 'Failed',
15
+ Aborted: 'Aborted',
16
+ Deleted: 'Deleted'
17
+ }.freeze
18
+
19
+ OPERATIONS = %i[insert delete hardDelete update upsert]
20
+
21
+ def initialize(operation:, object:, id: nil, records: nil)
22
+ @operation = operation.to_sym
23
+ @object = object
24
+ @id = id
25
+ @records = records
26
+ @state = nil
27
+ @content_url = nil
28
+ initialize_state_methods
29
+ end
30
+
31
+ def create
32
+ request_body = default_job_options.merge(operation: operation, object: object)
33
+ response = client.post("#{ingest_path}/", request_body)
34
+ update_attributes_from(response)
35
+ response
36
+ end
37
+
38
+ def upload
39
+ headers = {"Content-Type": 'text/csv'}
40
+ response = client.put(content_url, records.to_csv, headers)
41
+ response
42
+ end
43
+
44
+ def result
45
+ ActiveForce::Bulk::JobResult.new(job: self)
46
+ end
47
+
48
+ def self.run(...)
49
+ job = new(...)
50
+ job.create
51
+ job.upload
52
+ job.run
53
+ job
54
+ end
55
+
56
+ def failed_results
57
+ client.get("#{ingest_path}/#{id}/failedResults/")
58
+ end
59
+
60
+ def successful_results
61
+ client.get("#{ingest_path}/#{id}/successfulResults/")
62
+ end
63
+
64
+ def info
65
+ response = client.get("#{ingest_path}/#{id}")
66
+ update_attributes_from(response)
67
+ response
68
+ end
69
+
70
+ def run
71
+ change_state(STATES[:UploadComplete])
72
+ end
73
+
74
+ def abort
75
+ change_state(STATES[:Aborted])
76
+ end
77
+
78
+ def delete
79
+ response = client.delete("#{ingest_path}/#{id}")
80
+ response
81
+ end
82
+
83
+ def finished?
84
+ job_complete? || failed? || aborted?
85
+ end
86
+
87
+ private
88
+ attr_writer :state, :object, :operation, :content_url
89
+
90
+
91
+ def ingest_path
92
+ "/services/data/v#{client.options[:api_version]}/jobs/ingest"
93
+ end
94
+
95
+ def client
96
+ @client ||= ActiveForce.sfdc_client
97
+ end
98
+
99
+ def default_job_options
100
+ {
101
+ columnDelimiter: 'COMMA',
102
+ contentType: 'CSV',
103
+ lineEnding: 'LF',
104
+ }
105
+ end
106
+
107
+ def change_state(value)
108
+ request_body = {state: value}
109
+ headers = {"Content-Type": "application/json"}
110
+ response = client.patch("#{ingest_path}/#{id}", request_body, headers)
111
+ update_attributes_from(response)
112
+ response
113
+ end
114
+
115
+ def update_attributes_from(response)
116
+ return unless response.body.present?
117
+
118
+ %i[id state object operation contentUrl].each do |attr|
119
+ self.send("#{attr.to_s.underscore}=", response.body[attr.to_s]) if response.body[attr.to_s].present?
120
+ end
121
+ end
122
+
123
+ # Defines question methods for various states of the job, e.g. #open?, #in_progress?, etc.
124
+ def initialize_state_methods
125
+ STATES.values.each do |state|
126
+ state_method = <<-STATE_METHOD
127
+ def #{state.to_s.underscore}?
128
+ @state == '#{state}'
129
+ end
130
+ STATE_METHOD
131
+ self.class_eval(state_method)
132
+ end
133
+ end
134
+ end
135
+ end
136
+ end
@@ -0,0 +1,48 @@
1
+ module ActiveForce
2
+ module Bulk
3
+ class JobResult
4
+ attr_reader :job, :failed, :successful, :stats, :errors
5
+
6
+ def initialize(job:)
7
+ @job = job
8
+ @stats = result_from_job_info
9
+ @failed = failed_results
10
+ @successful = successful_results
11
+ @errors = errors_from_failed_results
12
+ end
13
+
14
+ def success?
15
+ failed.blank? && successful.present?
16
+ end
17
+
18
+ private
19
+ attr_writer :errors, :failed, :successful
20
+
21
+ def errors_from_failed_results
22
+ return [] if @stats[:number_records_failed].zero? || self.failed.blank?
23
+
24
+ self.errors = self.failed.pluck('sf__Error').uniq
25
+ end
26
+
27
+ def failed_results
28
+ return [] if @stats[:number_records_failed].zero?
29
+
30
+ response = job.failed_results
31
+ self.failed = CSV.parse(response.body, headers: true).map(&:to_h)
32
+ end
33
+
34
+ def successful_results
35
+ response = job.successful_results
36
+ self.successful = CSV.parse(response.body, headers: true).map(&:to_h)
37
+ end
38
+
39
+ def job_info
40
+ job.info
41
+ end
42
+
43
+ def result_from_job_info
44
+ job_info&.body.slice('numberRecordsProcessed', 'numberRecordsFailed', 'totalProcessingTime').transform_keys { |k| k.to_s.underscore.to_sym }
45
+ end
46
+ end
47
+ end
48
+ end
@@ -0,0 +1,42 @@
1
+ require 'csv'
2
+
3
+ module ActiveForce
4
+ module Bulk
5
+ class Records
6
+ NULL_VALUE = '#N/A'.freeze
7
+
8
+ attr_reader :headers, :data
9
+ def initialize(headers:, data:)
10
+ @headers = headers
11
+ @data = data
12
+ end
13
+
14
+ def to_csv
15
+ CSV.generate(String.new, headers: headers, write_headers: true) do |csv|
16
+ data.each { |row| csv << row }
17
+ end
18
+ end
19
+
20
+ def self.parse_from_attributes(records)
21
+ # Sorting ensures that the headers line up with the values for the CSV
22
+ headers = records.first.keys.sort.map(&:to_s)
23
+ data = records.map do |r|
24
+ r.transform_values { |v| transform_value_for_sf(v) }.sort.pluck(-1)
25
+ end
26
+ new(headers: headers, data: data)
27
+ end
28
+
29
+ # SF expects a special value for setting a column to be NULL.
30
+ def self.transform_value_for_sf(value)
31
+ case value
32
+ when NilClass
33
+ NULL_VALUE
34
+ when Time
35
+ value.iso8601
36
+ else
37
+ value.to_s
38
+ end
39
+ end
40
+ end
41
+ end
42
+ end
@@ -0,0 +1,47 @@
1
+ require 'active_force/bulk/job'
2
+ require 'active_force/bulk/records'
3
+
4
+ module ActiveForce
5
+ module Bulk
6
+ class TimeoutError < Timeout::Error; end
7
+ TIMEOUT_MESSAGE = 'Bulk job execution expired based on timeout of %{timeout} seconds'.freeze
8
+
9
+ def bulk_insert_all(attributes, options={})
10
+ run_bulk_job(:insert, attributes, options)
11
+ end
12
+
13
+ def bulk_update_all(attributes, options={})
14
+ run_bulk_job(:update, attributes, options)
15
+ end
16
+
17
+ def bulk_delete_all(attributes, options={})
18
+ run_bulk_job(:delete, attributes, options)
19
+ end
20
+
21
+ private
22
+
23
+ def default_options
24
+ {
25
+ timeout: 30,
26
+ sleep: 0.02 # short sleep so we can end our poll loop more quickly
27
+ }
28
+ end
29
+
30
+ def run_bulk_job(operation, attributes, options)
31
+ runtime_options = default_options.merge(options)
32
+ records = Records.parse_from_attributes(translate_to_sf(attributes))
33
+ job = Job.run(operation: operation, object: self.table_name, records: records)
34
+ Timeout.timeout(runtime_options[:timeout], ActiveForce::Bulk::TimeoutError, TIMEOUT_MESSAGE % runtime_options) do
35
+ until job.finished? do
36
+ job.info
37
+ sleep(runtime_options[:sleep])
38
+ end
39
+ end
40
+ job.result
41
+ end
42
+
43
+ def translate_to_sf(attributes)
44
+ attributes.map{ |r| self.mapping.translate_to_sf(r) }
45
+ end
46
+ end
47
+ end
@@ -78,7 +78,15 @@ module ActiveForce
78
78
  end
79
79
 
80
80
  def first
81
- limit 1
81
+ if @records
82
+ clone_and_set_instance_variables(
83
+ size: 1,
84
+ records: [@records.first],
85
+ decorated_records: [@decorated_records&.first]
86
+ )
87
+ else
88
+ limit(1)
89
+ end
82
90
  end
83
91
 
84
92
  def last(limit = 1)
@@ -126,9 +134,9 @@ module ActiveForce
126
134
 
127
135
  def clone_and_set_instance_variables instance_variable_hash={}
128
136
  clone = self.clone
129
- clone.instance_variable_set(:@decorated_records, nil)
130
- clone.instance_variable_set(:@records, nil)
131
- instance_variable_hash.each { |k,v| clone.instance_variable_set("@#{k.to_s}", v) }
137
+ { decorated_records: nil, records: nil }
138
+ .merge(instance_variable_hash)
139
+ .each { |k,v| clone.instance_variable_set("@#{k.to_s}", v) }
132
140
  clone
133
141
  end
134
142
  end
@@ -1,6 +1,7 @@
1
1
  require 'active_model'
2
2
  require 'active_force/active_query'
3
3
  require 'active_force/association'
4
+ require 'active_force/bulk'
4
5
  require 'active_force/mapping'
5
6
  require 'yaml'
6
7
  require 'forwardable'
@@ -19,6 +20,7 @@ module ActiveForce
19
20
  extend ActiveModel::Callbacks
20
21
  include ActiveModel::Serializers::JSON
21
22
  extend ActiveForce::Association
23
+ extend ActiveForce::Bulk
22
24
 
23
25
 
24
26
  define_model_callbacks :build, :create, :update, :save, :destroy
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module ActiveForce
4
- VERSION = '0.18.0'
4
+ VERSION = '0.20.0'
5
5
  end
data/lib/active_force.rb CHANGED
@@ -3,6 +3,7 @@ require 'active_model/type/salesforce/percent'
3
3
  require 'active_force/version'
4
4
  require 'active_force/sobject'
5
5
  require 'active_force/query'
6
+ require 'active_force/bulk'
6
7
 
7
8
  module ActiveForce
8
9
 
@@ -0,0 +1,116 @@
1
+ require 'spec_helper'
2
+
3
+ describe ActiveForce::Bulk::JobResult do
4
+ subject { described_class.new(job: job)}
5
+ let(:job) { instance_double(ActiveForce::Bulk::Job)}
6
+
7
+ let(:failed_results_response) do
8
+ Faraday::Response.new(status: 200, response_body: "\"sf__Id\",\"sf__Error\"\n\"45678\",\"Something failed.\"")
9
+ end
10
+
11
+ let(:successful_results_response) do
12
+ Faraday::Response.new(status: 200, response_body: "\"sf__Id\",\"sf__Error\"\n\"12354\",\"false\"\n")
13
+ end
14
+
15
+ let(:successful_info) do
16
+ {"id"=>"22222222222222",
17
+ "operation"=>"update",
18
+ "object"=>"FooBar__c",
19
+ "createdById"=>"11111111111111111",
20
+ "createdDate"=>"2023-09-22T16:39:09.000+0000",
21
+ "systemModstamp"=>"2023-09-22T16:39:13.000+0000",
22
+ "state"=>"JobComplete",
23
+ "concurrencyMode"=>"Parallel",
24
+ "contentType"=>"CSV",
25
+ "apiVersion"=>58.0,
26
+ "jobType"=>"V2Ingest",
27
+ "lineEnding"=>"LF",
28
+ "columnDelimiter"=>"COMMA",
29
+ "numberRecordsProcessed"=>1,
30
+ "numberRecordsFailed"=>0,
31
+ "retries"=>0,
32
+ "totalProcessingTime"=>713,
33
+ "apiActiveProcessingTime"=>323,
34
+ "apexProcessingTime"=>3}
35
+ end
36
+
37
+ let(:failed_info) do
38
+ {"id"=>"33333333333333",
39
+ "operation"=>"update",
40
+ "object"=>"FooBar__c",
41
+ "createdById"=>"11111111111111111",
42
+ "createdDate"=>"2023-09-22T16:39:09.000+0000",
43
+ "systemModstamp"=>"2023-09-22T16:39:13.000+0000",
44
+ "state"=>"JobComplete",
45
+ "concurrencyMode"=>"Parallel",
46
+ "contentType"=>"CSV",
47
+ "apiVersion"=>58.0,
48
+ "jobType"=>"V2Ingest",
49
+ "lineEnding"=>"LF",
50
+ "columnDelimiter"=>"COMMA",
51
+ "numberRecordsProcessed"=>1,
52
+ "numberRecordsFailed"=>1,
53
+ "retries"=>0,
54
+ "totalProcessingTime"=>713,
55
+ "apiActiveProcessingTime"=>323,
56
+ "apexProcessingTime"=>3}
57
+ end
58
+
59
+ let(:successful_job_info_response) do
60
+ Faraday::Response.new(status: 200, response_body: successful_info)
61
+ end
62
+
63
+ let(:failed_job_info_response) do
64
+ Faraday::Response.new(status: 200, response_body: failed_info)
65
+ end
66
+
67
+ before do
68
+ allow(job).to receive(:failed_results).and_return(failed_results_response)
69
+ allow(job).to receive(:successful_results).and_return(successful_results_response)
70
+ allow(job).to receive(:info).and_return(successful_job_info_response)
71
+ end
72
+
73
+ describe '::new' do
74
+ it 'uses the passed in job object to pull job info' do
75
+ expect(subject.stats[:total_processing_time]).to eq successful_info['totalProcessingTime']
76
+ expect(subject.stats[:number_records_processed]).to eq successful_info['numberRecordsProcessed']
77
+ expect(subject.stats[:number_records_failed]).to eq successful_info['numberRecordsFailed']
78
+ expect(subject.failed).to be_empty
79
+ expect(subject.successful.size).to eq 1
80
+ end
81
+ end
82
+
83
+ describe '#success?' do
84
+ context 'when no failed results' do
85
+ it 'returns true' do
86
+ expect(subject.success?).to be true
87
+ end
88
+ end
89
+ context 'when there are failed results' do
90
+ before do
91
+ allow(job).to receive(:info).and_return(failed_job_info_response)
92
+ end
93
+ it 'returns false' do
94
+ expect(subject.success?).to be false
95
+ end
96
+ end
97
+ end
98
+
99
+ describe '#errors' do
100
+ context 'when there are failed results' do
101
+ before do
102
+ allow(job).to receive(:info).and_return(failed_job_info_response)
103
+ end
104
+ it 'returns an array of errors' do
105
+ expect(subject.errors).to eq ['Something failed.']
106
+ end
107
+ end
108
+
109
+ context 'when there are no failed results' do
110
+ it 'returns an empty array' do
111
+ expect(subject.errors).to eq []
112
+ end
113
+ end
114
+ end
115
+ end
116
+
@@ -0,0 +1,113 @@
1
+ require 'spec_helper'
2
+
3
+ describe ActiveForce::Bulk::Job do
4
+ subject { described_class.new(**args)}
5
+ let(:args) { { operation: operation, object: object, id: id, records: records } }
6
+ let(:operation) { described_class::OPERATIONS.sample }
7
+ let(:object) { 'Whizbang__c' }
8
+ let(:id) { nil }
9
+ let(:records) { instance_double(ActiveForce::Bulk::Records) }
10
+ let(:sfdc_client) { spy(ActiveForce.sfdc_client.class)}
11
+ let(:state) { 'Open' }
12
+ let(:api_version) { '58.0' }
13
+ let(:ingest_url) { "/services/data/v#{api_version}/jobs/ingest" }
14
+
15
+ before do
16
+ allow(ActiveForce).to receive(:sfdc_client).and_return(sfdc_client)
17
+ allow(sfdc_client).to receive(:options).and_return({api_version: api_version})
18
+ allow(records).to receive(:to_csv)
19
+ end
20
+
21
+ describe '#finished?' do
22
+ let(:finished_state) { described_class::STATES.slice(:JobComplete, :Failed, :Aborted).values.sample }
23
+ let(:unfinished_state) { described_class::STATES.except(:JobComplete, :Failed, :Aborted).values.sample }
24
+
25
+ context 'when job is in one of the finished states' do
26
+ it 'returns true' do
27
+ job = subject
28
+ job.instance_variable_set(:@state, finished_state)
29
+ expect(job.finished?).to be true
30
+ end
31
+ end
32
+
33
+ context 'when job is NOT in one of the finished states' do
34
+ it 'returns false' do
35
+ job = subject
36
+ job.instance_variable_set(:@state, unfinished_state)
37
+ expect(job.finished?).to be false
38
+ end
39
+ end
40
+ end
41
+
42
+ describe '::run' do
43
+ let(:job) { spy(described_class)}
44
+
45
+ before do
46
+ allow(described_class).to receive(:new).and_return(job)
47
+ end
48
+
49
+ it 'creates, uploads, and runs the job via the SF API' do
50
+ described_class.run(**args)
51
+ expect(job).to have_received(:create)
52
+ expect(job).to have_received(:upload)
53
+ expect(job).to have_received(:run)
54
+ end
55
+ end
56
+
57
+ describe '#run' do
58
+ it 'PATCHs the job state to SF API to run the job' do
59
+ subject.run
60
+ expect(sfdc_client).to have_received(:patch).with("#{ingest_url}/#{id}", {state: described_class::STATES[:UploadComplete]}, anything)
61
+ end
62
+ end
63
+
64
+ describe '#abort' do
65
+ it 'PATCHs the job state to SF API to abort the job' do
66
+ subject.abort
67
+ expect(sfdc_client).to have_received(:patch).with("#{ingest_url}/#{id}", {state: described_class::STATES[:Aborted]}, anything)
68
+ end
69
+ end
70
+
71
+ describe '#delete' do
72
+ it 'DELETEs the job from SF API' do
73
+ subject.delete
74
+ expect(sfdc_client).to have_received(:delete).with("#{ingest_url}/#{id}")
75
+ end
76
+ end
77
+
78
+ describe '#failed_results' do
79
+ it 'GETs job info from SF API' do
80
+ subject.failed_results
81
+ expect(sfdc_client).to have_received(:get).with("#{ingest_url}/#{id}/failedResults/")
82
+ end
83
+ end
84
+
85
+ describe '#successful_results' do
86
+ it 'GETs successful results from SF API' do
87
+ subject.successful_results
88
+ expect(sfdc_client).to have_received(:get).with("#{ingest_url}/#{id}/successfulResults/")
89
+ end
90
+ end
91
+
92
+ describe '#info' do
93
+ it 'GETs job info from SF API' do
94
+ subject.info
95
+ expect(sfdc_client).to have_received(:get).with("#{ingest_url}/#{id}")
96
+ end
97
+ end
98
+
99
+ describe '#upload' do
100
+ it 'PUTs CSV to SF API to upload records to job' do
101
+ subject.upload
102
+ expect(sfdc_client).to have_received(:put).with(subject.content_url, anything, hash_including("Content-Type": 'text/csv'))
103
+ end
104
+ end
105
+
106
+ describe '#create' do
107
+ it 'POSTs to SF API to create a job' do
108
+ subject.create
109
+ expect(sfdc_client).to have_received(:post).with("#{ingest_url}/", hash_including(operation: operation, object: object))
110
+ end
111
+ end
112
+ end
113
+
@@ -0,0 +1,54 @@
1
+ require 'spec_helper'
2
+
3
+ describe ActiveForce::Bulk::Records do
4
+ subject { described_class.new(headers: headers, data: data)}
5
+ let(:headers) { %w[header1 header2] }
6
+ let(:data) do
7
+ [
8
+ %w[value1 value2],
9
+ %w[value3 value4],
10
+ ]
11
+ end
12
+
13
+ describe '#to_csv' do
14
+ it 'returns CSV with headers' do
15
+ expect(subject.to_csv).to eq "header1,header2\nvalue1,value2\nvalue3,value4\n"
16
+ end
17
+ end
18
+
19
+ describe '::parse_from_attributes' do
20
+ subject { described_class.parse_from_attributes(attributes) }
21
+ let(:attributes) do
22
+ [
23
+ { header1: 'value1', header2: 'value2'},
24
+ { header1: 'value3', header2: 'value4'},
25
+ ]
26
+ end
27
+ it 'parses array of hash attributes into Records object with headers and data' do
28
+ records = subject
29
+ expect(records).to be_a described_class
30
+ expect(records.headers).to eq headers
31
+ expect(records.data).to eq data
32
+ end
33
+
34
+ context 'when there are NULL values' do
35
+ let(:attributes) do
36
+ [
37
+ { header1: nil, header2: 'value2'},
38
+ { header1: 'value3', header2: nil},
39
+ ]
40
+ end
41
+ let(:data) do
42
+ [
43
+ %w[#N/A value2],
44
+ %w[value3 #N/A],
45
+ ]
46
+ end
47
+
48
+ it 'substitutes the expected SF value for NULL' do
49
+ records = subject
50
+ expect(records.data).to eq data
51
+ end
52
+ end
53
+ end
54
+ end
@@ -0,0 +1,110 @@
1
+ require 'spec_helper'
2
+
3
+ # Set up a new SObject to test the mixin.
4
+ class FooBarSObject < ActiveForce::SObject
5
+ field :id, from: 'Id'
6
+ field :baz_id, from: 'Baz_id__c'
7
+ end
8
+
9
+ describe ActiveForce::Bulk do
10
+ subject { FooBarSObject }
11
+ let(:attributes) do
12
+ [
13
+ {id: 1, baz_id: 1 },
14
+ {id: 2, baz_id: 1 },
15
+ {id: 3, baz_id: 2 },
16
+ ]
17
+ end
18
+ let(:job) { double(ActiveForce::Bulk::Job) }
19
+ let(:timeout_message) { /Bulk job execution expired based on timeout of/ }
20
+ before do
21
+ allow(job).to receive(:result)
22
+ end
23
+
24
+ describe '::bulk_insert_all' do
25
+ before do
26
+ allow(ActiveForce::Bulk::Job).to receive(:run).with(hash_including(operation: :insert)).and_return(job)
27
+ end
28
+
29
+ it 'runs a bulk insert job' do
30
+ allow(job).to receive(:finished?).and_return(true)
31
+ subject.bulk_insert_all(attributes)
32
+ end
33
+
34
+ context 'when job takes a while to run' do
35
+ it 'polls job info until job is finished' do
36
+ allow(job).to receive(:finished?).and_return(false, false, true)
37
+ allow(job).to receive(:info)
38
+ subject.bulk_insert_all(attributes)
39
+ end
40
+ end
41
+
42
+ context 'when job run exceeds timeout' do
43
+ it 'raises error' do
44
+ allow(job).to receive(:finished?).and_return(false)
45
+ allow(job).to receive(:info)
46
+ expect do
47
+ subject.bulk_insert_all(attributes, timeout: 0.1)
48
+ end.to raise_error(ActiveForce::Bulk::TimeoutError, timeout_message)
49
+ end
50
+ end
51
+ end
52
+
53
+ describe '::bulk_update_all' do
54
+ before do
55
+ allow(ActiveForce::Bulk::Job).to receive(:run).with(hash_including(operation: :update)).and_return(job)
56
+ end
57
+
58
+ it 'runs a bulk insert job' do
59
+ allow(job).to receive(:finished?).and_return(true)
60
+ subject.bulk_update_all(attributes)
61
+ end
62
+
63
+ context 'when job takes a while to run' do
64
+ it 'polls job info until job is finished' do
65
+ allow(job).to receive(:finished?).and_return(false, false, true)
66
+ allow(job).to receive(:info)
67
+ subject.bulk_update_all(attributes)
68
+ end
69
+ end
70
+
71
+ context 'when job run exceeds timeout' do
72
+ it 'raises error' do
73
+ allow(job).to receive(:finished?).and_return(false)
74
+ allow(job).to receive(:info)
75
+ expect do
76
+ subject.bulk_update_all(attributes, timeout: 0.1)
77
+ end.to raise_error(ActiveForce::Bulk::TimeoutError, timeout_message)
78
+ end
79
+ end
80
+ end
81
+
82
+ describe '::bulk_delete_all' do
83
+ before do
84
+ allow(ActiveForce::Bulk::Job).to receive(:run).with(hash_including(operation: :delete)).and_return(job)
85
+ end
86
+
87
+ it 'runs a bulk insert job' do
88
+ allow(job).to receive(:finished?).and_return(true)
89
+ subject.bulk_delete_all(attributes)
90
+ end
91
+
92
+ context 'when job takes a while to run' do
93
+ it 'polls job info until job is finished' do
94
+ allow(job).to receive(:finished?).and_return(false, false, true)
95
+ allow(job).to receive(:info)
96
+ subject.bulk_delete_all(attributes)
97
+ end
98
+ end
99
+
100
+ context 'when job run exceeds timeout' do
101
+ it 'raises error' do
102
+ allow(job).to receive(:finished?).and_return(false)
103
+ allow(job).to receive(:info)
104
+ expect do
105
+ subject.bulk_delete_all(attributes, timeout: 0.1)
106
+ end.to raise_error(ActiveForce::Bulk::TimeoutError, timeout_message)
107
+ end
108
+ end
109
+ end
110
+ end
@@ -165,6 +165,22 @@ describe ActiveForce::Query do
165
165
  expect(query.to_s).to eq "SELECT Id, name, etc FROM table_name"
166
166
  expect(new_query.to_s).to eq 'SELECT Id, name, etc FROM table_name LIMIT 1'
167
167
  end
168
+
169
+ it "does not query if records have already been fetched" do
170
+ query = ActiveForce::Query.new 'table_name'
171
+ query.instance_variable_set(:@records, %w[foo bar])
172
+ query.instance_variable_set(:@decorated_records, %w[foo bar])
173
+ expect(query).not_to receive(:limit)
174
+ expect(query).to receive(:clone_and_set_instance_variables).with(size: 1, records: ['foo'], decorated_records: ['foo'])
175
+ query.first
176
+ end
177
+
178
+ it 'queries the api if it has not been queried yet' do
179
+ query = ActiveForce::Query.new 'table_name'
180
+ query.instance_variable_set(:@records, nil)
181
+ expect(query).to receive(:limit)
182
+ query.first
183
+ end
168
184
  end
169
185
 
170
186
  describe '.last' do
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: active_force
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.18.0
4
+ version: 0.20.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Eloy Espinaco
@@ -11,7 +11,7 @@ authors:
11
11
  autorequire:
12
12
  bindir: bin
13
13
  cert_chain: []
14
- date: 2023-10-02 00:00:00.000000000 Z
14
+ date: 2023-11-27 00:00:00.000000000 Z
15
15
  dependencies:
16
16
  - !ruby/object:Gem::Dependency
17
17
  name: activemodel
@@ -155,6 +155,10 @@ files:
155
155
  - lib/active_force/association/has_many_association.rb
156
156
  - lib/active_force/association/has_one_association.rb
157
157
  - lib/active_force/association/relation_model_builder.rb
158
+ - lib/active_force/bulk.rb
159
+ - lib/active_force/bulk/job.rb
160
+ - lib/active_force/bulk/job_result.rb
161
+ - lib/active_force/bulk/records.rb
158
162
  - lib/active_force/field.rb
159
163
  - lib/active_force/mapping.rb
160
164
  - lib/active_force/query.rb
@@ -170,6 +174,10 @@ files:
170
174
  - spec/active_force/active_query_spec.rb
171
175
  - spec/active_force/association/relation_model_builder_spec.rb
172
176
  - spec/active_force/association_spec.rb
177
+ - spec/active_force/bulk/job_result_spec.rb
178
+ - spec/active_force/bulk/job_spec.rb
179
+ - spec/active_force/bulk/record_spec.rb
180
+ - spec/active_force/bulk_spec.rb
173
181
  - spec/active_force/callbacks_spec.rb
174
182
  - spec/active_force/field_spec.rb
175
183
  - spec/active_force/mapping_spec.rb
@@ -216,6 +224,10 @@ test_files:
216
224
  - spec/active_force/active_query_spec.rb
217
225
  - spec/active_force/association/relation_model_builder_spec.rb
218
226
  - spec/active_force/association_spec.rb
227
+ - spec/active_force/bulk/job_result_spec.rb
228
+ - spec/active_force/bulk/job_spec.rb
229
+ - spec/active_force/bulk/record_spec.rb
230
+ - spec/active_force/bulk_spec.rb
219
231
  - spec/active_force/callbacks_spec.rb
220
232
  - spec/active_force/field_spec.rb
221
233
  - spec/active_force/mapping_spec.rb