@simplysf/simply-data 2.3.27 → 2.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +58 -6
- package/lib/commands/simply/data/file/upload.d.ts +1 -0
- package/lib/commands/simply/data/file/upload.js +4 -0
- package/lib/commands/simply/data/file/upload.js.map +1 -1
- package/lib/commands/simply/data/files/download.d.ts +1 -0
- package/lib/commands/simply/data/files/download.js +6 -0
- package/lib/commands/simply/data/files/download.js.map +1 -1
- package/lib/commands/simply/data/files/upload.d.ts +1 -0
- package/lib/commands/simply/data/files/upload.js +9 -0
- package/lib/commands/simply/data/files/upload.js.map +1 -1
- package/lib/common/apiBudgetFlag.d.ts +26 -0
- package/lib/common/apiBudgetFlag.js +61 -0
- package/lib/common/apiBudgetFlag.js.map +1 -0
- package/lib/common/apiCost.d.ts +23 -0
- package/lib/common/apiCost.js +41 -0
- package/lib/common/apiCost.js.map +1 -0
- package/lib/common/contentVersionUtils.js +29 -0
- package/lib/common/contentVersionUtils.js.map +1 -1
- package/lib/common/countCsvRows.d.ts +15 -0
- package/lib/common/countCsvRows.js +42 -0
- package/lib/common/countCsvRows.js.map +1 -0
- package/messages/apiBudget.md +17 -0
- package/package.json +3 -3
package/README.md
CHANGED
|
@@ -31,7 +31,7 @@ Upload a file to a Salesforce org.
|
|
|
31
31
|
```
|
|
32
32
|
USAGE
|
|
33
33
|
$ sf simply data file upload -o <value> --file-path <value> [--json] [--flags-dir <value>] [--api-version <value>]
|
|
34
|
-
[--first-publish-location-id <value>] [--title <value>]
|
|
34
|
+
[--max-api-usage <value>] [--first-publish-location-id <value>] [--title <value>]
|
|
35
35
|
|
|
36
36
|
FLAGS
|
|
37
37
|
-o, --target-org=<value> (required) Username or alias of the target org. Not required if the
|
|
@@ -40,6 +40,8 @@ FLAGS
|
|
|
40
40
|
--file-path=<value> (required) Path to the file to upload. May be relative or absolute; only the
|
|
41
41
|
file's name is sent to the org.
|
|
42
42
|
--first-publish-location-id=<value> Specify a record Id that the file should be linked to.
|
|
43
|
+
--max-api-usage=<value> [default: 20] Maximum percentage of the org's remaining API requests this run
|
|
44
|
+
may consume.
|
|
43
45
|
--title=<value> Specify the title for the file being uploaded.
|
|
44
46
|
|
|
45
47
|
GLOBAL FLAGS
|
|
@@ -58,9 +60,25 @@ EXAMPLES
|
|
|
58
60
|
$ sf simply data file upload --file-path fileToUpload.txt --target-org myTargetOrg
|
|
59
61
|
|
|
60
62
|
$ sf simply data file upload --file-path fileToUpload.txt --first-publish-location-id 0019000000DmehK --target-org myTargetOrg
|
|
63
|
+
|
|
64
|
+
FLAG DESCRIPTIONS
|
|
65
|
+
--max-api-usage=<value> Maximum percentage of the org's remaining API requests this run may consume.
|
|
66
|
+
|
|
67
|
+
Checked before any request is made, so a run that would exceed its budget stops without doing partial work. The
|
|
68
|
+
percentage applies to the requests the org has left today, not to its daily maximum — an org that has already used
|
|
69
|
+
most of its allocation gets a proportionally smaller budget.
|
|
70
|
+
|
|
71
|
+
Note that uploading a file costs two API requests, not one: the upload itself, and a follow-up query for the
|
|
72
|
+
resulting ContentDocumentId.
|
|
73
|
+
|
|
74
|
+
A run that cannot finish within the org's remaining requests is refused regardless of this value. To allow a larger
|
|
75
|
+
share, raise it; to allow the maximum, pass 100.
|
|
76
|
+
|
|
77
|
+
If the org's remaining allocation can't be read — reading it falls back to the limits API, which needs the "View
|
|
78
|
+
Setup and Configuration" permission — the command warns and proceeds rather than failing.
|
|
61
79
|
```
|
|
62
80
|
|
|
63
|
-
_See code: [lib/commands/simply/data/file/upload.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.
|
|
81
|
+
_See code: [lib/commands/simply/data/file/upload.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.4.1/packages/simply-data/lib/commands/simply/data/file/upload.js)_
|
|
64
82
|
|
|
65
83
|
## `sf simply data files download`
|
|
66
84
|
|
|
@@ -69,12 +87,14 @@ Download files from a Salesforce org.
|
|
|
69
87
|
```
|
|
70
88
|
USAGE
|
|
71
89
|
$ sf simply data files download -o <value> --where-content-version <value> [--json] [--flags-dir <value>] [--api-version
|
|
72
|
-
<value>] [--max-parallel-jobs <value>]
|
|
90
|
+
<value>] [--max-api-usage <value>] [--max-parallel-jobs <value>]
|
|
73
91
|
|
|
74
92
|
FLAGS
|
|
75
93
|
-o, --target-org=<value> (required) Username or alias of the target org. Not required if the `target-org`
|
|
76
94
|
configuration variable is already set.
|
|
77
95
|
--api-version=<value> Override the api version used for api requests made by this command
|
|
96
|
+
--max-api-usage=<value> [default: 20] Maximum percentage of the org's remaining API requests this run may
|
|
97
|
+
consume.
|
|
78
98
|
--max-parallel-jobs=<value> [default: 1] Maximum number of parallel jobs.
|
|
79
99
|
--where-content-version=<value> (required) WHERE clause for ContentVersion query.
|
|
80
100
|
|
|
@@ -95,6 +115,21 @@ EXAMPLES
|
|
|
95
115
|
$ sf simply data files download --where-content-version "IsLatest=true" --max-parallel-jobs 5 --target-org myTargetOrg
|
|
96
116
|
|
|
97
117
|
FLAG DESCRIPTIONS
|
|
118
|
+
--max-api-usage=<value> Maximum percentage of the org's remaining API requests this run may consume.
|
|
119
|
+
|
|
120
|
+
Checked before any request is made, so a run that would exceed its budget stops without doing partial work. The
|
|
121
|
+
percentage applies to the requests the org has left today, not to its daily maximum — an org that has already used
|
|
122
|
+
most of its allocation gets a proportionally smaller budget.
|
|
123
|
+
|
|
124
|
+
Note that uploading a file costs two API requests, not one: the upload itself, and a follow-up query for the
|
|
125
|
+
resulting ContentDocumentId.
|
|
126
|
+
|
|
127
|
+
A run that cannot finish within the org's remaining requests is refused regardless of this value. To allow a larger
|
|
128
|
+
share, raise it; to allow the maximum, pass 100.
|
|
129
|
+
|
|
130
|
+
If the org's remaining allocation can't be read — reading it falls back to the limits API, which needs the "View
|
|
131
|
+
Setup and Configuration" permission — the command warns and proceeds rather than failing.
|
|
132
|
+
|
|
98
133
|
--max-parallel-jobs=<value> Maximum number of parallel jobs.
|
|
99
134
|
|
|
100
135
|
By default the plugin will only process a single file download at a time. You can increase this value to allow for
|
|
@@ -105,7 +140,7 @@ FLAG DESCRIPTIONS
|
|
|
105
140
|
Provide a WHERE clause to allow the plugin to specify which ContentVersion records should be downloaded.
|
|
106
141
|
```
|
|
107
142
|
|
|
108
|
-
_See code: [lib/commands/simply/data/files/download.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.
|
|
143
|
+
_See code: [lib/commands/simply/data/files/download.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.4.1/packages/simply-data/lib/commands/simply/data/files/download.js)_
|
|
109
144
|
|
|
110
145
|
## `sf simply data files upload`
|
|
111
146
|
|
|
@@ -114,13 +149,15 @@ Upload files to a Salesforce org.
|
|
|
114
149
|
```
|
|
115
150
|
USAGE
|
|
116
151
|
$ sf simply data files upload -o <value> --file-path <value> [--json] [--flags-dir <value>] [--api-version <value>]
|
|
117
|
-
[--max-parallel-jobs <value>]
|
|
152
|
+
[--max-api-usage <value>] [--max-parallel-jobs <value>]
|
|
118
153
|
|
|
119
154
|
FLAGS
|
|
120
155
|
-o, --target-org=<value> (required) Username or alias of the target org. Not required if the `target-org`
|
|
121
156
|
configuration variable is already set.
|
|
122
157
|
--api-version=<value> Override the api version used for api requests made by this command
|
|
123
158
|
--file-path=<value> (required) Path to the csv file that specifies the upload.
|
|
159
|
+
--max-api-usage=<value> [default: 20] Maximum percentage of the org's remaining API requests this run may
|
|
160
|
+
consume.
|
|
124
161
|
--max-parallel-jobs=<value> [default: 1] Maximum number of parallel jobs.
|
|
125
162
|
|
|
126
163
|
GLOBAL FLAGS
|
|
@@ -148,13 +185,28 @@ FLAG DESCRIPTIONS
|
|
|
148
185
|
to the org — Salesforce stores it as the ContentVersion's PathOnClient and derives FileExtension and FileType from
|
|
149
186
|
it — so the local directory the file came from is never uploaded.
|
|
150
187
|
|
|
188
|
+
--max-api-usage=<value> Maximum percentage of the org's remaining API requests this run may consume.
|
|
189
|
+
|
|
190
|
+
Checked before any request is made, so a run that would exceed its budget stops without doing partial work. The
|
|
191
|
+
percentage applies to the requests the org has left today, not to its daily maximum — an org that has already used
|
|
192
|
+
most of its allocation gets a proportionally smaller budget.
|
|
193
|
+
|
|
194
|
+
Note that uploading a file costs two API requests, not one: the upload itself, and a follow-up query for the
|
|
195
|
+
resulting ContentDocumentId.
|
|
196
|
+
|
|
197
|
+
A run that cannot finish within the org's remaining requests is refused regardless of this value. To allow a larger
|
|
198
|
+
share, raise it; to allow the maximum, pass 100.
|
|
199
|
+
|
|
200
|
+
If the org's remaining allocation can't be read — reading it falls back to the limits API, which needs the "View
|
|
201
|
+
Setup and Configuration" permission — the command warns and proceeds rather than failing.
|
|
202
|
+
|
|
151
203
|
--max-parallel-jobs=<value> Maximum number of parallel jobs.
|
|
152
204
|
|
|
153
205
|
By default the plugin will only process a single file upload at a time. You can increase this value to allow for
|
|
154
206
|
quasi concurrent uploads. Please note that setting this value too high can cause performance issues.
|
|
155
207
|
```
|
|
156
208
|
|
|
157
|
-
_See code: [lib/commands/simply/data/files/upload.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.
|
|
209
|
+
_See code: [lib/commands/simply/data/files/upload.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.4.1/packages/simply-data/lib/commands/simply/data/files/upload.js)_
|
|
158
210
|
<!-- commandsstop -->
|
|
159
211
|
|
|
160
212
|
## License
|
|
@@ -9,6 +9,7 @@ export default class DataFileUpload extends SfCommand<ContentVersion> {
|
|
|
9
9
|
'file-path': import("@oclif/core/interfaces").OptionFlag<string, import("@oclif/core/interfaces").CustomOptions>;
|
|
10
10
|
'first-publish-location-id': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
|
|
11
11
|
title: import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
|
|
12
|
+
'max-api-usage': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
12
13
|
'api-version': import("@oclif/core/interfaces").OptionFlag<string | undefined>;
|
|
13
14
|
'target-org': import("@oclif/core/interfaces").OptionFlag<import("@salesforce/core").Org>;
|
|
14
15
|
'flags-dir': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
|
|
@@ -17,6 +17,8 @@ import { Messages } from '@salesforce/core';
|
|
|
17
17
|
import { SfCommand, Flags } from '@salesforce/sf-plugins-core';
|
|
18
18
|
import { requireConnection, targetOrgFlags } from '@simplysf/simply-plugin-kit';
|
|
19
19
|
import { uploadContentVersion } from '../../../../common/contentVersionUtils.js';
|
|
20
|
+
import { apiBudgetFlags, assertApiBudget } from '../../../../common/apiBudgetFlag.js';
|
|
21
|
+
import { REQUESTS_PER_UPLOAD } from '../../../../common/apiCost.js';
|
|
20
22
|
Messages.importMessagesDirectoryFromMetaUrl(import.meta.url);
|
|
21
23
|
const messages = Messages.loadMessages('@simplysf/simply-data', 'simply.data.file.upload');
|
|
22
24
|
/** Uploads a single local file to a Salesforce org as a `ContentVersion`. */
|
|
@@ -27,6 +29,7 @@ export default class DataFileUpload extends SfCommand {
|
|
|
27
29
|
static flags = {
|
|
28
30
|
...SfCommand.baseFlags,
|
|
29
31
|
...targetOrgFlags,
|
|
32
|
+
...apiBudgetFlags,
|
|
30
33
|
'file-path': Flags.directory({
|
|
31
34
|
summary: messages.getMessage('flags.file-path.summary'),
|
|
32
35
|
required: true,
|
|
@@ -43,6 +46,7 @@ export default class DataFileUpload extends SfCommand {
|
|
|
43
46
|
const { flags } = await this.parse(DataFileUpload);
|
|
44
47
|
// Authorize to the target org
|
|
45
48
|
const targetOrgConnection = requireConnection(flags);
|
|
49
|
+
await assertApiBudget(targetOrgConnection, REQUESTS_PER_UPLOAD, flags['max-api-usage'], (message) => this.warn(message));
|
|
46
50
|
this.spinner.start('Uploading file', '', { stdout: true });
|
|
47
51
|
const contentVersion = await uploadContentVersion(targetOrgConnection, flags['file-path'], flags['title'], flags['first-publish-location-id']);
|
|
48
52
|
this.spinner.stop();
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"upload.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/file/upload.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;
|
|
1
|
+
{"version":3,"file":"upload.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/file/upload.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AACjF,OAAO,EAAE,cAAc,EAAE,eAAe,EAAE,MAAM,qCAAqC,CAAC;AACtF,OAAO,EAAE,mBAAmB,EAAE,MAAM,+BAA+B,CAAC;AAGpE,QAAQ,CAAC,kCAAkC,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAC7D,MAAM,QAAQ,GAAG,QAAQ,CAAC,YAAY,CAAC,uBAAuB,EAAE,yBAAyB,CAAC,CAAC;AAE3F,6EAA6E;AAC7E,MAAM,CAAC,OAAO,OAAO,cAAe,SAAQ,SAAyB;IAC5D,MAAM,CAAU,OAAO,GAAG,QAAQ,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;IACzD,MAAM,CAAU,WAAW,GAAG,QAAQ,CAAC,UAAU,CAAC,aAAa,CAAC,CAAC;IACjE,MAAM,CAAU,QAAQ,GAAG,QAAQ,CAAC,WAAW,CAAC,UAAU,CAAC,CAAC;IAE5D,MAAM,CAAU,KAAK,GAAG;QAC7B,GAAG,SAAS,CAAC,SAAS;QACtB,GAAG,cAAc;QACjB,GAAG,cAAc;QACjB,WAAW,EAAE,KAAK,CAAC,SAAS,CAAC;YAC3B,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,yBAAyB,CAAC;YACvD,QAAQ,EAAE,IAAI;SACf,CAAC;QACF,2BAA2B,EAAE,KAAK,CAAC,MAAM,CAAC;YACxC,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,yCAAyC,CAAC;SACxE,CAAC;QACF,KAAK,EAAE,KAAK,CAAC,MAAM,CAAC;YAClB,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,qBAAqB,CAAC;SACpD,CAAC;KACH,CAAC;IAEF,6CAA6C;IACtC,KAAK,CAAC,GAAG;QACd,MAAM,EAAE,KAAK,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,cAAc,CAAC,CAAC;QAEnD,8BAA8B;QAC9B,MAAM,mBAAmB,GAAG,iBAAiB,CAAC,KAAK,CAAC,CAAC;QAErD,MAAM,eAAe,CAAC,mBAAmB,EAAE,mBAAmB,EAAE,KAAK,CAAC,eAAe,CAAC,EAAE,CAAC,OAAO,EAAE,EAAE,CAClG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,CACnB,CAAC;QAEF,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,gBAAgB,EAAE,EAAE,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAE3D,MAAM,cAAc,GAAG,MAAM,oBAAoB,CAC/C,mBAAmB,EACnB,KAAK,CAAC,WAAW,CAAC,EAClB,KAAK,CAAC,OAAO,CAAC,EACd,KAAK,CAAC,2BAA2B,CAAC,CACnC,CAAC;QAEF,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;QAEpB,OAAO,cAAc,CAAC;IACxB,CAAC"}
|
|
@@ -14,6 +14,7 @@ export default class DataFilesDownload extends SfCommand<void> {
|
|
|
14
14
|
static readonly flags: {
|
|
15
15
|
'max-parallel-jobs': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
16
16
|
'where-content-version': import("@oclif/core/interfaces").OptionFlag<string, import("@oclif/core/interfaces").CustomOptions>;
|
|
17
|
+
'max-api-usage': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
17
18
|
'api-version': import("@oclif/core/interfaces").OptionFlag<string | undefined>;
|
|
18
19
|
'target-org': import("@oclif/core/interfaces").OptionFlag<import("@salesforce/core").Org>;
|
|
19
20
|
'flags-dir': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
|
|
@@ -20,6 +20,8 @@ import { SfCommand, Flags } from '@salesforce/sf-plugins-core';
|
|
|
20
20
|
import { requireConnection, targetOrgFlags } from '@simplysf/simply-plugin-kit';
|
|
21
21
|
import { createCsvFileWriter } from '@simplysf/simply-core';
|
|
22
22
|
import { downloadContentVersion } from '../../../../common/contentVersionUtils.js';
|
|
23
|
+
import { apiBudgetFlags, assertApiBudget } from '../../../../common/apiBudgetFlag.js';
|
|
24
|
+
import { REQUESTS_PER_DOWNLOAD, requestsForQuery } from '../../../../common/apiCost.js';
|
|
23
25
|
Messages.importMessagesDirectoryFromMetaUrl(import.meta.url);
|
|
24
26
|
const messages = Messages.loadMessages('@simplysf/simply-data', 'simply.data.files.download');
|
|
25
27
|
/**
|
|
@@ -37,6 +39,7 @@ export default class DataFilesDownload extends SfCommand {
|
|
|
37
39
|
static flags = {
|
|
38
40
|
...SfCommand.baseFlags,
|
|
39
41
|
...targetOrgFlags,
|
|
42
|
+
...apiBudgetFlags,
|
|
40
43
|
'max-parallel-jobs': Flags.integer({
|
|
41
44
|
summary: messages.getMessage('flags.max-parallel-jobs.summary'),
|
|
42
45
|
description: messages.getMessage('flags.max-parallel-jobs.description'),
|
|
@@ -59,6 +62,9 @@ export default class DataFilesDownload extends SfCommand {
|
|
|
59
62
|
scanAll: false,
|
|
60
63
|
});
|
|
61
64
|
const contentVersionDownloads = result.records;
|
|
65
|
+
// The query above already ran through jsforce, so `connection.limitInfo` is populated and the
|
|
66
|
+
// budget check gets the free header path rather than spending a request on the limits API.
|
|
67
|
+
await assertApiBudget(targetOrgConnection, contentVersionDownloads.length * REQUESTS_PER_DOWNLOAD + requestsForQuery(contentVersionDownloads.length), flags['max-api-usage'], (message) => this.warn(message));
|
|
62
68
|
this.spinner.start('Initializing file download', '\n', { stdout: true });
|
|
63
69
|
if (!fs.existsSync('download')) {
|
|
64
70
|
fs.mkdirSync('download');
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"download.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/files/download.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,MAAM,MAAM,SAAS,CAAC;AAC7B,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,mBAAmB,EAAE,MAAM,uBAAuB,CAAC;AAC5D,OAAO,EAAE,sBAAsB,EAAE,MAAM,2CAA2C,CAAC;
|
|
1
|
+
{"version":3,"file":"download.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/files/download.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,MAAM,MAAM,SAAS,CAAC;AAC7B,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,mBAAmB,EAAE,MAAM,uBAAuB,CAAC;AAC5D,OAAO,EAAE,sBAAsB,EAAE,MAAM,2CAA2C,CAAC;AACnF,OAAO,EAAE,cAAc,EAAE,eAAe,EAAE,MAAM,qCAAqC,CAAC;AACtF,OAAO,EAAE,qBAAqB,EAAE,gBAAgB,EAAE,MAAM,+BAA+B,CAAC;AAGxF,QAAQ,CAAC,kCAAkC,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAC7D,MAAM,QAAQ,GAAG,QAAQ,CAAC,YAAY,CAAC,uBAAuB,EAAE,4BAA4B,CAAC,CAAC;AAE9F;;;;;;;GAOG;AACH,MAAM,CAAC,OAAO,OAAO,iBAAkB,SAAQ,SAAe;IACrD,MAAM,CAAU,OAAO,GAAG,QAAQ,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;IACzD,MAAM,CAAU,WAAW,GAAG,QAAQ,CAAC,UAAU,CAAC,aAAa,CAAC,CAAC;IACjE,MAAM,CAAU,QAAQ,GAAG,QAAQ,CAAC,WAAW,CAAC,UAAU,CAAC,CAAC;IAE5D,MAAM,CAAU,KAAK,GAAG;QAC7B,GAAG,SAAS,CAAC,SAAS;QACtB,GAAG,cAAc;QACjB,GAAG,cAAc;QACjB,mBAAmB,EAAE,KAAK,CAAC,OAAO,CAAC;YACjC,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,iCAAiC,CAAC;YAC/D,WAAW,EAAE,QAAQ,CAAC,UAAU,CAAC,qCAAqC,CAAC;YACvE,OAAO,EAAE,CAAC;SACX,CAAC;QACF,uBAAuB,EAAE,KAAK,CAAC,MAAM,CAAC;YACpC,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,qCAAqC,CAAC;YACnE,WAAW,EAAE,QAAQ,CAAC,UAAU,CAAC,yCAAyC,CAAC;YAC3E,QAAQ,EAAE,IAAI;SACf,CAAC;KACH,CAAC;IAEK,KAAK,CAAC,GAAG;QACd,MAAM,EAAE,KAAK,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,iBAAiB,CAAC,CAAC;QAEtD,8BAA8B;QAC9B,MAAM,mBAAmB,GAAG,iBAAiB,CAAC,KAAK,CAAC,CAAC;QAErD,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,oBAAoB,EAAE,IAAI,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAEjE,MAAM,MAAM,GAAG,MAAM,mBAAmB,CAAC,KAAK,CAC5C,oKAAoK,KAAK,CAAC,uBAAuB,CAAC,EAAE,EACpM;YACE,SAAS,EAAE,IAAI;YACf,QAAQ,EAAG,IAAI,CAAC,gBAAgB,CAAC,OAAO,CAAC,qBAAqB,CAAC,CAAC,KAAgB,IAAI,MAAM;YAC1F,OAAO,EAAE,KAAK;SACf,CACF,CAAC;QAEF,MAAM,uBAAuB,GAAG,MAAM,CAAC,OAAO,CAAC;QAE/C,8FAA8F;QAC9F,2FAA2F;QAC3F,MAAM,eAAe,CACnB,mBAAmB,EACnB,uBAAuB,CAAC,MAAM,GAAG,qBAAqB,GAAG,gBAAgB,CAAC,uBAAuB,CAAC,MAAM,CAAC,EACzG,KAAK,CAAC,eAAe,CAAC,EACtB,CAAC,OAAO,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,CAChC,CAAC;QAEF,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,4BAA4B,EAAE,IAAI,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAEzE,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,UAAU,CAAC,EAAE,CAAC;YAC/B,EAAE,CAAC,SAAS,CAAC,UAAU,CAAC,CAAC;QAC3B,CAAC;QAED,MAAM,aAAa,GAAG,mBAAmB,CAAC,sBAAsB,EAAE;YAChE,IAAI;YACJ,mBAAmB;YACnB,aAAa;YACb,aAAa;YACb,eAAe;YACf,UAAU;YACV,wBAAwB;YACxB,UAAU;YACV,cAAc;YACd,OAAO;YACP,UAAU;SACX,CAAC,CAAC;QAEH,MAAM,WAAW,GAAG,mBAAmB,CAAC,oBAAoB,EAAE;YAC5D,IAAI;YACJ,mBAAmB;YACnB,aAAa;YACb,aAAa;YACb,eAAe;YACf,UAAU;YACV,wBAAwB;YACxB,UAAU;YACV,cAAc;YACd,OAAO;YACP,OAAO;SACR,CAAC,CAAC;QAEH,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,mBAAmB,EAAE,IAAI,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAEhE,MAAM,aAAa,GAAG,IAAI,MAAM,CAAC,EAAE,WAAW,EAAE,KAAK,CAAC,mBAAmB,CAAC,EAAE,CAAC,CAAC;QAE9E,IAAI,KAAK,GAAG,CAAC,CAAC;QACd,aAAa,CAAC,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE;YAC3B,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,cAAc,KAAK,WAAW,aAAa,CAAC,IAAI,cAAc,aAAa,CAAC,OAAO,IAAI,CAAC;QAChH,CAAC,CAAC,CAAC;QAEH,aAAa,CAAC,EAAE,CAAC,WAAW,EAAE,GAAG,EAAE;YACjC,KAAK,EAAE,CAAC;YACR,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,cAAc,KAAK,WAAW,aAAa,CAAC,IAAI,cAAc,aAAa,CAAC,OAAO,IAAI,CAAC;QAChH,CAAC,CAAC,CAAC;QAEH,KAAK,MAAM,sBAAsB,IAAI,uBAAuB,EAAE,CAAC;YAC7D,KAAK,aAAa,CAAC,GAAG,CAAC,KAAK,IAAI,EAAE;gBAChC,IAAI,CAAC;oBACH,sBAAsB,CAAC,QAAQ,GAAG,MAAM,sBAAsB,CAC5D,mBAAmB,EACnB,sBAAsB,EACtB,UAAU,CACX,CAAC;oBACF,MAAM,aAAa,CAAC,KAAK,CAAC,sBAAsB,CAAC,CAAC;gBACpD,CAAC;gBAAC,OAAO,KAAK,EAAE,CAAC;oBACf,sBAAsB,CAAC,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;oBAC7C,MAAM,WAAW,CAAC,KAAK,CAAC,sBAAsB,CAAC,CAAC;gBAClD,CAAC;YACH,CAAC,CAAC,CAAC;QACL,CAAC;QAED,MAAM,aAAa,CAAC,MAAM,EAAE,CAAC;QAC7B,MAAM,OAAO,CAAC,GAAG,CAAC,CAAC,aAAa,CAAC,GAAG,EAAE,EAAE,WAAW,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC;QAE5D,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;IACtB,CAAC"}
|
|
@@ -15,6 +15,7 @@ export default class DataFilesUpload extends SfCommand<void> {
|
|
|
15
15
|
static readonly flags: {
|
|
16
16
|
'file-path': import("@oclif/core/interfaces").OptionFlag<string, import("@oclif/core/interfaces").CustomOptions>;
|
|
17
17
|
'max-parallel-jobs': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
18
|
+
'max-api-usage': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
18
19
|
'api-version': import("@oclif/core/interfaces").OptionFlag<string | undefined>;
|
|
19
20
|
'target-org': import("@oclif/core/interfaces").OptionFlag<import("@salesforce/core").Org>;
|
|
20
21
|
'flags-dir': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
|
|
@@ -21,6 +21,9 @@ import { SfCommand, Flags } from '@salesforce/sf-plugins-core';
|
|
|
21
21
|
import { requireConnection, targetOrgFlags } from '@simplysf/simply-plugin-kit';
|
|
22
22
|
import { createCsvFileWriter } from '@simplysf/simply-core';
|
|
23
23
|
import { uploadContentVersion } from '../../../../common/contentVersionUtils.js';
|
|
24
|
+
import { apiBudgetFlags, assertApiBudget } from '../../../../common/apiBudgetFlag.js';
|
|
25
|
+
import { countCsvRows } from '../../../../common/countCsvRows.js';
|
|
26
|
+
import { REQUESTS_PER_UPLOAD } from '../../../../common/apiCost.js';
|
|
24
27
|
Messages.importMessagesDirectoryFromMetaUrl(import.meta.url);
|
|
25
28
|
const messages = Messages.loadMessages('@simplysf/simply-data', 'simply.data.files.upload');
|
|
26
29
|
/**
|
|
@@ -39,6 +42,7 @@ export default class DataFilesUpload extends SfCommand {
|
|
|
39
42
|
static flags = {
|
|
40
43
|
...SfCommand.baseFlags,
|
|
41
44
|
...targetOrgFlags,
|
|
45
|
+
...apiBudgetFlags,
|
|
42
46
|
'file-path': Flags.directory({
|
|
43
47
|
summary: messages.getMessage('flags.file-path.summary'),
|
|
44
48
|
description: messages.getMessage('flags.file-path.description'),
|
|
@@ -54,6 +58,11 @@ export default class DataFilesUpload extends SfCommand {
|
|
|
54
58
|
const { flags } = await this.parse(DataFilesUpload);
|
|
55
59
|
// Authorize to the target org
|
|
56
60
|
const targetOrgConnection = requireConnection(flags);
|
|
61
|
+
// The upload streams this CSV row by row, so the row count is not otherwise known until the
|
|
62
|
+
// work is already underway. Counting first costs one extra local read and no API requests,
|
|
63
|
+
// and it is what lets the budget be checked before anything is sent.
|
|
64
|
+
const rowCount = await countCsvRows(flags['file-path']);
|
|
65
|
+
await assertApiBudget(targetOrgConnection, rowCount * REQUESTS_PER_UPLOAD, flags['max-api-usage'], (message) => this.warn(message));
|
|
57
66
|
this.spinner.start('Initializing file upload', '\n', { stdout: true });
|
|
58
67
|
if (!fs.existsSync('upload')) {
|
|
59
68
|
fs.mkdirSync('upload');
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"upload.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/files/upload.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AAClC,OAAO,MAAM,MAAM,SAAS,CAAC;AAC7B,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,mBAAmB,EAAE,MAAM,uBAAuB,CAAC;AAC5D,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;
|
|
1
|
+
{"version":3,"file":"upload.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/files/upload.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AAClC,OAAO,MAAM,MAAM,SAAS,CAAC;AAC7B,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,mBAAmB,EAAE,MAAM,uBAAuB,CAAC;AAC5D,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AACjF,OAAO,EAAE,cAAc,EAAE,eAAe,EAAE,MAAM,qCAAqC,CAAC;AACtF,OAAO,EAAE,YAAY,EAAE,MAAM,oCAAoC,CAAC;AAClE,OAAO,EAAE,mBAAmB,EAAE,MAAM,+BAA+B,CAAC;AAGpE,QAAQ,CAAC,kCAAkC,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAC7D,MAAM,QAAQ,GAAG,QAAQ,CAAC,YAAY,CAAC,uBAAuB,EAAE,0BAA0B,CAAC,CAAC;AAE5F;;;;;;;;GAQG;AACH,MAAM,CAAC,OAAO,OAAO,eAAgB,SAAQ,SAAe;IACnD,MAAM,CAAU,OAAO,GAAG,QAAQ,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;IACzD,MAAM,CAAU,WAAW,GAAG,QAAQ,CAAC,UAAU,CAAC,aAAa,CAAC,CAAC;IACjE,MAAM,CAAU,QAAQ,GAAG,QAAQ,CAAC,WAAW,CAAC,UAAU,CAAC,CAAC;IAE5D,MAAM,CAAU,KAAK,GAAG;QAC7B,GAAG,SAAS,CAAC,SAAS;QACtB,GAAG,cAAc;QACjB,GAAG,cAAc;QACjB,WAAW,EAAE,KAAK,CAAC,SAAS,CAAC;YAC3B,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,yBAAyB,CAAC;YACvD,WAAW,EAAE,QAAQ,CAAC,UAAU,CAAC,6BAA6B,CAAC;YAC/D,QAAQ,EAAE,IAAI;SACf,CAAC;QACF,mBAAmB,EAAE,KAAK,CAAC,OAAO,CAAC;YACjC,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,iCAAiC,CAAC;YAC/D,WAAW,EAAE,QAAQ,CAAC,UAAU,CAAC,qCAAqC,CAAC;YACvE,OAAO,EAAE,CAAC;SACX,CAAC;KACH,CAAC;IAEK,KAAK,CAAC,GAAG;QACd,MAAM,EAAE,KAAK,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,eAAe,CAAC,CAAC;QAEpD,8BAA8B;QAC9B,MAAM,mBAAmB,GAAG,iBAAiB,CAAC,KAAK,CAAC,CAAC;QAErD,4FAA4F;QAC5F,2FAA2F;QAC3F,qEAAqE;QACrE,MAAM,QAAQ,GAAG,MAAM,YAAY,CAAC,KAAK,CAAC,WAAW,CAAC,CAAC,CAAC;QAExD,MAAM,eAAe,CAAC,mBAAmB,EAAE,QAAQ,GAAG,mBAAmB,EAAE,KAAK,CAAC,eAAe,CAAC,EAAE,CAAC,OAAO,EAAE,EAAE,CAC7G,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,CACnB,CAAC;QAEF,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,0BAA0B,EAAE,IAAI,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAEvE,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;YAC7B,EAAE,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC;QACzB,CAAC;QAED,MAAM,aAAa,GAAG,mBAAmB,CAAC,oBAAoB,EAAE;YAC9D,cAAc;YACd,OAAO;YACP,wBAAwB;YACxB,mBAAmB;SACpB,CAAC,CAAC;QAEH,MAAM,WAAW,GAAG,mBAAmB,CAAC,kBAAkB,EAAE;YAC1D,cAAc;YACd,OAAO;YACP,wBAAwB;YACxB,OAAO;SACR,CAAC,CAAC;QAEH,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;QAEpB,MAAM,SAAS,GAAG,IAAI,MAAM,CAAC,EAAE,WAAW,EAAE,KAAK,CAAC,mBAAmB,CAAC,EAAE,CAAC,CAAC;QAE1E,MAAM,MAAM,GAAG,EAAE,CAAC,gBAAgB,CAAC,KAAK,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;QAEjG,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,iBAAiB,EAAE,cAAc,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAExE,IAAI,KAAK,GAAG,CAAC,CAAC;QACd,SAAS,CAAC,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE;YACvB,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,cAAc,KAAK,WAAW,SAAS,CAAC,IAAI,cAAc,SAAS,CAAC,OAAO,EAAE,CAAC;QACtG,CAAC,CAAC,CAAC;QAEH,SAAS,CAAC,EAAE,CAAC,WAAW,EAAE,GAAG,EAAE;YAC7B,KAAK,EAAE,CAAC;YACR,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,cAAc,KAAK,WAAW,SAAS,CAAC,IAAI,cAAc,SAAS,CAAC,OAAO,EAAE,CAAC;QACtG,CAAC,CAAC,CAAC;QAEH,IAAI,KAAK,EAAE,MAAM,MAAM,IAAI,MAAM,EAAE,CAAC;YAClC,KAAK,SAAS,CAAC,GAAG,CAAC,KAAK,IAAI,EAAE;gBAC5B,MAAM,sBAAsB,GAAG,MAAgC,CAAC;gBAChE,IAAI,CAAC;oBACH,MAAM,cAAc,GAAG,MAAM,oBAAoB,CAC/C,mBAAmB,EACnB,sBAAsB,CAAC,YAAY,EACnC,sBAAsB,CAAC,KAAK,EAC5B,sBAAsB,CAAC,sBAAsB,CAC9C,CAAC;oBACF,sBAAsB,CAAC,iBAAiB,GAAG,cAAc,CAAC,iBAAiB,CAAC;oBAC5E,MAAM,aAAa,CAAC,KAAK,CAAC,sBAAsB,CAAC,CAAC;gBACpD,CAAC;gBAAC,OAAO,KAAK,EAAE,CAAC;oBACf,sBAAsB,CAAC,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;oBAC7C,MAAM,WAAW,CAAC,KAAK,CAAC,sBAAsB,CAAC,CAAC;gBAClD,CAAC;YACH,CAAC,CAAC,CAAC;QACL,CAAC;QAED,MAAM,SAAS,CAAC,MAAM,EAAE,CAAC;QACzB,MAAM,OAAO,CAAC,GAAG,CAAC,CAAC,aAAa,CAAC,GAAG,EAAE,EAAE,WAAW,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC;QAE5D,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;IACtB,CAAC"}
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
import { type ApiBudgetResult } from '@simplysf/simply-core';
|
|
2
|
+
import type { Connection } from '@salesforce/core';
|
|
3
|
+
/** Default share of the org's remaining API requests a single run may consume. */
|
|
4
|
+
export declare const DEFAULT_MAX_API_USAGE = 20;
|
|
5
|
+
/**
|
|
6
|
+
* `--max-api-usage`, shared by every command that can count its requests in advance.
|
|
7
|
+
*
|
|
8
|
+
* Declared once so the three file commands cannot drift apart on the default or the bounds.
|
|
9
|
+
*/
|
|
10
|
+
export declare const apiBudgetFlags: {
|
|
11
|
+
'max-api-usage': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
12
|
+
};
|
|
13
|
+
/**
|
|
14
|
+
* Check a planned run against the org's remaining API allocation, and stop it if it does not fit.
|
|
15
|
+
*
|
|
16
|
+
* Call this before making any of the requests being counted — the value of the check is entirely
|
|
17
|
+
* in nothing having happened when it fails.
|
|
18
|
+
*
|
|
19
|
+
* @param connection - The org connection.
|
|
20
|
+
* @param plannedRequests - How many API requests the run is about to make.
|
|
21
|
+
* @param maxUsagePercent - The `--max-api-usage` value.
|
|
22
|
+
* @param warn - Called when the allocation could not be read, so the command can surface it.
|
|
23
|
+
* @returns The check result, for inclusion in the command's JSON output.
|
|
24
|
+
* @throws {SfError} If the run would exceed its budget.
|
|
25
|
+
*/
|
|
26
|
+
export declare function assertApiBudget(connection: Connection, plannedRequests: number, maxUsagePercent: number, warn: (message: string) => void): Promise<ApiBudgetResult>;
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
/*
|
|
2
|
+
* Copyright (c) 2026, Clay Chipps.
|
|
3
|
+
*
|
|
4
|
+
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
5
|
+
* you may not use this file except in compliance with the License.
|
|
6
|
+
* You may obtain a copy of the License at
|
|
7
|
+
*
|
|
8
|
+
* http://www.apache.org/licenses/LICENSE-2.0
|
|
9
|
+
*
|
|
10
|
+
* Unless required by applicable law or agreed to in writing, software
|
|
11
|
+
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
12
|
+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
13
|
+
* See the License for the specific language governing permissions and
|
|
14
|
+
* limitations under the License.
|
|
15
|
+
*/
|
|
16
|
+
import { Messages } from '@salesforce/core';
|
|
17
|
+
import { Flags } from '@salesforce/sf-plugins-core';
|
|
18
|
+
import { apiBudgetError, checkApiBudget } from '@simplysf/simply-core';
|
|
19
|
+
Messages.importMessagesDirectoryFromMetaUrl(import.meta.url);
|
|
20
|
+
const messages = Messages.loadMessages('@simplysf/simply-data', 'apiBudget');
|
|
21
|
+
/** Default share of the org's remaining API requests a single run may consume. */
|
|
22
|
+
export const DEFAULT_MAX_API_USAGE = 20;
|
|
23
|
+
/**
|
|
24
|
+
* `--max-api-usage`, shared by every command that can count its requests in advance.
|
|
25
|
+
*
|
|
26
|
+
* Declared once so the three file commands cannot drift apart on the default or the bounds.
|
|
27
|
+
*/
|
|
28
|
+
export const apiBudgetFlags = {
|
|
29
|
+
'max-api-usage': Flags.integer({
|
|
30
|
+
summary: messages.getMessage('flags.max-api-usage.summary'),
|
|
31
|
+
description: messages.getMessage('flags.max-api-usage.description'),
|
|
32
|
+
default: DEFAULT_MAX_API_USAGE,
|
|
33
|
+
min: 1,
|
|
34
|
+
max: 100,
|
|
35
|
+
}),
|
|
36
|
+
};
|
|
37
|
+
/**
|
|
38
|
+
* Check a planned run against the org's remaining API allocation, and stop it if it does not fit.
|
|
39
|
+
*
|
|
40
|
+
* Call this before making any of the requests being counted — the value of the check is entirely
|
|
41
|
+
* in nothing having happened when it fails.
|
|
42
|
+
*
|
|
43
|
+
* @param connection - The org connection.
|
|
44
|
+
* @param plannedRequests - How many API requests the run is about to make.
|
|
45
|
+
* @param maxUsagePercent - The `--max-api-usage` value.
|
|
46
|
+
* @param warn - Called when the allocation could not be read, so the command can surface it.
|
|
47
|
+
* @returns The check result, for inclusion in the command's JSON output.
|
|
48
|
+
* @throws {SfError} If the run would exceed its budget.
|
|
49
|
+
*/
|
|
50
|
+
export async function assertApiBudget(connection, plannedRequests, maxUsagePercent, warn) {
|
|
51
|
+
const result = await checkApiBudget(connection, plannedRequests, { maxUsagePercent });
|
|
52
|
+
if (result.status === 'unavailable') {
|
|
53
|
+
warn(messages.getMessage('warning.budgetUnavailable', [result.reason ?? 'unknown reason']));
|
|
54
|
+
return result;
|
|
55
|
+
}
|
|
56
|
+
if (result.status === 'exceeded') {
|
|
57
|
+
throw apiBudgetError(result, maxUsagePercent);
|
|
58
|
+
}
|
|
59
|
+
return result;
|
|
60
|
+
}
|
|
61
|
+
//# sourceMappingURL=apiBudgetFlag.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"apiBudgetFlag.js","sourceRoot":"","sources":["../../src/common/apiBudgetFlag.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AACpD,OAAO,EAAE,cAAc,EAAE,cAAc,EAAwB,MAAM,uBAAuB,CAAC;AAG7F,QAAQ,CAAC,kCAAkC,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAC7D,MAAM,QAAQ,GAAG,QAAQ,CAAC,YAAY,CAAC,uBAAuB,EAAE,WAAW,CAAC,CAAC;AAE7E,kFAAkF;AAClF,MAAM,CAAC,MAAM,qBAAqB,GAAG,EAAE,CAAC;AAExC;;;;GAIG;AACH,MAAM,CAAC,MAAM,cAAc,GAAG;IAC5B,eAAe,EAAE,KAAK,CAAC,OAAO,CAAC;QAC7B,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,6BAA6B,CAAC;QAC3D,WAAW,EAAE,QAAQ,CAAC,UAAU,CAAC,iCAAiC,CAAC;QACnE,OAAO,EAAE,qBAAqB;QAC9B,GAAG,EAAE,CAAC;QACN,GAAG,EAAE,GAAG;KACT,CAAC;CACH,CAAC;AAEF;;;;;;;;;;;;GAYG;AACH,MAAM,CAAC,KAAK,UAAU,eAAe,CACnC,UAAsB,EACtB,eAAuB,EACvB,eAAuB,EACvB,IAA+B;IAE/B,MAAM,MAAM,GAAG,MAAM,cAAc,CAAC,UAAU,EAAE,eAAe,EAAE,EAAE,eAAe,EAAE,CAAC,CAAC;IAEtF,IAAI,MAAM,CAAC,MAAM,KAAK,aAAa,EAAE,CAAC;QACpC,IAAI,CAAC,QAAQ,CAAC,UAAU,CAAC,2BAA2B,EAAE,CAAC,MAAM,CAAC,MAAM,IAAI,gBAAgB,CAAC,CAAC,CAAC,CAAC;QAE5F,OAAO,MAAM,CAAC;IAChB,CAAC;IAED,IAAI,MAAM,CAAC,MAAM,KAAK,UAAU,EAAE,CAAC;QACjC,MAAM,cAAc,CAAC,MAAM,EAAE,eAAe,CAAC,CAAC;IAChD,CAAC;IAED,OAAO,MAAM,CAAC;AAChB,CAAC"}
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* API requests consumed per file uploaded.
|
|
3
|
+
*
|
|
4
|
+
* **Two, not one.** Salesforce's create response returns only the new `ContentVersion` id, so
|
|
5
|
+
* `uploadContentVersion` follows it with a query for `ContentDocumentId`. Both count toward the
|
|
6
|
+
* org's daily allocation. Counting one per file here would under-budget every upload by half.
|
|
7
|
+
*/
|
|
8
|
+
export declare const REQUESTS_PER_UPLOAD = 2;
|
|
9
|
+
/** API requests consumed per file downloaded — a single GET of the `VersionData` blob. */
|
|
10
|
+
export declare const REQUESTS_PER_DOWNLOAD = 1;
|
|
11
|
+
/**
|
|
12
|
+
* Records jsforce returns per query round trip before it has to fetch another batch.
|
|
13
|
+
*
|
|
14
|
+
* Used to estimate how many requests a record query costs on top of the per-file work.
|
|
15
|
+
*/
|
|
16
|
+
export declare const QUERY_BATCH_SIZE = 2000;
|
|
17
|
+
/**
|
|
18
|
+
* Estimate the API requests a record query costs, including its `queryMore` round trips.
|
|
19
|
+
*
|
|
20
|
+
* @param recordCount - How many records the query returns.
|
|
21
|
+
* @returns The number of requests, at least 1.
|
|
22
|
+
*/
|
|
23
|
+
export declare function requestsForQuery(recordCount: number): number;
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
/*
|
|
2
|
+
* Copyright (c) 2026, Clay Chipps.
|
|
3
|
+
*
|
|
4
|
+
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
5
|
+
* you may not use this file except in compliance with the License.
|
|
6
|
+
* You may obtain a copy of the License at
|
|
7
|
+
*
|
|
8
|
+
* http://www.apache.org/licenses/LICENSE-2.0
|
|
9
|
+
*
|
|
10
|
+
* Unless required by applicable law or agreed to in writing, software
|
|
11
|
+
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
12
|
+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
13
|
+
* See the License for the specific language governing permissions and
|
|
14
|
+
* limitations under the License.
|
|
15
|
+
*/
|
|
16
|
+
/**
|
|
17
|
+
* API requests consumed per file uploaded.
|
|
18
|
+
*
|
|
19
|
+
* **Two, not one.** Salesforce's create response returns only the new `ContentVersion` id, so
|
|
20
|
+
* `uploadContentVersion` follows it with a query for `ContentDocumentId`. Both count toward the
|
|
21
|
+
* org's daily allocation. Counting one per file here would under-budget every upload by half.
|
|
22
|
+
*/
|
|
23
|
+
export const REQUESTS_PER_UPLOAD = 2;
|
|
24
|
+
/** API requests consumed per file downloaded — a single GET of the `VersionData` blob. */
|
|
25
|
+
export const REQUESTS_PER_DOWNLOAD = 1;
|
|
26
|
+
/**
|
|
27
|
+
* Records jsforce returns per query round trip before it has to fetch another batch.
|
|
28
|
+
*
|
|
29
|
+
* Used to estimate how many requests a record query costs on top of the per-file work.
|
|
30
|
+
*/
|
|
31
|
+
export const QUERY_BATCH_SIZE = 2000;
|
|
32
|
+
/**
|
|
33
|
+
* Estimate the API requests a record query costs, including its `queryMore` round trips.
|
|
34
|
+
*
|
|
35
|
+
* @param recordCount - How many records the query returns.
|
|
36
|
+
* @returns The number of requests, at least 1.
|
|
37
|
+
*/
|
|
38
|
+
export function requestsForQuery(recordCount) {
|
|
39
|
+
return Math.max(1, Math.ceil(recordCount / QUERY_BATCH_SIZE));
|
|
40
|
+
}
|
|
41
|
+
//# sourceMappingURL=apiCost.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"apiCost.js","sourceRoot":"","sources":["../../src/common/apiCost.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH;;;;;;GAMG;AACH,MAAM,CAAC,MAAM,mBAAmB,GAAG,CAAC,CAAC;AAErC,0FAA0F;AAC1F,MAAM,CAAC,MAAM,qBAAqB,GAAG,CAAC,CAAC;AAEvC;;;;GAIG;AACH,MAAM,CAAC,MAAM,gBAAgB,GAAG,IAAI,CAAC;AAErC;;;;;GAKG;AACH,MAAM,UAAU,gBAAgB,CAAC,WAAmB;IAClD,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,IAAI,CAAC,WAAW,GAAG,gBAAgB,CAAC,CAAC,CAAC;AAChE,CAAC"}
|
|
@@ -20,6 +20,33 @@ import { SfError } from '@salesforce/core';
|
|
|
20
20
|
import { contentVersionMultipart } from './multipart.js';
|
|
21
21
|
/** How much of an error response body to include in a thrown error before truncating. */
|
|
22
22
|
const MAX_ERROR_BODY_LENGTH = 2000;
|
|
23
|
+
/** Matches the `api-usage=used/limit` portion of a `Sforce-Limit-Info` response header. */
|
|
24
|
+
const API_USAGE_PATTERN = /api-usage=(?<used>\d+)\/(?<limit>\d+)/;
|
|
25
|
+
/**
|
|
26
|
+
* Feed a raw `fetch` response's `Sforce-Limit-Info` header back into the connection.
|
|
27
|
+
*
|
|
28
|
+
* Salesforce reports API consumption on every REST response, and jsforce normally parses it into
|
|
29
|
+
* `connection.limitInfo`. These two functions call `fetch` directly, so jsforce never sees their
|
|
30
|
+
* responses and the readings would otherwise be lost — leaving `limitInfo` stale at whatever the
|
|
31
|
+
* last jsforce-issued request reported, which for an upload run is "nothing at all".
|
|
32
|
+
*
|
|
33
|
+
* Keeping it current costs one header parse and makes the reading available to the API budget
|
|
34
|
+
* check and to any later running check.
|
|
35
|
+
*
|
|
36
|
+
* @param connection - The connection whose `limitInfo` should be updated.
|
|
37
|
+
* @param response - The response to read the header from.
|
|
38
|
+
*/
|
|
39
|
+
function recordApiUsage(connection, response) {
|
|
40
|
+
const match = API_USAGE_PATTERN.exec(response.headers.get('sforce-limit-info') ?? '');
|
|
41
|
+
if (match?.groups) {
|
|
42
|
+
connection.limitInfo = {
|
|
43
|
+
apiUsage: {
|
|
44
|
+
used: Number.parseInt(match.groups.used, 10),
|
|
45
|
+
limit: Number.parseInt(match.groups.limit, 10),
|
|
46
|
+
},
|
|
47
|
+
};
|
|
48
|
+
}
|
|
49
|
+
}
|
|
23
50
|
/**
|
|
24
51
|
* Turn a non-2xx response into an `SfError`.
|
|
25
52
|
*
|
|
@@ -58,6 +85,7 @@ export async function downloadContentVersion(targetOrgConnection, contentVersion
|
|
|
58
85
|
Authorization: `Bearer ${targetOrgConnection.accessToken}`,
|
|
59
86
|
},
|
|
60
87
|
});
|
|
88
|
+
recordApiUsage(targetOrgConnection, response);
|
|
61
89
|
// Checked before the write stream is opened, so a failed download leaves nothing behind.
|
|
62
90
|
if (!response.ok) {
|
|
63
91
|
throw await responseError(response, `Download of ContentVersion ${contentVersionDownload.Id}`);
|
|
@@ -108,6 +136,7 @@ export async function uploadContentVersion(targetOrgConnection, filePath, title,
|
|
|
108
136
|
'Content-Type': contentType,
|
|
109
137
|
},
|
|
110
138
|
});
|
|
139
|
+
recordApiUsage(targetOrgConnection, response);
|
|
111
140
|
if (!response.ok) {
|
|
112
141
|
throw await responseError(response, `Upload of "${filePath}"`);
|
|
113
142
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"contentVersionUtils.js","sourceRoot":"","sources":["../../src/common/contentVersionUtils.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,IAAI,MAAM,WAAW,CAAC;AAC7B,OAAO,MAAM,EAAE,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AAC/C,OAAO,EAAc,OAAO,EAAE,MAAM,kBAAkB,CAAC;AACvD,OAAO,EAAE,uBAAuB,EAAE,MAAM,gBAAgB,CAAC;AAQzD,yFAAyF;AACzF,MAAM,qBAAqB,GAAG,IAAI,CAAC;AAEnC;;;;;;;;;;GAUG;AACH,KAAK,UAAU,aAAa,CAAC,QAAkB,EAAE,OAAe;IAC9D,IAAI,IAAI,GAAG,EAAE,CAAC;IACd,IAAI,CAAC;QACH,IAAI,GAAG,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,qBAAqB,CAAC,CAAC;IACjE,CAAC;IAAC,MAAM,CAAC;QACP,gGAAgG;IAClG,CAAC;IAED,OAAO,IAAI,OAAO,CAChB,GAAG,OAAO,qBAAqB,QAAQ,CAAC,MAAM,IAAI,QAAQ,CAAC,UAAU,GAAG,IAAI,CAAC,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,EAAE,EACjG,4BAA4B,CAC7B,CAAC;AACJ,CAAC;AAED;;;;;;;;;GASG;AACH,MAAM,CAAC,KAAK,UAAU,sBAAsB,CAC1C,mBAA+B,EAC/B,sBAA8C,EAC9C,iBAAyB;IAEzB,MAAM,QAAQ,GAAG,GAAG,iBAAiB,IACnC,sBAAsB,CAAC,iBACzB,IAAI,sBAAsB,CAAC,KAAK,CAAC,UAAU,CAAC,GAAG,EAAE,GAAG,CAAC,IAAI,sBAAsB,CAAC,aAAa,EAAE,CAAC;IAEhG,MAAM,QAAQ,GAAG,MAAM,KAAK,CAC1B,GAAG,mBAAmB,CAAC,OAAO,EAAE,4BAA4B,sBAAsB,CAAC,EAAE,cAAc,EACnG;QACE,OAAO,EAAE;YACP,aAAa,EAAE,UAAU,mBAAmB,CAAC,WAAqB,EAAE;SACrE;KACF,CACF,CAAC;IAEF,yFAAyF;IACzF,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;QACjB,MAAM,MAAM,aAAa,CAAC,QAAQ,EAAE,8BAA8B,sBAAsB,CAAC,EAAE,EAAE,CAAC,CAAC;IACjG,CAAC;IAED,IAAI,CAAC,QAAQ,CAAC,IAAI,EAAE,CAAC;QACnB,MAAM,IAAI,OAAO,CACf,8BAA8B,sBAAsB,CAAC,EAAE,mCAAmC,EAC1F,4BAA4B,CAC7B,CAAC;IACJ,CAAC;IAED,MAAM,MAAM,CAAC,QAAQ,CAAC,QAAQ,CAAC,QAAQ,CAAC,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,EAAE,CAAC,iBAAiB,CAAC,QAAQ,CAAC,CAAC,CAAC;IAEhG,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED;;;;;;;;;;;;;;;GAeG;AACH,MAAM,CAAC,KAAK,UAAU,oBAAoB,CACxC,mBAA+B,EAC/B,QAAgB,EAChB,KAAc,EACd,sBAA+B;IAE/B,wCAAwC;IACxC,MAAM,EAAE,CAAC,QAAQ,CAAC,MAAM,CAAC,QAAQ,EAAE,EAAE,CAAC,SAAS,CAAC,IAAI,CAAC,CAAC;IAEtD,MAAM,QAAQ,GAAG,IAAI,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;IAEzC,MAAM,2BAA2B,GAAgC;QAC/D,sBAAsB,EAAE,sBAAsB;QAC9C,YAAY,EAAE,QAAQ;QACtB,KAAK,EAAE,KAAK,IAAI,QAAQ;KACzB,CAAC;IAEF,MAAM,EAAE,WAAW,EAAE,UAAU,EAAE,GAAG,uBAAuB,CAAC;QAC1D,MAAM,EAAE,2BAA2B;QACnC,QAAQ;QACR,QAAQ,EAAE,QAAQ;KACnB,CAAC,CAAC;IAEH,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,GAAG,mBAAmB,CAAC,OAAO,EAAE,0BAA0B,EAAE;QACvF,MAAM,EAAE,MAAM;QACd,wEAAwE;QACxE,IAAI,EAAE,QAAQ,CAAC,KAAK,CAAC,UAAU,EAAE,CAA+B;QAChE,MAAM,EAAE,MAAM;QACd,OAAO,EAAE;YACP,aAAa,EAAE,UAAU,mBAAmB,CAAC,WAAqB,EAAE;YACpE,cAAc,EAAE,WAAW;SAC5B;KACF,CAAC,CAAC;IAEH,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;QACjB,MAAM,MAAM,aAAa,CAAC,QAAQ,EAAE,cAAc,QAAQ,GAAG,CAAC,CAAC;IACjE,CAAC;IAED,MAAM,IAAI,GAAG,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAA+B,CAAC;IAEnE,MAAM,WAAW,GAAG,MAAM,mBAAmB,CAAC,iBAAiB,CAC7D,oFAAoF,IAAI,CAAC,EAAE,GAAG,CAC/F,CAAC;IAEF,OAAO,WAA6B,CAAC;AACvC,CAAC"}
|
|
1
|
+
{"version":3,"file":"contentVersionUtils.js","sourceRoot":"","sources":["../../src/common/contentVersionUtils.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,IAAI,MAAM,WAAW,CAAC;AAC7B,OAAO,MAAM,EAAE,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AAC/C,OAAO,EAAc,OAAO,EAAE,MAAM,kBAAkB,CAAC;AACvD,OAAO,EAAE,uBAAuB,EAAE,MAAM,gBAAgB,CAAC;AAQzD,yFAAyF;AACzF,MAAM,qBAAqB,GAAG,IAAI,CAAC;AAEnC,2FAA2F;AAC3F,MAAM,iBAAiB,GAAG,uCAAuC,CAAC;AAElE;;;;;;;;;;;;;GAaG;AACH,SAAS,cAAc,CAAC,UAAsB,EAAE,QAAkB;IAChE,MAAM,KAAK,GAAG,iBAAiB,CAAC,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,mBAAmB,CAAC,IAAI,EAAE,CAAC,CAAC;IAEtF,IAAI,KAAK,EAAE,MAAM,EAAE,CAAC;QAClB,UAAU,CAAC,SAAS,GAAG;YACrB,QAAQ,EAAE;gBACR,IAAI,EAAE,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;gBAC5C,KAAK,EAAE,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,MAAM,CAAC,KAAK,EAAE,EAAE,CAAC;aAC/C;SACF,CAAC;IACJ,CAAC;AACH,CAAC;AAED;;;;;;;;;;GAUG;AACH,KAAK,UAAU,aAAa,CAAC,QAAkB,EAAE,OAAe;IAC9D,IAAI,IAAI,GAAG,EAAE,CAAC;IACd,IAAI,CAAC;QACH,IAAI,GAAG,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,qBAAqB,CAAC,CAAC;IACjE,CAAC;IAAC,MAAM,CAAC;QACP,gGAAgG;IAClG,CAAC;IAED,OAAO,IAAI,OAAO,CAChB,GAAG,OAAO,qBAAqB,QAAQ,CAAC,MAAM,IAAI,QAAQ,CAAC,UAAU,GAAG,IAAI,CAAC,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,EAAE,EACjG,4BAA4B,CAC7B,CAAC;AACJ,CAAC;AAED;;;;;;;;;GASG;AACH,MAAM,CAAC,KAAK,UAAU,sBAAsB,CAC1C,mBAA+B,EAC/B,sBAA8C,EAC9C,iBAAyB;IAEzB,MAAM,QAAQ,GAAG,GAAG,iBAAiB,IACnC,sBAAsB,CAAC,iBACzB,IAAI,sBAAsB,CAAC,KAAK,CAAC,UAAU,CAAC,GAAG,EAAE,GAAG,CAAC,IAAI,sBAAsB,CAAC,aAAa,EAAE,CAAC;IAEhG,MAAM,QAAQ,GAAG,MAAM,KAAK,CAC1B,GAAG,mBAAmB,CAAC,OAAO,EAAE,4BAA4B,sBAAsB,CAAC,EAAE,cAAc,EACnG;QACE,OAAO,EAAE;YACP,aAAa,EAAE,UAAU,mBAAmB,CAAC,WAAqB,EAAE;SACrE;KACF,CACF,CAAC;IAEF,cAAc,CAAC,mBAAmB,EAAE,QAAQ,CAAC,CAAC;IAE9C,yFAAyF;IACzF,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;QACjB,MAAM,MAAM,aAAa,CAAC,QAAQ,EAAE,8BAA8B,sBAAsB,CAAC,EAAE,EAAE,CAAC,CAAC;IACjG,CAAC;IAED,IAAI,CAAC,QAAQ,CAAC,IAAI,EAAE,CAAC;QACnB,MAAM,IAAI,OAAO,CACf,8BAA8B,sBAAsB,CAAC,EAAE,mCAAmC,EAC1F,4BAA4B,CAC7B,CAAC;IACJ,CAAC;IAED,MAAM,MAAM,CAAC,QAAQ,CAAC,QAAQ,CAAC,QAAQ,CAAC,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,EAAE,CAAC,iBAAiB,CAAC,QAAQ,CAAC,CAAC,CAAC;IAEhG,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED;;;;;;;;;;;;;;;GAeG;AACH,MAAM,CAAC,KAAK,UAAU,oBAAoB,CACxC,mBAA+B,EAC/B,QAAgB,EAChB,KAAc,EACd,sBAA+B;IAE/B,wCAAwC;IACxC,MAAM,EAAE,CAAC,QAAQ,CAAC,MAAM,CAAC,QAAQ,EAAE,EAAE,CAAC,SAAS,CAAC,IAAI,CAAC,CAAC;IAEtD,MAAM,QAAQ,GAAG,IAAI,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;IAEzC,MAAM,2BAA2B,GAAgC;QAC/D,sBAAsB,EAAE,sBAAsB;QAC9C,YAAY,EAAE,QAAQ;QACtB,KAAK,EAAE,KAAK,IAAI,QAAQ;KACzB,CAAC;IAEF,MAAM,EAAE,WAAW,EAAE,UAAU,EAAE,GAAG,uBAAuB,CAAC;QAC1D,MAAM,EAAE,2BAA2B;QACnC,QAAQ;QACR,QAAQ,EAAE,QAAQ;KACnB,CAAC,CAAC;IAEH,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,GAAG,mBAAmB,CAAC,OAAO,EAAE,0BAA0B,EAAE;QACvF,MAAM,EAAE,MAAM;QACd,wEAAwE;QACxE,IAAI,EAAE,QAAQ,CAAC,KAAK,CAAC,UAAU,EAAE,CAA+B;QAChE,MAAM,EAAE,MAAM;QACd,OAAO,EAAE;YACP,aAAa,EAAE,UAAU,mBAAmB,CAAC,WAAqB,EAAE;YACpE,cAAc,EAAE,WAAW;SAC5B;KACF,CAAC,CAAC;IAEH,cAAc,CAAC,mBAAmB,EAAE,QAAQ,CAAC,CAAC;IAE9C,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;QACjB,MAAM,MAAM,aAAa,CAAC,QAAQ,EAAE,cAAc,QAAQ,GAAG,CAAC,CAAC;IACjE,CAAC;IAED,MAAM,IAAI,GAAG,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAA+B,CAAC;IAEnE,MAAM,WAAW,GAAG,MAAM,mBAAmB,CAAC,iBAAiB,CAC7D,oFAAoF,IAAI,CAAC,EAAE,GAAG,CAC/F,CAAC;IAEF,OAAO,WAA6B,CAAC;AACvC,CAAC"}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Count the data rows in a CSV, excluding its header.
|
|
3
|
+
*
|
|
4
|
+
* Parses rather than counting newlines: a quoted field may contain line breaks, so `\n` counting
|
|
5
|
+
* over-reports and would budget for uploads that never happen. The parser options match the ones
|
|
6
|
+
* the upload itself uses, so the count and the work cannot disagree.
|
|
7
|
+
*
|
|
8
|
+
* This reads the file a second time rather than buffering the parsed rows. The trade is a cheap
|
|
9
|
+
* sequential local read against unbounded memory on exactly the large inputs where checking the
|
|
10
|
+
* budget matters most.
|
|
11
|
+
*
|
|
12
|
+
* @param filePath - Path to the CSV.
|
|
13
|
+
* @returns The number of data rows.
|
|
14
|
+
*/
|
|
15
|
+
export declare function countCsvRows(filePath: string): Promise<number>;
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
/*
|
|
2
|
+
* Copyright (c) 2026, Clay Chipps.
|
|
3
|
+
*
|
|
4
|
+
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
5
|
+
* you may not use this file except in compliance with the License.
|
|
6
|
+
* You may obtain a copy of the License at
|
|
7
|
+
*
|
|
8
|
+
* http://www.apache.org/licenses/LICENSE-2.0
|
|
9
|
+
*
|
|
10
|
+
* Unless required by applicable law or agreed to in writing, software
|
|
11
|
+
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
12
|
+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
13
|
+
* See the License for the specific language governing permissions and
|
|
14
|
+
* limitations under the License.
|
|
15
|
+
*/
|
|
16
|
+
import fs from 'node:fs';
|
|
17
|
+
import { parse } from 'csv-parse';
|
|
18
|
+
/**
|
|
19
|
+
* Count the data rows in a CSV, excluding its header.
|
|
20
|
+
*
|
|
21
|
+
* Parses rather than counting newlines: a quoted field may contain line breaks, so `\n` counting
|
|
22
|
+
* over-reports and would budget for uploads that never happen. The parser options match the ones
|
|
23
|
+
* the upload itself uses, so the count and the work cannot disagree.
|
|
24
|
+
*
|
|
25
|
+
* This reads the file a second time rather than buffering the parsed rows. The trade is a cheap
|
|
26
|
+
* sequential local read against unbounded memory on exactly the large inputs where checking the
|
|
27
|
+
* budget matters most.
|
|
28
|
+
*
|
|
29
|
+
* @param filePath - Path to the CSV.
|
|
30
|
+
* @returns The number of data rows.
|
|
31
|
+
*/
|
|
32
|
+
export async function countCsvRows(filePath) {
|
|
33
|
+
const parser = fs.createReadStream(filePath).pipe(parse({ bom: true, columns: true }));
|
|
34
|
+
let rows = 0;
|
|
35
|
+
for await (const record of parser) {
|
|
36
|
+
// The record itself is not needed, only that the parser produced one.
|
|
37
|
+
void record;
|
|
38
|
+
rows++;
|
|
39
|
+
}
|
|
40
|
+
return rows;
|
|
41
|
+
}
|
|
42
|
+
//# sourceMappingURL=countCsvRows.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"countCsvRows.js","sourceRoot":"","sources":["../../src/common/countCsvRows.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AAElC;;;;;;;;;;;;;GAaG;AACH,MAAM,CAAC,KAAK,UAAU,YAAY,CAAC,QAAgB;IACjD,MAAM,MAAM,GAAG,EAAE,CAAC,gBAAgB,CAAC,QAAQ,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;IAEvF,IAAI,IAAI,GAAG,CAAC,CAAC;IACb,IAAI,KAAK,EAAE,MAAM,MAAM,IAAI,MAAM,EAAE,CAAC;QAClC,sEAAsE;QACtE,KAAK,MAAM,CAAC;QACZ,IAAI,EAAE,CAAC;IACT,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC"}
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# flags.max-api-usage.summary
|
|
2
|
+
|
|
3
|
+
Maximum percentage of the org's remaining API requests this run may consume.
|
|
4
|
+
|
|
5
|
+
# flags.max-api-usage.description
|
|
6
|
+
|
|
7
|
+
Checked before any request is made, so a run that would exceed its budget stops without doing partial work. The percentage applies to the requests the org has left today, not to its daily maximum — an org that has already used most of its allocation gets a proportionally smaller budget.
|
|
8
|
+
|
|
9
|
+
Note that uploading a file costs two API requests, not one: the upload itself, and a follow-up query for the resulting ContentDocumentId.
|
|
10
|
+
|
|
11
|
+
A run that cannot finish within the org's remaining requests is refused regardless of this value. To allow a larger share, raise it; to allow the maximum, pass 100.
|
|
12
|
+
|
|
13
|
+
If the org's remaining allocation can't be read — reading it falls back to the limits API, which needs the "View Setup and Configuration" permission — the command warns and proceeds rather than failing.
|
|
14
|
+
|
|
15
|
+
# warning.budgetUnavailable
|
|
16
|
+
|
|
17
|
+
Could not read the org's remaining API requests, so the --max-api-usage budget was not checked: %s
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@simplysf/simply-data",
|
|
3
3
|
"description": "Utilities for working with data",
|
|
4
|
-
"version": "2.
|
|
4
|
+
"version": "2.4.1",
|
|
5
5
|
"author": "@ClayChipps",
|
|
6
6
|
"homepage": "https://github.com/SimplySF/simply-node",
|
|
7
7
|
"bugs": "https://github.com/SimplySF/simply-node/issues",
|
|
@@ -65,7 +65,7 @@
|
|
|
65
65
|
"@salesforce/core": "^8.30.0",
|
|
66
66
|
"@salesforce/kit": "^3.2.6",
|
|
67
67
|
"@salesforce/sf-plugins-core": "^12.2.16",
|
|
68
|
-
"@simplysf/simply-core": "^1.
|
|
68
|
+
"@simplysf/simply-core": "^1.5.0",
|
|
69
69
|
"@simplysf/simply-plugin-kit": "^1.0.4",
|
|
70
70
|
"csv-parse": "^7.0.2",
|
|
71
71
|
"p-queue": "^9.2.0"
|
|
@@ -214,5 +214,5 @@
|
|
|
214
214
|
"output": []
|
|
215
215
|
}
|
|
216
216
|
},
|
|
217
|
-
"gitHead": "
|
|
217
|
+
"gitHead": "7027cc7497e4b5852f51abb3713757f9eae85cd7"
|
|
218
218
|
}
|