@simplysf/simply-data 2.3.26 → 2.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +67 -7
- package/lib/commands/simply/data/file/upload.d.ts +1 -0
- package/lib/commands/simply/data/file/upload.js +4 -0
- package/lib/commands/simply/data/file/upload.js.map +1 -1
- package/lib/commands/simply/data/files/download.d.ts +1 -0
- package/lib/commands/simply/data/files/download.js +6 -0
- package/lib/commands/simply/data/files/download.js.map +1 -1
- package/lib/commands/simply/data/files/upload.d.ts +1 -0
- package/lib/commands/simply/data/files/upload.js +9 -0
- package/lib/commands/simply/data/files/upload.js.map +1 -1
- package/lib/common/apiBudgetFlag.d.ts +26 -0
- package/lib/common/apiBudgetFlag.js +61 -0
- package/lib/common/apiBudgetFlag.js.map +1 -0
- package/lib/common/apiCost.d.ts +23 -0
- package/lib/common/apiCost.js +41 -0
- package/lib/common/apiCost.js.map +1 -0
- package/lib/common/contentVersionUtils.d.ts +8 -3
- package/lib/common/contentVersionUtils.js +44 -9
- package/lib/common/contentVersionUtils.js.map +1 -1
- package/lib/common/countCsvRows.d.ts +15 -0
- package/lib/common/countCsvRows.js +42 -0
- package/lib/common/countCsvRows.js.map +1 -0
- package/messages/apiBudget.md +17 -0
- package/messages/simply.data.file.upload.md +3 -1
- package/messages/simply.data.files.upload.md +2 -0
- package/package.json +3 -3
package/README.md
CHANGED
|
@@ -31,14 +31,17 @@ Upload a file to a Salesforce org.
|
|
|
31
31
|
```
|
|
32
32
|
USAGE
|
|
33
33
|
$ sf simply data file upload -o <value> --file-path <value> [--json] [--flags-dir <value>] [--api-version <value>]
|
|
34
|
-
[--first-publish-location-id <value>] [--title <value>]
|
|
34
|
+
[--max-api-usage <value>] [--first-publish-location-id <value>] [--title <value>]
|
|
35
35
|
|
|
36
36
|
FLAGS
|
|
37
37
|
-o, --target-org=<value> (required) Username or alias of the target org. Not required if the
|
|
38
38
|
`target-org` configuration variable is already set.
|
|
39
39
|
--api-version=<value> Override the api version used for api requests made by this command
|
|
40
|
-
--file-path=<value> (required) Path to the file to upload.
|
|
40
|
+
--file-path=<value> (required) Path to the file to upload. May be relative or absolute; only the
|
|
41
|
+
file's name is sent to the org.
|
|
41
42
|
--first-publish-location-id=<value> Specify a record Id that the file should be linked to.
|
|
43
|
+
--max-api-usage=<value> [default: 20] Maximum percentage of the org's remaining API requests this run
|
|
44
|
+
may consume.
|
|
42
45
|
--title=<value> Specify the title for the file being uploaded.
|
|
43
46
|
|
|
44
47
|
GLOBAL FLAGS
|
|
@@ -50,13 +53,32 @@ DESCRIPTION
|
|
|
50
53
|
|
|
51
54
|
Uploads a file to a Salesforce org.
|
|
52
55
|
|
|
56
|
+
Only the file's name is sent to the org — Salesforce stores it as the ContentVersion's PathOnClient and derives
|
|
57
|
+
FileExtension and FileType from it — so the local directory the file came from is never uploaded.
|
|
58
|
+
|
|
53
59
|
EXAMPLES
|
|
54
60
|
$ sf simply data file upload --file-path fileToUpload.txt --target-org myTargetOrg
|
|
55
61
|
|
|
56
62
|
$ sf simply data file upload --file-path fileToUpload.txt --first-publish-location-id 0019000000DmehK --target-org myTargetOrg
|
|
63
|
+
|
|
64
|
+
FLAG DESCRIPTIONS
|
|
65
|
+
--max-api-usage=<value> Maximum percentage of the org's remaining API requests this run may consume.
|
|
66
|
+
|
|
67
|
+
Checked before any request is made, so a run that would exceed its budget stops without doing partial work. The
|
|
68
|
+
percentage applies to the requests the org has left today, not to its daily maximum — an org that has already used
|
|
69
|
+
most of its allocation gets a proportionally smaller budget.
|
|
70
|
+
|
|
71
|
+
Note that uploading a file costs two API requests, not one: the upload itself, and a follow-up query for the
|
|
72
|
+
resulting ContentDocumentId.
|
|
73
|
+
|
|
74
|
+
A run that cannot finish within the org's remaining requests is refused regardless of this value. To allow a larger
|
|
75
|
+
share, raise it; to allow the maximum, pass 100.
|
|
76
|
+
|
|
77
|
+
If the org's remaining allocation can't be read — reading it falls back to the limits API, which needs the "View
|
|
78
|
+
Setup and Configuration" permission — the command warns and proceeds rather than failing.
|
|
57
79
|
```
|
|
58
80
|
|
|
59
|
-
_See code: [lib/commands/simply/data/file/upload.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.
|
|
81
|
+
_See code: [lib/commands/simply/data/file/upload.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.4.0/packages/simply-data/lib/commands/simply/data/file/upload.js)_
|
|
60
82
|
|
|
61
83
|
## `sf simply data files download`
|
|
62
84
|
|
|
@@ -65,12 +87,14 @@ Download files from a Salesforce org.
|
|
|
65
87
|
```
|
|
66
88
|
USAGE
|
|
67
89
|
$ sf simply data files download -o <value> --where-content-version <value> [--json] [--flags-dir <value>] [--api-version
|
|
68
|
-
<value>] [--max-parallel-jobs <value>]
|
|
90
|
+
<value>] [--max-api-usage <value>] [--max-parallel-jobs <value>]
|
|
69
91
|
|
|
70
92
|
FLAGS
|
|
71
93
|
-o, --target-org=<value> (required) Username or alias of the target org. Not required if the `target-org`
|
|
72
94
|
configuration variable is already set.
|
|
73
95
|
--api-version=<value> Override the api version used for api requests made by this command
|
|
96
|
+
--max-api-usage=<value> [default: 20] Maximum percentage of the org's remaining API requests this run may
|
|
97
|
+
consume.
|
|
74
98
|
--max-parallel-jobs=<value> [default: 1] Maximum number of parallel jobs.
|
|
75
99
|
--where-content-version=<value> (required) WHERE clause for ContentVersion query.
|
|
76
100
|
|
|
@@ -91,6 +115,21 @@ EXAMPLES
|
|
|
91
115
|
$ sf simply data files download --where-content-version "IsLatest=true" --max-parallel-jobs 5 --target-org myTargetOrg
|
|
92
116
|
|
|
93
117
|
FLAG DESCRIPTIONS
|
|
118
|
+
--max-api-usage=<value> Maximum percentage of the org's remaining API requests this run may consume.
|
|
119
|
+
|
|
120
|
+
Checked before any request is made, so a run that would exceed its budget stops without doing partial work. The
|
|
121
|
+
percentage applies to the requests the org has left today, not to its daily maximum — an org that has already used
|
|
122
|
+
most of its allocation gets a proportionally smaller budget.
|
|
123
|
+
|
|
124
|
+
Note that uploading a file costs two API requests, not one: the upload itself, and a follow-up query for the
|
|
125
|
+
resulting ContentDocumentId.
|
|
126
|
+
|
|
127
|
+
A run that cannot finish within the org's remaining requests is refused regardless of this value. To allow a larger
|
|
128
|
+
share, raise it; to allow the maximum, pass 100.
|
|
129
|
+
|
|
130
|
+
If the org's remaining allocation can't be read — reading it falls back to the limits API, which needs the "View
|
|
131
|
+
Setup and Configuration" permission — the command warns and proceeds rather than failing.
|
|
132
|
+
|
|
94
133
|
--max-parallel-jobs=<value> Maximum number of parallel jobs.
|
|
95
134
|
|
|
96
135
|
By default the plugin will only process a single file download at a time. You can increase this value to allow for
|
|
@@ -101,7 +140,7 @@ FLAG DESCRIPTIONS
|
|
|
101
140
|
Provide a WHERE clause to allow the plugin to specify which ContentVersion records should be downloaded.
|
|
102
141
|
```
|
|
103
142
|
|
|
104
|
-
_See code: [lib/commands/simply/data/files/download.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.
|
|
143
|
+
_See code: [lib/commands/simply/data/files/download.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.4.0/packages/simply-data/lib/commands/simply/data/files/download.js)_
|
|
105
144
|
|
|
106
145
|
## `sf simply data files upload`
|
|
107
146
|
|
|
@@ -110,13 +149,15 @@ Upload files to a Salesforce org.
|
|
|
110
149
|
```
|
|
111
150
|
USAGE
|
|
112
151
|
$ sf simply data files upload -o <value> --file-path <value> [--json] [--flags-dir <value>] [--api-version <value>]
|
|
113
|
-
[--max-parallel-jobs <value>]
|
|
152
|
+
[--max-api-usage <value>] [--max-parallel-jobs <value>]
|
|
114
153
|
|
|
115
154
|
FLAGS
|
|
116
155
|
-o, --target-org=<value> (required) Username or alias of the target org. Not required if the `target-org`
|
|
117
156
|
configuration variable is already set.
|
|
118
157
|
--api-version=<value> Override the api version used for api requests made by this command
|
|
119
158
|
--file-path=<value> (required) Path to the csv file that specifies the upload.
|
|
159
|
+
--max-api-usage=<value> [default: 20] Maximum percentage of the org's remaining API requests this run may
|
|
160
|
+
consume.
|
|
120
161
|
--max-parallel-jobs=<value> [default: 1] Maximum number of parallel jobs.
|
|
121
162
|
|
|
122
163
|
GLOBAL FLAGS
|
|
@@ -140,13 +181,32 @@ FLAG DESCRIPTIONS
|
|
|
140
181
|
The csv file must specify the columns PathOnClient and Title. Optionally, a FirstPublishLocationId can be specified
|
|
141
182
|
to have it linked directly to a Salesforce record after upload.
|
|
142
183
|
|
|
184
|
+
PathOnClient is the local path each file is read from, and may be relative or absolute. Only the file's name is sent
|
|
185
|
+
to the org — Salesforce stores it as the ContentVersion's PathOnClient and derives FileExtension and FileType from
|
|
186
|
+
it — so the local directory the file came from is never uploaded.
|
|
187
|
+
|
|
188
|
+
--max-api-usage=<value> Maximum percentage of the org's remaining API requests this run may consume.
|
|
189
|
+
|
|
190
|
+
Checked before any request is made, so a run that would exceed its budget stops without doing partial work. The
|
|
191
|
+
percentage applies to the requests the org has left today, not to its daily maximum — an org that has already used
|
|
192
|
+
most of its allocation gets a proportionally smaller budget.
|
|
193
|
+
|
|
194
|
+
Note that uploading a file costs two API requests, not one: the upload itself, and a follow-up query for the
|
|
195
|
+
resulting ContentDocumentId.
|
|
196
|
+
|
|
197
|
+
A run that cannot finish within the org's remaining requests is refused regardless of this value. To allow a larger
|
|
198
|
+
share, raise it; to allow the maximum, pass 100.
|
|
199
|
+
|
|
200
|
+
If the org's remaining allocation can't be read — reading it falls back to the limits API, which needs the "View
|
|
201
|
+
Setup and Configuration" permission — the command warns and proceeds rather than failing.
|
|
202
|
+
|
|
143
203
|
--max-parallel-jobs=<value> Maximum number of parallel jobs.
|
|
144
204
|
|
|
145
205
|
By default the plugin will only process a single file upload at a time. You can increase this value to allow for
|
|
146
206
|
quasi concurrent uploads. Please note that setting this value too high can cause performance issues.
|
|
147
207
|
```
|
|
148
208
|
|
|
149
|
-
_See code: [lib/commands/simply/data/files/upload.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.
|
|
209
|
+
_See code: [lib/commands/simply/data/files/upload.js](https://github.com/SimplySF/simply-node/blob/@simplysf/simply-data@2.4.0/packages/simply-data/lib/commands/simply/data/files/upload.js)_
|
|
150
210
|
<!-- commandsstop -->
|
|
151
211
|
|
|
152
212
|
## License
|
|
@@ -9,6 +9,7 @@ export default class DataFileUpload extends SfCommand<ContentVersion> {
|
|
|
9
9
|
'file-path': import("@oclif/core/interfaces").OptionFlag<string, import("@oclif/core/interfaces").CustomOptions>;
|
|
10
10
|
'first-publish-location-id': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
|
|
11
11
|
title: import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
|
|
12
|
+
'max-api-usage': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
12
13
|
'api-version': import("@oclif/core/interfaces").OptionFlag<string | undefined>;
|
|
13
14
|
'target-org': import("@oclif/core/interfaces").OptionFlag<import("@salesforce/core").Org>;
|
|
14
15
|
'flags-dir': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
|
|
@@ -17,6 +17,8 @@ import { Messages } from '@salesforce/core';
|
|
|
17
17
|
import { SfCommand, Flags } from '@salesforce/sf-plugins-core';
|
|
18
18
|
import { requireConnection, targetOrgFlags } from '@simplysf/simply-plugin-kit';
|
|
19
19
|
import { uploadContentVersion } from '../../../../common/contentVersionUtils.js';
|
|
20
|
+
import { apiBudgetFlags, assertApiBudget } from '../../../../common/apiBudgetFlag.js';
|
|
21
|
+
import { REQUESTS_PER_UPLOAD } from '../../../../common/apiCost.js';
|
|
20
22
|
Messages.importMessagesDirectoryFromMetaUrl(import.meta.url);
|
|
21
23
|
const messages = Messages.loadMessages('@simplysf/simply-data', 'simply.data.file.upload');
|
|
22
24
|
/** Uploads a single local file to a Salesforce org as a `ContentVersion`. */
|
|
@@ -27,6 +29,7 @@ export default class DataFileUpload extends SfCommand {
|
|
|
27
29
|
static flags = {
|
|
28
30
|
...SfCommand.baseFlags,
|
|
29
31
|
...targetOrgFlags,
|
|
32
|
+
...apiBudgetFlags,
|
|
30
33
|
'file-path': Flags.directory({
|
|
31
34
|
summary: messages.getMessage('flags.file-path.summary'),
|
|
32
35
|
required: true,
|
|
@@ -43,6 +46,7 @@ export default class DataFileUpload extends SfCommand {
|
|
|
43
46
|
const { flags } = await this.parse(DataFileUpload);
|
|
44
47
|
// Authorize to the target org
|
|
45
48
|
const targetOrgConnection = requireConnection(flags);
|
|
49
|
+
await assertApiBudget(targetOrgConnection, REQUESTS_PER_UPLOAD, flags['max-api-usage'], (message) => this.warn(message));
|
|
46
50
|
this.spinner.start('Uploading file', '', { stdout: true });
|
|
47
51
|
const contentVersion = await uploadContentVersion(targetOrgConnection, flags['file-path'], flags['title'], flags['first-publish-location-id']);
|
|
48
52
|
this.spinner.stop();
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"upload.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/file/upload.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;
|
|
1
|
+
{"version":3,"file":"upload.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/file/upload.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AACjF,OAAO,EAAE,cAAc,EAAE,eAAe,EAAE,MAAM,qCAAqC,CAAC;AACtF,OAAO,EAAE,mBAAmB,EAAE,MAAM,+BAA+B,CAAC;AAGpE,QAAQ,CAAC,kCAAkC,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAC7D,MAAM,QAAQ,GAAG,QAAQ,CAAC,YAAY,CAAC,uBAAuB,EAAE,yBAAyB,CAAC,CAAC;AAE3F,6EAA6E;AAC7E,MAAM,CAAC,OAAO,OAAO,cAAe,SAAQ,SAAyB;IAC5D,MAAM,CAAU,OAAO,GAAG,QAAQ,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;IACzD,MAAM,CAAU,WAAW,GAAG,QAAQ,CAAC,UAAU,CAAC,aAAa,CAAC,CAAC;IACjE,MAAM,CAAU,QAAQ,GAAG,QAAQ,CAAC,WAAW,CAAC,UAAU,CAAC,CAAC;IAE5D,MAAM,CAAU,KAAK,GAAG;QAC7B,GAAG,SAAS,CAAC,SAAS;QACtB,GAAG,cAAc;QACjB,GAAG,cAAc;QACjB,WAAW,EAAE,KAAK,CAAC,SAAS,CAAC;YAC3B,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,yBAAyB,CAAC;YACvD,QAAQ,EAAE,IAAI;SACf,CAAC;QACF,2BAA2B,EAAE,KAAK,CAAC,MAAM,CAAC;YACxC,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,yCAAyC,CAAC;SACxE,CAAC;QACF,KAAK,EAAE,KAAK,CAAC,MAAM,CAAC;YAClB,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,qBAAqB,CAAC;SACpD,CAAC;KACH,CAAC;IAEF,6CAA6C;IACtC,KAAK,CAAC,GAAG;QACd,MAAM,EAAE,KAAK,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,cAAc,CAAC,CAAC;QAEnD,8BAA8B;QAC9B,MAAM,mBAAmB,GAAG,iBAAiB,CAAC,KAAK,CAAC,CAAC;QAErD,MAAM,eAAe,CAAC,mBAAmB,EAAE,mBAAmB,EAAE,KAAK,CAAC,eAAe,CAAC,EAAE,CAAC,OAAO,EAAE,EAAE,CAClG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,CACnB,CAAC;QAEF,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,gBAAgB,EAAE,EAAE,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAE3D,MAAM,cAAc,GAAG,MAAM,oBAAoB,CAC/C,mBAAmB,EACnB,KAAK,CAAC,WAAW,CAAC,EAClB,KAAK,CAAC,OAAO,CAAC,EACd,KAAK,CAAC,2BAA2B,CAAC,CACnC,CAAC;QAEF,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;QAEpB,OAAO,cAAc,CAAC;IACxB,CAAC"}
|
|
@@ -14,6 +14,7 @@ export default class DataFilesDownload extends SfCommand<void> {
|
|
|
14
14
|
static readonly flags: {
|
|
15
15
|
'max-parallel-jobs': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
16
16
|
'where-content-version': import("@oclif/core/interfaces").OptionFlag<string, import("@oclif/core/interfaces").CustomOptions>;
|
|
17
|
+
'max-api-usage': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
17
18
|
'api-version': import("@oclif/core/interfaces").OptionFlag<string | undefined>;
|
|
18
19
|
'target-org': import("@oclif/core/interfaces").OptionFlag<import("@salesforce/core").Org>;
|
|
19
20
|
'flags-dir': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
|
|
@@ -20,6 +20,8 @@ import { SfCommand, Flags } from '@salesforce/sf-plugins-core';
|
|
|
20
20
|
import { requireConnection, targetOrgFlags } from '@simplysf/simply-plugin-kit';
|
|
21
21
|
import { createCsvFileWriter } from '@simplysf/simply-core';
|
|
22
22
|
import { downloadContentVersion } from '../../../../common/contentVersionUtils.js';
|
|
23
|
+
import { apiBudgetFlags, assertApiBudget } from '../../../../common/apiBudgetFlag.js';
|
|
24
|
+
import { REQUESTS_PER_DOWNLOAD, requestsForQuery } from '../../../../common/apiCost.js';
|
|
23
25
|
Messages.importMessagesDirectoryFromMetaUrl(import.meta.url);
|
|
24
26
|
const messages = Messages.loadMessages('@simplysf/simply-data', 'simply.data.files.download');
|
|
25
27
|
/**
|
|
@@ -37,6 +39,7 @@ export default class DataFilesDownload extends SfCommand {
|
|
|
37
39
|
static flags = {
|
|
38
40
|
...SfCommand.baseFlags,
|
|
39
41
|
...targetOrgFlags,
|
|
42
|
+
...apiBudgetFlags,
|
|
40
43
|
'max-parallel-jobs': Flags.integer({
|
|
41
44
|
summary: messages.getMessage('flags.max-parallel-jobs.summary'),
|
|
42
45
|
description: messages.getMessage('flags.max-parallel-jobs.description'),
|
|
@@ -59,6 +62,9 @@ export default class DataFilesDownload extends SfCommand {
|
|
|
59
62
|
scanAll: false,
|
|
60
63
|
});
|
|
61
64
|
const contentVersionDownloads = result.records;
|
|
65
|
+
// The query above already ran through jsforce, so `connection.limitInfo` is populated and the
|
|
66
|
+
// budget check gets the free header path rather than spending a request on the limits API.
|
|
67
|
+
await assertApiBudget(targetOrgConnection, contentVersionDownloads.length * REQUESTS_PER_DOWNLOAD + requestsForQuery(contentVersionDownloads.length), flags['max-api-usage'], (message) => this.warn(message));
|
|
62
68
|
this.spinner.start('Initializing file download', '\n', { stdout: true });
|
|
63
69
|
if (!fs.existsSync('download')) {
|
|
64
70
|
fs.mkdirSync('download');
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"download.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/files/download.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,MAAM,MAAM,SAAS,CAAC;AAC7B,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,mBAAmB,EAAE,MAAM,uBAAuB,CAAC;AAC5D,OAAO,EAAE,sBAAsB,EAAE,MAAM,2CAA2C,CAAC;
|
|
1
|
+
{"version":3,"file":"download.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/files/download.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,MAAM,MAAM,SAAS,CAAC;AAC7B,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,mBAAmB,EAAE,MAAM,uBAAuB,CAAC;AAC5D,OAAO,EAAE,sBAAsB,EAAE,MAAM,2CAA2C,CAAC;AACnF,OAAO,EAAE,cAAc,EAAE,eAAe,EAAE,MAAM,qCAAqC,CAAC;AACtF,OAAO,EAAE,qBAAqB,EAAE,gBAAgB,EAAE,MAAM,+BAA+B,CAAC;AAGxF,QAAQ,CAAC,kCAAkC,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAC7D,MAAM,QAAQ,GAAG,QAAQ,CAAC,YAAY,CAAC,uBAAuB,EAAE,4BAA4B,CAAC,CAAC;AAE9F;;;;;;;GAOG;AACH,MAAM,CAAC,OAAO,OAAO,iBAAkB,SAAQ,SAAe;IACrD,MAAM,CAAU,OAAO,GAAG,QAAQ,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;IACzD,MAAM,CAAU,WAAW,GAAG,QAAQ,CAAC,UAAU,CAAC,aAAa,CAAC,CAAC;IACjE,MAAM,CAAU,QAAQ,GAAG,QAAQ,CAAC,WAAW,CAAC,UAAU,CAAC,CAAC;IAE5D,MAAM,CAAU,KAAK,GAAG;QAC7B,GAAG,SAAS,CAAC,SAAS;QACtB,GAAG,cAAc;QACjB,GAAG,cAAc;QACjB,mBAAmB,EAAE,KAAK,CAAC,OAAO,CAAC;YACjC,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,iCAAiC,CAAC;YAC/D,WAAW,EAAE,QAAQ,CAAC,UAAU,CAAC,qCAAqC,CAAC;YACvE,OAAO,EAAE,CAAC;SACX,CAAC;QACF,uBAAuB,EAAE,KAAK,CAAC,MAAM,CAAC;YACpC,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,qCAAqC,CAAC;YACnE,WAAW,EAAE,QAAQ,CAAC,UAAU,CAAC,yCAAyC,CAAC;YAC3E,QAAQ,EAAE,IAAI;SACf,CAAC;KACH,CAAC;IAEK,KAAK,CAAC,GAAG;QACd,MAAM,EAAE,KAAK,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,iBAAiB,CAAC,CAAC;QAEtD,8BAA8B;QAC9B,MAAM,mBAAmB,GAAG,iBAAiB,CAAC,KAAK,CAAC,CAAC;QAErD,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,oBAAoB,EAAE,IAAI,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAEjE,MAAM,MAAM,GAAG,MAAM,mBAAmB,CAAC,KAAK,CAC5C,oKAAoK,KAAK,CAAC,uBAAuB,CAAC,EAAE,EACpM;YACE,SAAS,EAAE,IAAI;YACf,QAAQ,EAAG,IAAI,CAAC,gBAAgB,CAAC,OAAO,CAAC,qBAAqB,CAAC,CAAC,KAAgB,IAAI,MAAM;YAC1F,OAAO,EAAE,KAAK;SACf,CACF,CAAC;QAEF,MAAM,uBAAuB,GAAG,MAAM,CAAC,OAAO,CAAC;QAE/C,8FAA8F;QAC9F,2FAA2F;QAC3F,MAAM,eAAe,CACnB,mBAAmB,EACnB,uBAAuB,CAAC,MAAM,GAAG,qBAAqB,GAAG,gBAAgB,CAAC,uBAAuB,CAAC,MAAM,CAAC,EACzG,KAAK,CAAC,eAAe,CAAC,EACtB,CAAC,OAAO,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,CAChC,CAAC;QAEF,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,4BAA4B,EAAE,IAAI,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAEzE,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,UAAU,CAAC,EAAE,CAAC;YAC/B,EAAE,CAAC,SAAS,CAAC,UAAU,CAAC,CAAC;QAC3B,CAAC;QAED,MAAM,aAAa,GAAG,mBAAmB,CAAC,sBAAsB,EAAE;YAChE,IAAI;YACJ,mBAAmB;YACnB,aAAa;YACb,aAAa;YACb,eAAe;YACf,UAAU;YACV,wBAAwB;YACxB,UAAU;YACV,cAAc;YACd,OAAO;YACP,UAAU;SACX,CAAC,CAAC;QAEH,MAAM,WAAW,GAAG,mBAAmB,CAAC,oBAAoB,EAAE;YAC5D,IAAI;YACJ,mBAAmB;YACnB,aAAa;YACb,aAAa;YACb,eAAe;YACf,UAAU;YACV,wBAAwB;YACxB,UAAU;YACV,cAAc;YACd,OAAO;YACP,OAAO;SACR,CAAC,CAAC;QAEH,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,mBAAmB,EAAE,IAAI,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAEhE,MAAM,aAAa,GAAG,IAAI,MAAM,CAAC,EAAE,WAAW,EAAE,KAAK,CAAC,mBAAmB,CAAC,EAAE,CAAC,CAAC;QAE9E,IAAI,KAAK,GAAG,CAAC,CAAC;QACd,aAAa,CAAC,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE;YAC3B,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,cAAc,KAAK,WAAW,aAAa,CAAC,IAAI,cAAc,aAAa,CAAC,OAAO,IAAI,CAAC;QAChH,CAAC,CAAC,CAAC;QAEH,aAAa,CAAC,EAAE,CAAC,WAAW,EAAE,GAAG,EAAE;YACjC,KAAK,EAAE,CAAC;YACR,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,cAAc,KAAK,WAAW,aAAa,CAAC,IAAI,cAAc,aAAa,CAAC,OAAO,IAAI,CAAC;QAChH,CAAC,CAAC,CAAC;QAEH,KAAK,MAAM,sBAAsB,IAAI,uBAAuB,EAAE,CAAC;YAC7D,KAAK,aAAa,CAAC,GAAG,CAAC,KAAK,IAAI,EAAE;gBAChC,IAAI,CAAC;oBACH,sBAAsB,CAAC,QAAQ,GAAG,MAAM,sBAAsB,CAC5D,mBAAmB,EACnB,sBAAsB,EACtB,UAAU,CACX,CAAC;oBACF,MAAM,aAAa,CAAC,KAAK,CAAC,sBAAsB,CAAC,CAAC;gBACpD,CAAC;gBAAC,OAAO,KAAK,EAAE,CAAC;oBACf,sBAAsB,CAAC,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;oBAC7C,MAAM,WAAW,CAAC,KAAK,CAAC,sBAAsB,CAAC,CAAC;gBAClD,CAAC;YACH,CAAC,CAAC,CAAC;QACL,CAAC;QAED,MAAM,aAAa,CAAC,MAAM,EAAE,CAAC;QAC7B,MAAM,OAAO,CAAC,GAAG,CAAC,CAAC,aAAa,CAAC,GAAG,EAAE,EAAE,WAAW,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC;QAE5D,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;IACtB,CAAC"}
|
|
@@ -15,6 +15,7 @@ export default class DataFilesUpload extends SfCommand<void> {
|
|
|
15
15
|
static readonly flags: {
|
|
16
16
|
'file-path': import("@oclif/core/interfaces").OptionFlag<string, import("@oclif/core/interfaces").CustomOptions>;
|
|
17
17
|
'max-parallel-jobs': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
18
|
+
'max-api-usage': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
18
19
|
'api-version': import("@oclif/core/interfaces").OptionFlag<string | undefined>;
|
|
19
20
|
'target-org': import("@oclif/core/interfaces").OptionFlag<import("@salesforce/core").Org>;
|
|
20
21
|
'flags-dir': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
|
|
@@ -21,6 +21,9 @@ import { SfCommand, Flags } from '@salesforce/sf-plugins-core';
|
|
|
21
21
|
import { requireConnection, targetOrgFlags } from '@simplysf/simply-plugin-kit';
|
|
22
22
|
import { createCsvFileWriter } from '@simplysf/simply-core';
|
|
23
23
|
import { uploadContentVersion } from '../../../../common/contentVersionUtils.js';
|
|
24
|
+
import { apiBudgetFlags, assertApiBudget } from '../../../../common/apiBudgetFlag.js';
|
|
25
|
+
import { countCsvRows } from '../../../../common/countCsvRows.js';
|
|
26
|
+
import { REQUESTS_PER_UPLOAD } from '../../../../common/apiCost.js';
|
|
24
27
|
Messages.importMessagesDirectoryFromMetaUrl(import.meta.url);
|
|
25
28
|
const messages = Messages.loadMessages('@simplysf/simply-data', 'simply.data.files.upload');
|
|
26
29
|
/**
|
|
@@ -39,6 +42,7 @@ export default class DataFilesUpload extends SfCommand {
|
|
|
39
42
|
static flags = {
|
|
40
43
|
...SfCommand.baseFlags,
|
|
41
44
|
...targetOrgFlags,
|
|
45
|
+
...apiBudgetFlags,
|
|
42
46
|
'file-path': Flags.directory({
|
|
43
47
|
summary: messages.getMessage('flags.file-path.summary'),
|
|
44
48
|
description: messages.getMessage('flags.file-path.description'),
|
|
@@ -54,6 +58,11 @@ export default class DataFilesUpload extends SfCommand {
|
|
|
54
58
|
const { flags } = await this.parse(DataFilesUpload);
|
|
55
59
|
// Authorize to the target org
|
|
56
60
|
const targetOrgConnection = requireConnection(flags);
|
|
61
|
+
// The upload streams this CSV row by row, so the row count is not otherwise known until the
|
|
62
|
+
// work is already underway. Counting first costs one extra local read and no API requests,
|
|
63
|
+
// and it is what lets the budget be checked before anything is sent.
|
|
64
|
+
const rowCount = await countCsvRows(flags['file-path']);
|
|
65
|
+
await assertApiBudget(targetOrgConnection, rowCount * REQUESTS_PER_UPLOAD, flags['max-api-usage'], (message) => this.warn(message));
|
|
57
66
|
this.spinner.start('Initializing file upload', '\n', { stdout: true });
|
|
58
67
|
if (!fs.existsSync('upload')) {
|
|
59
68
|
fs.mkdirSync('upload');
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"upload.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/files/upload.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AAClC,OAAO,MAAM,MAAM,SAAS,CAAC;AAC7B,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,mBAAmB,EAAE,MAAM,uBAAuB,CAAC;AAC5D,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;
|
|
1
|
+
{"version":3,"file":"upload.js","sourceRoot":"","sources":["../../../../../src/commands/simply/data/files/upload.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AAClC,OAAO,MAAM,MAAM,SAAS,CAAC;AAC7B,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AAC/D,OAAO,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,6BAA6B,CAAC;AAChF,OAAO,EAAE,mBAAmB,EAAE,MAAM,uBAAuB,CAAC;AAC5D,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AACjF,OAAO,EAAE,cAAc,EAAE,eAAe,EAAE,MAAM,qCAAqC,CAAC;AACtF,OAAO,EAAE,YAAY,EAAE,MAAM,oCAAoC,CAAC;AAClE,OAAO,EAAE,mBAAmB,EAAE,MAAM,+BAA+B,CAAC;AAGpE,QAAQ,CAAC,kCAAkC,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAC7D,MAAM,QAAQ,GAAG,QAAQ,CAAC,YAAY,CAAC,uBAAuB,EAAE,0BAA0B,CAAC,CAAC;AAE5F;;;;;;;;GAQG;AACH,MAAM,CAAC,OAAO,OAAO,eAAgB,SAAQ,SAAe;IACnD,MAAM,CAAU,OAAO,GAAG,QAAQ,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;IACzD,MAAM,CAAU,WAAW,GAAG,QAAQ,CAAC,UAAU,CAAC,aAAa,CAAC,CAAC;IACjE,MAAM,CAAU,QAAQ,GAAG,QAAQ,CAAC,WAAW,CAAC,UAAU,CAAC,CAAC;IAE5D,MAAM,CAAU,KAAK,GAAG;QAC7B,GAAG,SAAS,CAAC,SAAS;QACtB,GAAG,cAAc;QACjB,GAAG,cAAc;QACjB,WAAW,EAAE,KAAK,CAAC,SAAS,CAAC;YAC3B,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,yBAAyB,CAAC;YACvD,WAAW,EAAE,QAAQ,CAAC,UAAU,CAAC,6BAA6B,CAAC;YAC/D,QAAQ,EAAE,IAAI;SACf,CAAC;QACF,mBAAmB,EAAE,KAAK,CAAC,OAAO,CAAC;YACjC,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,iCAAiC,CAAC;YAC/D,WAAW,EAAE,QAAQ,CAAC,UAAU,CAAC,qCAAqC,CAAC;YACvE,OAAO,EAAE,CAAC;SACX,CAAC;KACH,CAAC;IAEK,KAAK,CAAC,GAAG;QACd,MAAM,EAAE,KAAK,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,eAAe,CAAC,CAAC;QAEpD,8BAA8B;QAC9B,MAAM,mBAAmB,GAAG,iBAAiB,CAAC,KAAK,CAAC,CAAC;QAErD,4FAA4F;QAC5F,2FAA2F;QAC3F,qEAAqE;QACrE,MAAM,QAAQ,GAAG,MAAM,YAAY,CAAC,KAAK,CAAC,WAAW,CAAC,CAAC,CAAC;QAExD,MAAM,eAAe,CAAC,mBAAmB,EAAE,QAAQ,GAAG,mBAAmB,EAAE,KAAK,CAAC,eAAe,CAAC,EAAE,CAAC,OAAO,EAAE,EAAE,CAC7G,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,CACnB,CAAC;QAEF,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,0BAA0B,EAAE,IAAI,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAEvE,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;YAC7B,EAAE,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC;QACzB,CAAC;QAED,MAAM,aAAa,GAAG,mBAAmB,CAAC,oBAAoB,EAAE;YAC9D,cAAc;YACd,OAAO;YACP,wBAAwB;YACxB,mBAAmB;SACpB,CAAC,CAAC;QAEH,MAAM,WAAW,GAAG,mBAAmB,CAAC,kBAAkB,EAAE;YAC1D,cAAc;YACd,OAAO;YACP,wBAAwB;YACxB,OAAO;SACR,CAAC,CAAC;QAEH,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;QAEpB,MAAM,SAAS,GAAG,IAAI,MAAM,CAAC,EAAE,WAAW,EAAE,KAAK,CAAC,mBAAmB,CAAC,EAAE,CAAC,CAAC;QAE1E,MAAM,MAAM,GAAG,EAAE,CAAC,gBAAgB,CAAC,KAAK,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;QAEjG,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,iBAAiB,EAAE,cAAc,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QAExE,IAAI,KAAK,GAAG,CAAC,CAAC;QACd,SAAS,CAAC,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE;YACvB,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,cAAc,KAAK,WAAW,SAAS,CAAC,IAAI,cAAc,SAAS,CAAC,OAAO,EAAE,CAAC;QACtG,CAAC,CAAC,CAAC;QAEH,SAAS,CAAC,EAAE,CAAC,WAAW,EAAE,GAAG,EAAE;YAC7B,KAAK,EAAE,CAAC;YACR,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,cAAc,KAAK,WAAW,SAAS,CAAC,IAAI,cAAc,SAAS,CAAC,OAAO,EAAE,CAAC;QACtG,CAAC,CAAC,CAAC;QAEH,IAAI,KAAK,EAAE,MAAM,MAAM,IAAI,MAAM,EAAE,CAAC;YAClC,KAAK,SAAS,CAAC,GAAG,CAAC,KAAK,IAAI,EAAE;gBAC5B,MAAM,sBAAsB,GAAG,MAAgC,CAAC;gBAChE,IAAI,CAAC;oBACH,MAAM,cAAc,GAAG,MAAM,oBAAoB,CAC/C,mBAAmB,EACnB,sBAAsB,CAAC,YAAY,EACnC,sBAAsB,CAAC,KAAK,EAC5B,sBAAsB,CAAC,sBAAsB,CAC9C,CAAC;oBACF,sBAAsB,CAAC,iBAAiB,GAAG,cAAc,CAAC,iBAAiB,CAAC;oBAC5E,MAAM,aAAa,CAAC,KAAK,CAAC,sBAAsB,CAAC,CAAC;gBACpD,CAAC;gBAAC,OAAO,KAAK,EAAE,CAAC;oBACf,sBAAsB,CAAC,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;oBAC7C,MAAM,WAAW,CAAC,KAAK,CAAC,sBAAsB,CAAC,CAAC;gBAClD,CAAC;YACH,CAAC,CAAC,CAAC;QACL,CAAC;QAED,MAAM,SAAS,CAAC,MAAM,EAAE,CAAC;QACzB,MAAM,OAAO,CAAC,GAAG,CAAC,CAAC,aAAa,CAAC,GAAG,EAAE,EAAE,WAAW,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC;QAE5D,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;IACtB,CAAC"}
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
import { type ApiBudgetResult } from '@simplysf/simply-core';
|
|
2
|
+
import type { Connection } from '@salesforce/core';
|
|
3
|
+
/** Default share of the org's remaining API requests a single run may consume. */
|
|
4
|
+
export declare const DEFAULT_MAX_API_USAGE = 20;
|
|
5
|
+
/**
|
|
6
|
+
* `--max-api-usage`, shared by every command that can count its requests in advance.
|
|
7
|
+
*
|
|
8
|
+
* Declared once so the three file commands cannot drift apart on the default or the bounds.
|
|
9
|
+
*/
|
|
10
|
+
export declare const apiBudgetFlags: {
|
|
11
|
+
'max-api-usage': import("@oclif/core/interfaces").OptionFlag<number, import("@oclif/core/interfaces").CustomOptions>;
|
|
12
|
+
};
|
|
13
|
+
/**
|
|
14
|
+
* Check a planned run against the org's remaining API allocation, and stop it if it does not fit.
|
|
15
|
+
*
|
|
16
|
+
* Call this before making any of the requests being counted — the value of the check is entirely
|
|
17
|
+
* in nothing having happened when it fails.
|
|
18
|
+
*
|
|
19
|
+
* @param connection - The org connection.
|
|
20
|
+
* @param plannedRequests - How many API requests the run is about to make.
|
|
21
|
+
* @param maxUsagePercent - The `--max-api-usage` value.
|
|
22
|
+
* @param warn - Called when the allocation could not be read, so the command can surface it.
|
|
23
|
+
* @returns The check result, for inclusion in the command's JSON output.
|
|
24
|
+
* @throws {SfError} If the run would exceed its budget.
|
|
25
|
+
*/
|
|
26
|
+
export declare function assertApiBudget(connection: Connection, plannedRequests: number, maxUsagePercent: number, warn: (message: string) => void): Promise<ApiBudgetResult>;
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
/*
|
|
2
|
+
* Copyright (c) 2026, Clay Chipps.
|
|
3
|
+
*
|
|
4
|
+
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
5
|
+
* you may not use this file except in compliance with the License.
|
|
6
|
+
* You may obtain a copy of the License at
|
|
7
|
+
*
|
|
8
|
+
* http://www.apache.org/licenses/LICENSE-2.0
|
|
9
|
+
*
|
|
10
|
+
* Unless required by applicable law or agreed to in writing, software
|
|
11
|
+
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
12
|
+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
13
|
+
* See the License for the specific language governing permissions and
|
|
14
|
+
* limitations under the License.
|
|
15
|
+
*/
|
|
16
|
+
import { Messages } from '@salesforce/core';
|
|
17
|
+
import { Flags } from '@salesforce/sf-plugins-core';
|
|
18
|
+
import { apiBudgetError, checkApiBudget } from '@simplysf/simply-core';
|
|
19
|
+
Messages.importMessagesDirectoryFromMetaUrl(import.meta.url);
|
|
20
|
+
const messages = Messages.loadMessages('@simplysf/simply-data', 'apiBudget');
|
|
21
|
+
/** Default share of the org's remaining API requests a single run may consume. */
|
|
22
|
+
export const DEFAULT_MAX_API_USAGE = 20;
|
|
23
|
+
/**
|
|
24
|
+
* `--max-api-usage`, shared by every command that can count its requests in advance.
|
|
25
|
+
*
|
|
26
|
+
* Declared once so the three file commands cannot drift apart on the default or the bounds.
|
|
27
|
+
*/
|
|
28
|
+
export const apiBudgetFlags = {
|
|
29
|
+
'max-api-usage': Flags.integer({
|
|
30
|
+
summary: messages.getMessage('flags.max-api-usage.summary'),
|
|
31
|
+
description: messages.getMessage('flags.max-api-usage.description'),
|
|
32
|
+
default: DEFAULT_MAX_API_USAGE,
|
|
33
|
+
min: 1,
|
|
34
|
+
max: 100,
|
|
35
|
+
}),
|
|
36
|
+
};
|
|
37
|
+
/**
|
|
38
|
+
* Check a planned run against the org's remaining API allocation, and stop it if it does not fit.
|
|
39
|
+
*
|
|
40
|
+
* Call this before making any of the requests being counted — the value of the check is entirely
|
|
41
|
+
* in nothing having happened when it fails.
|
|
42
|
+
*
|
|
43
|
+
* @param connection - The org connection.
|
|
44
|
+
* @param plannedRequests - How many API requests the run is about to make.
|
|
45
|
+
* @param maxUsagePercent - The `--max-api-usage` value.
|
|
46
|
+
* @param warn - Called when the allocation could not be read, so the command can surface it.
|
|
47
|
+
* @returns The check result, for inclusion in the command's JSON output.
|
|
48
|
+
* @throws {SfError} If the run would exceed its budget.
|
|
49
|
+
*/
|
|
50
|
+
export async function assertApiBudget(connection, plannedRequests, maxUsagePercent, warn) {
|
|
51
|
+
const result = await checkApiBudget(connection, plannedRequests, { maxUsagePercent });
|
|
52
|
+
if (result.status === 'unavailable') {
|
|
53
|
+
warn(messages.getMessage('warning.budgetUnavailable', [result.reason ?? 'unknown reason']));
|
|
54
|
+
return result;
|
|
55
|
+
}
|
|
56
|
+
if (result.status === 'exceeded') {
|
|
57
|
+
throw apiBudgetError(result, maxUsagePercent);
|
|
58
|
+
}
|
|
59
|
+
return result;
|
|
60
|
+
}
|
|
61
|
+
//# sourceMappingURL=apiBudgetFlag.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"apiBudgetFlag.js","sourceRoot":"","sources":["../../src/common/apiBudgetFlag.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,KAAK,EAAE,MAAM,6BAA6B,CAAC;AACpD,OAAO,EAAE,cAAc,EAAE,cAAc,EAAwB,MAAM,uBAAuB,CAAC;AAG7F,QAAQ,CAAC,kCAAkC,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAC7D,MAAM,QAAQ,GAAG,QAAQ,CAAC,YAAY,CAAC,uBAAuB,EAAE,WAAW,CAAC,CAAC;AAE7E,kFAAkF;AAClF,MAAM,CAAC,MAAM,qBAAqB,GAAG,EAAE,CAAC;AAExC;;;;GAIG;AACH,MAAM,CAAC,MAAM,cAAc,GAAG;IAC5B,eAAe,EAAE,KAAK,CAAC,OAAO,CAAC;QAC7B,OAAO,EAAE,QAAQ,CAAC,UAAU,CAAC,6BAA6B,CAAC;QAC3D,WAAW,EAAE,QAAQ,CAAC,UAAU,CAAC,iCAAiC,CAAC;QACnE,OAAO,EAAE,qBAAqB;QAC9B,GAAG,EAAE,CAAC;QACN,GAAG,EAAE,GAAG;KACT,CAAC;CACH,CAAC;AAEF;;;;;;;;;;;;GAYG;AACH,MAAM,CAAC,KAAK,UAAU,eAAe,CACnC,UAAsB,EACtB,eAAuB,EACvB,eAAuB,EACvB,IAA+B;IAE/B,MAAM,MAAM,GAAG,MAAM,cAAc,CAAC,UAAU,EAAE,eAAe,EAAE,EAAE,eAAe,EAAE,CAAC,CAAC;IAEtF,IAAI,MAAM,CAAC,MAAM,KAAK,aAAa,EAAE,CAAC;QACpC,IAAI,CAAC,QAAQ,CAAC,UAAU,CAAC,2BAA2B,EAAE,CAAC,MAAM,CAAC,MAAM,IAAI,gBAAgB,CAAC,CAAC,CAAC,CAAC;QAE5F,OAAO,MAAM,CAAC;IAChB,CAAC;IAED,IAAI,MAAM,CAAC,MAAM,KAAK,UAAU,EAAE,CAAC;QACjC,MAAM,cAAc,CAAC,MAAM,EAAE,eAAe,CAAC,CAAC;IAChD,CAAC;IAED,OAAO,MAAM,CAAC;AAChB,CAAC"}
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* API requests consumed per file uploaded.
|
|
3
|
+
*
|
|
4
|
+
* **Two, not one.** Salesforce's create response returns only the new `ContentVersion` id, so
|
|
5
|
+
* `uploadContentVersion` follows it with a query for `ContentDocumentId`. Both count toward the
|
|
6
|
+
* org's daily allocation. Counting one per file here would under-budget every upload by half.
|
|
7
|
+
*/
|
|
8
|
+
export declare const REQUESTS_PER_UPLOAD = 2;
|
|
9
|
+
/** API requests consumed per file downloaded — a single GET of the `VersionData` blob. */
|
|
10
|
+
export declare const REQUESTS_PER_DOWNLOAD = 1;
|
|
11
|
+
/**
|
|
12
|
+
* Records jsforce returns per query round trip before it has to fetch another batch.
|
|
13
|
+
*
|
|
14
|
+
* Used to estimate how many requests a record query costs on top of the per-file work.
|
|
15
|
+
*/
|
|
16
|
+
export declare const QUERY_BATCH_SIZE = 2000;
|
|
17
|
+
/**
|
|
18
|
+
* Estimate the API requests a record query costs, including its `queryMore` round trips.
|
|
19
|
+
*
|
|
20
|
+
* @param recordCount - How many records the query returns.
|
|
21
|
+
* @returns The number of requests, at least 1.
|
|
22
|
+
*/
|
|
23
|
+
export declare function requestsForQuery(recordCount: number): number;
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
/*
|
|
2
|
+
* Copyright (c) 2026, Clay Chipps.
|
|
3
|
+
*
|
|
4
|
+
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
5
|
+
* you may not use this file except in compliance with the License.
|
|
6
|
+
* You may obtain a copy of the License at
|
|
7
|
+
*
|
|
8
|
+
* http://www.apache.org/licenses/LICENSE-2.0
|
|
9
|
+
*
|
|
10
|
+
* Unless required by applicable law or agreed to in writing, software
|
|
11
|
+
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
12
|
+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
13
|
+
* See the License for the specific language governing permissions and
|
|
14
|
+
* limitations under the License.
|
|
15
|
+
*/
|
|
16
|
+
/**
|
|
17
|
+
* API requests consumed per file uploaded.
|
|
18
|
+
*
|
|
19
|
+
* **Two, not one.** Salesforce's create response returns only the new `ContentVersion` id, so
|
|
20
|
+
* `uploadContentVersion` follows it with a query for `ContentDocumentId`. Both count toward the
|
|
21
|
+
* org's daily allocation. Counting one per file here would under-budget every upload by half.
|
|
22
|
+
*/
|
|
23
|
+
export const REQUESTS_PER_UPLOAD = 2;
|
|
24
|
+
/** API requests consumed per file downloaded — a single GET of the `VersionData` blob. */
|
|
25
|
+
export const REQUESTS_PER_DOWNLOAD = 1;
|
|
26
|
+
/**
|
|
27
|
+
* Records jsforce returns per query round trip before it has to fetch another batch.
|
|
28
|
+
*
|
|
29
|
+
* Used to estimate how many requests a record query costs on top of the per-file work.
|
|
30
|
+
*/
|
|
31
|
+
export const QUERY_BATCH_SIZE = 2000;
|
|
32
|
+
/**
|
|
33
|
+
* Estimate the API requests a record query costs, including its `queryMore` round trips.
|
|
34
|
+
*
|
|
35
|
+
* @param recordCount - How many records the query returns.
|
|
36
|
+
* @returns The number of requests, at least 1.
|
|
37
|
+
*/
|
|
38
|
+
export function requestsForQuery(recordCount) {
|
|
39
|
+
return Math.max(1, Math.ceil(recordCount / QUERY_BATCH_SIZE));
|
|
40
|
+
}
|
|
41
|
+
//# sourceMappingURL=apiCost.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"apiCost.js","sourceRoot":"","sources":["../../src/common/apiCost.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH;;;;;;GAMG;AACH,MAAM,CAAC,MAAM,mBAAmB,GAAG,CAAC,CAAC;AAErC,0FAA0F;AAC1F,MAAM,CAAC,MAAM,qBAAqB,GAAG,CAAC,CAAC;AAEvC;;;;GAIG;AACH,MAAM,CAAC,MAAM,gBAAgB,GAAG,IAAI,CAAC;AAErC;;;;;GAKG;AACH,MAAM,UAAU,gBAAgB,CAAC,WAAmB;IAClD,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,IAAI,CAAC,WAAW,GAAG,gBAAgB,CAAC,CAAC,CAAC;AAChE,CAAC"}
|
|
@@ -14,12 +14,17 @@ export declare function downloadContentVersion(targetOrgConnection: Connection,
|
|
|
14
14
|
/**
|
|
15
15
|
* Upload a local file as a new `ContentVersion`.
|
|
16
16
|
*
|
|
17
|
+
* Only the file's name is sent to the org as `PathOnClient`, not the local path it was read from.
|
|
18
|
+
* Salesforce derives `FileExtension` and `FileType` from that value, so the name is all it needs —
|
|
19
|
+
* and sending the rest would publish the uploader's local directory layout into the org, where it
|
|
20
|
+
* is visible to anyone who can query `ContentVersion`.
|
|
21
|
+
*
|
|
17
22
|
* @param targetOrgConnection - The org connection to upload to.
|
|
18
|
-
* @param
|
|
19
|
-
* @param title - The `ContentVersion` title. Defaults to the file's
|
|
23
|
+
* @param filePath - The local filesystem path of the file to upload. Stays local.
|
|
24
|
+
* @param title - The `ContentVersion` title. Defaults to the file's name.
|
|
20
25
|
* @param firstPublishLocationId - The ID of the record/library to attach the resulting
|
|
21
26
|
* `ContentDocument` to, if any.
|
|
22
27
|
* @returns The created `ContentVersion`, re-queried to include `ContentDocumentId`.
|
|
23
28
|
* @throws {SfError} If the org returns a non-2xx response.
|
|
24
29
|
*/
|
|
25
|
-
export declare function uploadContentVersion(targetOrgConnection: Connection,
|
|
30
|
+
export declare function uploadContentVersion(targetOrgConnection: Connection, filePath: string, title?: string, firstPublishLocationId?: string): Promise<ContentVersion>;
|
|
@@ -20,6 +20,33 @@ import { SfError } from '@salesforce/core';
|
|
|
20
20
|
import { contentVersionMultipart } from './multipart.js';
|
|
21
21
|
/** How much of an error response body to include in a thrown error before truncating. */
|
|
22
22
|
const MAX_ERROR_BODY_LENGTH = 2000;
|
|
23
|
+
/** Matches the `api-usage=used/limit` portion of a `Sforce-Limit-Info` response header. */
|
|
24
|
+
const API_USAGE_PATTERN = /api-usage=(?<used>\d+)\/(?<limit>\d+)/;
|
|
25
|
+
/**
|
|
26
|
+
* Feed a raw `fetch` response's `Sforce-Limit-Info` header back into the connection.
|
|
27
|
+
*
|
|
28
|
+
* Salesforce reports API consumption on every REST response, and jsforce normally parses it into
|
|
29
|
+
* `connection.limitInfo`. These two functions call `fetch` directly, so jsforce never sees their
|
|
30
|
+
* responses and the readings would otherwise be lost — leaving `limitInfo` stale at whatever the
|
|
31
|
+
* last jsforce-issued request reported, which for an upload run is "nothing at all".
|
|
32
|
+
*
|
|
33
|
+
* Keeping it current costs one header parse and makes the reading available to the API budget
|
|
34
|
+
* check and to any later running check.
|
|
35
|
+
*
|
|
36
|
+
* @param connection - The connection whose `limitInfo` should be updated.
|
|
37
|
+
* @param response - The response to read the header from.
|
|
38
|
+
*/
|
|
39
|
+
function recordApiUsage(connection, response) {
|
|
40
|
+
const match = API_USAGE_PATTERN.exec(response.headers.get('sforce-limit-info') ?? '');
|
|
41
|
+
if (match?.groups) {
|
|
42
|
+
connection.limitInfo = {
|
|
43
|
+
apiUsage: {
|
|
44
|
+
used: Number.parseInt(match.groups.used, 10),
|
|
45
|
+
limit: Number.parseInt(match.groups.limit, 10),
|
|
46
|
+
},
|
|
47
|
+
};
|
|
48
|
+
}
|
|
49
|
+
}
|
|
23
50
|
/**
|
|
24
51
|
* Turn a non-2xx response into an `SfError`.
|
|
25
52
|
*
|
|
@@ -58,6 +85,7 @@ export async function downloadContentVersion(targetOrgConnection, contentVersion
|
|
|
58
85
|
Authorization: `Bearer ${targetOrgConnection.accessToken}`,
|
|
59
86
|
},
|
|
60
87
|
});
|
|
88
|
+
recordApiUsage(targetOrgConnection, response);
|
|
61
89
|
// Checked before the write stream is opened, so a failed download leaves nothing behind.
|
|
62
90
|
if (!response.ok) {
|
|
63
91
|
throw await responseError(response, `Download of ContentVersion ${contentVersionDownload.Id}`);
|
|
@@ -71,26 +99,32 @@ export async function downloadContentVersion(targetOrgConnection, contentVersion
|
|
|
71
99
|
/**
|
|
72
100
|
* Upload a local file as a new `ContentVersion`.
|
|
73
101
|
*
|
|
102
|
+
* Only the file's name is sent to the org as `PathOnClient`, not the local path it was read from.
|
|
103
|
+
* Salesforce derives `FileExtension` and `FileType` from that value, so the name is all it needs —
|
|
104
|
+
* and sending the rest would publish the uploader's local directory layout into the org, where it
|
|
105
|
+
* is visible to anyone who can query `ContentVersion`.
|
|
106
|
+
*
|
|
74
107
|
* @param targetOrgConnection - The org connection to upload to.
|
|
75
|
-
* @param
|
|
76
|
-
* @param title - The `ContentVersion` title. Defaults to the file's
|
|
108
|
+
* @param filePath - The local filesystem path of the file to upload. Stays local.
|
|
109
|
+
* @param title - The `ContentVersion` title. Defaults to the file's name.
|
|
77
110
|
* @param firstPublishLocationId - The ID of the record/library to attach the resulting
|
|
78
111
|
* `ContentDocument` to, if any.
|
|
79
112
|
* @returns The created `ContentVersion`, re-queried to include `ContentDocumentId`.
|
|
80
113
|
* @throws {SfError} If the org returns a non-2xx response.
|
|
81
114
|
*/
|
|
82
|
-
export async function uploadContentVersion(targetOrgConnection,
|
|
115
|
+
export async function uploadContentVersion(targetOrgConnection, filePath, title, firstPublishLocationId) {
|
|
83
116
|
// Check that we have access to the file
|
|
84
|
-
await fs.promises.access(
|
|
117
|
+
await fs.promises.access(filePath, fs.constants.F_OK);
|
|
118
|
+
const fileName = path.basename(filePath);
|
|
85
119
|
const contentVersionCreateRequest = {
|
|
86
120
|
FirstPublishLocationId: firstPublishLocationId,
|
|
87
|
-
PathOnClient:
|
|
88
|
-
Title: title ??
|
|
121
|
+
PathOnClient: fileName,
|
|
122
|
+
Title: title ?? fileName,
|
|
89
123
|
};
|
|
90
124
|
const { contentType, createBody } = contentVersionMultipart({
|
|
91
125
|
entity: contentVersionCreateRequest,
|
|
92
|
-
filePath
|
|
93
|
-
filename:
|
|
126
|
+
filePath,
|
|
127
|
+
filename: fileName,
|
|
94
128
|
});
|
|
95
129
|
const response = await fetch(`${targetOrgConnection.baseUrl()}/sobjects/ContentVersion`, {
|
|
96
130
|
method: 'POST',
|
|
@@ -102,8 +136,9 @@ export async function uploadContentVersion(targetOrgConnection, pathOnClient, ti
|
|
|
102
136
|
'Content-Type': contentType,
|
|
103
137
|
},
|
|
104
138
|
});
|
|
139
|
+
recordApiUsage(targetOrgConnection, response);
|
|
105
140
|
if (!response.ok) {
|
|
106
|
-
throw await responseError(response, `Upload of "${
|
|
141
|
+
throw await responseError(response, `Upload of "${filePath}"`);
|
|
107
142
|
}
|
|
108
143
|
const data = (await response.json());
|
|
109
144
|
const queryResult = await targetOrgConnection.singleRecordQuery(`SELECT ContentDocumentId, FileExtension, Id, Title FROM ContentVersion WHERE Id='${data.id}'`);
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"contentVersionUtils.js","sourceRoot":"","sources":["../../src/common/contentVersionUtils.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,IAAI,MAAM,WAAW,CAAC;AAC7B,OAAO,MAAM,EAAE,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AAC/C,OAAO,EAAc,OAAO,EAAE,MAAM,kBAAkB,CAAC;AACvD,OAAO,EAAE,uBAAuB,EAAE,MAAM,gBAAgB,CAAC;AAQzD,yFAAyF;AACzF,MAAM,qBAAqB,GAAG,IAAI,CAAC;AAEnC;;;;;;;;;;GAUG;AACH,KAAK,UAAU,aAAa,CAAC,QAAkB,EAAE,OAAe;IAC9D,IAAI,IAAI,GAAG,EAAE,CAAC;IACd,IAAI,CAAC;QACH,IAAI,GAAG,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,qBAAqB,CAAC,CAAC;IACjE,CAAC;IAAC,MAAM,CAAC;QACP,gGAAgG;IAClG,CAAC;IAED,OAAO,IAAI,OAAO,CAChB,GAAG,OAAO,qBAAqB,QAAQ,CAAC,MAAM,IAAI,QAAQ,CAAC,UAAU,GAAG,IAAI,CAAC,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,EAAE,EACjG,4BAA4B,CAC7B,CAAC;AACJ,CAAC;AAED;;;;;;;;;GASG;AACH,MAAM,CAAC,KAAK,UAAU,sBAAsB,CAC1C,mBAA+B,EAC/B,sBAA8C,EAC9C,iBAAyB;IAEzB,MAAM,QAAQ,GAAG,GAAG,iBAAiB,IACnC,sBAAsB,CAAC,iBACzB,IAAI,sBAAsB,CAAC,KAAK,CAAC,UAAU,CAAC,GAAG,EAAE,GAAG,CAAC,IAAI,sBAAsB,CAAC,aAAa,EAAE,CAAC;IAEhG,MAAM,QAAQ,GAAG,MAAM,KAAK,CAC1B,GAAG,mBAAmB,CAAC,OAAO,EAAE,4BAA4B,sBAAsB,CAAC,EAAE,cAAc,EACnG;QACE,OAAO,EAAE;YACP,aAAa,EAAE,UAAU,mBAAmB,CAAC,WAAqB,EAAE;SACrE;KACF,CACF,CAAC;IAEF,yFAAyF;IACzF,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;QACjB,MAAM,MAAM,aAAa,CAAC,QAAQ,EAAE,8BAA8B,sBAAsB,CAAC,EAAE,EAAE,CAAC,CAAC;IACjG,CAAC;IAED,IAAI,CAAC,QAAQ,CAAC,IAAI,EAAE,CAAC;QACnB,MAAM,IAAI,OAAO,CACf,8BAA8B,sBAAsB,CAAC,EAAE,mCAAmC,EAC1F,4BAA4B,CAC7B,CAAC;IACJ,CAAC;IAED,MAAM,MAAM,CAAC,QAAQ,CAAC,QAAQ,CAAC,QAAQ,CAAC,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,EAAE,CAAC,iBAAiB,CAAC,QAAQ,CAAC,CAAC,CAAC;IAEhG,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED
|
|
1
|
+
{"version":3,"file":"contentVersionUtils.js","sourceRoot":"","sources":["../../src/common/contentVersionUtils.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,IAAI,MAAM,WAAW,CAAC;AAC7B,OAAO,MAAM,EAAE,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AAC/C,OAAO,EAAc,OAAO,EAAE,MAAM,kBAAkB,CAAC;AACvD,OAAO,EAAE,uBAAuB,EAAE,MAAM,gBAAgB,CAAC;AAQzD,yFAAyF;AACzF,MAAM,qBAAqB,GAAG,IAAI,CAAC;AAEnC,2FAA2F;AAC3F,MAAM,iBAAiB,GAAG,uCAAuC,CAAC;AAElE;;;;;;;;;;;;;GAaG;AACH,SAAS,cAAc,CAAC,UAAsB,EAAE,QAAkB;IAChE,MAAM,KAAK,GAAG,iBAAiB,CAAC,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,mBAAmB,CAAC,IAAI,EAAE,CAAC,CAAC;IAEtF,IAAI,KAAK,EAAE,MAAM,EAAE,CAAC;QAClB,UAAU,CAAC,SAAS,GAAG;YACrB,QAAQ,EAAE;gBACR,IAAI,EAAE,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;gBAC5C,KAAK,EAAE,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,MAAM,CAAC,KAAK,EAAE,EAAE,CAAC;aAC/C;SACF,CAAC;IACJ,CAAC;AACH,CAAC;AAED;;;;;;;;;;GAUG;AACH,KAAK,UAAU,aAAa,CAAC,QAAkB,EAAE,OAAe;IAC9D,IAAI,IAAI,GAAG,EAAE,CAAC;IACd,IAAI,CAAC;QACH,IAAI,GAAG,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,qBAAqB,CAAC,CAAC;IACjE,CAAC;IAAC,MAAM,CAAC;QACP,gGAAgG;IAClG,CAAC;IAED,OAAO,IAAI,OAAO,CAChB,GAAG,OAAO,qBAAqB,QAAQ,CAAC,MAAM,IAAI,QAAQ,CAAC,UAAU,GAAG,IAAI,CAAC,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,EAAE,EACjG,4BAA4B,CAC7B,CAAC;AACJ,CAAC;AAED;;;;;;;;;GASG;AACH,MAAM,CAAC,KAAK,UAAU,sBAAsB,CAC1C,mBAA+B,EAC/B,sBAA8C,EAC9C,iBAAyB;IAEzB,MAAM,QAAQ,GAAG,GAAG,iBAAiB,IACnC,sBAAsB,CAAC,iBACzB,IAAI,sBAAsB,CAAC,KAAK,CAAC,UAAU,CAAC,GAAG,EAAE,GAAG,CAAC,IAAI,sBAAsB,CAAC,aAAa,EAAE,CAAC;IAEhG,MAAM,QAAQ,GAAG,MAAM,KAAK,CAC1B,GAAG,mBAAmB,CAAC,OAAO,EAAE,4BAA4B,sBAAsB,CAAC,EAAE,cAAc,EACnG;QACE,OAAO,EAAE;YACP,aAAa,EAAE,UAAU,mBAAmB,CAAC,WAAqB,EAAE;SACrE;KACF,CACF,CAAC;IAEF,cAAc,CAAC,mBAAmB,EAAE,QAAQ,CAAC,CAAC;IAE9C,yFAAyF;IACzF,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;QACjB,MAAM,MAAM,aAAa,CAAC,QAAQ,EAAE,8BAA8B,sBAAsB,CAAC,EAAE,EAAE,CAAC,CAAC;IACjG,CAAC;IAED,IAAI,CAAC,QAAQ,CAAC,IAAI,EAAE,CAAC;QACnB,MAAM,IAAI,OAAO,CACf,8BAA8B,sBAAsB,CAAC,EAAE,mCAAmC,EAC1F,4BAA4B,CAC7B,CAAC;IACJ,CAAC;IAED,MAAM,MAAM,CAAC,QAAQ,CAAC,QAAQ,CAAC,QAAQ,CAAC,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,EAAE,CAAC,iBAAiB,CAAC,QAAQ,CAAC,CAAC,CAAC;IAEhG,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED;;;;;;;;;;;;;;;GAeG;AACH,MAAM,CAAC,KAAK,UAAU,oBAAoB,CACxC,mBAA+B,EAC/B,QAAgB,EAChB,KAAc,EACd,sBAA+B;IAE/B,wCAAwC;IACxC,MAAM,EAAE,CAAC,QAAQ,CAAC,MAAM,CAAC,QAAQ,EAAE,EAAE,CAAC,SAAS,CAAC,IAAI,CAAC,CAAC;IAEtD,MAAM,QAAQ,GAAG,IAAI,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;IAEzC,MAAM,2BAA2B,GAAgC;QAC/D,sBAAsB,EAAE,sBAAsB;QAC9C,YAAY,EAAE,QAAQ;QACtB,KAAK,EAAE,KAAK,IAAI,QAAQ;KACzB,CAAC;IAEF,MAAM,EAAE,WAAW,EAAE,UAAU,EAAE,GAAG,uBAAuB,CAAC;QAC1D,MAAM,EAAE,2BAA2B;QACnC,QAAQ;QACR,QAAQ,EAAE,QAAQ;KACnB,CAAC,CAAC;IAEH,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,GAAG,mBAAmB,CAAC,OAAO,EAAE,0BAA0B,EAAE;QACvF,MAAM,EAAE,MAAM;QACd,wEAAwE;QACxE,IAAI,EAAE,QAAQ,CAAC,KAAK,CAAC,UAAU,EAAE,CAA+B;QAChE,MAAM,EAAE,MAAM;QACd,OAAO,EAAE;YACP,aAAa,EAAE,UAAU,mBAAmB,CAAC,WAAqB,EAAE;YACpE,cAAc,EAAE,WAAW;SAC5B;KACF,CAAC,CAAC;IAEH,cAAc,CAAC,mBAAmB,EAAE,QAAQ,CAAC,CAAC;IAE9C,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;QACjB,MAAM,MAAM,aAAa,CAAC,QAAQ,EAAE,cAAc,QAAQ,GAAG,CAAC,CAAC;IACjE,CAAC;IAED,MAAM,IAAI,GAAG,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAA+B,CAAC;IAEnE,MAAM,WAAW,GAAG,MAAM,mBAAmB,CAAC,iBAAiB,CAC7D,oFAAoF,IAAI,CAAC,EAAE,GAAG,CAC/F,CAAC;IAEF,OAAO,WAA6B,CAAC;AACvC,CAAC"}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Count the data rows in a CSV, excluding its header.
|
|
3
|
+
*
|
|
4
|
+
* Parses rather than counting newlines: a quoted field may contain line breaks, so `\n` counting
|
|
5
|
+
* over-reports and would budget for uploads that never happen. The parser options match the ones
|
|
6
|
+
* the upload itself uses, so the count and the work cannot disagree.
|
|
7
|
+
*
|
|
8
|
+
* This reads the file a second time rather than buffering the parsed rows. The trade is a cheap
|
|
9
|
+
* sequential local read against unbounded memory on exactly the large inputs where checking the
|
|
10
|
+
* budget matters most.
|
|
11
|
+
*
|
|
12
|
+
* @param filePath - Path to the CSV.
|
|
13
|
+
* @returns The number of data rows.
|
|
14
|
+
*/
|
|
15
|
+
export declare function countCsvRows(filePath: string): Promise<number>;
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
/*
|
|
2
|
+
* Copyright (c) 2026, Clay Chipps.
|
|
3
|
+
*
|
|
4
|
+
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
5
|
+
* you may not use this file except in compliance with the License.
|
|
6
|
+
* You may obtain a copy of the License at
|
|
7
|
+
*
|
|
8
|
+
* http://www.apache.org/licenses/LICENSE-2.0
|
|
9
|
+
*
|
|
10
|
+
* Unless required by applicable law or agreed to in writing, software
|
|
11
|
+
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
12
|
+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
13
|
+
* See the License for the specific language governing permissions and
|
|
14
|
+
* limitations under the License.
|
|
15
|
+
*/
|
|
16
|
+
import fs from 'node:fs';
|
|
17
|
+
import { parse } from 'csv-parse';
|
|
18
|
+
/**
|
|
19
|
+
* Count the data rows in a CSV, excluding its header.
|
|
20
|
+
*
|
|
21
|
+
* Parses rather than counting newlines: a quoted field may contain line breaks, so `\n` counting
|
|
22
|
+
* over-reports and would budget for uploads that never happen. The parser options match the ones
|
|
23
|
+
* the upload itself uses, so the count and the work cannot disagree.
|
|
24
|
+
*
|
|
25
|
+
* This reads the file a second time rather than buffering the parsed rows. The trade is a cheap
|
|
26
|
+
* sequential local read against unbounded memory on exactly the large inputs where checking the
|
|
27
|
+
* budget matters most.
|
|
28
|
+
*
|
|
29
|
+
* @param filePath - Path to the CSV.
|
|
30
|
+
* @returns The number of data rows.
|
|
31
|
+
*/
|
|
32
|
+
export async function countCsvRows(filePath) {
|
|
33
|
+
const parser = fs.createReadStream(filePath).pipe(parse({ bom: true, columns: true }));
|
|
34
|
+
let rows = 0;
|
|
35
|
+
for await (const record of parser) {
|
|
36
|
+
// The record itself is not needed, only that the parser produced one.
|
|
37
|
+
void record;
|
|
38
|
+
rows++;
|
|
39
|
+
}
|
|
40
|
+
return rows;
|
|
41
|
+
}
|
|
42
|
+
//# sourceMappingURL=countCsvRows.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"countCsvRows.js","sourceRoot":"","sources":["../../src/common/countCsvRows.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AACzB,OAAO,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AAElC;;;;;;;;;;;;;GAaG;AACH,MAAM,CAAC,KAAK,UAAU,YAAY,CAAC,QAAgB;IACjD,MAAM,MAAM,GAAG,EAAE,CAAC,gBAAgB,CAAC,QAAQ,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;IAEvF,IAAI,IAAI,GAAG,CAAC,CAAC;IACb,IAAI,KAAK,EAAE,MAAM,MAAM,IAAI,MAAM,EAAE,CAAC;QAClC,sEAAsE;QACtE,KAAK,MAAM,CAAC;QACZ,IAAI,EAAE,CAAC;IACT,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC"}
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# flags.max-api-usage.summary
|
|
2
|
+
|
|
3
|
+
Maximum percentage of the org's remaining API requests this run may consume.
|
|
4
|
+
|
|
5
|
+
# flags.max-api-usage.description
|
|
6
|
+
|
|
7
|
+
Checked before any request is made, so a run that would exceed its budget stops without doing partial work. The percentage applies to the requests the org has left today, not to its daily maximum — an org that has already used most of its allocation gets a proportionally smaller budget.
|
|
8
|
+
|
|
9
|
+
Note that uploading a file costs two API requests, not one: the upload itself, and a follow-up query for the resulting ContentDocumentId.
|
|
10
|
+
|
|
11
|
+
A run that cannot finish within the org's remaining requests is refused regardless of this value. To allow a larger share, raise it; to allow the maximum, pass 100.
|
|
12
|
+
|
|
13
|
+
If the org's remaining allocation can't be read — reading it falls back to the limits API, which needs the "View Setup and Configuration" permission — the command warns and proceeds rather than failing.
|
|
14
|
+
|
|
15
|
+
# warning.budgetUnavailable
|
|
16
|
+
|
|
17
|
+
Could not read the org's remaining API requests, so the --max-api-usage budget was not checked: %s
|
|
@@ -6,9 +6,11 @@ Upload a file to a Salesforce org.
|
|
|
6
6
|
|
|
7
7
|
Uploads a file to a Salesforce org.
|
|
8
8
|
|
|
9
|
+
Only the file's name is sent to the org — Salesforce stores it as the ContentVersion's PathOnClient and derives FileExtension and FileType from it — so the local directory the file came from is never uploaded.
|
|
10
|
+
|
|
9
11
|
# flags.file-path.summary
|
|
10
12
|
|
|
11
|
-
Path to the file to upload.
|
|
13
|
+
Path to the file to upload. May be relative or absolute; only the file's name is sent to the org.
|
|
12
14
|
|
|
13
15
|
# flags.first-publish-location-id.summary
|
|
14
16
|
|
|
@@ -14,6 +14,8 @@ Path to the csv file that specifies the upload.
|
|
|
14
14
|
|
|
15
15
|
The csv file must specify the columns PathOnClient and Title. Optionally, a FirstPublishLocationId can be specified to have it linked directly to a Salesforce record after upload.
|
|
16
16
|
|
|
17
|
+
PathOnClient is the local path each file is read from, and may be relative or absolute. Only the file's name is sent to the org — Salesforce stores it as the ContentVersion's PathOnClient and derives FileExtension and FileType from it — so the local directory the file came from is never uploaded.
|
|
18
|
+
|
|
17
19
|
# flags.max-parallel-jobs.summary
|
|
18
20
|
|
|
19
21
|
Maximum number of parallel jobs.
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@simplysf/simply-data",
|
|
3
3
|
"description": "Utilities for working with data",
|
|
4
|
-
"version": "2.
|
|
4
|
+
"version": "2.4.0",
|
|
5
5
|
"author": "@ClayChipps",
|
|
6
6
|
"homepage": "https://github.com/SimplySF/simply-node",
|
|
7
7
|
"bugs": "https://github.com/SimplySF/simply-node/issues",
|
|
@@ -65,7 +65,7 @@
|
|
|
65
65
|
"@salesforce/core": "^8.30.0",
|
|
66
66
|
"@salesforce/kit": "^3.2.6",
|
|
67
67
|
"@salesforce/sf-plugins-core": "^12.2.16",
|
|
68
|
-
"@simplysf/simply-core": "^1.
|
|
68
|
+
"@simplysf/simply-core": "^1.4.0",
|
|
69
69
|
"@simplysf/simply-plugin-kit": "^1.0.4",
|
|
70
70
|
"csv-parse": "^7.0.2",
|
|
71
71
|
"p-queue": "^9.2.0"
|
|
@@ -214,5 +214,5 @@
|
|
|
214
214
|
"output": []
|
|
215
215
|
}
|
|
216
216
|
},
|
|
217
|
-
"gitHead": "
|
|
217
|
+
"gitHead": "714cbc02c9cac7b8eb745a5be607e281af238751"
|
|
218
218
|
}
|