@celigo/api-specs 0.2.175 → 0.2.177
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/api.yml +3 -2
- package/dist/connection.yml +16 -2
- package/dist/enduser.yml +3 -2
- package/dist/export.yml +17 -8
- package/dist/flow.yml +21 -6
- package/dist/import.yml +7 -6
- package/dist/job.yml +11 -4
- package/dist/mcp-server.yml +3 -2
- package/dist/tool.yml +27 -16
- package/package.json +1 -1
- package/schemas.json +1 -1
package/dist/api.yml
CHANGED
|
@@ -5460,7 +5460,8 @@ components:
|
|
|
5460
5460
|
- To change throughput while this is true, write `targetConcurrencyLevel`; a `concurrencyLevel` sent in a PUT returns 200 and changes nothing.
|
|
5461
5461
|
- Turning it on copies `concurrencyLevel` into `targetConcurrencyLevel`; turning it off unsets the target and leaves `concurrencyLevel` at its current value, which may be below the old target if a recovery was in progress. A `targetConcurrencyLevel` written while this is false is unset by the save.
|
|
5462
5462
|
- These four settings (`concurrencyLevel`, `targetConcurrencyLevel`, `_borrowConcurrencyFromConnectionId`, this flag) accept a credential-free `PATCH /v1/connections/{_id}`. A PUT replaces the whole document, so send each secret back as its masked `******` value (that keeps it) or as a new value, and never drop one to keep it — an omitted password or `http.encrypted` is cleared, and a static token left out (with no `refreshMethod`) fails the PUT with 422.
|
|
5463
|
-
- On
|
|
5463
|
+
- On a create, while this is true, set the level with `targetConcurrencyLevel`: on `http`, `ftp`, `rdbms` and `mongodb` connections the save stores the target sent, or the adaptor's default without one (1 for `ftp`, 5 for `rdbms` and `mongodb`, 25 for `http` where the license's `concurrency` is at least 25), as both fields, whatever `concurrencyLevel` the body carried. A create that sends this as `false` keeps its `concurrencyLevel`.
|
|
5464
|
+
- On an `ftp` connection recovery never runs, and transfer threads are `ftp.multiThreadCount`.
|
|
5464
5465
|
enableMicroBatchForOneToMany:
|
|
5465
5466
|
type: boolean
|
|
5466
5467
|
default: true
|
|
@@ -6827,7 +6828,7 @@ components:
|
|
|
6827
6828
|
Send no more than the account's licensed maximum (25 standard, 50 with an Environments license).
|
|
6828
6829
|
A decimal is rejected with 422 `invalid_field_value` on every write (the same rule applies to every
|
|
6829
6830
|
adaptor's `concurrencyLevel` and `targetConcurrencyLevel`, and to `ftp.multiThreadCount`).
|
|
6830
|
-
Platform-managed while `autoRecoverRateLimitErrors` is true — a PUT ignores the value and the recovery machinery moves it toward the target; set `targetConcurrencyLevel` instead.
|
|
6831
|
+
Platform-managed while `autoRecoverRateLimitErrors` is true — a create or PUT ignores the value (a create without `targetConcurrencyLevel` stores 25 where the license's `concurrency` is at least 25) and the recovery machinery moves it toward the target; set `targetConcurrencyLevel` instead.
|
|
6831
6832
|
At level 1 an import's pages run one at a time, so a record's existence lookup and write finish before the next record's start (measured on a single import); the import's `idLockTemplate` serializes only same-key records and leaves the rest parallel.
|
|
6832
6833
|
minimum: 1
|
|
6833
6834
|
maximum: 50
|
package/dist/connection.yml
CHANGED
|
@@ -721,6 +721,12 @@ paths:
|
|
|
721
721
|
in clear on create and come back masked as "******"; the response also carries the
|
|
722
722
|
server-populated fields (`connectionRole: standard`, `concurrencyLevel`,
|
|
723
723
|
`targetConcurrencyLevel`, `microServices`).
|
|
724
|
+
- |-
|
|
725
|
+
Choose the level with `targetConcurrencyLevel`, or send `autoRecoverRateLimitErrors: false`
|
|
726
|
+
with `concurrencyLevel`. While auto-recovery is on, which a create gets unless it sends
|
|
727
|
+
`false`, an `http`, `ftp`, `rdbms` or `mongodb` create replaces a `concurrencyLevel` sent
|
|
728
|
+
without a target with the adaptor's default, so an `http` create that asks for 10 is stored at
|
|
729
|
+
25 where the license's `concurrency` is at least 25 (see `autoRecoverRateLimitErrors`).
|
|
724
730
|
/v1/connections/{_id}:
|
|
725
731
|
get:
|
|
726
732
|
x-internal: false
|
|
@@ -4311,7 +4317,8 @@ components:
|
|
|
4311
4317
|
- To change throughput while this is true, write `targetConcurrencyLevel`; a `concurrencyLevel` sent in a PUT returns 200 and changes nothing.
|
|
4312
4318
|
- Turning it on copies `concurrencyLevel` into `targetConcurrencyLevel`; turning it off unsets the target and leaves `concurrencyLevel` at its current value, which may be below the old target if a recovery was in progress. A `targetConcurrencyLevel` written while this is false is unset by the save.
|
|
4313
4319
|
- These four settings (`concurrencyLevel`, `targetConcurrencyLevel`, `_borrowConcurrencyFromConnectionId`, this flag) accept a credential-free `PATCH /v1/connections/{_id}`. A PUT replaces the whole document, so send each secret back as its masked `******` value (that keeps it) or as a new value, and never drop one to keep it — an omitted password or `http.encrypted` is cleared, and a static token left out (with no `refreshMethod`) fails the PUT with 422.
|
|
4314
|
-
- On
|
|
4320
|
+
- On a create, while this is true, set the level with `targetConcurrencyLevel`: on `http`, `ftp`, `rdbms` and `mongodb` connections the save stores the target sent, or the adaptor's default without one (1 for `ftp`, 5 for `rdbms` and `mongodb`, 25 for `http` where the license's `concurrency` is at least 25), as both fields, whatever `concurrencyLevel` the body carried. A create that sends this as `false` keeps its `concurrencyLevel`.
|
|
4321
|
+
- On an `ftp` connection recovery never runs, and transfer threads are `ftp.multiThreadCount`.
|
|
4315
4322
|
enableMicroBatchForOneToMany:
|
|
4316
4323
|
type: boolean
|
|
4317
4324
|
default: true
|
|
@@ -5678,7 +5685,7 @@ components:
|
|
|
5678
5685
|
Send no more than the account's licensed maximum (25 standard, 50 with an Environments license).
|
|
5679
5686
|
A decimal is rejected with 422 `invalid_field_value` on every write (the same rule applies to every
|
|
5680
5687
|
adaptor's `concurrencyLevel` and `targetConcurrencyLevel`, and to `ftp.multiThreadCount`).
|
|
5681
|
-
Platform-managed while `autoRecoverRateLimitErrors` is true — a PUT ignores the value and the recovery machinery moves it toward the target; set `targetConcurrencyLevel` instead.
|
|
5688
|
+
Platform-managed while `autoRecoverRateLimitErrors` is true — a create or PUT ignores the value (a create without `targetConcurrencyLevel` stores 25 where the license's `concurrency` is at least 25) and the recovery machinery moves it toward the target; set `targetConcurrencyLevel` instead.
|
|
5682
5689
|
At level 1 an import's pages run one at a time, so a record's existence lookup and write finish before the next record's start (measured on a single import); the import's `idLockTemplate` serializes only same-key records and leaves the rest parallel.
|
|
5683
5690
|
minimum: 1
|
|
5684
5691
|
maximum: 50
|
|
@@ -9663,6 +9670,13 @@ components:
|
|
|
9663
9670
|
drops to 1 and climbs back toward the adaptor block's `targetConcurrencyLevel`, which is the
|
|
9664
9671
|
throughput lever while this is on. Turning it on copies the current `concurrencyLevel` into
|
|
9665
9672
|
the target; turning it off unsets the target and leaves `concurrencyLevel` at its current value.
|
|
9673
|
+
A create gets `true` unless it sends `false`; while it is true, set a new connection's level
|
|
9674
|
+
with `targetConcurrencyLevel`. On `http`, `ftp`, `rdbms` and `mongodb` connections the create
|
|
9675
|
+
stores the target sent, or the adaptor's default without one, as both `concurrencyLevel` and
|
|
9676
|
+
`targetConcurrencyLevel`, whatever `concurrencyLevel` the body carried: 1 for `ftp`, 5 for
|
|
9677
|
+
`rdbms` and `mongodb`, and 25 for `http` where the license's `concurrency` is at least 25, so
|
|
9678
|
+
such an `http` create that sends only `concurrencyLevel: 10` is stored at 25. To keep the
|
|
9679
|
+
`concurrencyLevel` you send, create with this `false`.
|
|
9666
9680
|
Recovery never runs on `ftp` connections: FTP errors aren't treated as rate limits.
|
|
9667
9681
|
__integrationIds:
|
|
9668
9682
|
type: array
|
package/dist/enduser.yml
CHANGED
|
@@ -2368,7 +2368,8 @@ components:
|
|
|
2368
2368
|
- To change throughput while this is true, write `targetConcurrencyLevel`; a `concurrencyLevel` sent in a PUT returns 200 and changes nothing.
|
|
2369
2369
|
- Turning it on copies `concurrencyLevel` into `targetConcurrencyLevel`; turning it off unsets the target and leaves `concurrencyLevel` at its current value, which may be below the old target if a recovery was in progress. A `targetConcurrencyLevel` written while this is false is unset by the save.
|
|
2370
2370
|
- These four settings (`concurrencyLevel`, `targetConcurrencyLevel`, `_borrowConcurrencyFromConnectionId`, this flag) accept a credential-free `PATCH /v1/connections/{_id}`. A PUT replaces the whole document, so send each secret back as its masked `******` value (that keeps it) or as a new value, and never drop one to keep it — an omitted password or `http.encrypted` is cleared, and a static token left out (with no `refreshMethod`) fails the PUT with 422.
|
|
2371
|
-
- On
|
|
2371
|
+
- On a create, while this is true, set the level with `targetConcurrencyLevel`: on `http`, `ftp`, `rdbms` and `mongodb` connections the save stores the target sent, or the adaptor's default without one (1 for `ftp`, 5 for `rdbms` and `mongodb`, 25 for `http` where the license's `concurrency` is at least 25), as both fields, whatever `concurrencyLevel` the body carried. A create that sends this as `false` keeps its `concurrencyLevel`.
|
|
2372
|
+
- On an `ftp` connection recovery never runs, and transfer threads are `ftp.multiThreadCount`.
|
|
2372
2373
|
enableMicroBatchForOneToMany:
|
|
2373
2374
|
type: boolean
|
|
2374
2375
|
default: true
|
|
@@ -3735,7 +3736,7 @@ components:
|
|
|
3735
3736
|
Send no more than the account's licensed maximum (25 standard, 50 with an Environments license).
|
|
3736
3737
|
A decimal is rejected with 422 `invalid_field_value` on every write (the same rule applies to every
|
|
3737
3738
|
adaptor's `concurrencyLevel` and `targetConcurrencyLevel`, and to `ftp.multiThreadCount`).
|
|
3738
|
-
Platform-managed while `autoRecoverRateLimitErrors` is true — a PUT ignores the value and the recovery machinery moves it toward the target; set `targetConcurrencyLevel` instead.
|
|
3739
|
+
Platform-managed while `autoRecoverRateLimitErrors` is true — a create or PUT ignores the value (a create without `targetConcurrencyLevel` stores 25 where the license's `concurrency` is at least 25) and the recovery machinery moves it toward the target; set `targetConcurrencyLevel` instead.
|
|
3739
3740
|
At level 1 an import's pages run one at a time, so a record's existence lookup and write finish before the next record's start (measured on a single import); the import's `idLockTemplate` serializes only same-key records and leaves the rest parallel.
|
|
3740
3741
|
minimum: 1
|
|
3741
3742
|
maximum: 50
|
package/dist/export.yml
CHANGED
|
@@ -5608,31 +5608,37 @@ components:
|
|
|
5608
5608
|
description: |-
|
|
5609
5609
|
Selects which files are processed: files matching the filter criteria are exported,
|
|
5610
5610
|
and non-matching files are skipped entirely before processing begins. Filterable
|
|
5611
|
-
fields come from each file's
|
|
5612
|
-
and
|
|
5611
|
+
fields come from each file's listing entry: the name as `fileName`, the size as
|
|
5612
|
+
`filesize`, and the modified time as `lastModifiedTime` on FTP, SFTP and S3 or
|
|
5613
|
+
`modifiedTime` on cloud-storage folders (`http.type: "file"`)
|
|
5614
|
+
(e.g. ["endswith", ["string", ["extract", "fileName"]], ".csv"]). There is no
|
|
5615
|
+
`lastmodified` key.
|
|
5613
5616
|
allOf:
|
|
5614
5617
|
- $ref: '#/components/schemas/Filter'
|
|
5615
5618
|
x-celigo-ai-guidance:
|
|
5616
5619
|
- |-
|
|
5617
5620
|
## AVAILABLE FILTER FIELDS
|
|
5618
5621
|
|
|
5619
|
-
The
|
|
5622
|
+
The fields a file filter reads are on each file's listing entry, before the file is
|
|
5623
|
+
downloaded. They are not the records' `fileMeta`, though some names match.
|
|
5620
5624
|
|
|
5621
5625
|
### Common Filter Fields
|
|
5622
5626
|
|
|
5623
5627
|
These are the most commonly available fields across most file providers:
|
|
5624
5628
|
|
|
5625
|
-
1. **
|
|
5629
|
+
1. **fileName**: The name of the file (with extension)
|
|
5626
5630
|
- Example filter: Match files with specific extensions or naming patterns
|
|
5627
|
-
- Usage: `["endswith", ["string", ["extract", "
|
|
5631
|
+
- Usage: `["endswith", ["string", ["extract", "fileName"]], ".csv"]`
|
|
5628
5632
|
|
|
5629
5633
|
2. **filesize**: The size of the file in bytes
|
|
5630
5634
|
- Example filter: Skip files that are too large or too small
|
|
5631
5635
|
- Usage: `["lessthan", ["number", ["extract", "filesize"]], 1000000]`
|
|
5632
5636
|
|
|
5633
|
-
3. **
|
|
5637
|
+
3. **lastModifiedTime** (FTP, SFTP, S3) or **modifiedTime** (cloud-storage folders): The
|
|
5638
|
+
last modification timestamp of the file. There is no `lastmodified` key.
|
|
5634
5639
|
- Example filter: Process only files created/modified within a specific date range
|
|
5635
|
-
- Usage: `["greaterthan", ["epochtime", ["extract", "
|
|
5640
|
+
- Usage (FTP, SFTP, S3): `["greaterthan", ["epochtime", ["extract", "lastModifiedTime"]], 1672531200000]` (2023-01-01T00:00:00Z in epoch milliseconds)
|
|
5641
|
+
- Usage (cloud-storage folders): `["greaterthan", ["epochtime", ["extract", "modifiedTime"]], 1672531200000]`
|
|
5636
5642
|
backupPath:
|
|
5637
5643
|
type: string
|
|
5638
5644
|
description: Path where backup copies of source files are stored before processing.
|
|
@@ -6636,7 +6642,10 @@ components:
|
|
|
6636
6642
|
- async_helper
|
|
6637
6643
|
once:
|
|
6638
6644
|
type: object
|
|
6639
|
-
description:
|
|
6645
|
+
description: |-
|
|
6646
|
+
Callback configuration for once exports, used to mark records as exported in the source system after successful processing.
|
|
6647
|
+
Called only while the export's `type` is `once`: an export with no `type` keeps a saved `once` and never calls it, so every
|
|
6648
|
+
run reads every record again (live-verified 2026-10-22).
|
|
6640
6649
|
properties:
|
|
6641
6650
|
relativeURI:
|
|
6642
6651
|
type: string
|
package/dist/flow.yml
CHANGED
|
@@ -7208,7 +7208,10 @@ components:
|
|
|
7208
7208
|
- retry
|
|
7209
7209
|
- bulk_retry
|
|
7210
7210
|
x-enumDescriptions:
|
|
7211
|
-
export:
|
|
7211
|
+
export: |-
|
|
7212
|
+
A child job for an export step within a flow run: the flow's source export, or a
|
|
7213
|
+
lookup step (an export with `isLookup`), whose job counts the pages it looked up in
|
|
7214
|
+
`numPagesProcessed`.
|
|
7212
7215
|
import: A child job for an import step within a flow run.
|
|
7213
7216
|
flow: A parent flow-run job that orchestrates the full pipeline.
|
|
7214
7217
|
retry: A retry of a single failed record from a previous job.
|
|
@@ -7438,8 +7441,9 @@ components:
|
|
|
7438
7441
|
type: integer
|
|
7439
7442
|
readOnly: true
|
|
7440
7443
|
description: |-
|
|
7441
|
-
Number of records skipped
|
|
7442
|
-
rule
|
|
7444
|
+
Number of records skipped: by a filter, by an import's ignore-existing or ignore-missing
|
|
7445
|
+
rule, by an import's preMap hook that returned no data for the record, or by a lookup step's
|
|
7446
|
+
input filter (the record continues without the lookup's data). Records an import's filter skips count in that import's child job; records an export's
|
|
7443
7447
|
output filter drops count in the export's child job, whose `numSuccess` then counts only the
|
|
7444
7448
|
records that passed (an export that read 6 records and dropped 3 reads `numSuccess` 3,
|
|
7445
7449
|
`numIgnore` 3; live-verified 2026-10-22). Filtered records raise no error.
|
|
@@ -7466,7 +7470,10 @@ components:
|
|
|
7466
7470
|
numPagesProcessed:
|
|
7467
7471
|
type: integer
|
|
7468
7472
|
readOnly: true
|
|
7469
|
-
description:
|
|
7473
|
+
description: |-
|
|
7474
|
+
Number of pages processed by downstream steps: imports and lookup steps. A source export's
|
|
7475
|
+
child job reads 0, a lookup step's and an import's count the pages they processed, and a
|
|
7476
|
+
parent flow job sums its children.
|
|
7470
7477
|
examples:
|
|
7471
7478
|
- 10
|
|
7472
7479
|
oIndex:
|
|
@@ -8080,11 +8087,13 @@ components:
|
|
|
8080
8087
|
enum:
|
|
8081
8088
|
- success
|
|
8082
8089
|
- error
|
|
8083
|
-
-
|
|
8090
|
+
- ignore
|
|
8084
8091
|
x-enumDescriptions:
|
|
8085
8092
|
success: The record was processed successfully at this step.
|
|
8086
8093
|
error: The record encountered an error during processing.
|
|
8087
|
-
|
|
8094
|
+
ignore: |-
|
|
8095
|
+
The step skipped the record: its filter, a preMap hook that returned no data for the
|
|
8096
|
+
record, or an ignore-existing or ignore-missing rule. `stage` names which.
|
|
8088
8097
|
timeTaken:
|
|
8089
8098
|
type: integer
|
|
8090
8099
|
description: Milliseconds spent at this step for this record.
|
|
@@ -8106,8 +8115,14 @@ components:
|
|
|
8106
8115
|
Pipeline stage at which this step entry was recorded. Observed values include `apiCall`
|
|
8107
8116
|
(the adaptor invocation itself). Useful for filtering metadata responses when a step has
|
|
8108
8117
|
multiple stage entries across a single record's pass.
|
|
8118
|
+
|
|
8119
|
+
On an entry with `status: ignore`, the stage that skipped the record, recorded even on a run
|
|
8120
|
+
logged at `basic` (whose stage data is empty): `inputFilter` for the step's filter,
|
|
8121
|
+
`preMapHook` for a preMap hook that returned no data for the record, and an empty string for
|
|
8122
|
+
an ignore-existing rule.
|
|
8109
8123
|
examples:
|
|
8110
8124
|
- apiCall
|
|
8125
|
+
- inputFilter
|
|
8111
8126
|
ExecutionLogMetadataResponse:
|
|
8112
8127
|
type: object
|
|
8113
8128
|
description: |-
|
package/dist/import.yml
CHANGED
|
@@ -5418,7 +5418,7 @@ components:
|
|
|
5418
5418
|
- bulk_load
|
|
5419
5419
|
x-enumDescriptions:
|
|
5420
5420
|
INSERT: Legacy value for insert operations; use bulk_insert or per_record instead.
|
|
5421
|
-
UPDATE: Legacy value for update operations; use per_record instead.
|
|
5421
|
+
UPDATE: Legacy value for update operations; use per_page or per_record instead.
|
|
5422
5422
|
bulk_insert: Batch INSERT via multi-row VALUES clause for efficient bulk data loading.
|
|
5423
5423
|
per_record: Execute one SQL statement per record, giving full control over individual record SQL.
|
|
5424
5424
|
per_page: Execute one SQL statement per page (batch) of records.
|
|
@@ -5428,7 +5428,7 @@ components:
|
|
|
5428
5428
|
description: |-
|
|
5429
5429
|
Execution strategy for the SQL operation, which determines the required companion field: per_record, per_page, and first_page require `query`; bulk_insert requires `bulkInsert`; bulk_load requires `bulkLoad`.
|
|
5430
5430
|
Prefer the highest-performance type the operation supports: bulk_load (Snowflake, Google BigQuery, Microsoft SQL Server on a cloud connection, Azure Synapse and Amazon Redshift — handles INSERT, upsert via `bulkLoad.primaryKeys`, and custom merge or ignore-existing logic via `bulkLoad.overrideMergeQuery`), then bulk_insert for pure INSERTs, then per_page, then per_record only when the logic cannot be expressed as a batch operation.
|
|
5431
|
-
For UPDATE, UPSERT, or ignore-existing logic on databases without bulk_load support, use per_page or per_record with `query` — bulk_insert has no duplicate-checking logic.
|
|
5431
|
+
For UPDATE, UPSERT, or ignore-existing logic on databases without bulk_load support, use per_page or per_record with `query` — bulk_insert has no duplicate-checking logic. Prefer per_page when the statement can loop `batch_of_records`, and use per_record when the SQL can't be written as one batch statement.
|
|
5432
5432
|
Set a single value; do not combine types or use the legacy INSERT and UPDATE values.
|
|
5433
5433
|
per_page statements render against the whole batch (`batch_of_records`) rather than a single record — loop with `{{#each batch_of_records}}...{{record.fieldName}}...{{/each}}`, with the same quoting as `query` (double braces for strings, triple for numbers).
|
|
5434
5434
|
|
|
@@ -5490,9 +5490,10 @@ components:
|
|
|
5490
5490
|
### UPDATE Operation
|
|
5491
5491
|
Prompt: "Update inventory counts"
|
|
5492
5492
|
```json
|
|
5493
|
-
"queryType": ["
|
|
5494
|
-
"query": ["UPDATE inventory SET qty = {{{record.qty}}} WHERE sku = {{record.sku}}"]
|
|
5493
|
+
"queryType": ["per_page"],
|
|
5494
|
+
"query": ["{{#each batch_of_records}}UPDATE inventory SET qty = {{{record.qty}}} WHERE sku = {{record.sku}};{{/each}}"]
|
|
5495
5495
|
```
|
|
5496
|
+
Use `"queryType": ["per_record"]` with `"query": ["UPDATE inventory SET qty = {{{record.qty}}} WHERE sku = {{record.sku}}"]` when the SQL can't be written as one batch statement.
|
|
5496
5497
|
|
|
5497
5498
|
### Run Once Per Page
|
|
5498
5499
|
Prompt: "Execute this stored procedure for each page of results"
|
|
@@ -5510,7 +5511,7 @@ components:
|
|
|
5510
5511
|
type: object
|
|
5511
5512
|
description: |-
|
|
5512
5513
|
Configuration for batch INSERTs via a multi-row VALUES clause; required when queryType is `["bulk_insert"]` and must not be set otherwise.
|
|
5513
|
-
Only suited to pure INSERTs — for UPDATE
|
|
5514
|
+
Only suited to pure INSERTs — for UPDATE or UPSERT, use queryType `["per_page"]` (a statement that loops `batch_of_records`) or `["per_record"]` with the `query` field instead, and for ignore-existing/skip-duplicates logic `["per_record"]`.
|
|
5514
5515
|
x-celigo-ai-guidance:
|
|
5515
5516
|
- |-
|
|
5516
5517
|
**CRITICAL: This object is ONLY used when `queryType` is `["bulk_insert"]`.**
|
|
@@ -5520,7 +5521,7 @@ components:
|
|
|
5520
5521
|
- Pure INSERT operation with NO ignoreExisting/duplicate checking
|
|
5521
5522
|
|
|
5522
5523
|
❌ **DO NOT SET THIS** when:
|
|
5523
|
-
- `queryType` is `["per_record"]` (use `query` field instead)
|
|
5524
|
+
- `queryType` is `["per_page"]` or `["per_record"]` (use the `query` field instead)
|
|
5524
5525
|
- Operation involves UPDATE, UPSERT, or ignoreExisting logic
|
|
5525
5526
|
|
|
5526
5527
|
If the prompt mentions "ignore existing", "skip duplicates", or "match on field",
|
package/dist/job.yml
CHANGED
|
@@ -1341,7 +1341,10 @@ components:
|
|
|
1341
1341
|
- retry
|
|
1342
1342
|
- bulk_retry
|
|
1343
1343
|
x-enumDescriptions:
|
|
1344
|
-
export:
|
|
1344
|
+
export: |-
|
|
1345
|
+
A child job for an export step within a flow run: the flow's source export, or a
|
|
1346
|
+
lookup step (an export with `isLookup`), whose job counts the pages it looked up in
|
|
1347
|
+
`numPagesProcessed`.
|
|
1345
1348
|
import: A child job for an import step within a flow run.
|
|
1346
1349
|
flow: A parent flow-run job that orchestrates the full pipeline.
|
|
1347
1350
|
retry: A retry of a single failed record from a previous job.
|
|
@@ -1571,8 +1574,9 @@ components:
|
|
|
1571
1574
|
type: integer
|
|
1572
1575
|
readOnly: true
|
|
1573
1576
|
description: |-
|
|
1574
|
-
Number of records skipped
|
|
1575
|
-
rule
|
|
1577
|
+
Number of records skipped: by a filter, by an import's ignore-existing or ignore-missing
|
|
1578
|
+
rule, by an import's preMap hook that returned no data for the record, or by a lookup step's
|
|
1579
|
+
input filter (the record continues without the lookup's data). Records an import's filter skips count in that import's child job; records an export's
|
|
1576
1580
|
output filter drops count in the export's child job, whose `numSuccess` then counts only the
|
|
1577
1581
|
records that passed (an export that read 6 records and dropped 3 reads `numSuccess` 3,
|
|
1578
1582
|
`numIgnore` 3; live-verified 2026-10-22). Filtered records raise no error.
|
|
@@ -1599,7 +1603,10 @@ components:
|
|
|
1599
1603
|
numPagesProcessed:
|
|
1600
1604
|
type: integer
|
|
1601
1605
|
readOnly: true
|
|
1602
|
-
description:
|
|
1606
|
+
description: |-
|
|
1607
|
+
Number of pages processed by downstream steps: imports and lookup steps. A source export's
|
|
1608
|
+
child job reads 0, a lookup step's and an import's count the pages they processed, and a
|
|
1609
|
+
parent flow job sums its children.
|
|
1603
1610
|
examples:
|
|
1604
1611
|
- 10
|
|
1605
1612
|
oIndex:
|
package/dist/mcp-server.yml
CHANGED
|
@@ -4204,7 +4204,8 @@ components:
|
|
|
4204
4204
|
- To change throughput while this is true, write `targetConcurrencyLevel`; a `concurrencyLevel` sent in a PUT returns 200 and changes nothing.
|
|
4205
4205
|
- Turning it on copies `concurrencyLevel` into `targetConcurrencyLevel`; turning it off unsets the target and leaves `concurrencyLevel` at its current value, which may be below the old target if a recovery was in progress. A `targetConcurrencyLevel` written while this is false is unset by the save.
|
|
4206
4206
|
- These four settings (`concurrencyLevel`, `targetConcurrencyLevel`, `_borrowConcurrencyFromConnectionId`, this flag) accept a credential-free `PATCH /v1/connections/{_id}`. A PUT replaces the whole document, so send each secret back as its masked `******` value (that keeps it) or as a new value, and never drop one to keep it — an omitted password or `http.encrypted` is cleared, and a static token left out (with no `refreshMethod`) fails the PUT with 422.
|
|
4207
|
-
- On
|
|
4207
|
+
- On a create, while this is true, set the level with `targetConcurrencyLevel`: on `http`, `ftp`, `rdbms` and `mongodb` connections the save stores the target sent, or the adaptor's default without one (1 for `ftp`, 5 for `rdbms` and `mongodb`, 25 for `http` where the license's `concurrency` is at least 25), as both fields, whatever `concurrencyLevel` the body carried. A create that sends this as `false` keeps its `concurrencyLevel`.
|
|
4208
|
+
- On an `ftp` connection recovery never runs, and transfer threads are `ftp.multiThreadCount`.
|
|
4208
4209
|
enableMicroBatchForOneToMany:
|
|
4209
4210
|
type: boolean
|
|
4210
4211
|
default: true
|
|
@@ -5571,7 +5572,7 @@ components:
|
|
|
5571
5572
|
Send no more than the account's licensed maximum (25 standard, 50 with an Environments license).
|
|
5572
5573
|
A decimal is rejected with 422 `invalid_field_value` on every write (the same rule applies to every
|
|
5573
5574
|
adaptor's `concurrencyLevel` and `targetConcurrencyLevel`, and to `ftp.multiThreadCount`).
|
|
5574
|
-
Platform-managed while `autoRecoverRateLimitErrors` is true — a PUT ignores the value and the recovery machinery moves it toward the target; set `targetConcurrencyLevel` instead.
|
|
5575
|
+
Platform-managed while `autoRecoverRateLimitErrors` is true — a create or PUT ignores the value (a create without `targetConcurrencyLevel` stores 25 where the license's `concurrency` is at least 25) and the recovery machinery moves it toward the target; set `targetConcurrencyLevel` instead.
|
|
5575
5576
|
At level 1 an import's pages run one at a time, so a record's existence lookup and write finish before the next record's start (measured on a single import); the import's `idLockTemplate` serializes only same-key records and leaves the rest parallel.
|
|
5576
5577
|
minimum: 1
|
|
5577
5578
|
maximum: 50
|
package/dist/tool.yml
CHANGED
|
@@ -7738,7 +7738,8 @@ components:
|
|
|
7738
7738
|
- To change throughput while this is true, write `targetConcurrencyLevel`; a `concurrencyLevel` sent in a PUT returns 200 and changes nothing.
|
|
7739
7739
|
- Turning it on copies `concurrencyLevel` into `targetConcurrencyLevel`; turning it off unsets the target and leaves `concurrencyLevel` at its current value, which may be below the old target if a recovery was in progress. A `targetConcurrencyLevel` written while this is false is unset by the save.
|
|
7740
7740
|
- These four settings (`concurrencyLevel`, `targetConcurrencyLevel`, `_borrowConcurrencyFromConnectionId`, this flag) accept a credential-free `PATCH /v1/connections/{_id}`. A PUT replaces the whole document, so send each secret back as its masked `******` value (that keeps it) or as a new value, and never drop one to keep it — an omitted password or `http.encrypted` is cleared, and a static token left out (with no `refreshMethod`) fails the PUT with 422.
|
|
7741
|
-
- On
|
|
7741
|
+
- On a create, while this is true, set the level with `targetConcurrencyLevel`: on `http`, `ftp`, `rdbms` and `mongodb` connections the save stores the target sent, or the adaptor's default without one (1 for `ftp`, 5 for `rdbms` and `mongodb`, 25 for `http` where the license's `concurrency` is at least 25), as both fields, whatever `concurrencyLevel` the body carried. A create that sends this as `false` keeps its `concurrencyLevel`.
|
|
7742
|
+
- On an `ftp` connection recovery never runs, and transfer threads are `ftp.multiThreadCount`.
|
|
7742
7743
|
enableMicroBatchForOneToMany:
|
|
7743
7744
|
type: boolean
|
|
7744
7745
|
default: true
|
|
@@ -9105,7 +9106,7 @@ components:
|
|
|
9105
9106
|
Send no more than the account's licensed maximum (25 standard, 50 with an Environments license).
|
|
9106
9107
|
A decimal is rejected with 422 `invalid_field_value` on every write (the same rule applies to every
|
|
9107
9108
|
adaptor's `concurrencyLevel` and `targetConcurrencyLevel`, and to `ftp.multiThreadCount`).
|
|
9108
|
-
Platform-managed while `autoRecoverRateLimitErrors` is true — a PUT ignores the value and the recovery machinery moves it toward the target; set `targetConcurrencyLevel` instead.
|
|
9109
|
+
Platform-managed while `autoRecoverRateLimitErrors` is true — a create or PUT ignores the value (a create without `targetConcurrencyLevel` stores 25 where the license's `concurrency` is at least 25) and the recovery machinery moves it toward the target; set `targetConcurrencyLevel` instead.
|
|
9109
9110
|
At level 1 an import's pages run one at a time, so a record's existence lookup and write finish before the next record's start (measured on a single import); the import's `idLockTemplate` serializes only same-key records and leaves the rest parallel.
|
|
9110
9111
|
minimum: 1
|
|
9111
9112
|
maximum: 50
|
|
@@ -16995,7 +16996,7 @@ components:
|
|
|
16995
16996
|
- bulk_load
|
|
16996
16997
|
x-enumDescriptions:
|
|
16997
16998
|
INSERT: Legacy value for insert operations; use bulk_insert or per_record instead.
|
|
16998
|
-
UPDATE: Legacy value for update operations; use per_record instead.
|
|
16999
|
+
UPDATE: Legacy value for update operations; use per_page or per_record instead.
|
|
16999
17000
|
bulk_insert: Batch INSERT via multi-row VALUES clause for efficient bulk data loading.
|
|
17000
17001
|
per_record: Execute one SQL statement per record, giving full control over individual record SQL.
|
|
17001
17002
|
per_page: Execute one SQL statement per page (batch) of records.
|
|
@@ -17005,7 +17006,7 @@ components:
|
|
|
17005
17006
|
description: |-
|
|
17006
17007
|
Execution strategy for the SQL operation, which determines the required companion field: per_record, per_page, and first_page require `query`; bulk_insert requires `bulkInsert`; bulk_load requires `bulkLoad`.
|
|
17007
17008
|
Prefer the highest-performance type the operation supports: bulk_load (Snowflake, Google BigQuery, Microsoft SQL Server on a cloud connection, Azure Synapse and Amazon Redshift — handles INSERT, upsert via `bulkLoad.primaryKeys`, and custom merge or ignore-existing logic via `bulkLoad.overrideMergeQuery`), then bulk_insert for pure INSERTs, then per_page, then per_record only when the logic cannot be expressed as a batch operation.
|
|
17008
|
-
For UPDATE, UPSERT, or ignore-existing logic on databases without bulk_load support, use per_page or per_record with `query` — bulk_insert has no duplicate-checking logic.
|
|
17009
|
+
For UPDATE, UPSERT, or ignore-existing logic on databases without bulk_load support, use per_page or per_record with `query` — bulk_insert has no duplicate-checking logic. Prefer per_page when the statement can loop `batch_of_records`, and use per_record when the SQL can't be written as one batch statement.
|
|
17009
17010
|
Set a single value; do not combine types or use the legacy INSERT and UPDATE values.
|
|
17010
17011
|
per_page statements render against the whole batch (`batch_of_records`) rather than a single record — loop with `{{#each batch_of_records}}...{{record.fieldName}}...{{/each}}`, with the same quoting as `query` (double braces for strings, triple for numbers).
|
|
17011
17012
|
|
|
@@ -17067,9 +17068,10 @@ components:
|
|
|
17067
17068
|
### UPDATE Operation
|
|
17068
17069
|
Prompt: "Update inventory counts"
|
|
17069
17070
|
```json
|
|
17070
|
-
"queryType": ["
|
|
17071
|
-
"query": ["UPDATE inventory SET qty = {{{record.qty}}} WHERE sku = {{record.sku}}"]
|
|
17071
|
+
"queryType": ["per_page"],
|
|
17072
|
+
"query": ["{{#each batch_of_records}}UPDATE inventory SET qty = {{{record.qty}}} WHERE sku = {{record.sku}};{{/each}}"]
|
|
17072
17073
|
```
|
|
17074
|
+
Use `"queryType": ["per_record"]` with `"query": ["UPDATE inventory SET qty = {{{record.qty}}} WHERE sku = {{record.sku}}"]` when the SQL can't be written as one batch statement.
|
|
17073
17075
|
|
|
17074
17076
|
### Run Once Per Page
|
|
17075
17077
|
Prompt: "Execute this stored procedure for each page of results"
|
|
@@ -17087,7 +17089,7 @@ components:
|
|
|
17087
17089
|
type: object
|
|
17088
17090
|
description: |-
|
|
17089
17091
|
Configuration for batch INSERTs via a multi-row VALUES clause; required when queryType is `["bulk_insert"]` and must not be set otherwise.
|
|
17090
|
-
Only suited to pure INSERTs — for UPDATE
|
|
17092
|
+
Only suited to pure INSERTs — for UPDATE or UPSERT, use queryType `["per_page"]` (a statement that loops `batch_of_records`) or `["per_record"]` with the `query` field instead, and for ignore-existing/skip-duplicates logic `["per_record"]`.
|
|
17091
17093
|
x-celigo-ai-guidance:
|
|
17092
17094
|
- |-
|
|
17093
17095
|
**CRITICAL: This object is ONLY used when `queryType` is `["bulk_insert"]`.**
|
|
@@ -17097,7 +17099,7 @@ components:
|
|
|
17097
17099
|
- Pure INSERT operation with NO ignoreExisting/duplicate checking
|
|
17098
17100
|
|
|
17099
17101
|
❌ **DO NOT SET THIS** when:
|
|
17100
|
-
- `queryType` is `["per_record"]` (use `query` field instead)
|
|
17102
|
+
- `queryType` is `["per_page"]` or `["per_record"]` (use the `query` field instead)
|
|
17101
17103
|
- Operation involves UPDATE, UPSERT, or ignoreExisting logic
|
|
17102
17104
|
|
|
17103
17105
|
If the prompt mentions "ignore existing", "skip duplicates", or "match on field",
|
|
@@ -30620,31 +30622,37 @@ components:
|
|
|
30620
30622
|
description: |-
|
|
30621
30623
|
Selects which files are processed: files matching the filter criteria are exported,
|
|
30622
30624
|
and non-matching files are skipped entirely before processing begins. Filterable
|
|
30623
|
-
fields come from each file's
|
|
30624
|
-
and
|
|
30625
|
+
fields come from each file's listing entry: the name as `fileName`, the size as
|
|
30626
|
+
`filesize`, and the modified time as `lastModifiedTime` on FTP, SFTP and S3 or
|
|
30627
|
+
`modifiedTime` on cloud-storage folders (`http.type: "file"`)
|
|
30628
|
+
(e.g. ["endswith", ["string", ["extract", "fileName"]], ".csv"]). There is no
|
|
30629
|
+
`lastmodified` key.
|
|
30625
30630
|
allOf:
|
|
30626
30631
|
- $ref: '#/components/schemas/Filter'
|
|
30627
30632
|
x-celigo-ai-guidance:
|
|
30628
30633
|
- |-
|
|
30629
30634
|
## AVAILABLE FILTER FIELDS
|
|
30630
30635
|
|
|
30631
|
-
The
|
|
30636
|
+
The fields a file filter reads are on each file's listing entry, before the file is
|
|
30637
|
+
downloaded. They are not the records' `fileMeta`, though some names match.
|
|
30632
30638
|
|
|
30633
30639
|
### Common Filter Fields
|
|
30634
30640
|
|
|
30635
30641
|
These are the most commonly available fields across most file providers:
|
|
30636
30642
|
|
|
30637
|
-
1. **
|
|
30643
|
+
1. **fileName**: The name of the file (with extension)
|
|
30638
30644
|
- Example filter: Match files with specific extensions or naming patterns
|
|
30639
|
-
- Usage: `["endswith", ["string", ["extract", "
|
|
30645
|
+
- Usage: `["endswith", ["string", ["extract", "fileName"]], ".csv"]`
|
|
30640
30646
|
|
|
30641
30647
|
2. **filesize**: The size of the file in bytes
|
|
30642
30648
|
- Example filter: Skip files that are too large or too small
|
|
30643
30649
|
- Usage: `["lessthan", ["number", ["extract", "filesize"]], 1000000]`
|
|
30644
30650
|
|
|
30645
|
-
3. **
|
|
30651
|
+
3. **lastModifiedTime** (FTP, SFTP, S3) or **modifiedTime** (cloud-storage folders): The
|
|
30652
|
+
last modification timestamp of the file. There is no `lastmodified` key.
|
|
30646
30653
|
- Example filter: Process only files created/modified within a specific date range
|
|
30647
|
-
- Usage: `["greaterthan", ["epochtime", ["extract", "
|
|
30654
|
+
- Usage (FTP, SFTP, S3): `["greaterthan", ["epochtime", ["extract", "lastModifiedTime"]], 1672531200000]` (2023-01-01T00:00:00Z in epoch milliseconds)
|
|
30655
|
+
- Usage (cloud-storage folders): `["greaterthan", ["epochtime", ["extract", "modifiedTime"]], 1672531200000]`
|
|
30648
30656
|
backupPath:
|
|
30649
30657
|
type: string
|
|
30650
30658
|
description: Path where backup copies of source files are stored before processing.
|
|
@@ -31648,7 +31656,10 @@ components:
|
|
|
31648
31656
|
- async_helper
|
|
31649
31657
|
once:
|
|
31650
31658
|
type: object
|
|
31651
|
-
description:
|
|
31659
|
+
description: |-
|
|
31660
|
+
Callback configuration for once exports, used to mark records as exported in the source system after successful processing.
|
|
31661
|
+
Called only while the export's `type` is `once`: an export with no `type` keeps a saved `once` and never calls it, so every
|
|
31662
|
+
run reads every record again (live-verified 2026-10-22).
|
|
31652
31663
|
properties:
|
|
31653
31664
|
relativeURI:
|
|
31654
31665
|
type: string
|
package/package.json
CHANGED