@celigo/api-specs 0.2.138 → 0.2.139

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/dist/export.yml CHANGED
@@ -2825,7 +2825,7 @@ components:
2825
2825
 
2826
2826
  **Common filter patterns**
2827
2827
 
2828
- 1. **Pattern matching**: Using wildcards like `*` and `?`
2828
+ 1. **Pattern matching**: Regular expressions with `matches`
2829
2829
  2. **Value range filtering**: Numbers between min and max values
2830
2830
  3. **Date range filtering**: Items created/modified within specific time ranges
2831
2831
  4. **Status checking**: Items with specific status values or properties
@@ -2875,9 +2875,12 @@ components:
2875
2875
  - `"endswith"`: String ends with specified suffix (ends with)
2876
2876
  - `"contains"`: String contains specified substring (contains)
2877
2877
  - `"doesnotcontain"`: String does not contain specified substring (does not contain)
2878
- - `"isempty"`: Field is empty or null (is empty)
2879
- - `"isnotempty"`: Field contains a value (is not empty)
2880
- - `"matches"`: Matches specified pattern (matches)
2878
+ - `"empty"`: Field is missing or empty (is empty)
2879
+ - `"notempty"`: Field contains a value (is not empty)
2880
+ - `"matches"`: Matches a regular expression (matches)
2881
+
2882
+ `isempty`, `isnotempty` and `anyof` are not operators: they are refused with 422
2883
+ "Invalid operation: <name>".
2881
2884
 
2882
2885
  **Logical Operators**
2883
2886
  - `"and"`: All conditions must be true
@@ -2897,7 +2900,11 @@ components:
2897
2900
  **Using `extract` to access record fields:**
2898
2901
  - Retrieves values from the current record being processed
2899
2902
  - Can access nested properties using dot notation (e.g., `"customer.email"`)
2900
- - Returns the raw field value which may need type conversion
2903
+ - In a record filter, must be wrapped in a type conversion (`string`, `number`,
2904
+ `boolean` or `epochtime`): a bare `["extract", ...]` operand is refused with 422
2905
+ "Field operation extract must be called by a Type Conversion Operation but was called
2906
+ by <operator>", both when an import's filter is saved and by
2907
+ `POST /v1/processors/filter`
2901
2908
 
2902
2909
  **Using `settings` to access configuration values:**
2903
2910
  - Retrieves values from the integration's configuration settings
@@ -2915,7 +2922,20 @@ components:
2915
2922
  - `"floor"`: Round number down to the nearest integer
2916
2923
  - `"abs"`: Get absolute value of a number
2917
2924
 
2918
- Type conversion operators are often necessary when comparing extracted field values against literals or when the field type doesn't match the comparison operator's expected type. For example:
2925
+ **Handlebars operands**
2926
+ - An operand can also be a string holding exactly one Handlebars expression, such as
2927
+ `"{{record.status}}"` or `"{{lowercase record.code}}"`, on either side of the
2928
+ operator. It renders against the record (`record.<field>`; an unprefixed
2929
+ `{{status}}` renders empty) before the comparison.
2930
+ - The rendered value is a string: wrap it in `["number", ...]` to compare it with a
2931
+ number, or the rule is refused with 422.
2932
+ - `epochtime` does not render Handlebars: `["epochtime", "{{timestamp}}"]` is refused
2933
+ with 422 "invalid datetime format".
2934
+ - A string with text around the expression, or with more than one expression, is
2935
+ compared as a literal.
2936
+
2937
+ Choose the type conversion that matches the comparison — `number` for numeric
2938
+ comparisons, `epochtime` for dates, `string` for text. For example:
2919
2939
 
2920
2940
  ```json
2921
2941
  [
@@ -3006,8 +3026,11 @@ components:
3006
3026
  [
3007
3027
  "notequals",
3008
3028
  [
3009
- "extract",
3010
- "status"
3029
+ "string",
3030
+ [
3031
+ "extract",
3032
+ "status"
3033
+ ]
3011
3034
  ],
3012
3035
  "cancelled"
3013
3036
  ]
@@ -3018,8 +3041,11 @@ components:
3018
3041
  [
3019
3042
  "startswith",
3020
3043
  [
3021
- "extract",
3022
- "filename"
3044
+ "string",
3045
+ [
3046
+ "extract",
3047
+ "filename"
3048
+ ]
3023
3049
  ],
3024
3050
  "HC"
3025
3051
  ]
@@ -3040,29 +3066,26 @@ components:
3040
3066
  ]
3041
3067
  ```
3042
3068
 
3043
- Example 4: Order date is after January 1, 2023
3069
+ Example 4: Order date is after January 1, 2023 (UTC). Compare dates as `epochtime`
3070
+ against epoch milliseconds: a `string` comparison of ISO timestamps orders values with
3071
+ different UTC offsets wrongly.
3044
3072
  ```json
3045
3073
  [
3046
3074
  "greaterthan",
3047
3075
  [
3048
- "extract",
3049
- "orderDate"
3050
- ],
3051
- "2023-01-01T00:00:00Z"
3052
- ]
3053
- ```
3054
-
3055
- Example 5: Category contains any of ["Urgent", "High Priority"]
3056
- ```json
3057
- [
3058
- "anyof",
3059
- [
3060
- "extract",
3061
- "category"
3076
+ "epochtime",
3077
+ [
3078
+ "extract",
3079
+ "orderDate"
3080
+ ]
3062
3081
  ],
3063
- ["Urgent", "High Priority"]
3082
+ 1672531200000
3064
3083
  ]
3065
3084
  ```
3085
+ x-celigo-ai-guidance:
3086
+ - |-
3087
+ To test whether a field has a value, use `notempty` on the wrapped extract, not a
3088
+ `matches` pattern: a record without the field matches `.*\S.*`.
3066
3089
  items:
3067
3090
  oneOf:
3068
3091
  - title: String
@@ -12381,14 +12404,15 @@ components:
12381
12404
  - `objectarray`: Array of objects (most common array type)
12382
12405
  - `arrayarray`: Array of arrays (for matrix/table structures)
12383
12406
 
12384
- Array dataTypes can be populated two ways: pass a source array through
12385
- unchanged via `extract` alone (when the source is already an array of
12386
- the right shape), or construct/iterate via `buildArrayHelper`.
12407
+ Array dataTypes can be populated three ways: pass a source array
12408
+ through unchanged via a JSONPath `extract` alone (when the source is
12409
+ already an array of the right shape), construct/iterate via
12410
+ `buildArrayHelper`, or set a constant with `hardCodedValue`.
12387
12411
 
12388
12412
  IMPORTANT: The dataType controls which additional fields are relevant:
12389
12413
  - For date-like string fields: extractDateFormat, generateDateFormat, etc. become relevant
12390
12414
  - For object types: 'mappings' array becomes relevant
12391
- - For array types: `buildArrayHelper` is one option (see above)
12415
+ - For array types: a JSONPath `extract` alone, `buildArrayHelper`, or `hardCodedValue` (see above)
12392
12416
 
12393
12417
  When analyzing existing mappings or generating new ones, always check dataType first
12394
12418
  to understand what additional fields should be present.
@@ -12451,9 +12475,12 @@ components:
12451
12475
  - Access input record fields with {{record.fieldName}} notation; a grouped
12452
12476
  (array) record binds as `rows` instead — {{rows.0.fieldName}} / {{#each rows}}
12453
12477
  - Examples: "{{record.firstName}} {{record.lastName}}", "{{#if record.isActive}}Active{{else}}Inactive{{/if}}"
12454
- - Valid on `object`, `objectarray`, and primitive-array dataTypes as
12455
- well as scalars — the rendered template output must parse into the
12456
- declared shape
12478
+ - Valid on scalars and on `object` rows, where the rendered output must
12479
+ parse into an object. On an array row the template goes in a
12480
+ `buildArrayHelper` entry instead: on an `objectarray` row's own
12481
+ `extract` it is refused with 422 ("buildArrayHelper" must be present
12482
+ for object array datatype), and on a primitive-array row's own
12483
+ `extract` it renders null without an error
12457
12484
 
12458
12485
  ```json
12459
12486
  "extract": "{{record.price}} {{record.currency}}" // Combines two fields
@@ -12652,6 +12679,19 @@ components:
12652
12679
  - countryCodeToName
12653
12680
  - statusMapping
12654
12681
  - departmentCodes
12682
+ hardCodedValue:
12683
+ type: string
12684
+ description: |
12685
+ Constant written to the destination instead of an extracted value. On a primitive
12686
+ array (`stringarray`, `numberarray`, `booleanarray`) the string is split on commas,
12687
+ verbatim and untrimmed, into elements of the array's type (`"A,B"` gives `["A", "B"]`).
12688
+ On an `object` it must be a JSON object string and on an `objectarray` a JSON array
12689
+ string (an object string becomes one element); the save parses it and refuses non-JSON
12690
+ or the wrong shape with 422, and an array row with a constant needs no `buildArrayHelper`.
12691
+ examples:
12692
+ - USD
12693
+ - A,B
12694
+ - '[{"code": "A"}, {"code": "B"}]'
12655
12695
  description:
12656
12696
  type: string
12657
12697
  description: |
@@ -12787,15 +12827,25 @@ components:
12787
12827
  processor renders the entire destination array as null with no
12788
12828
  validation error. To emit a fixed number of static elements, add
12789
12829
  that many sibling constructed entries — never index the extract.
12830
+ - |-
12831
+ An entry's `extract` can be a Handlebars template. On an
12832
+ `objectarray` it must render a JSON object (one element) or a JSON
12833
+ array (one element per item). On a primitive array each entry
12834
+ renders one element, so a serialized array becomes one string
12835
+ element. A bare literal entry extract is refused with 422 ("extract
12836
+ only supports json path or handlebars expression"); set a constant
12837
+ with the row's `hardCodedValue` instead.
12790
12838
  description: |
12791
12839
  **Purpose**
12792
12840
  Configures how to construct arrays in the output record, handling various array types and inputs.
12793
12841
 
12794
12842
  **Guidance**
12795
- This is the REQUIRED mechanism for ALL array data types:
12843
+ This is the mechanism for building array data types:
12796
12844
 
12797
12845
  **When to Use**
12798
- - REQUIRED when dataType ends with "array" (stringarray, objectarray, etc.)
12846
+ - REQUIRED when dataType ends with "array" (stringarray, objectarray, etc.), unless the
12847
+ row passes a source array through with a JSONPath `extract` alone or sets
12848
+ `hardCodedValue`
12799
12849
  - Each entry in this array contributes elements to the output array
12800
12850
  - Multiple entries allow combining data from different input arrays
12801
12851
 
@@ -13420,14 +13470,15 @@ components:
13420
13470
  - `objectarray`: Array of objects (most common array type)
13421
13471
  - `arrayarray`: Array of arrays (for matrix/table structures)
13422
13472
 
13423
- Array dataTypes can be populated two ways: pass a source array through
13424
- unchanged via `extract` alone (when the source is already an array of
13425
- the right shape), or construct/iterate via `buildArrayHelper`.
13473
+ Array dataTypes can be populated three ways: pass a source array
13474
+ through unchanged via a JSONPath `extract` alone (when the source is
13475
+ already an array of the right shape), construct/iterate via
13476
+ `buildArrayHelper`, or set a constant with `hardCodedValue`.
13426
13477
 
13427
13478
  IMPORTANT: The dataType controls which additional fields are relevant:
13428
13479
  - For date-like string fields: extractDateFormat, generateDateFormat, etc. become relevant
13429
13480
  - For object types: 'mappings' array becomes relevant
13430
- - For array types: `buildArrayHelper` is one option (see above)
13481
+ - For array types: a JSONPath `extract` alone, `buildArrayHelper`, or `hardCodedValue` (see above)
13431
13482
 
13432
13483
  When analyzing existing mappings or generating new ones, always check dataType first
13433
13484
  to understand what additional fields should be present.
@@ -13490,9 +13541,12 @@ components:
13490
13541
  - Access input record fields with {{record.fieldName}} notation; a grouped
13491
13542
  (array) record binds as `rows` instead — {{rows.0.fieldName}} / {{#each rows}}
13492
13543
  - Examples: "{{record.firstName}} {{record.lastName}}", "{{#if record.isActive}}Active{{else}}Inactive{{/if}}"
13493
- - Valid on `object`, `objectarray`, and primitive-array dataTypes as
13494
- well as scalars — the rendered template output must parse into the
13495
- declared shape
13544
+ - Valid on scalars and on `object` rows, where the rendered output must
13545
+ parse into an object. On an array row the template goes in a
13546
+ `buildArrayHelper` entry instead: on an `objectarray` row's own
13547
+ `extract` it is refused with 422 ("buildArrayHelper" must be present
13548
+ for object array datatype), and on a primitive-array row's own
13549
+ `extract` it renders null without an error
13496
13550
 
13497
13551
  ```json
13498
13552
  "extract": "{{record.price}} {{record.currency}}" // Combines two fields
@@ -13691,6 +13745,19 @@ components:
13691
13745
  - countryCodeToName
13692
13746
  - statusMapping
13693
13747
  - departmentCodes
13748
+ hardCodedValue:
13749
+ type: string
13750
+ description: |
13751
+ Constant written to the destination instead of an extracted value. On a primitive
13752
+ array (`stringarray`, `numberarray`, `booleanarray`) the string is split on commas,
13753
+ verbatim and untrimmed, into elements of the array's type (`"A,B"` gives `["A", "B"]`).
13754
+ On an `object` it must be a JSON object string and on an `objectarray` a JSON array
13755
+ string (an object string becomes one element); the save parses it and refuses non-JSON
13756
+ or the wrong shape with 422, and an array row with a constant needs no `buildArrayHelper`.
13757
+ examples:
13758
+ - USD
13759
+ - A,B
13760
+ - '[{"code": "A"}, {"code": "B"}]'
13694
13761
  description:
13695
13762
  type: string
13696
13763
  description: |
@@ -13826,15 +13893,25 @@ components:
13826
13893
  processor renders the entire destination array as null with no
13827
13894
  validation error. To emit a fixed number of static elements, add
13828
13895
  that many sibling constructed entries — never index the extract.
13896
+ - |-
13897
+ An entry's `extract` can be a Handlebars template. On an
13898
+ `objectarray` it must render a JSON object (one element) or a JSON
13899
+ array (one element per item). On a primitive array each entry
13900
+ renders one element, so a serialized array becomes one string
13901
+ element. A bare literal entry extract is refused with 422 ("extract
13902
+ only supports json path or handlebars expression"); set a constant
13903
+ with the row's `hardCodedValue` instead.
13829
13904
  description: |
13830
13905
  **Purpose**
13831
13906
  Configures how to construct arrays in the output record, handling various array types and inputs.
13832
13907
 
13833
13908
  **Guidance**
13834
- This is the REQUIRED mechanism for ALL array data types:
13909
+ This is the mechanism for building array data types:
13835
13910
 
13836
13911
  **When to Use**
13837
- - REQUIRED when dataType ends with "array" (stringarray, objectarray, etc.)
13912
+ - REQUIRED when dataType ends with "array" (stringarray, objectarray, etc.), unless the
13913
+ row passes a source array through with a JSONPath `extract` alone or sets
13914
+ `hardCodedValue`
13838
13915
  - Each entry in this array contributes elements to the output array
13839
13916
  - Multiple entries allow combining data from different input arrays
13840
13917
 
@@ -14865,7 +14942,7 @@ components:
14865
14942
  - envelope
14866
14943
  x-enumDescriptions:
14867
14944
  record: The transform processor evaluates `mappings[].extract` JSONPath against the flat record itself (`$.field`, no `record.` segment) — the default when absent.
14868
- envelope: The transform processor evaluates JSONPath against the wrapped runtime envelope `{record, settings, job, …}` (file-provider steps also carry `fileMeta`) — extracts must include the `record.` segment (`$.record.field`).
14945
+ envelope: The transform processor evaluates JSONPath against the wrapped runtime envelope `{record, job, testMode, settings, headers, queryParams}` (file-provider steps also carry `fileMeta`) — record fields need the `record.` segment (`$.record.field`), while `headers` and `queryParams` sit beside `record` (`$.headers['X-Id']`, `$.queryParams.src`).
14869
14946
  description: |
14870
14947
  Controls the JSON shape the transformTwoDotZero processor
14871
14948
  evaluates `mappings[].extract` JSONPath values against at
@@ -14885,10 +14962,12 @@ components:
14885
14962
  `"$.PLACEHOLDER_<field>"`.
14886
14963
  This is the dominant Celigo idiom and the right choice for new transforms.
14887
14964
  - `envelope`: the processor sees a wrapped object
14888
- `{record: {...}, settings: ..., job: ...}` (plus
14889
- `fileMeta` on file-provider steps).
14965
+ `{record: {...}, job, testMode, settings, headers,
14966
+ queryParams}` (plus `fileMeta` on file-provider steps).
14890
14967
  `mappings[].extract` JSONPath MUST include the
14891
- `record.` segment — e.g. `"$.record.orderId"`;
14968
+ `record.` segment for record fields — e.g.
14969
+ `"$.record.orderId"` — and reads `headers` and
14970
+ `queryParams` beside it (`$.headers['X-Id']`);
14892
14971
  PLACEHOLDER extracts use `"$.record.PLACEHOLDER_<field>"`.
14893
14972
  Pick this only when the transform was authored against
14894
14973
  the envelope shape on purpose.
@@ -14897,6 +14976,11 @@ components:
14897
14976
  (`{{record.field}}`, `{{dateFormat ... record.field}}`)
14898
14977
  ALWAYS reference the `record` root regardless of
14899
14978
  `inputContext` — only JSONPath anchoring flips.
14979
+ - |-
14980
+ In `envelope` mode a `modify` transform merges its mapped
14981
+ fields into `record` and keeps the rest of the envelope
14982
+ around it; a `create` transform returns only the mapped
14983
+ fields, as a bare record.
14900
14984
  - |-
14901
14985
  Emit `inputContext` explicitly on new v2 transforms:
14902
14986
  unset is read as `record`, but stating it makes the
@@ -14917,13 +15001,17 @@ components:
14917
15001
 
14918
15002
  - ``envelope``: the transform processor sees a wrapped
14919
15003
  object of the shape
14920
- ``{record: {...}, settings: ..., job: ...}`` (plus
14921
- ``fileMeta`` on file-provider steps) — the
15004
+ ``{record: {...}, job, testMode, settings, headers,
15005
+ queryParams}`` (plus ``fileMeta`` on file-provider
15006
+ steps; ``headers`` and ``queryParams`` sit beside
15007
+ ``record``) — the
14922
15008
  same runtime-context envelope Handlebars / SQL /
14923
15009
  GraphQL templates render against. ``mappings[].extract``
14924
- JSONPath values MUST include the ``record.`` segment
14925
- — for example ``"$.record.orderId"`` or
14926
- ``"$.record.customer.email"``. PLACEHOLDER extracts
15010
+ JSONPath values for record fields MUST include the
15011
+ ``record.`` segment — for example ``"$.record.orderId"``
15012
+ or ``"$.record.customer.email"`` — while ``headers`` and
15013
+ ``queryParams`` are read beside it
15014
+ (``"$.headers['X-Id']"``). PLACEHOLDER extracts
14927
15015
  use ``"$.record.PLACEHOLDER_<field>"``. Pick this
14928
15016
  mode only when the export was authored against the
14929
15017
  envelope shape on purpose (typically because an
package/dist/flow.yml CHANGED
@@ -4662,7 +4662,7 @@ components:
4662
4662
 
4663
4663
  Uses the same expression language as export/import filters
4664
4664
  (`Filter.expression.rules`): comparison operators (`equals`, `notequals`,
4665
- `greaterthan`, `lessthan`, `startswith`, `contains`, `isempty`, `matches`, ...),
4665
+ `greaterthan`, `lessthan`, `startswith`, `contains`, `empty`, `matches`, ...),
4666
4666
  logical operators (`and`, `or`, `not`), field access (`extract`, `settings`)
4667
4667
  and type conversions (`string`, `number`, `boolean`, `epochtime`).
4668
4668
 
@@ -1249,7 +1249,7 @@ components:
1249
1249
 
1250
1250
  **Common filter patterns**
1251
1251
 
1252
- 1. **Pattern matching**: Using wildcards like `*` and `?`
1252
+ 1. **Pattern matching**: Regular expressions with `matches`
1253
1253
  2. **Value range filtering**: Numbers between min and max values
1254
1254
  3. **Date range filtering**: Items created/modified within specific time ranges
1255
1255
  4. **Status checking**: Items with specific status values or properties
@@ -1299,9 +1299,12 @@ components:
1299
1299
  - `"endswith"`: String ends with specified suffix (ends with)
1300
1300
  - `"contains"`: String contains specified substring (contains)
1301
1301
  - `"doesnotcontain"`: String does not contain specified substring (does not contain)
1302
- - `"isempty"`: Field is empty or null (is empty)
1303
- - `"isnotempty"`: Field contains a value (is not empty)
1304
- - `"matches"`: Matches specified pattern (matches)
1302
+ - `"empty"`: Field is missing or empty (is empty)
1303
+ - `"notempty"`: Field contains a value (is not empty)
1304
+ - `"matches"`: Matches a regular expression (matches)
1305
+
1306
+ `isempty`, `isnotempty` and `anyof` are not operators: they are refused with 422
1307
+ "Invalid operation: <name>".
1305
1308
 
1306
1309
  **Logical Operators**
1307
1310
  - `"and"`: All conditions must be true
@@ -1321,7 +1324,11 @@ components:
1321
1324
  **Using `extract` to access record fields:**
1322
1325
  - Retrieves values from the current record being processed
1323
1326
  - Can access nested properties using dot notation (e.g., `"customer.email"`)
1324
- - Returns the raw field value which may need type conversion
1327
+ - In a record filter, must be wrapped in a type conversion (`string`, `number`,
1328
+ `boolean` or `epochtime`): a bare `["extract", ...]` operand is refused with 422
1329
+ "Field operation extract must be called by a Type Conversion Operation but was called
1330
+ by <operator>", both when an import's filter is saved and by
1331
+ `POST /v1/processors/filter`
1325
1332
 
1326
1333
  **Using `settings` to access configuration values:**
1327
1334
  - Retrieves values from the integration's configuration settings
@@ -1339,7 +1346,20 @@ components:
1339
1346
  - `"floor"`: Round number down to the nearest integer
1340
1347
  - `"abs"`: Get absolute value of a number
1341
1348
 
1342
- Type conversion operators are often necessary when comparing extracted field values against literals or when the field type doesn't match the comparison operator's expected type. For example:
1349
+ **Handlebars operands**
1350
+ - An operand can also be a string holding exactly one Handlebars expression, such as
1351
+ `"{{record.status}}"` or `"{{lowercase record.code}}"`, on either side of the
1352
+ operator. It renders against the record (`record.<field>`; an unprefixed
1353
+ `{{status}}` renders empty) before the comparison.
1354
+ - The rendered value is a string: wrap it in `["number", ...]` to compare it with a
1355
+ number, or the rule is refused with 422.
1356
+ - `epochtime` does not render Handlebars: `["epochtime", "{{timestamp}}"]` is refused
1357
+ with 422 "invalid datetime format".
1358
+ - A string with text around the expression, or with more than one expression, is
1359
+ compared as a literal.
1360
+
1361
+ Choose the type conversion that matches the comparison — `number` for numeric
1362
+ comparisons, `epochtime` for dates, `string` for text. For example:
1343
1363
 
1344
1364
  ```json
1345
1365
  [
@@ -1430,8 +1450,11 @@ components:
1430
1450
  [
1431
1451
  "notequals",
1432
1452
  [
1433
- "extract",
1434
- "status"
1453
+ "string",
1454
+ [
1455
+ "extract",
1456
+ "status"
1457
+ ]
1435
1458
  ],
1436
1459
  "cancelled"
1437
1460
  ]
@@ -1442,8 +1465,11 @@ components:
1442
1465
  [
1443
1466
  "startswith",
1444
1467
  [
1445
- "extract",
1446
- "filename"
1468
+ "string",
1469
+ [
1470
+ "extract",
1471
+ "filename"
1472
+ ]
1447
1473
  ],
1448
1474
  "HC"
1449
1475
  ]
@@ -1464,29 +1490,26 @@ components:
1464
1490
  ]
1465
1491
  ```
1466
1492
 
1467
- Example 4: Order date is after January 1, 2023
1493
+ Example 4: Order date is after January 1, 2023 (UTC). Compare dates as `epochtime`
1494
+ against epoch milliseconds: a `string` comparison of ISO timestamps orders values with
1495
+ different UTC offsets wrongly.
1468
1496
  ```json
1469
1497
  [
1470
1498
  "greaterthan",
1471
1499
  [
1472
- "extract",
1473
- "orderDate"
1474
- ],
1475
- "2023-01-01T00:00:00Z"
1476
- ]
1477
- ```
1478
-
1479
- Example 5: Category contains any of ["Urgent", "High Priority"]
1480
- ```json
1481
- [
1482
- "anyof",
1483
- [
1484
- "extract",
1485
- "category"
1500
+ "epochtime",
1501
+ [
1502
+ "extract",
1503
+ "orderDate"
1504
+ ]
1486
1505
  ],
1487
- ["Urgent", "High Priority"]
1506
+ 1672531200000
1488
1507
  ]
1489
1508
  ```
1509
+ x-celigo-ai-guidance:
1510
+ - |-
1511
+ To test whether a field has a value, use `notempty` on the wrapped extract, not a
1512
+ `matches` pattern: a record without the field matches `.*\S.*`.
1490
1513
  items:
1491
1514
  oneOf:
1492
1515
  - title: String
@@ -695,7 +695,7 @@ components:
695
695
 
696
696
  **Common filter patterns**
697
697
 
698
- 1. **Pattern matching**: Using wildcards like `*` and `?`
698
+ 1. **Pattern matching**: Regular expressions with `matches`
699
699
  2. **Value range filtering**: Numbers between min and max values
700
700
  3. **Date range filtering**: Items created/modified within specific time ranges
701
701
  4. **Status checking**: Items with specific status values or properties
@@ -745,9 +745,12 @@ components:
745
745
  - `"endswith"`: String ends with specified suffix (ends with)
746
746
  - `"contains"`: String contains specified substring (contains)
747
747
  - `"doesnotcontain"`: String does not contain specified substring (does not contain)
748
- - `"isempty"`: Field is empty or null (is empty)
749
- - `"isnotempty"`: Field contains a value (is not empty)
750
- - `"matches"`: Matches specified pattern (matches)
748
+ - `"empty"`: Field is missing or empty (is empty)
749
+ - `"notempty"`: Field contains a value (is not empty)
750
+ - `"matches"`: Matches a regular expression (matches)
751
+
752
+ `isempty`, `isnotempty` and `anyof` are not operators: they are refused with 422
753
+ "Invalid operation: <name>".
751
754
 
752
755
  **Logical Operators**
753
756
  - `"and"`: All conditions must be true
@@ -767,7 +770,11 @@ components:
767
770
  **Using `extract` to access record fields:**
768
771
  - Retrieves values from the current record being processed
769
772
  - Can access nested properties using dot notation (e.g., `"customer.email"`)
770
- - Returns the raw field value which may need type conversion
773
+ - In a record filter, must be wrapped in a type conversion (`string`, `number`,
774
+ `boolean` or `epochtime`): a bare `["extract", ...]` operand is refused with 422
775
+ "Field operation extract must be called by a Type Conversion Operation but was called
776
+ by <operator>", both when an import's filter is saved and by
777
+ `POST /v1/processors/filter`
771
778
 
772
779
  **Using `settings` to access configuration values:**
773
780
  - Retrieves values from the integration's configuration settings
@@ -785,7 +792,20 @@ components:
785
792
  - `"floor"`: Round number down to the nearest integer
786
793
  - `"abs"`: Get absolute value of a number
787
794
 
788
- Type conversion operators are often necessary when comparing extracted field values against literals or when the field type doesn't match the comparison operator's expected type. For example:
795
+ **Handlebars operands**
796
+ - An operand can also be a string holding exactly one Handlebars expression, such as
797
+ `"{{record.status}}"` or `"{{lowercase record.code}}"`, on either side of the
798
+ operator. It renders against the record (`record.<field>`; an unprefixed
799
+ `{{status}}` renders empty) before the comparison.
800
+ - The rendered value is a string: wrap it in `["number", ...]` to compare it with a
801
+ number, or the rule is refused with 422.
802
+ - `epochtime` does not render Handlebars: `["epochtime", "{{timestamp}}"]` is refused
803
+ with 422 "invalid datetime format".
804
+ - A string with text around the expression, or with more than one expression, is
805
+ compared as a literal.
806
+
807
+ Choose the type conversion that matches the comparison — `number` for numeric
808
+ comparisons, `epochtime` for dates, `string` for text. For example:
789
809
 
790
810
  ```json
791
811
  [
@@ -876,8 +896,11 @@ components:
876
896
  [
877
897
  "notequals",
878
898
  [
879
- "extract",
880
- "status"
899
+ "string",
900
+ [
901
+ "extract",
902
+ "status"
903
+ ]
881
904
  ],
882
905
  "cancelled"
883
906
  ]
@@ -888,8 +911,11 @@ components:
888
911
  [
889
912
  "startswith",
890
913
  [
891
- "extract",
892
- "filename"
914
+ "string",
915
+ [
916
+ "extract",
917
+ "filename"
918
+ ]
893
919
  ],
894
920
  "HC"
895
921
  ]
@@ -910,29 +936,26 @@ components:
910
936
  ]
911
937
  ```
912
938
 
913
- Example 4: Order date is after January 1, 2023
939
+ Example 4: Order date is after January 1, 2023 (UTC). Compare dates as `epochtime`
940
+ against epoch milliseconds: a `string` comparison of ISO timestamps orders values with
941
+ different UTC offsets wrongly.
914
942
  ```json
915
943
  [
916
944
  "greaterthan",
917
945
  [
918
- "extract",
919
- "orderDate"
920
- ],
921
- "2023-01-01T00:00:00Z"
922
- ]
923
- ```
924
-
925
- Example 5: Category contains any of ["Urgent", "High Priority"]
926
- ```json
927
- [
928
- "anyof",
929
- [
930
- "extract",
931
- "category"
946
+ "epochtime",
947
+ [
948
+ "extract",
949
+ "orderDate"
950
+ ]
932
951
  ],
933
- ["Urgent", "High Priority"]
952
+ 1672531200000
934
953
  ]
935
954
  ```
955
+ x-celigo-ai-guidance:
956
+ - |-
957
+ To test whether a field has a value, use `notempty` on the wrapped extract, not a
958
+ `matches` pattern: a record without the field matches `.*\S.*`.
936
959
  items:
937
960
  oneOf:
938
961
  - title: String