@celigo/api-specs 0.2.154 → 0.2.156

package/dist/tool.yml

@@ -3524,9 +3524,8 @@ components:
             - For regular fields: Set to the exact property name (e.g., "firstName", "price", "isActive")
             - For object fields: Set to the object property name, then add child mappings in the 'mappings' array
             - For array fields: Set to the array property name, then configure 'buildArrayHelper'
-            - Two structural blanks: the inner array item of an array of arrays (it has no field name),
-              and the single top-level `objectarray` item of a `rows [ ]` mapping (the array output
-              structure — see the array's description). A child-bearing wrapper and a source-only row that
+            - One structural blank: the single top-level `objectarray` item of a `rows [ ]` mapping (the
+              array output structure — see the array's description). A child-bearing wrapper and a source-only row that
               already exist in a saved document also stay unnamed when the tree is re-sent — filling in a
               name changes the output; it does not complete the row
 
@@ -3543,7 +3542,9 @@ components:
             }
             ```
 
-            When parsing existing mappings, an empty `generate` inside a parent array is an inner array; an
+            When parsing existing mappings, an empty `generate` on a row inside an `arrayarray` row's
+            `buildArrayHelper` entry is that row's inner array (`arrayarray` fails at run time; see
+            `dataType`); other unnamed rows are the wrappers and source-only rows described above; an
             empty `generate` on the ONLY top-level item, with `dataType: objectarray`, is the `rows [ ]` JSON
             structure — array output, bound as `rows` in templates — and is intentional, not a missing name.
           examples:
@@ -3577,17 +3578,25 @@ components:
             - `numberarray`: Array of numbers
             - `booleanarray`: Array of booleans
             - `objectarray`: Array of objects (most common array type)
-            - `arrayarray`: Array of arrays (for matrix/table structures)
 
-            Array dataTypes can be populated three ways: pass a source array
-            through unchanged via a JSONPath `extract` alone (when the source is
-            already an array of the right shape), construct/iterate via
-            `buildArrayHelper`, or set a constant with `hardCodedValue`.
+            `arrayarray` is not a type the mapper recognizes: a save accepts it, but every record
+            then fails with `Destination datatype is a required field.` Build an array of arrays
+            in a script hook instead.
+
+            Array dataTypes are populated two ways: construct or iterate through
+            `buildArrayHelper` entries, or set a constant with `hardCodedValue`. The
+            row's own `extract` is never read on an array type: a primitive-array row
+            that has only an `extract` (JSONPath, Handlebars or a literal) yields
+            nothing, with no error, and an `objectarray` row without `buildArrayHelper`
+            or `hardCodedValue` is refused with 422 ("buildArrayHelper" or
+            "hardCodedValue" must be present for object array datatype.). To copy a
+            source array through unchanged, use one entry that iterates it
+            (`buildArrayHelper: [{"extract": "$.items[*]"}]`).
 
             IMPORTANT: The dataType controls which additional fields are relevant:
             - For date-like string fields: extractDateFormat, generateDateFormat, etc. become relevant
             - For object types: 'mappings' array becomes relevant
-            - For array types: a JSONPath `extract` alone, `buildArrayHelper`, or `hardCodedValue` (see above)
+            - For array types: `buildArrayHelper` or `hardCodedValue` (see above)
 
             When analyzing existing mappings or generating new ones, always check dataType first
             to understand what additional fields should be present.
@@ -3606,7 +3615,6 @@ components:
             - numberarray
             - booleanarray
             - objectarray
-            - arrayarray
           x-enumDescriptions:
             string: Text values; converts other types to their string representation.
             number: Numeric values; attempts conversion from strings.
@@ -3616,7 +3624,6 @@ components:
             numberarray: Array of numeric values.
             booleanarray: Array of boolean values.
             objectarray: Array of objects, the most common array type.
-            arrayarray: Array of arrays, used for matrix or table structures.
           examples:
             - string
             - number
@@ -3659,9 +3666,10 @@ components:
             - Valid on scalars and on `object` rows, where the rendered output must
               parse into an object. On an array row the template goes in a
               `buildArrayHelper` entry instead: on an `objectarray` row's own
-              `extract` it is refused with 422 ("buildArrayHelper" must be present
-              for object array datatype), and on a primitive-array row's own
-              `extract` it renders null without an error
+              `extract` it is refused with 422 ("buildArrayHelper" or "hardCodedValue"
+              must be present for object array datatype.), and on a primitive-array
+              row's own `extract` it yields nothing (the field is left out) without an
+              error
 
             ```json
             "extract": "{{record.price}} {{record.currency}}"  // Combines two fields
@@ -3803,8 +3811,10 @@ components:
             Explicit JSON `null` is itself a valid fallback: the destination field is written as
             JSON null when the extract yields nothing (the Mapper UI's "Use null as default value"
             action). Omitting the key entirely means no fallback — the field is left out of the
-            output when the extract is empty (the UI's "Do nothing" action). Failing the record
-            instead is `conditional.when: fail_record_when_empty`, which excludes a default.
+            output when the extract is missing or an empty string (the UI's "Do nothing" action),
+            while a null source is still written as null unless `conditional.when` is
+            `extract_not_empty`. Failing the record instead is `conditional.when:
+            fail_record_when_empty`, which excludes a default.
           x-celigo-ai-guidance:
             - |-
               `"default": null` and `"default": ""` are distinct saved values, not
@@ -3812,11 +3822,14 @@ components:
               empty string renders it as `""` when the extract yields nothing.
               Preserve them verbatim when editing or migrating a mapping — dropping
               the key changes runtime behavior from "use null/empty string as
-              default value" to "do nothing" (field omitted from the output).
+              default value" to "do nothing" (a missing or `""` source is left out
+              of the output, while a null source is written as null unless
+              `conditional.when` is `extract_not_empty`).
             - |-
               The four "Action to take if source field has no value" choices map to:
-              do nothing → no `default` key (optionally `conditional.when:
-              extract_not_empty`); use null → `"default": null`; use empty string or
+              do nothing → no `default` key (add `conditional.when:
+              extract_not_empty` to leave a null source out too; without it a null
+              source is written as null); use null → `"default": null`; use empty string or
               a custom value → `"default": "<value>"`; cause the record to fail →
               `conditional.when: fail_record_when_empty` with no `default` key.
           examples:
@@ -4031,8 +4044,7 @@ components:
 
             **When to Use**
             - REQUIRED when dataType ends with "array" (stringarray, objectarray, etc.), unless the
-              row passes a source array through with a JSONPath `extract` alone or sets
-              `hardCodedValue`
+              row sets `hardCodedValue`: an array row's own `extract` is never read (see `dataType`)
             - Each entry in this array contributes elements to the output array
             - Multiple entries allow combining data from different input arrays
 
@@ -4116,28 +4128,10 @@ components:
             }
             ```
 
-            **For Arrays of Arrays (arrayarray)**
-            - Similar to objectarray, but inner arrays have empty 'generate' fields
-            - Used for matrix/table structures
-            ```json
-            {
-              "generate": "matrix",
-              "dataType": "arrayarray",
-              "buildArrayHelper": [
-                {
-                  "extract": "$.rows[*]",  // For each row in the rows array
-                  "mappings": [
-                    {
-                      "dataType": "numberarray",  // Note: No generate field for inner arrays
-                      "buildArrayHelper": [
-                        {"extract": "$.rows.columns[*]"}  // Notice: "rows" is singular in the composite object
-                      ]
-                    }
-                  ]
-                }
-              ]
-            }
-            ```
+            **Arrays of Arrays**
+            - The mapper has no working type for them: `arrayarray` saves, but every record
+              fails with `Destination datatype is a required field.` Build an array of arrays
+              in a script hook instead.
 
             **Important details**
 
@@ -4555,8 +4549,8 @@ components:
         nature is essential for building effective mappings:
 
         1. The schema is self-referential by design - a mapping can contain nested mappings of the same structure
-        2. Complex data structures (nested objects, arrays of objects, arrays of arrays of objects) are ALL
-           handled through this recursive pattern
+        2. Complex data structures (nested objects and arrays of objects, to any depth) are ALL
+           handled through this recursive pattern; an array of arrays is not (build one in a script hook)
         3. Each mapping handles one level of the data structure; deeper levels are handled by nested mappings
 
         When generating mappings programmatically:
@@ -4610,9 +4604,8 @@ components:
               - For regular fields: Set to the exact property name (e.g., "firstName", "price", "isActive")
               - For object fields: Set to the object property name, then add child mappings in the 'mappings' array
               - For array fields: Set to the array property name, then configure 'buildArrayHelper'
-              - Two structural blanks: the inner array item of an array of arrays (it has no field name),
-                and the single top-level `objectarray` item of a `rows [ ]` mapping (the array output
-                structure — see the array's description). A child-bearing wrapper and a source-only row that
+              - One structural blank: the single top-level `objectarray` item of a `rows [ ]` mapping (the
+                array output structure — see the array's description). A child-bearing wrapper and a source-only row that
                 already exist in a saved document also stay unnamed when the tree is re-sent — filling in a
                 name changes the output; it does not complete the row
 
@@ -4629,7 +4622,9 @@ components:
               }
               ```
 
-              When parsing existing mappings, an empty `generate` inside a parent array is an inner array; an
+              When parsing existing mappings, an empty `generate` on a row inside an `arrayarray` row's
+              `buildArrayHelper` entry is that row's inner array (`arrayarray` fails at run time; see
+              `dataType`); other unnamed rows are the wrappers and source-only rows described above; an
               empty `generate` on the ONLY top-level item, with `dataType: objectarray`, is the `rows [ ]` JSON
               structure — array output, bound as `rows` in templates — and is intentional, not a missing name.
             examples:
@@ -4663,17 +4658,25 @@ components:
               - `numberarray`: Array of numbers
               - `booleanarray`: Array of booleans
               - `objectarray`: Array of objects (most common array type)
-              - `arrayarray`: Array of arrays (for matrix/table structures)
 
-              Array dataTypes can be populated three ways: pass a source array
-              through unchanged via a JSONPath `extract` alone (when the source is
-              already an array of the right shape), construct/iterate via
-              `buildArrayHelper`, or set a constant with `hardCodedValue`.
+              `arrayarray` is not a type the mapper recognizes: a save accepts it, but every record
+              then fails with `Destination datatype is a required field.` Build an array of arrays
+              in a script hook instead.
+
+              Array dataTypes are populated two ways: construct or iterate through
+              `buildArrayHelper` entries, or set a constant with `hardCodedValue`. The
+              row's own `extract` is never read on an array type: a primitive-array row
+              that has only an `extract` (JSONPath, Handlebars or a literal) yields
+              nothing, with no error, and an `objectarray` row without `buildArrayHelper`
+              or `hardCodedValue` is refused with 422 ("buildArrayHelper" or
+              "hardCodedValue" must be present for object array datatype.). To copy a
+              source array through unchanged, use one entry that iterates it
+              (`buildArrayHelper: [{"extract": "$.items[*]"}]`).
 
               IMPORTANT: The dataType controls which additional fields are relevant:
               - For date-like string fields: extractDateFormat, generateDateFormat, etc. become relevant
               - For object types: 'mappings' array becomes relevant
-              - For array types: a JSONPath `extract` alone, `buildArrayHelper`, or `hardCodedValue` (see above)
+              - For array types: `buildArrayHelper` or `hardCodedValue` (see above)
 
               When analyzing existing mappings or generating new ones, always check dataType first
               to understand what additional fields should be present.
@@ -4692,7 +4695,6 @@ components:
               - numberarray
               - booleanarray
               - objectarray
-              - arrayarray
             x-enumDescriptions:
               string: Text values; converts other types to their string representation.
               number: Numeric values; attempts conversion from strings.
@@ -4702,7 +4704,6 @@ components:
               numberarray: Array of numeric values.
               booleanarray: Array of boolean values.
               objectarray: Array of objects, the most common array type.
-              arrayarray: Array of arrays, used for matrix or table structures.
             examples:
               - string
               - number
@@ -4745,9 +4746,10 @@ components:
               - Valid on scalars and on `object` rows, where the rendered output must
                 parse into an object. On an array row the template goes in a
                 `buildArrayHelper` entry instead: on an `objectarray` row's own
-                `extract` it is refused with 422 ("buildArrayHelper" must be present
-                for object array datatype), and on a primitive-array row's own
-                `extract` it renders null without an error
+                `extract` it is refused with 422 ("buildArrayHelper" or "hardCodedValue"
+                must be present for object array datatype.), and on a primitive-array
+                row's own `extract` it yields nothing (the field is left out) without an
+                error
 
               ```json
               "extract": "{{record.price}} {{record.currency}}"  // Combines two fields
@@ -4889,8 +4891,10 @@ components:
               Explicit JSON `null` is itself a valid fallback: the destination field is written as
               JSON null when the extract yields nothing (the Mapper UI's "Use null as default value"
               action). Omitting the key entirely means no fallback — the field is left out of the
-              output when the extract is empty (the UI's "Do nothing" action). Failing the record
-              instead is `conditional.when: fail_record_when_empty`, which excludes a default.
+              output when the extract is missing or an empty string (the UI's "Do nothing" action),
+              while a null source is still written as null unless `conditional.when` is
+              `extract_not_empty`. Failing the record instead is `conditional.when:
+              fail_record_when_empty`, which excludes a default.
             x-celigo-ai-guidance:
               - |-
                 `"default": null` and `"default": ""` are distinct saved values, not
@@ -4898,11 +4902,14 @@ components:
                 empty string renders it as `""` when the extract yields nothing.
                 Preserve them verbatim when editing or migrating a mapping — dropping
                 the key changes runtime behavior from "use null/empty string as
-                default value" to "do nothing" (field omitted from the output).
+                default value" to "do nothing" (a missing or `""` source is left out
+                of the output, while a null source is written as null unless
+                `conditional.when` is `extract_not_empty`).
               - |-
                 The four "Action to take if source field has no value" choices map to:
-                do nothing → no `default` key (optionally `conditional.when:
-                extract_not_empty`); use null → `"default": null`; use empty string or
+                do nothing → no `default` key (add `conditional.when:
+                extract_not_empty` to leave a null source out too; without it a null
+                source is written as null); use null → `"default": null`; use empty string or
                 a custom value → `"default": "<value>"`; cause the record to fail →
                 `conditional.when: fail_record_when_empty` with no `default` key.
             examples:
@@ -5117,8 +5124,7 @@ components:
 
               **When to Use**
               - REQUIRED when dataType ends with "array" (stringarray, objectarray, etc.), unless the
-                row passes a source array through with a JSONPath `extract` alone or sets
-                `hardCodedValue`
+                row sets `hardCodedValue`: an array row's own `extract` is never read (see `dataType`)
               - Each entry in this array contributes elements to the output array
               - Multiple entries allow combining data from different input arrays
 
@@ -5202,28 +5208,10 @@ components:
               }
               ```
 
-              **For Arrays of Arrays (arrayarray)**
-              - Similar to objectarray, but inner arrays have empty 'generate' fields
-              - Used for matrix/table structures
-              ```json
-              {
-                "generate": "matrix",
-                "dataType": "arrayarray",
-                "buildArrayHelper": [
-                  {
-                    "extract": "$.rows[*]",  // For each row in the rows array
-                    "mappings": [
-                      {
-                        "dataType": "numberarray",  // Note: No generate field for inner arrays
-                        "buildArrayHelper": [
-                          {"extract": "$.rows.columns[*]"}  // Notice: "rows" is singular in the composite object
-                        ]
-                      }
-                    ]
-                  }
-                ]
-              }
-              ```
+              **Arrays of Arrays**
+              - The mapper has no working type for them: `arrayarray` saves, but every record
+                fails with `Destination datatype is a required field.` Build an array of arrays
+                in a script hook instead.
 
               **Important details**
 
@@ -5867,7 +5855,7 @@ components:
               - Unmapped Value
               - '{Original Value}'
             x-celigo-ai-guidance:
-              - 'Omit this field entirely when no fallback value is intended. The API rejects an empty string with [invalid_field] Invalid value '''' for field ''default'' — emitting default: "" to mean "no default" fails validation; leaving the key out is the valid spelling of that intent.'
+              - 'Omit this field entirely when no fallback value is intended: a miss the lookup lets through then leaves the mapped field out, and with `allowFailures: false` a miss fails the record whatever the default. An empty string is a value, not "no default" — the save accepts `default: ""`, and an allowed miss writes `""`.'
           allowFailures:
             type:
               - boolean
@@ -6026,6 +6014,11 @@ components:
                   dropped fields. Either map every field downstream logic
                   needs forward, or author Transform 2.0 with
                   ``mode: "modify"`` instead.
+                - |-
+                  An import's ``responseTransform`` is the exception: it accepts
+                  version ``"1"`` only (``"2"`` is refused with 422, verified live
+                  2026-10-21), so there, map every response field that response
+                  mapping needs.
               items:
                 type: array
                 items:
@@ -14595,7 +14588,7 @@ components:
                   unmatched lookup halts the record. Stored as null when no
                   fallback is configured.
                 x-celigo-ai-guidance:
-                  - 'Omit this field entirely when no fallback value is intended. The API rejects an empty string with [invalid_field] Invalid value '''' for field ''default'' — emitting default: "" to mean "no default" fails validation; leaving the key out is the valid spelling of that intent.'
+                  - 'Omit this field entirely when no fallback value is intended. An empty string is a value, not "no default": the save accepts `default: ""`, and a miss the lookup lets through writes `""`.'
                   - 'A fallback only takes effect on an allowed miss — pair a real default with allowFailures: true (or useDefaultOnMultipleMatches: true for the ambiguous-match case).'
               allowFailures:
                 type:
@@ -25479,10 +25472,11 @@ components:
 
                   Runtime contract: ``options.responseData[]`` (one per
                   submitted record; ``{statusCode, errors, _json, ...}`` —
-                  ``id`` is present on real runs but absent from the
-                  mock-built envelopes flow test runs substitute) → array of
-                  the same length, typically mutated to annotate or augment
-                  per-record errors.
+                  ``id`` is the destination record's on real runs; flow test
+                  runs substitute the saved ``mockResponse`` or the placeholder
+                  response, whose envelope carries the mock's id, such as
+                  ``1234567890``) → array of the same length, typically mutated
+                  to annotate or augment per-record errors.
               allOf:
                 - $ref: '#/components/schemas/Hook'
             postAggregate:
@@ -25521,21 +25515,55 @@ components:
             - transform
           description: |-
             Transformation that reshapes the destination system's response after records are
-            imported, before the response is processed by response mappings and downstream
-            steps. Commonly used to extract relevant fields from verbose API responses.
+            imported: it runs after the postSubmit hook, on each record's response body (`_json`),
+            and its output replaces that body, so response mapping can't read a field the
+            transform drops. Expression rules use version `"1"` (`rules`); version `"2"` is refused
+            on save. Commonly used to extract relevant fields from verbose API responses.
           allOf:
             - $ref: '#/components/schemas/Transform'
+            - type: object
+              properties:
+                expression:
+                  type: object
+                  properties:
+                    version:
+                      type: string
+                      enum:
+                        - '1'
+                      x-enumDescriptions:
+                        '1': Transformation 1.0 rules in `rules`, the only version an import's response transform accepts.
+                      description: Must be `"1"` on an import's response transform; `"2"` is refused with 422.
           x-celigo-ai-guidance:
+            - |-
+              Verified live 2026-10-21 on an HTTP import in a flow run: the postSubmit hook's changes
+              to `responseData[i]._json` were in the transform's input, the transform's output replaced
+              `_json` (only the mapped fields were kept), and response mapping then found nothing at a
+              dropped field's path. The execution-log stage is `responseTransformation`, and
+              `POST /v1/imports/{_id}/invoke` returns the transformed body.
+            - |-
+              A save with `expression.version: "2"` (`rulesTwoDotZero`) fails with 422 "`2` is not a
+              valid enum value for path `responseTransform.expression.version`." Write `version: "1"`
+              rules (`[[{extract, generate}, ...]]`).
             - |-
               ## IMPORT-SPECIFIC BEHAVIOR
 
-              **Response Transformation**: Transforms the response data returned by the destination system after records have been imported. This allows you to reshape, filter, or enrich the response before it is processed by downstream flow steps or returned to the client.
+              **Response Transformation**: Transforms the response data returned by the destination system after records have been imported. This allows you to reshape, filter, or enrich the response before response mapping reads it; the record passed downstream is the incoming record plus what response mapping copies out.
 
               **Common Use Cases**:
               - Extracting relevant fields from verbose API responses
               - Normalizing response formats across different destination systems
               - Adding computed fields based on response data
               - Filtering out sensitive information before logging or further processing
+            - |-
+              ## Transformation 1.0 rules only, and what response mapping reads
+
+              Expression rules here are Transformation 1.0 only (`expression.version: "1"`, the
+              `rules` array) or a script: a version "2" expression is refused with 422
+              (`` `2` is not a valid enum value for path `responseTransform.expression.version`. ``).
+              The rules read the parsed response body, and their output replaces `_json`. v1 rules
+              keep only the fields they generate, so the step's response mapping reads those
+              names (at the envelope root or under `_json`) and nothing the rules dropped; `id`
+              still resolves.
         modelMetadata:
           type: object
           description: Metadata about the destination data model captured for this import. Rarely set.
@@ -25565,6 +25593,11 @@ components:
               default to Mapper 2.0 at the top-level ``mappings`` (plural) field.
               Do not set this field unsolicited; the v2 default produces better
               output for modern connectors.
+
+              An import that carries both this field and ``mappings`` runs only
+              ``mappings``: the Mapper 1.0 rows are ignored, with no error. A
+              conversion to Mapper 2.0 therefore translates every row and removes
+              this field in the same save.
             - |-
               Allowed (but not the default) on other import adaptors such as
               ``HTTPImport``, ``RDBMSImport``, ``FTPImport``, and

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@celigo/api-specs",
-  "version": "0.2.154",
+  "version": "0.2.156",
   "private": false,
   "description": "Public OpenAPI bundles (dist/) and the celigo-cli lint schema pack (schemas.json).",
   "keywords": [