@cxtms/cx-schema 1.8.0 → 1.8.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,21 +1,11 @@
1
1
  ---
2
2
  name: cx-core
3
- description: >
4
- Provides shared CXTMS entity field reference — domain entities, field names, enums, and customValues patterns.
5
- Use when the user asks about CX entity fields, enums, customValues, entity relationships, or needs domain reference for Orders, Contacts, Commodities, Jobs, etc.
3
+ description: Shared CargoXplorer entity field reference — domain entities, field names, enums, and customValues patterns
6
4
  argument-hint: <entity name or question about fields>
7
5
  ---
8
6
 
9
7
  Shared domain reference for CargoXplorer entities. Used by `cx-workflow` and `cx-module` skills for entity field names, types, navigation properties, enums, and customValues extension patterns.
10
8
 
11
- ## CX Server Authentication & Management
12
-
13
- For CLI authentication, PAT tokens, org management, and releasing: see [ref-cli-auth.md](ref-cli-auth.md)
14
-
15
- ## GraphQL Querying & Audit History
16
-
17
- For running GraphQL queries via CLI, filter syntax (Lucene), pagination, audit change history, and field discovery: see [ref-graphql-query.md](ref-graphql-query.md)
18
-
19
9
  ## Feature File Layout
20
10
 
21
11
  All modules and workflows are organized under feature directories:
@@ -31,21 +21,49 @@ When creating new modules or workflows, always place them under the correct feat
31
21
  - `features/<feature_name>/modules/<name>-module.yaml`
32
22
  - `features/<feature_name>/workflows/<name>.yaml`
33
23
 
34
- Use `--feature <feature_name>` with `cxtms create` to automatically place files in the correct location.
24
+ Use `--feature <feature_name>` with `cx-cli create` to automatically place files in the correct location.
35
25
 
36
26
  ## Entity Field Reference
37
27
 
38
- Read the relevant entity reference file when needed for the current task. Do not load all files upfront.
28
+ ### Primary Entities
29
+
30
+ !cat .claude/skills/cx-core/ref-entity-order.md
31
+ !cat .claude/skills/cx-core/ref-entity-contact.md
32
+ !cat .claude/skills/cx-core/ref-entity-commodity.md
33
+ !cat .claude/skills/cx-core/ref-entity-accounting.md
34
+
35
+ ### Order Sub-Entities & Related
36
+
37
+ !cat .claude/skills/cx-core/ref-entity-order-sub.md
38
+ !cat .claude/skills/cx-core/ref-entity-job.md
39
+
40
+ ### Pricing & Accounting Lookups
41
+
42
+ !cat .claude/skills/cx-core/ref-entity-rate.md
43
+
44
+ ### Shared & Lookup Entities
45
+
46
+ !cat .claude/skills/cx-core/ref-entity-shared.md
47
+ !cat .claude/skills/cx-core/ref-entity-geography.md
48
+
49
+ ### Warehouse & Inventory
50
+
51
+ !cat .claude/skills/cx-core/ref-entity-warehouse.md
52
+
53
+ ### Notifications
54
+
55
+ !cat .claude/skills/cx-core/ref-entity-notification.md
39
56
 
40
57
  | Category | Entities | Reference |
41
58
  |----------|----------|-----------|
42
- | **Primary** | Order, Contact, Commodity, AccountingTransaction | [ref-entity-order.md](ref-entity-order.md), [ref-entity-contact.md](ref-entity-contact.md), [ref-entity-commodity.md](ref-entity-commodity.md), [ref-entity-accounting.md](ref-entity-accounting.md) |
43
- | **Order sub** | OrderEntity, TrackingEvent, EventDefinition, LinkedOrder, OrderDocument | [ref-entity-order-sub.md](ref-entity-order-sub.md) |
44
- | **Job** | Job, JobOrder, JobStatus | [ref-entity-job.md](ref-entity-job.md) |
45
- | **Pricing** | Rate, Lane, Discount, AccountingItem, AccountingAccount, PaymentTerm | [ref-entity-rate.md](ref-entity-rate.md) |
46
- | **Shared** | Tag, Attachment, Division, EquipmentType, PackageType, Note/NoteThread | [ref-entity-shared.md](ref-entity-shared.md) |
47
- | **Geography** | Country, State, City, Port, Vessel, CustomCode, ModeOfTransportation | [ref-entity-geography.md](ref-entity-geography.md) |
48
- | **Warehouse** | InventoryItem, WarehouseLocation, CargoMovement (Order variant) | [ref-entity-warehouse.md](ref-entity-warehouse.md) |
59
+ | **Primary** | Order, Contact, Commodity, AccountingTransaction | ref-entity-order/contact/commodity/accounting.md |
60
+ | **Order sub** | OrderEntity, TrackingEvent, EventDefinition, LinkedOrder, OrderDocument | ref-entity-order-sub.md |
61
+ | **Job** | Job, JobOrder, JobStatus | ref-entity-job.md |
62
+ | **Pricing** | Rate, Lane, Discount, AccountingItem, AccountingAccount, PaymentTerm | ref-entity-rate.md |
63
+ | **Shared** | Tag, Attachment, Division, EquipmentType, PackageType, Note/NoteThread | ref-entity-shared.md |
64
+ | **Geography** | Country, State, City, Port, Vessel, CustomCode, ModeOfTransportation | ref-entity-geography.md |
65
+ | **Warehouse** | InventoryItem, WarehouseLocation, CargoMovement (Order variant) | ref-entity-warehouse.md |
66
+ | **Notification** | Notification, UserNotification | ref-entity-notification.md |
49
67
 
50
68
  ## CustomValues Pattern
51
69
 
@@ -0,0 +1,85 @@
1
+ # Notification Entity Reference
2
+
3
+ Real-time notification system. Notifications are org-scoped with per-user read tracking via `UserNotification`.
4
+
5
+ ## Notification
6
+
7
+ | Field | Type | Notes |
8
+ |-------|------|-------|
9
+ | `notificationId` | `int` | PK |
10
+ | `organizationId` | `int` | FK to Organization |
11
+ | `title` | `string` | Required |
12
+ | `message` | `string?` | Body text, supports Markdown |
13
+ | `type` | `NotificationType` | System=0, OrderUpdate=1, TaskAssignment=2, Alert=3, Info=4 |
14
+ | `priority` | `NotificationPriority` | Low=0, Normal=1, High=2, Urgent=3 |
15
+ | `targetUserId` | `string?` | If set, targets one user; if null, broadcasts to all active org users |
16
+ | `entityType` | `string?` | Linked entity type (e.g. "Order", "Job") |
17
+ | `entityId` | `int?` | Linked entity PK |
18
+ | `expiresAt` | `DateTime?` | Optional expiration |
19
+ | `created` / `lastModified` | `DateTime` | Audit fields (from `AuditableEntity`) |
20
+ | `createdBy` / `lastModifiedBy` | `string` | Audit fields |
21
+
22
+ ### Domain Methods
23
+
24
+ - `ChangeTitle(string)`, `ChangeMessage(string?)`, `ChangeType(NotificationType)`, `ChangePriority(NotificationPriority)`, `ChangeExpiresAt(DateTime?)`
25
+
26
+ ---
27
+
28
+ ## UserNotification
29
+
30
+ Per-user read state join entity.
31
+
32
+ | Field | Type | Notes |
33
+ |-------|------|-------|
34
+ | `userNotificationId` | `int` | PK |
35
+ | `notificationId` | `int` | FK to Notification |
36
+ | `userId` | `string` | FK to ApplicationUser |
37
+ | `isRead` | `bool` | Default false |
38
+ | `readAt` | `DateTime?` | Set when marked read |
39
+
40
+ ### Domain Methods
41
+
42
+ - `MarkAsRead()` — sets `IsRead = true`, `ReadAt = UtcNow`
43
+ - `MarkAsUnread()` — resets both
44
+
45
+ ---
46
+
47
+ ## GraphQL
48
+
49
+ ### Queries
50
+
51
+ - `getNotification(organizationId, notificationId)` → single notification for current user (projected from UserNotification)
52
+ - `getNotifications(organizationId, filter?, search?, orderBy?)` → offset-paged list for current user; default sort: `-notification.created`
53
+ - `getUnreadNotificationCount(organizationId)` → int
54
+
55
+ ### Mutations
56
+
57
+ - `createNotification(organizationId, values)` → creates Notification + UserNotification rows; publishes to subscription topic per user
58
+ - `markNotificationRead(organizationId, notificationId)` → bool
59
+ - `markAllNotificationsRead(organizationId)` → int (count marked)
60
+ - `deleteNotification(organizationId, notificationId)` → DeleteResult
61
+
62
+ ### Subscriptions
63
+
64
+ - `onNotificationReceived(organizationId, userId)` → real-time via PostgreSQL NOTIFY/LISTEN
65
+ - Topic: `{organizationId}_{userId}_notifications`
66
+
67
+ ### GraphQL DTO (projected from UserNotification)
68
+
69
+ Flattens `Notification` + `UserNotification` into single DTO: `notificationId`, `organizationId`, `title`, `message`, `type`, `priority`, `targetUserId`, `entityType`, `entityId`, `expiresAt`, `isRead`, `readAt`, `created`, `createdBy`, `lastModified`, `lastModifiedBy`, plus resolved `createdUser` / `updatedUser`.
70
+
71
+ ---
72
+
73
+ ## Targeting Logic
74
+
75
+ On `CreateNotification`:
76
+ 1. If `targetUserId` is provided → single `UserNotification` row
77
+ 2. If null → queries all active `UserEmployee` records in org → one `UserNotification` per user
78
+ 3. After save, publishes `NotificationDto` to each user's subscription topic via `INotificationEventSender`
79
+
80
+ ## Infrastructure
81
+
82
+ - `NotificationConfiguration` — EF Core config: title max length, indexes on `OrganizationId`, composite index on `(OrganizationId, Created DESC)` for paging
83
+ - `UserNotificationConfiguration` — indexes on `UserId`, `NotificationId`, composite on `(UserId, IsRead)`
84
+ - `NotificationEventSender` — uses HotChocolate `ITopicEventSender` to publish to subscription topics
85
+ - Real-time delivery: PostgreSQL `NOTIFY/LISTEN` channel (configured in `DependencyInjection`)
@@ -40,6 +40,13 @@ Represents a party role (Shipper, Consignee, Carrier, etc.) on an order.
40
40
  - `attachments` — filterable collection
41
41
  - `getOrderEntityAttachments(idPropertyName, filter, orderBy, search)` — resolver
42
42
 
43
+ ### Import Behavior (Order/Import)
44
+
45
+ When importing order entities via `Order/Import@1`:
46
+ - The `contact` and `contactAddress` nested objects are excluded from direct field mapping and processed separately.
47
+ - **ContactAddress matching** uses `ImportOrderOptions.ContactAddressMatchByFields` — scoped to organization and (when persisted) to the resolved contact. Supports Lucene filter-based matching on any ContactAddress field (e.g., `["addressLine", "postalCode"]`). If null/empty, falls back to matching by `contactAddressId` only.
48
+ - Both contacts and contact addresses are cached per import session to prevent duplicate creation.
49
+
43
50
  ### EntityTypes Enum
44
51
 
45
52
  Shipper=0, Consignee=1, Carrier=2, Vendor=3, UltimateConsignee=4, NotifyParty=5, Intermediate=6, ForwardingAgent=7, DestinationAgent=8, PickupFrom=9, DeliverTo=10, DeliveryCarrier=11, ReceivedBy=12, USPPI=13
@@ -113,6 +113,7 @@ These are virtual fields that filter `orderEntities` by type:
113
113
  | `notesSummary` | `OrderNoteSummaryGqlDto` | `.totalCount` (int), `.hasAny` (bool) — batched DataLoader, backed by DB view |
114
114
  | `notesCount(threadFilter)` | `int` | |
115
115
  | `changeHistory(startDate, endDate, maxResults)` | `[ChangeHistory]` | Audit trail |
116
+ | `getCommoditiesWithRelatedOrder(orderType!, filter?)` | `[Commodity]` | Leaf commodities linked to related orders of specified type. Traverses commodity hierarchy, excludes wrappers. |
116
117
 
117
118
  ## OrderTypes Enum
118
119
 
@@ -154,3 +154,23 @@ No customValues. Linked to carriers via `CarrierEquipment` join.
154
154
  | `isDeleted` | `bool` | Soft delete |
155
155
 
156
156
  **Navigation:** `thread` (NoteThread)
157
+
158
+ ---
159
+
160
+ ## Secret
161
+
162
+ Encrypted key-value store for sensitive configuration (API keys, tokens, credentials). Encrypted at rest with AES-256; scoped to organizations via qualified naming.
163
+
164
+ | Field | Type | Notes |
165
+ |-------|------|-------|
166
+ | `id` | `int` | PK |
167
+ | `secretName` | `string` | Qualified name: `org/{organizationId}/{secretName}`. Unique index. Max 500 chars. |
168
+ | `encryptedValue` | `string` | AES-256 encrypted, base64-encoded (random IV prepended) |
169
+ | `createdAt` | `DateTime` | |
170
+ | `updatedAt` | `DateTime` | |
171
+
172
+ **Table:** `secrets` (snake_case columns)
173
+ **Provider:** `PostgresSecretManager` (default) or `AzureKeyVault` via `SecretManager:Provider` config.
174
+
175
+ **GraphQL mutations:** `setSecret(organizationId, secretName, secretValue)`, `deleteSecret(organizationId, secretName)`.
176
+ **Org scoping:** Commands validate user org membership; qualified name `org/{orgId}/{name}` is built by the command handler.
@@ -162,6 +162,7 @@ Polymorphic form field — renders different input types based on `type` prop.
162
162
  | `quill` | Rich text editor (Quill) |
163
163
  | `object` | JSON object editor |
164
164
  | `yaml` | YAML editor |
165
+ | `secret` | Encrypted secret input — stores `${secret:qualifiedName}` reference. Requires `secretPath` prop for naming. |
165
166
 
166
167
  **Select/Async options (under `options`):**
167
168
  | Prop | Type | Description |
@@ -279,13 +279,13 @@ Implicit variable: `iteration` (zero-based).
279
279
 
280
280
  | Category | Tasks | Load Reference |
281
281
  |----------|-------|----------------|
282
- | Utilities | SetVariable, Log, Error, HttpRequest, Map, Template, Import, Export, CsvParse | `!cat .claude/skills/cx-workflow/ref-utilities.md` |
282
+ | Utilities | SetVariable, Log, Error, HttpRequest, Map, Template, Import, Export, CsvParse, UnzipFile | `!cat .claude/skills/cx-workflow/ref-utilities.md` |
283
283
  | Query & Workflow | Query/GraphQL, Validation, Workflow/Execute | `!cat .claude/skills/cx-workflow/ref-query.md` |
284
- | Entity CRUD | Order, Contact, Commodity, Job, Charge, Discount, Inventory, Movement | `!cat .claude/skills/cx-workflow/ref-entity.md` |
284
+ | Entity CRUD | Order, Contact, Commodity, Job, Charge, Discount, Inventory, Movement, Transmission | `!cat .claude/skills/cx-workflow/ref-entity.md` |
285
285
  | Communication | Email/Send, Document/Render, Attachment, PdfDocument/Merge | `!cat .claude/skills/cx-workflow/ref-communication.md` |
286
286
  | File Transfer | Connect, Disconnect, ListFiles, Download, Upload, Move, Delete | `!cat .claude/skills/cx-workflow/ref-filetransfer.md` |
287
287
  | Accounting | AccountingTransaction, Payment, Number/Generate, SequenceNumber | `!cat .claude/skills/cx-workflow/ref-accounting.md` |
288
- | Other | User, Auth, Caching, EDI, Flow/Transition, Notes, AppModule, ActionEvent | `!cat .claude/skills/cx-workflow/ref-other.md` |
288
+ | Other | User, Auth, Caching, X12/Parse, EDIFACT, Flow/Transition, Notes, AppModule, ActionEvent | `!cat .claude/skills/cx-workflow/ref-other.md` |
289
289
 
290
290
  ## Entity Field Reference (cx-core)
291
291
 
@@ -102,6 +102,8 @@ Input priority: `stream` > `fileUrl` > `postalCodes`. Task catches exceptions an
102
102
  notes: "Updated by workflow"
103
103
  ```
104
104
 
105
+ **Order/Import commodity fields**: When importing commodities, you can supply `packageTypeName` (string) instead of `packageTypeId`. The import handler resolves the name to an ID using an N+1-safe per-import cache (one DB query per unique package type name).
106
+
105
107
  ## Contact
106
108
 
107
109
  | Task | Description |
@@ -273,6 +275,66 @@ Output `result`: `{ added, updated, skipped, failed, total, errors[] }`.
273
275
  | `Note/Export` | Export notes for an entity |
274
276
  | `Note/RenameThread` | Rename a note thread |
275
277
 
278
+ ## Transmission
279
+
280
+ | Task | Description |
281
+ |------|-------------|
282
+ | `Transmission/Create` | Create transmission record linked to orders |
283
+ | `Transmission/Update` | Update transmission fields (dynamic) |
284
+ | `Transmission/Delete` | Delete transmission record |
285
+
286
+ Records inbound/outbound message transmissions (EDI, API, Email, Webhook) linked to orders.
287
+
288
+ ```yaml
289
+ - task: "Transmission/Create@1"
290
+ name: CreateTransmission
291
+ inputs:
292
+ organizationId: "{{ int organizationId }}"
293
+ transmission:
294
+ orderIds: "{{ orderIds }}"
295
+ channel: "EDI"
296
+ direction: "Outbound"
297
+ messageType: "214"
298
+ sender: "{{ senderISA }}"
299
+ receiver: "{{ receiverISA }}"
300
+ status: "Pending"
301
+ endpoint: "{{ endpoint }}"
302
+ protocol: "SFTP"
303
+ outputs:
304
+ - name: transmission
305
+ mapping: "transmission"
306
+ ```
307
+
308
+ ```yaml
309
+ - task: "Transmission/Update@1"
310
+ name: UpdateTransmission
311
+ inputs:
312
+ organizationId: "{{ int organizationId }}"
313
+ transmissionId: "{{ int Main?.CreateTransmission?.transmission?.id? }}"
314
+ transmission:
315
+ status: "Sent"
316
+ completedAt: "{{ now() }}"
317
+ ```
318
+
319
+ ```yaml
320
+ - task: "Transmission/Delete@1"
321
+ name: DeleteTransmission
322
+ inputs:
323
+ organizationId: "{{ int organizationId }}"
324
+ transmissionId: "{{ int inputs.transmissionId }}"
325
+ outputs:
326
+ - name: success
327
+ mapping: "success"
328
+ ```
329
+
330
+ **Create inputs:** `organizationId` (int, required), `transmission` object — `orderIds` (required, at least one), `channel`, `direction` (Inbound/Outbound), `messageType`, `sender`, `receiver`, `status`, `endpoint`, `protocol`, `correlationId` (auto-generated if omitted), `parentId`, `httpStatus`, `byteSize`, `retryCount`, `maxRetries`, `nextRetryAt`, `errorCode`, `errorMessage`, `customValues`, `headers`, `payloadRef`, `scheduledAt`, `startedAt`, `completedAt`, `durationMs`.
331
+
332
+ **Create outputs:** `transmission` (full TransmissionDto).
333
+ **Update inputs:** `organizationId`, `transmissionId`, `transmission` (dynamic partial fields).
334
+ **Delete outputs:** `success` (boolean).
335
+
336
+ **Status enum:** Pending, InProgress, Sent, Received, Delivered, Acknowledged, Rejected, Error, RetryScheduled, Cancelled, Expired, Accepted.
337
+
276
338
  ## Accounting Transaction (Additional)
277
339
 
278
340
  | Task | Description |
@@ -98,6 +98,22 @@ Functions use two iterator variable names:
98
98
  | `dateToUtc([date])` or `dateToUtc([date], 'en-US')` | Convert to UTC. Optional culture for string parsing |
99
99
  | `toLocalTime([date], 'America/Chicago')` | Convert UTC datetime to local time in IANA timezone. Returns `null` if date or timezone is invalid |
100
100
 
101
+ ### Business Date Math (in Lucene filter expressions)
102
+
103
+ The filter engine (`FilterBy`) supports business-aware date math units in Lucene date expressions:
104
+
105
+ | Unit | Aliases | Description |
106
+ |------|---------|-------------|
107
+ | `BHOUR` | `BHOURS` | Add/subtract business hours (respects weekly schedule + holidays) |
108
+ | `BDAY` | `BDAYS` | Add/subtract business days (skips non-working days) |
109
+
110
+ **Usage**: These units are used in **Lucene filter strings** (not NCalc expressions). They require an `IBusinessDateMathResolver` and are resolved via the organization's business calendar.
111
+
112
+ ```
113
+ dueDate: [NOW TO NOW+3BDAYS]
114
+ pickupDate: [* TO NOW-8BHOURS]
115
+ ```
116
+
101
117
  ### Math Functions (NCalc built-in)
102
118
 
103
119
  `Abs(x)`, `Ceiling(x)`, `Floor(x)`, `Round(x, decimals)`, `Min(x, y)`, `Max(x, y)`, `Pow(x, y)`, `Sqrt(x)`, `Truncate(x)`
@@ -106,6 +106,16 @@ orderData:
106
106
  notes: "{{ newNotes }}"
107
107
  ```
108
108
 
109
+ **`resolve`** -- Entity ID lookup by querying a GraphQL collection:
110
+ ```yaml
111
+ customerId:
112
+ resolve:
113
+ entity: "Contact" # Entity type (auto-pluralized for query)
114
+ filter: "name:{{ customerName }}" # Lucene filter (template-parsed)
115
+ field: "contactId" # Field to return (default: <entity>Id)
116
+ ```
117
+ Results are batched and cached per unique `entity|filter|field` combination by `ResolvePreProcessor` before step execution. Cache misses return `null`. Useful inside `foreach` mappings where many items reference the same entity — only one query per unique filter value.
118
+
109
119
  **`$raw`** -- Prevent template parsing (pass as-is):
110
120
  ```yaml
111
121
  template:
@@ -152,10 +152,11 @@ transitions:
152
152
 
153
153
  ### Execution Order
154
154
  1. Validate transition from current state
155
- 2. Execute `onExit` steps (from source state)
156
- 3. Execute transition `steps`
157
- 4. Update entity status
158
- 5. Execute `onEnter` steps (on target state)
155
+ 2. Evaluate conditions (all must pass)
156
+ 3. Execute `onExit` steps (from source state)
157
+ 4. Execute transition `steps` (step inputs merged + template expressions resolved)
158
+ 5. Update entity status
159
+ 6. Execute `onEnter` steps (on target state)
159
160
 
160
161
  ## Aggregations Section
161
162
 
@@ -64,20 +64,48 @@
64
64
 
65
65
  | Task | Description |
66
66
  |------|-------------|
67
- | `EDI/Parse` | Parse EDI documents (X12, EDIFACT) |
67
+ | `X12/Parse` | Parse X12 EDI documents (850, 856) |
68
+ | `EDIFACT/Parse` | Parse EDIFACT messages (IFTMIN) |
69
+ | `EDIFACT/Generate` | Generate EDIFACT messages (IFTMIN) |
70
+ | `EDI/Parse` | **Deprecated** — alias for X12/Parse, use `X12/Parse` instead |
68
71
  | `StructuredFile/Parse` | Parse structured files |
69
72
 
70
73
  ```yaml
71
- - task: "EDI/Parse@1"
72
- name: ParseEdi
74
+ - task: "X12/Parse@1"
75
+ name: ParseX12
73
76
  inputs:
74
- content: "{{ Transfer.Download.content }}"
75
- format: "X12"
77
+ ediData: "{{ Transfer.Download.content }}"
78
+ transactionSet: "850"
79
+ validateSchema: true
76
80
  outputs:
77
81
  - name: parsed
78
82
  mapping: "document"
79
83
  ```
80
84
 
85
+ ```yaml
86
+ - task: "EDIFACT/Parse@1"
87
+ name: ParseEdifact
88
+ inputs:
89
+ edifactData: "{{ Transfer.Download.content }}"
90
+ messageType: "IFTMIN"
91
+ outputs:
92
+ - name: parsed
93
+ mapping: "document"
94
+ ```
95
+
96
+ ```yaml
97
+ - task: "EDIFACT/Generate@1"
98
+ name: GenerateEdifact
99
+ inputs:
100
+ messageType: "IFTMIN"
101
+ data: "{{ shipmentData }}"
102
+ outputs:
103
+ - name: edifactData
104
+ mapping: "edifactData"
105
+ - name: messageType
106
+ mapping: "messageType"
107
+ ```
108
+
81
109
  ## Flow
82
110
 
83
111
  | Task | Description |
@@ -1,17 +1,5 @@
1
1
  # Utilities Tasks Reference
2
2
 
3
- ## Contents
4
- - Available utility tasks (summary table of all Utilities/* tasks)
5
- - SetVariable task (set variables in activity and global scope)
6
- - Log task (log variables and messages to workflow logger)
7
- - Error task (throw workflow error to halt execution)
8
- - HttpRequest task (HTTP calls to external APIs with action events)
9
- - Map task (extract and reshape data into new variables)
10
- - Template task (Handlebars template rendering)
11
- - CsvParse task (parse CSV content into structured data)
12
- - Export task (export data to file format)
13
- - Import task (import data from file content)
14
-
15
3
  ## Available Tasks
16
4
 
17
5
  | Task | Description |
@@ -28,8 +16,64 @@
28
16
  | `Utilities/MoveFile@1` | Move file |
29
17
  | `Utilities/ValidateReCaptcha` | Validate reCAPTCHA |
30
18
  | `Utilities/ValidateHMAC` | Validate HMAC signatures |
19
+ | `Utilities/UnzipFile@1` | Extract files from ZIP archive (local path or URL) |
31
20
  | `Utilities/ResolveTimezone@1` | Resolve IANA timezone and UTC offset from lat/lng coordinates |
32
21
 
22
+ ## UnzipFile@1
23
+
24
+ Extracts files from a ZIP archive. Accepts a local file path (from `saveToFile` or previous step) or a URL (`file://`, `http://`, `https://`). Files are extracted to a workflow-scoped temp directory with auto-cleanup.
25
+
26
+ ```yaml
27
+ - task: "Utilities/UnzipFile@1"
28
+ name: ExtractArchive
29
+ inputs:
30
+ filePath: "{{ Main?.DownloadArchive?.result?.FilePath? }}"
31
+ filePattern: "*.csv"
32
+ outputs:
33
+ - name: files
34
+ mapping: "Files?"
35
+ - name: count
36
+ mapping: "Count?"
37
+ ```
38
+
39
+ **Inputs:** `filePath` (string, local path) OR `fileUrl` (string, URL — `file://`, `http://`, `https://`). Optional: `filePattern` (glob pattern, e.g. `*.csv`, `data_*.json`).
40
+ **Outputs:** `Files` (string[] — full paths to extracted files), `Count` (int — number of matched files).
41
+ Provide one of `filePath` or `fileUrl`. Common pattern: HttpRequest with `saveToFile: true` → UnzipFile with `filePath`.
42
+
43
+ ```yaml
44
+ # Download + unzip + import pipeline
45
+ - task: "Utilities/HttpRequest@1"
46
+ name: Download
47
+ inputs:
48
+ url: "{{ downloadUrl }}"
49
+ method: GET
50
+ saveToFile: true
51
+ outputs:
52
+ - name: result
53
+ mapping: "response?"
54
+
55
+ - task: "Utilities/UnzipFile@1"
56
+ name: Unzip
57
+ inputs:
58
+ filePath: "{{ Main?.Download?.result?.FilePath? }}"
59
+ filePattern: "*.csv"
60
+ outputs:
61
+ - name: files
62
+ mapping: "Files?"
63
+
64
+ - task: foreach
65
+ name: ProcessFiles
66
+ collection: "Main?.Unzip?.files?"
67
+ steps:
68
+ - task: "Utilities/Import@1"
69
+ name: ImportFile
70
+ inputs:
71
+ fileUrl: "file://{{ item }}"
72
+ format: "csv"
73
+ ```
74
+
75
+ ---
76
+
33
77
  ## ResolveTimezone@1
34
78
 
35
79
  Resolves IANA timezone ID and current UTC offset from geographic coordinates using offline boundary lookup (GeoTimeZone).
@@ -127,6 +171,21 @@ Performs HTTP requests to external APIs.
127
171
 
128
172
  Response available at `ActivityName?.CallApi?.result?`.
129
173
 
174
+ **`saveToFile` mode**: When `saveToFile: true`, the response body is saved to a temp file instead of being returned in memory. The response object changes to `{ StatusCode, Headers, FilePath }`. Use this for large file downloads, then pass `FilePath` to downstream tasks like `UnzipFile` or `Import`.
175
+
176
+ ```yaml
177
+ - task: "Utilities/HttpRequest@1"
178
+ name: DownloadArchive
179
+ inputs:
180
+ url: "{{ downloadUrl }}"
181
+ method: GET
182
+ saveToFile: true
183
+ outputs:
184
+ - name: result
185
+ mapping: "response?"
186
+ # result.FilePath contains the temp file path
187
+ ```
188
+
130
189
  **Action events**: When an HTTP request operates on a specific entity (e.g., sending parcel info for an order), enable `actionEvents` in the inputs so the system can track and notify about the request. Include `eventDataExt` with the entity ID to link the event to the entity.
131
190
 
132
191
  ```yaml
@@ -183,7 +242,7 @@ Renders a Handlebars template string with data.
183
242
 
184
243
  ## CsvParse@1
185
244
 
186
- Parses CSV content into structured data.
245
+ Parses CSV content into structured data. CSV headers are automatically trimmed of whitespace, BOM characters, and other special characters to ensure reliable column matching.
187
246
 
188
247
  ```yaml
189
248
  - task: "Utilities/CsvParse@1"
@@ -214,7 +273,7 @@ Exports data to file format.
214
273
 
215
274
  ## Import@1
216
275
 
217
- Imports data from file content.
276
+ Imports data from file content or URL. Supports `file://` URLs for local files (e.g. from UnzipFile output).
218
277
 
219
278
  ```yaml
220
279
  - task: "Utilities/Import@1"
@@ -226,3 +285,17 @@ Imports data from file content.
226
285
  - name: data
227
286
  mapping: "data?"
228
287
  ```
288
+
289
+ ```yaml
290
+ # Import from local file (e.g. extracted from ZIP)
291
+ - task: "Utilities/Import@1"
292
+ name: ImportLocalFile
293
+ inputs:
294
+ fileUrl: "file://{{ localFilePath }}"
295
+ format: "csv"
296
+ outputs:
297
+ - name: data
298
+ mapping: "data?"
299
+ ```
300
+
301
+ **`file://` URL support**: Import, Order/Import, PostalCodes/Import, and Notes/Import all accept `file://` URLs via UrlStreamHelper. This enables pipeline patterns: HttpRequest (saveToFile) → UnzipFile → Import (file://).
package/package.json CHANGED
@@ -1,12 +1,11 @@
1
1
  {
2
2
  "name": "@cxtms/cx-schema",
3
- "version": "1.8.0",
3
+ "version": "1.8.2",
4
4
  "description": "Schema validation package for CargoXplorer YAML modules",
5
5
  "main": "dist/index.js",
6
6
  "types": "dist/index.d.ts",
7
7
  "bin": {
8
- "cx-cli": "dist/cli.js",
9
- "cxtms": "dist/cli.js"
8
+ "cx-cli": "dist/cli.js"
10
9
  },
11
10
  "scripts": {
12
11
  "build": "tsc",
@@ -21,6 +21,7 @@
21
21
  { "$ref": "if.json" },
22
22
  { "$ref": "consoleLog.json" },
23
23
  { "$ref": "openBarcodeScanner.json" },
24
- { "$ref": "resetDirtyState.json" }
24
+ { "$ref": "resetDirtyState.json" },
25
+ { "$ref": "clipboard.json" }
25
26
  ]
26
27
  }
@@ -0,0 +1,46 @@
1
+ {
2
+ "$schema": "http://json-schema.org/draft-07/schema#",
3
+ "title": "clipboard Action",
4
+ "type": "object",
5
+ "properties": {
6
+ "clipboard": {
7
+ "type": "object",
8
+ "properties": {
9
+ "data": {
10
+ "description": "Data to copy — string, object, array, or template expression."
11
+ },
12
+ "path": {
13
+ "type": "string",
14
+ "description": "Dot-notation path to extract from each array item or single object."
15
+ },
16
+ "template": {
17
+ "type": "string",
18
+ "description": "Per-item template string with {{ }} expressions. Takes precedence over path."
19
+ },
20
+ "format": {
21
+ "type": "string",
22
+ "enum": ["lines", "json", "csv"],
23
+ "description": "Output format for arrays. Default: lines."
24
+ },
25
+ "separator": {
26
+ "type": "string",
27
+ "description": "Separator between items when format is lines. Default: newline."
28
+ },
29
+ "message": {
30
+ "oneOf": [
31
+ { "type": "string" },
32
+ { "$ref": "../schemas.json#/definitions/localized" }
33
+ ],
34
+ "description": "Custom success notification message."
35
+ }
36
+ },
37
+ "required": ["data"],
38
+ "x-examples": [
39
+ { "data": "{{ selectedRows }}", "path": "trackingNumber" },
40
+ { "data": "{{ selectedRows }}", "template": "{{ trackingNumber }}{{ eval (customValues?.spot ? 'S' : '') }}", "message": "Copied!" },
41
+ { "data": "{{ selectedRows }}", "format": "json" }
42
+ ]
43
+ }
44
+ },
45
+ "additionalProperties": false
46
+ }
@@ -92,6 +92,21 @@
92
92
  }
93
93
  }
94
94
  }
95
+ },
96
+ {
97
+ "if": {
98
+ "properties": {
99
+ "name": { "const": "Commodity" }
100
+ }
101
+ },
102
+ "then": {
103
+ "properties": {
104
+ "type": {
105
+ "type": "string",
106
+ "description": "CommodityType code. Optional — if omitted, the flow applies to all commodity types."
107
+ }
108
+ }
109
+ }
95
110
  }
96
111
  ],
97
112
  "additionalProperties": false,
@@ -105,6 +120,10 @@
105
120
  {
106
121
  "name": "Contact",
107
122
  "type": "Customer"
123
+ },
124
+ {
125
+ "name": "Commodity",
126
+ "includes": ["commodityStatus", "commodityEvents.trackingEvent.eventDefinition"]
108
127
  }
109
128
  ]
110
129
  }
@@ -118,9 +118,18 @@
118
118
  {
119
119
  "$ref": "template.json"
120
120
  },
121
+ {
122
+ "$ref": "resolve-timezone.json"
123
+ },
124
+ {
125
+ "$ref": "unzip-file.json"
126
+ },
121
127
  {
122
128
  "$ref": "tracking-event.json"
123
129
  },
130
+ {
131
+ "$ref": "transmission.json"
132
+ },
124
133
  {
125
134
  "$ref": "user.json"
126
135
  },
@@ -8,6 +8,8 @@
8
8
  "type": "string",
9
9
  "enum": [
10
10
  "EDI/Parse@1",
11
+ "EDIFACT/Parse@1",
12
+ "EDIFACT/Generate@1",
11
13
  "StructuredFile/Parse"
12
14
  ],
13
15
  "description": "Task type identifier"
@@ -39,11 +41,101 @@
39
41
  "properties": {
40
42
  "content": {
41
43
  "type": "string",
42
- "description": "File content to parse"
44
+ "description": "File content to parse (EDI/Parse)"
43
45
  },
44
46
  "format": {
45
47
  "type": "string",
46
48
  "description": "EDI format (e.g., X12, EDIFACT)"
49
+ },
50
+ "edifactData": {
51
+ "type": "string",
52
+ "description": "Raw EDIFACT data string (EDIFACT/Parse, EDIFACT/Generate)"
53
+ },
54
+ "messageType": {
55
+ "type": "string",
56
+ "enum": ["IFTMIN", "CONTRL", "APERAK"],
57
+ "description": "EDIFACT message type for parse/generate tasks"
58
+ },
59
+ "sender": {
60
+ "type": "object",
61
+ "description": "Sender identification for EDIFACT/Generate",
62
+ "properties": {
63
+ "id": { "type": "string", "description": "Sender ID (Trading Partner ID)" },
64
+ "qualifier": { "type": "string", "description": "Partner ID code qualifier (e.g., ZZZ)" }
65
+ }
66
+ },
67
+ "recipient": {
68
+ "type": "object",
69
+ "description": "Recipient identification for EDIFACT/Generate",
70
+ "properties": {
71
+ "id": { "type": "string", "description": "Recipient ID (e.g., INTTRA)" },
72
+ "qualifier": { "type": "string", "description": "Partner ID code qualifier" }
73
+ }
74
+ },
75
+ "documentNumber": {
76
+ "type": "string",
77
+ "description": "Shipment Identification Number"
78
+ },
79
+ "messageFunctionCode": {
80
+ "type": "string",
81
+ "enum": ["5", "9"],
82
+ "description": "9 = Original, 5 = Replace"
83
+ },
84
+ "references": {
85
+ "type": "array",
86
+ "description": "Business references (BN=Booking, BM=B/L, etc.)",
87
+ "items": {
88
+ "type": "object",
89
+ "properties": {
90
+ "qualifier": { "type": "string" },
91
+ "identifier": { "type": "string" }
92
+ }
93
+ }
94
+ },
95
+ "parties": {
96
+ "type": "array",
97
+ "description": "Party details (HI=Requestor, CZ=Shipper, CA=Carrier, CN=Consignee)",
98
+ "items": {
99
+ "type": "object",
100
+ "properties": {
101
+ "functionCode": { "type": "string" },
102
+ "identification": { "type": "string" },
103
+ "name": { "type": "string" }
104
+ }
105
+ }
106
+ },
107
+ "goodsItems": {
108
+ "type": "array",
109
+ "description": "Goods item details (GID segment group). Numeric fields (itemNumber, numberOfPackages) are serialized as strings.",
110
+ "items": {
111
+ "type": "object",
112
+ "properties": {
113
+ "itemNumber": { "type": ["integer", "string"], "description": "Goods item number (GID/0). Auto-incremented." },
114
+ "numberOfPackages": { "type": ["integer", "string"], "description": "Number of outer packages (C213/7224)" },
115
+ "packageType": { "type": "string", "description": "Package type code, UN/ECE rec. 21 (C213/7065, e.g. CT, PK, PL)" },
116
+ "packageTypeDescription": { "type": "string", "description": "Package type description (C213/7065 text)" },
117
+ "packageTypeAgencyCode": { "type": "string", "description": "Code list responsible agency (C213/3055). Auto-set to '6' (UN/ECE) when packageType is provided." },
118
+ "packageTypeDescriptionText": { "type": "string", "description": "Package type description text for B/L printing (C213/7064)" },
119
+ "innerPackageCount": { "type": ["integer", "string"], "description": "Number of inner packages (second C213/7224)" },
120
+ "innerPackageType": { "type": "string", "description": "Inner package type code (second C213/7065)" },
121
+ "grossWeight": { "type": ["number", "string"], "description": "Gross weight value" },
122
+ "weightUnit": { "type": "string", "description": "Weight unit (KGM, LBR)" },
123
+ "description": { "type": "string", "description": "Goods description free text" }
124
+ }
125
+ }
126
+ },
127
+ "equipment": {
128
+ "type": "array",
129
+ "description": "Container/equipment details",
130
+ "items": {
131
+ "type": "object",
132
+ "properties": {
133
+ "equipmentQualifier": { "type": "string" },
134
+ "equipmentIdentification": { "type": "string" },
135
+ "sizeTypeCode": { "type": "string" },
136
+ "fullEmptyIndicator": { "type": "string" }
137
+ }
138
+ }
47
139
  }
48
140
  },
49
141
  "additionalProperties": true
@@ -105,6 +105,10 @@
105
105
  }
106
106
  }
107
107
  },
108
+ "saveToFile": {
109
+ "type": "boolean",
110
+ "description": "When true, saves response body to a temp file instead of returning it in memory. Response returns { StatusCode, Headers, FilePath } instead of body content."
111
+ },
108
112
  "cache": {
109
113
  "type": "object",
110
114
  "properties": {
@@ -69,6 +69,51 @@
69
69
  "validateOnly": {
70
70
  "type": "boolean",
71
71
  "description": "Only validate, don't persist"
72
+ },
73
+ "data": {
74
+ "description": "Structured import data (for Order/Import@1)",
75
+ "additionalProperties": true
76
+ },
77
+ "options": {
78
+ "type": "object",
79
+ "description": "Import options (for Order/Import@1)",
80
+ "properties": {
81
+ "orderMatchByFields": {
82
+ "type": "array",
83
+ "items": { "type": "string" },
84
+ "description": "Fields to match existing orders. If null, creates new orders."
85
+ },
86
+ "contactMatchByFields": {
87
+ "type": "array",
88
+ "items": { "type": "string" },
89
+ "description": "Fields to match existing contacts for order entities."
90
+ },
91
+ "contactAddressMatchByFields": {
92
+ "type": "array",
93
+ "items": { "type": "string" },
94
+ "description": "Fields to match existing contact addresses within order entities."
95
+ },
96
+ "inventoryItemMatchByFields": {
97
+ "type": "array",
98
+ "items": { "type": "string" },
99
+ "description": "Fields to match existing inventory items on commodities."
100
+ },
101
+ "tagMatchByFields": {
102
+ "type": "array",
103
+ "items": { "type": "string" },
104
+ "description": "Fields to match existing tags."
105
+ },
106
+ "commodityMatchByFields": {
107
+ "type": "array",
108
+ "items": { "type": "string" },
109
+ "description": "Fields to match existing commodities."
110
+ },
111
+ "linkTrackingEventsToCommodities": {
112
+ "type": "boolean",
113
+ "description": "When true, imported tracking events are linked to the order's first-level (non-container) commodities. Default: false."
114
+ }
115
+ },
116
+ "additionalProperties": true
72
117
  }
73
118
  },
74
119
  "additionalProperties": true
@@ -0,0 +1,65 @@
1
+ {
2
+ "$schema": "http://json-schema.org/draft-07/schema#",
3
+ "title": "ResolveTimezone Task",
4
+ "description": "Resolve IANA timezone ID and UTC offset from geographic coordinates",
5
+ "type": "object",
6
+ "properties": {
7
+ "task": {
8
+ "type": "string",
9
+ "enum": ["Utilities/ResolveTimezone@1"],
10
+ "description": "Task type identifier"
11
+ },
12
+ "name": {
13
+ "type": "string",
14
+ "description": "Step name identifier"
15
+ },
16
+ "description": {
17
+ "type": "string",
18
+ "description": "Step description"
19
+ },
20
+ "conditions": {
21
+ "type": "array",
22
+ "items": {
23
+ "type": "object",
24
+ "properties": {
25
+ "expression": {
26
+ "type": "string"
27
+ }
28
+ },
29
+ "required": ["expression"]
30
+ }
31
+ },
32
+ "continueOnError": {
33
+ "type": "boolean"
34
+ },
35
+ "inputs": {
36
+ "type": "object",
37
+ "description": "ResolveTimezone inputs",
38
+ "properties": {
39
+ "latitude": {
40
+ "type": ["string", "number"],
41
+ "description": "Latitude coordinate (decimal degrees)"
42
+ },
43
+ "longitude": {
44
+ "type": ["string", "number"],
45
+ "description": "Longitude coordinate (decimal degrees)"
46
+ }
47
+ },
48
+ "required": ["latitude", "longitude"],
49
+ "additionalProperties": true
50
+ },
51
+ "outputs": {
52
+ "type": "array",
53
+ "items": {
54
+ "type": "object",
55
+ "properties": {
56
+ "name": { "type": "string" },
57
+ "mapping": { "type": "string" }
58
+ },
59
+ "required": ["name", "mapping"]
60
+ },
61
+ "description": "Outputs: timezoneId (IANA timezone string, e.g. 'America/Chicago'), utcOffset (number, hours from UTC, e.g. -5)"
62
+ }
63
+ },
64
+ "required": ["task", "name", "inputs"]
65
+ }
@@ -93,7 +93,7 @@
93
93
  },
94
94
  "status": {
95
95
  "type": ["string", "integer"],
96
- "enum": ["Pending", "InProgress", "Sent", "Received", "Delivered", "Acknowledged", "Rejected", "Error", "RetryScheduled", "Cancelled", "Expired", 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
96
+ "enum": ["Pending", "InProgress", "Sent", "Received", "Delivered", "Acknowledged", "Rejected", "Error", "RetryScheduled", "Cancelled", "Expired", "Accepted", 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
97
97
  "description": "Transmission status"
98
98
  },
99
99
  "endpoint": {
@@ -0,0 +1,68 @@
1
+ {
2
+ "$schema": "http://json-schema.org/draft-07/schema#",
3
+ "title": "UnzipFile Task",
4
+ "description": "Extract files from a ZIP archive (local path or URL)",
5
+ "type": "object",
6
+ "properties": {
7
+ "task": {
8
+ "type": "string",
9
+ "enum": ["Utilities/UnzipFile@1"],
10
+ "description": "Task type identifier"
11
+ },
12
+ "name": {
13
+ "type": "string",
14
+ "description": "Step name identifier"
15
+ },
16
+ "description": {
17
+ "type": "string",
18
+ "description": "Step description"
19
+ },
20
+ "conditions": {
21
+ "type": "array",
22
+ "items": {
23
+ "type": "object",
24
+ "properties": {
25
+ "expression": {
26
+ "type": "string"
27
+ }
28
+ },
29
+ "required": ["expression"]
30
+ }
31
+ },
32
+ "continueOnError": {
33
+ "type": "boolean"
34
+ },
35
+ "inputs": {
36
+ "type": "object",
37
+ "description": "UnzipFile inputs — provide either filePath or fileUrl",
38
+ "properties": {
39
+ "filePath": {
40
+ "type": "string",
41
+ "description": "Local file path to ZIP archive (from saveToFile or previous step)"
42
+ },
43
+ "fileUrl": {
44
+ "type": "string",
45
+ "description": "URL to ZIP archive (file://, http://, https://)"
46
+ },
47
+ "filePattern": {
48
+ "type": "string",
49
+ "description": "Glob-style pattern to filter extracted files (e.g. '*.csv', 'data_*.json'). If omitted, all files returned."
50
+ }
51
+ },
52
+ "additionalProperties": true
53
+ },
54
+ "outputs": {
55
+ "type": "array",
56
+ "items": {
57
+ "type": "object",
58
+ "properties": {
59
+ "name": { "type": "string" },
60
+ "mapping": { "type": "string" }
61
+ },
62
+ "required": ["name", "mapping"]
63
+ },
64
+ "description": "Outputs: Files (string[] — full paths to extracted files), Count (int — number of matched files)"
65
+ }
66
+ },
67
+ "required": ["task", "name", "inputs"]
68
+ }
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "$schema": "http://json-schema.org/draft-07/schema#",
3
3
  "title": "Workflow Variable",
4
- "description": "Definition of a workflow variable. Valid properties: name, value, fromConfig.",
4
+ "description": "Definition of a workflow variable. Valid properties: name, value, fromConfig, expression.",
5
5
  "type": "object",
6
6
  "properties": {
7
7
  "name": {
@@ -12,6 +12,10 @@
12
12
  "value": {
13
13
  "description": "Static variable value or template expression"
14
14
  },
15
+ "expression": {
16
+ "type": "string",
17
+ "description": "NCalc expression to evaluate at runtime. Has access to all previously resolved variables. Mutually exclusive with value and fromConfig."
18
+ },
15
19
  "fromConfig": {
16
20
  "oneOf": [
17
21
  {