@fluentcommerce/fc-connect-sdk 0.1.48 → 0.1.52
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +506 -379
- package/README.md +343 -0
- package/dist/cjs/clients/fluent-client.js +110 -14
- package/dist/cjs/data-sources/s3-data-source.js +1 -1
- package/dist/cjs/data-sources/sftp-data-source.js +1 -1
- package/dist/cjs/index.d.ts +1 -1
- package/dist/cjs/services/extraction/extraction-orchestrator.d.ts +4 -1
- package/dist/cjs/services/extraction/extraction-orchestrator.js +84 -11
- package/dist/cjs/types/index.d.ts +79 -10
- package/dist/cjs/versori/fluent-versori-client.d.ts +4 -1
- package/dist/cjs/versori/fluent-versori-client.js +131 -13
- package/dist/esm/clients/fluent-client.js +110 -14
- package/dist/esm/data-sources/s3-data-source.js +1 -1
- package/dist/esm/data-sources/sftp-data-source.js +1 -1
- package/dist/esm/index.d.ts +1 -1
- package/dist/esm/services/extraction/extraction-orchestrator.d.ts +4 -1
- package/dist/esm/services/extraction/extraction-orchestrator.js +84 -11
- package/dist/esm/types/index.d.ts +79 -10
- package/dist/esm/versori/fluent-versori-client.d.ts +4 -1
- package/dist/esm/versori/fluent-versori-client.js +131 -13
- package/dist/tsconfig.esm.tsbuildinfo +1 -1
- package/dist/tsconfig.tsbuildinfo +1 -1
- package/dist/tsconfig.types.tsbuildinfo +1 -1
- package/dist/types/index.d.ts +1 -1
- package/dist/types/services/extraction/extraction-orchestrator.d.ts +4 -1
- package/dist/types/types/index.d.ts +79 -10
- package/dist/types/versori/fluent-versori-client.d.ts +4 -1
- package/docs/02-CORE-GUIDES/api-reference/event-api-input-output-reference.md +478 -18
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-01-client-api.md +83 -0
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-08-types.md +52 -0
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-12-partial-responses.md +212 -0
- package/docs/02-CORE-GUIDES/api-reference/readme.md +1 -1
- package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-08-extraction-orchestrator.md +68 -4
- package/docs/02-CORE-GUIDES/mapping/modules/mapping-01-foundations.md +450 -448
- package/docs/02-CORE-GUIDES/mapping/modules/mapping-02-quick-start.md +476 -474
- package/docs/02-CORE-GUIDES/mapping/modules/mapping-03-schema-validation.md +464 -462
- package/docs/02-CORE-GUIDES/mapping/modules/mapping-05-advanced-patterns.md +1366 -1364
- package/docs/readme.md +245 -245
- package/package.json +17 -6
- package/docs/versori-apis/ACTIVATIONS-AND-VARIABLES-GUIDE.md +0 -60
- package/docs/versori-apis/JWT-GENERATION-GUIDE.md +0 -94
- package/docs/versori-apis/QUICK-WORKFLOW.md +0 -293
- package/docs/versori-apis/README.md +0 -73
- package/docs/versori-apis/VERSORI-PLATFORM-ARCHITECTURE.md +0 -880
- package/docs/versori-apis/Versori-Platform-API.postman_collection.json +0 -2925
- package/docs/versori-apis/Versori-Platform-API.postman_environment.example.json +0 -62
- package/docs/versori-apis/Versori-Platform-API.postman_environment.json +0 -178
|
@@ -1,448 +1,450 @@
|
|
|
1
|
-
# Module 1: Foundations
|
|
2
|
-
|
|
3
|
-
**Learn the fundamental concepts of universal mapping**
|
|
4
|
-
|
|
5
|
-
**Level:** Beginner
|
|
6
|
-
|
|
7
|
-
**Estimated Time:** 30 minutes
|
|
8
|
-
|
|
9
|
-
---
|
|
10
|
-
|
|
11
|
-
## What You'll Learn
|
|
12
|
-
|
|
13
|
-
By the end of this module, you'll understand:
|
|
14
|
-
|
|
15
|
-
- ✅ What universal mapping is and why it's powerful
|
|
16
|
-
- ✅ How mapping works with JavaScript objects (not file formats)
|
|
17
|
-
- ✅ The core concepts: fields, sources, resolvers, and validation
|
|
18
|
-
- ✅ How to read and write mapping configurations
|
|
19
|
-
- ✅ When to use UniversalMapper vs other mapping approaches
|
|
20
|
-
|
|
21
|
-
---
|
|
22
|
-
|
|
23
|
-
## What is Universal Mapping?
|
|
24
|
-
|
|
25
|
-
Universal mapping is a **single, unified pattern** for transforming data from any source format (CSV, JSON, XML, Parquet) into any target format (Fluent Commerce Batch API, GraphQL mutations, Parquet, JSON, XML).
|
|
26
|
-
|
|
27
|
-
### The Problem It Solves
|
|
28
|
-
|
|
29
|
-
Before universal mapping, you needed different tools for different transformations:
|
|
30
|
-
|
|
31
|
-
```typescript
|
|
32
|
-
// ❌ OLD WAY: Different tools for different formats
|
|
33
|
-
import { CSVMapper } from './csv-mapper';
|
|
34
|
-
import { XMLMapper } from './xml-mapper';
|
|
35
|
-
import { JSONMapper } from './json-mapper';
|
|
36
|
-
|
|
37
|
-
const csvMapper = new CSVMapper(csvConfig);
|
|
38
|
-
const xmlMapper = new XMLMapper(xmlConfig);
|
|
39
|
-
const jsonMapper = new JSONMapper(jsonConfig);
|
|
40
|
-
|
|
41
|
-
// Different APIs, different patterns, different learning curves
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
```typescript
|
|
45
|
-
// ✅ NEW WAY: One mapper for everything
|
|
46
|
-
import { UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
|
|
47
|
-
|
|
48
|
-
const mapper = new UniversalMapper({
|
|
49
|
-
fields: {
|
|
50
|
-
sku: { source: 'productRef', required: true },
|
|
51
|
-
quantity: { source: 'qty', resolver: 'sdk.parseInt' },
|
|
52
|
-
warehouse: { source: 'locationRef' }
|
|
53
|
-
}
|
|
54
|
-
});
|
|
55
|
-
|
|
56
|
-
// Same API, same pattern, works with CSV, JSON, XML, Parquet, GraphQL
|
|
57
|
-
const result = await mapper.map(data);
|
|
58
|
-
```
|
|
59
|
-
|
|
60
|
-
### Key Benefits
|
|
61
|
-
|
|
62
|
-
1. **One Pattern for All Formats** - Learn once, use everywhere
|
|
63
|
-
2. **Consistent API** - Same methods, same error handling, same validation
|
|
64
|
-
3. **Reusable Configurations** - Write once, use with CSV, JSON, XML, Parquet
|
|
65
|
-
4. **Type Safety** - Full TypeScript support with type inference
|
|
66
|
-
5. **Built-in Resolvers** - Common transformations (trim, parseInt, formatDate) included
|
|
67
|
-
6. **Custom Resolvers** - Extend with your own transformation logic
|
|
68
|
-
|
|
69
|
-
---
|
|
70
|
-
|
|
71
|
-
## Core Concepts
|
|
72
|
-
|
|
73
|
-
### 1. Fields
|
|
74
|
-
|
|
75
|
-
A **field** is a target property in your output data. Each field defines:
|
|
76
|
-
- **Source**: Where to get the data from (JavaScript object path)
|
|
77
|
-
- **Resolver**: How to transform it (optional)
|
|
78
|
-
- **Required**: Whether it must be present (optional)
|
|
79
|
-
- **Default**: Fallback value if missing (optional)
|
|
80
|
-
|
|
81
|
-
```typescript
|
|
82
|
-
{
|
|
83
|
-
fields: {
|
|
84
|
-
sku: { // ← Field name (target)
|
|
85
|
-
source: 'productRef', // ← Source path
|
|
86
|
-
required: true, // ← Validation rule
|
|
87
|
-
resolver: 'sdk.trim' // ← Transformation
|
|
88
|
-
}
|
|
89
|
-
}
|
|
90
|
-
}
|
|
91
|
-
```
|
|
92
|
-
|
|
93
|
-
### 2. Source Paths
|
|
94
|
-
|
|
95
|
-
Source paths use **JavaScript object notation** (dot notation), not JSONPath or XPath:
|
|
96
|
-
|
|
97
|
-
```typescript
|
|
98
|
-
// ✅ CORRECT: JavaScript object paths
|
|
99
|
-
source: 'productRef' // Simple property
|
|
100
|
-
source: 'order.items[0].sku' // Array access
|
|
101
|
-
source: 'customer.address.city' // Nested object
|
|
102
|
-
|
|
103
|
-
// ❌ WRONG: JSONPath syntax
|
|
104
|
-
source: '$.productRef' // Don't use $ prefix
|
|
105
|
-
source: '$..items[*].sku' // Don't use JSONPath wildcards
|
|
106
|
-
```
|
|
107
|
-
|
|
108
|
-
**Important:** Source paths work on **parsed JavaScript objects**, not raw file formats. The SDK parsers (CSVParserService, XMLParserService, etc.) convert files to JavaScript objects first.
|
|
109
|
-
|
|
110
|
-
### 3. Resolvers
|
|
111
|
-
|
|
112
|
-
Resolvers transform source values before mapping:
|
|
113
|
-
|
|
114
|
-
```typescript
|
|
115
|
-
{
|
|
116
|
-
fields: {
|
|
117
|
-
quantity: {
|
|
118
|
-
source: 'qty',
|
|
119
|
-
resolver: 'sdk.parseInt' // ← Converts string "100" to number 100
|
|
120
|
-
},
|
|
121
|
-
status: {
|
|
122
|
-
source: 'status',
|
|
123
|
-
resolver: 'sdk.uppercase' // ← Converts "available" to "AVAILABLE"
|
|
124
|
-
}
|
|
125
|
-
}
|
|
126
|
-
}
|
|
127
|
-
```
|
|
128
|
-
|
|
129
|
-
**Built-in SDK Resolvers:**
|
|
130
|
-
- **String**: `sdk.trim`, `sdk.uppercase`, `sdk.lowercase`, `sdk.toString`
|
|
131
|
-
- **Number**: `sdk.parseInt`, `sdk.parseFloat`, `sdk.number`
|
|
132
|
-
- **Date**: `sdk.formatDate`, `sdk.parseDate`
|
|
133
|
-
- **Boolean**: `sdk.boolean`
|
|
134
|
-
- **Utility**: `sdk.identity`, `sdk.default`, `sdk.coalesce`
|
|
135
|
-
|
|
136
|
-
**Custom Resolvers:** You can also create your own (see [Module 6: Custom Resolvers](./mapping-06-helpers-resolvers.md)).
|
|
137
|
-
|
|
138
|
-
### 4. Validation
|
|
139
|
-
|
|
140
|
-
UniversalMapper validates data automatically:
|
|
141
|
-
|
|
142
|
-
```typescript
|
|
143
|
-
const result = await mapper.map(data);
|
|
144
|
-
|
|
145
|
-
if (!result.success) {
|
|
146
|
-
// Validation failed
|
|
147
|
-
console.error('Errors:', result.errors);
|
|
148
|
-
// [
|
|
149
|
-
// { field: 'sku', error: 'Required field missing' },
|
|
150
|
-
// { field: 'quantity', error: 'Invalid number format' }
|
|
151
|
-
// ]
|
|
152
|
-
} else {
|
|
153
|
-
// Validation passed
|
|
154
|
-
console.log('Mapped data:', result.data);
|
|
155
|
-
}
|
|
156
|
-
```
|
|
157
|
-
|
|
158
|
-
---
|
|
159
|
-
|
|
160
|
-
## How Mapping Works
|
|
161
|
-
|
|
162
|
-
### Step-by-Step Process
|
|
163
|
-
|
|
164
|
-
```mermaid
|
|
165
|
-
graph LR
|
|
166
|
-
A[Source Data] --> B[Parser]
|
|
167
|
-
B --> C[JavaScript Object]
|
|
168
|
-
C --> D[UniversalMapper]
|
|
169
|
-
D --> E[Field Mapping]
|
|
170
|
-
E --> F[Resolver Transform]
|
|
171
|
-
F --> G[Validation]
|
|
172
|
-
G --> H[Output Data]
|
|
173
|
-
```
|
|
174
|
-
|
|
175
|
-
**Example Flow:**
|
|
176
|
-
|
|
177
|
-
1. **Source**: CSV file with columns `sku`, `qty`, `warehouse`
|
|
178
|
-
2. **Parser**: `CSVParserService` converts CSV → JavaScript array:
|
|
179
|
-
```javascript
|
|
180
|
-
[
|
|
181
|
-
{ sku: 'SKU-001', qty: '100', warehouse: 'WH-001' },
|
|
182
|
-
{ sku: 'SKU-002', qty: '200', warehouse: 'WH-002' }
|
|
183
|
-
]
|
|
184
|
-
```
|
|
185
|
-
3. **Mapper**: `UniversalMapper` transforms using config:
|
|
186
|
-
```typescript
|
|
187
|
-
{
|
|
188
|
-
fields: {
|
|
189
|
-
productRef: { source: 'sku' },
|
|
190
|
-
quantity: { source: 'qty', resolver: 'sdk.parseInt' },
|
|
191
|
-
locationRef: { source: 'warehouse' }
|
|
192
|
-
}
|
|
193
|
-
}
|
|
194
|
-
```
|
|
195
|
-
4. **Output**: Transformed data:
|
|
196
|
-
```javascript
|
|
197
|
-
[
|
|
198
|
-
{ productRef: 'SKU-001', quantity: 100, locationRef: 'WH-001' },
|
|
199
|
-
{ productRef: 'SKU-002', quantity: 200, locationRef: 'WH-002' }
|
|
200
|
-
]
|
|
201
|
-
```
|
|
202
|
-
|
|
203
|
-
---
|
|
204
|
-
|
|
205
|
-
## Mapping Configuration Format
|
|
206
|
-
|
|
207
|
-
### Basic Structure
|
|
208
|
-
|
|
209
|
-
```typescript
|
|
210
|
-
{
|
|
211
|
-
fields: {
|
|
212
|
-
targetField1: {
|
|
213
|
-
source: 'sourcePath1',
|
|
214
|
-
resolver: 'sdk.resolverName', // Optional
|
|
215
|
-
required: true, // Optional
|
|
216
|
-
defaultValue: 'default' // Optional
|
|
217
|
-
},
|
|
218
|
-
targetField2: {
|
|
219
|
-
source: 'sourcePath2'
|
|
220
|
-
}
|
|
221
|
-
}
|
|
222
|
-
}
|
|
223
|
-
```
|
|
224
|
-
|
|
225
|
-
### External JSON Configuration (Recommended)
|
|
226
|
-
|
|
227
|
-
For production, store mapping configs in external JSON files:
|
|
228
|
-
|
|
229
|
-
```json
|
|
230
|
-
// config/inventory-mapping.json
|
|
231
|
-
{
|
|
232
|
-
"fields": {
|
|
233
|
-
"ref": {
|
|
234
|
-
"source": "sku",
|
|
235
|
-
"required": true
|
|
236
|
-
},
|
|
237
|
-
"productRef": {
|
|
238
|
-
"source": "sku",
|
|
239
|
-
"required": true
|
|
240
|
-
},
|
|
241
|
-
"locationRef": {
|
|
242
|
-
"source": "warehouse",
|
|
243
|
-
"required": true
|
|
244
|
-
},
|
|
245
|
-
"qty": {
|
|
246
|
-
"source": "quantity",
|
|
247
|
-
"resolver": "sdk.parseInt",
|
|
248
|
-
"required": true
|
|
249
|
-
}
|
|
250
|
-
}
|
|
251
|
-
}
|
|
252
|
-
```
|
|
253
|
-
|
|
254
|
-
```typescript
|
|
255
|
-
// Load and use
|
|
256
|
-
import mappingConfig from './config/inventory-mapping.json' with { type: 'json' };
|
|
257
|
-
const mapper = new UniversalMapper(mappingConfig);
|
|
258
|
-
```
|
|
259
|
-
|
|
260
|
-
**Benefits of External JSON:**
|
|
261
|
-
- ✅ Environment-specific configurations
|
|
262
|
-
- ✅ No code changes for field updates
|
|
263
|
-
- ✅ Easier testing and validation
|
|
264
|
-
- ✅ Better separation of concerns
|
|
265
|
-
|
|
266
|
-
---
|
|
267
|
-
|
|
268
|
-
## When to Use UniversalMapper
|
|
269
|
-
|
|
270
|
-
### ✅ Use UniversalMapper For:
|
|
271
|
-
|
|
272
|
-
- **CSV → Batch API** - Inventory ingestion from CSV files
|
|
273
|
-
- **JSON → Batch API** - API responses to Fluent Commerce
|
|
274
|
-
- **GraphQL → Parquet** - Data extraction and export
|
|
275
|
-
- **XML → JSON** - Format conversion
|
|
276
|
-
- **Any data transformation** - When you need field mapping
|
|
277
|
-
|
|
278
|
-
### ❌ Don't Use UniversalMapper For:
|
|
279
|
-
|
|
280
|
-
- **XML → GraphQL Mutations** - Use `GraphQLMutationMapper` instead (specialized tool)
|
|
281
|
-
- **Simple field copying** - If no transformation needed, direct assignment might be simpler
|
|
282
|
-
- **Complex business logic** - Use custom resolvers or separate transformation functions
|
|
283
|
-
|
|
284
|
-
**See:** [Mapper Comparison Guide](../mapping-mapper-comparison-guide.md) for detailed decision matrix.
|
|
285
|
-
|
|
286
|
-
---
|
|
287
|
-
|
|
288
|
-
## Quick Example
|
|
289
|
-
|
|
290
|
-
Here's a complete, working example:
|
|
291
|
-
|
|
292
|
-
```typescript
|
|
293
|
-
import { UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
|
|
294
|
-
|
|
295
|
-
// Define mapping configuration
|
|
296
|
-
const mappingConfig = {
|
|
297
|
-
fields: {
|
|
298
|
-
sku: {
|
|
299
|
-
source: 'productRef',
|
|
300
|
-
required: true
|
|
301
|
-
},
|
|
302
|
-
quantity: {
|
|
303
|
-
source: 'qty',
|
|
304
|
-
resolver: 'sdk.parseInt',
|
|
305
|
-
required: true
|
|
306
|
-
},
|
|
307
|
-
warehouse: {
|
|
308
|
-
source: 'locationRef',
|
|
309
|
-
defaultValue: 'DEFAULT-WH'
|
|
310
|
-
},
|
|
311
|
-
status: {
|
|
312
|
-
source: 'status',
|
|
313
|
-
resolver: 'sdk.uppercase',
|
|
314
|
-
defaultValue: 'AVAILABLE'
|
|
315
|
-
}
|
|
316
|
-
}
|
|
317
|
-
};
|
|
318
|
-
|
|
319
|
-
// Create mapper
|
|
320
|
-
const mapper = new UniversalMapper(mappingConfig);
|
|
321
|
-
|
|
322
|
-
// Source data (could be from CSV, JSON, XML, etc.)
|
|
323
|
-
const sourceData = [
|
|
324
|
-
{ productRef: 'SKU-001', qty: '100', locationRef: 'WH-001', status: 'available' },
|
|
325
|
-
{ productRef: 'SKU-002', qty: '200', status: 'in_stock' } // Missing locationRef
|
|
326
|
-
];
|
|
327
|
-
|
|
328
|
-
// Map data
|
|
329
|
-
const result = await mapper.map(sourceData);
|
|
330
|
-
|
|
331
|
-
if (result.success) {
|
|
332
|
-
console.log('Mapped data:', result.data);
|
|
333
|
-
// [
|
|
334
|
-
// { sku: 'SKU-001', quantity: 100, warehouse: 'WH-001', status: 'AVAILABLE' },
|
|
335
|
-
// { sku: 'SKU-002', quantity: 200, warehouse: 'DEFAULT-WH', status: 'IN_STOCK' }
|
|
336
|
-
// ]
|
|
337
|
-
} else {
|
|
338
|
-
console.error('Mapping errors:', result.errors);
|
|
339
|
-
}
|
|
340
|
-
```
|
|
341
|
-
|
|
342
|
-
---
|
|
343
|
-
|
|
344
|
-
## Common Patterns
|
|
345
|
-
|
|
346
|
-
### Pattern 1: Simple Field Mapping
|
|
347
|
-
|
|
348
|
-
```typescript
|
|
349
|
-
{
|
|
350
|
-
fields: {
|
|
351
|
-
targetField: { source: 'sourceField' }
|
|
352
|
-
}
|
|
353
|
-
}
|
|
354
|
-
```
|
|
355
|
-
|
|
356
|
-
### Pattern 2: With Type Conversion
|
|
357
|
-
|
|
358
|
-
```typescript
|
|
359
|
-
{
|
|
360
|
-
fields: {
|
|
361
|
-
quantity: {
|
|
362
|
-
source: 'qty',
|
|
363
|
-
resolver: 'sdk.parseInt' // String → Number
|
|
364
|
-
}
|
|
365
|
-
}
|
|
366
|
-
}
|
|
367
|
-
```
|
|
368
|
-
|
|
369
|
-
### Pattern 3: With Default Value
|
|
370
|
-
|
|
371
|
-
```typescript
|
|
372
|
-
{
|
|
373
|
-
fields: {
|
|
374
|
-
status: {
|
|
375
|
-
source: 'status',
|
|
376
|
-
defaultValue: 'AVAILABLE' // Use if source is missing
|
|
377
|
-
}
|
|
378
|
-
}
|
|
379
|
-
}
|
|
380
|
-
```
|
|
381
|
-
|
|
382
|
-
### Pattern 4: Required Field
|
|
383
|
-
|
|
384
|
-
```typescript
|
|
385
|
-
{
|
|
386
|
-
fields: {
|
|
387
|
-
sku: {
|
|
388
|
-
source: 'productRef',
|
|
389
|
-
required: true // Validation: must be present
|
|
390
|
-
}
|
|
391
|
-
}
|
|
392
|
-
}
|
|
393
|
-
```
|
|
394
|
-
|
|
395
|
-
### Pattern 5: Nested Object Access
|
|
396
|
-
|
|
397
|
-
```typescript
|
|
398
|
-
{
|
|
399
|
-
fields: {
|
|
400
|
-
city: {
|
|
401
|
-
source: 'address.city' // Access nested property
|
|
402
|
-
}
|
|
403
|
-
}
|
|
404
|
-
}
|
|
405
|
-
```
|
|
406
|
-
|
|
407
|
-
---
|
|
408
|
-
|
|
409
|
-
## Next Steps
|
|
410
|
-
|
|
411
|
-
Now that you understand the foundations:
|
|
412
|
-
|
|
413
|
-
1. **Module 2: Quick Start** - Build your first mapping workflow
|
|
414
|
-
2. **Module 3: Schema Validation** - Validate mappings against Fluent Commerce schema
|
|
415
|
-
3. **Module 4: Use Cases** - Real-world mapping scenarios
|
|
416
|
-
4. **Module 5: Advanced Patterns** - Arrays, nested objects, conditional mapping
|
|
417
|
-
|
|
418
|
-
---
|
|
419
|
-
|
|
420
|
-
## Key Takeaways
|
|
421
|
-
|
|
422
|
-
✅ **Universal mapping** = One pattern for all data transformations
|
|
423
|
-
|
|
424
|
-
✅ **Source paths** = JavaScript object notation (not JSONPath/XPath)
|
|
425
|
-
|
|
426
|
-
✅ **Resolvers** = Transform values before mapping (built-in + custom)
|
|
427
|
-
|
|
428
|
-
✅ **Validation** = Automatic validation of required fields and types
|
|
429
|
-
|
|
430
|
-
✅ **External JSON** = Recommended for production configurations
|
|
431
|
-
|
|
432
|
-
✅ **Works with any format** = CSV, JSON, XML, Parquet, GraphQL
|
|
433
|
-
|
|
434
|
-
---
|
|
435
|
-
|
|
436
|
-
**Ready to build your first mapping?** → [Module 2: Quick Start](./mapping-02-quick-start.md)
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
1
|
+
# Module 1: Foundations
|
|
2
|
+
|
|
3
|
+
**Learn the fundamental concepts of universal mapping**
|
|
4
|
+
|
|
5
|
+
**Level:** Beginner
|
|
6
|
+
|
|
7
|
+
**Estimated Time:** 30 minutes
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## What You'll Learn
|
|
12
|
+
|
|
13
|
+
By the end of this module, you'll understand:
|
|
14
|
+
|
|
15
|
+
- ✅ What universal mapping is and why it's powerful
|
|
16
|
+
- ✅ How mapping works with JavaScript objects (not file formats)
|
|
17
|
+
- ✅ The core concepts: fields, sources, resolvers, and validation
|
|
18
|
+
- ✅ How to read and write mapping configurations
|
|
19
|
+
- ✅ When to use UniversalMapper vs other mapping approaches
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## What is Universal Mapping?
|
|
24
|
+
|
|
25
|
+
Universal mapping is a **single, unified pattern** for transforming data from any source format (CSV, JSON, XML, Parquet) into any target format (Fluent Commerce Batch API, GraphQL mutations, Parquet, JSON, XML).
|
|
26
|
+
|
|
27
|
+
### The Problem It Solves
|
|
28
|
+
|
|
29
|
+
Before universal mapping, you needed different tools for different transformations:
|
|
30
|
+
|
|
31
|
+
```typescript
|
|
32
|
+
// ❌ OLD WAY: Different tools for different formats
|
|
33
|
+
import { CSVMapper } from './csv-mapper';
|
|
34
|
+
import { XMLMapper } from './xml-mapper';
|
|
35
|
+
import { JSONMapper } from './json-mapper';
|
|
36
|
+
|
|
37
|
+
const csvMapper = new CSVMapper(csvConfig);
|
|
38
|
+
const xmlMapper = new XMLMapper(xmlConfig);
|
|
39
|
+
const jsonMapper = new JSONMapper(jsonConfig);
|
|
40
|
+
|
|
41
|
+
// Different APIs, different patterns, different learning curves
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
```typescript
|
|
45
|
+
// ✅ NEW WAY: One mapper for everything
|
|
46
|
+
import { UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
|
|
47
|
+
|
|
48
|
+
const mapper = new UniversalMapper({
|
|
49
|
+
fields: {
|
|
50
|
+
sku: { source: 'productRef', required: true },
|
|
51
|
+
quantity: { source: 'qty', resolver: 'sdk.parseInt' },
|
|
52
|
+
warehouse: { source: 'locationRef' }
|
|
53
|
+
}
|
|
54
|
+
});
|
|
55
|
+
|
|
56
|
+
// Same API, same pattern, works with CSV, JSON, XML, Parquet, GraphQL
|
|
57
|
+
const result = await mapper.map(data);
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
### Key Benefits
|
|
61
|
+
|
|
62
|
+
1. **One Pattern for All Formats** - Learn once, use everywhere
|
|
63
|
+
2. **Consistent API** - Same methods, same error handling, same validation
|
|
64
|
+
3. **Reusable Configurations** - Write once, use with CSV, JSON, XML, Parquet
|
|
65
|
+
4. **Type Safety** - Full TypeScript support with type inference
|
|
66
|
+
5. **Built-in Resolvers** - Common transformations (trim, parseInt, formatDate) included
|
|
67
|
+
6. **Custom Resolvers** - Extend with your own transformation logic
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## Core Concepts
|
|
72
|
+
|
|
73
|
+
### 1. Fields
|
|
74
|
+
|
|
75
|
+
A **field** is a target property in your output data. Each field defines:
|
|
76
|
+
- **Source**: Where to get the data from (JavaScript object path)
|
|
77
|
+
- **Resolver**: How to transform it (optional)
|
|
78
|
+
- **Required**: Whether it must be present (optional)
|
|
79
|
+
- **Default**: Fallback value if missing (optional)
|
|
80
|
+
|
|
81
|
+
```typescript
|
|
82
|
+
{
|
|
83
|
+
fields: {
|
|
84
|
+
sku: { // ← Field name (target)
|
|
85
|
+
source: 'productRef', // ← Source path
|
|
86
|
+
required: true, // ← Validation rule
|
|
87
|
+
resolver: 'sdk.trim' // ← Transformation
|
|
88
|
+
}
|
|
89
|
+
}
|
|
90
|
+
}
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
### 2. Source Paths
|
|
94
|
+
|
|
95
|
+
Source paths use **JavaScript object notation** (dot notation), not JSONPath or XPath:
|
|
96
|
+
|
|
97
|
+
```typescript
|
|
98
|
+
// ✅ CORRECT: JavaScript object paths
|
|
99
|
+
source: 'productRef' // Simple property
|
|
100
|
+
source: 'order.items[0].sku' // Array access
|
|
101
|
+
source: 'customer.address.city' // Nested object
|
|
102
|
+
|
|
103
|
+
// ❌ WRONG: JSONPath syntax
|
|
104
|
+
source: '$.productRef' // Don't use $ prefix
|
|
105
|
+
source: '$..items[*].sku' // Don't use JSONPath wildcards
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
**Important:** Source paths work on **parsed JavaScript objects**, not raw file formats. The SDK parsers (CSVParserService, XMLParserService, etc.) convert files to JavaScript objects first.
|
|
109
|
+
|
|
110
|
+
### 3. Resolvers
|
|
111
|
+
|
|
112
|
+
Resolvers transform source values before mapping:
|
|
113
|
+
|
|
114
|
+
```typescript
|
|
115
|
+
{
|
|
116
|
+
fields: {
|
|
117
|
+
quantity: {
|
|
118
|
+
source: 'qty',
|
|
119
|
+
resolver: 'sdk.parseInt' // ← Converts string "100" to number 100
|
|
120
|
+
},
|
|
121
|
+
status: {
|
|
122
|
+
source: 'status',
|
|
123
|
+
resolver: 'sdk.uppercase' // ← Converts "available" to "AVAILABLE"
|
|
124
|
+
}
|
|
125
|
+
}
|
|
126
|
+
}
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
**Built-in SDK Resolvers:**
|
|
130
|
+
- **String**: `sdk.trim`, `sdk.uppercase`, `sdk.lowercase`, `sdk.toString`
|
|
131
|
+
- **Number**: `sdk.parseInt`, `sdk.parseFloat`, `sdk.number`
|
|
132
|
+
- **Date**: `sdk.formatDate`, `sdk.parseDate`
|
|
133
|
+
- **Boolean**: `sdk.boolean`
|
|
134
|
+
- **Utility**: `sdk.identity`, `sdk.default`, `sdk.coalesce`
|
|
135
|
+
|
|
136
|
+
**Custom Resolvers:** You can also create your own (see [Module 6: Custom Resolvers](./mapping-06-helpers-resolvers.md)).
|
|
137
|
+
|
|
138
|
+
### 4. Validation
|
|
139
|
+
|
|
140
|
+
UniversalMapper validates data automatically:
|
|
141
|
+
|
|
142
|
+
```typescript
|
|
143
|
+
const result = await mapper.map(data);
|
|
144
|
+
|
|
145
|
+
if (!result.success) {
|
|
146
|
+
// Validation failed
|
|
147
|
+
console.error('Errors:', result.errors);
|
|
148
|
+
// [
|
|
149
|
+
// { field: 'sku', error: 'Required field missing' },
|
|
150
|
+
// { field: 'quantity', error: 'Invalid number format' }
|
|
151
|
+
// ]
|
|
152
|
+
} else {
|
|
153
|
+
// Validation passed
|
|
154
|
+
console.log('Mapped data:', result.data);
|
|
155
|
+
}
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
---
|
|
159
|
+
|
|
160
|
+
## How Mapping Works
|
|
161
|
+
|
|
162
|
+
### Step-by-Step Process
|
|
163
|
+
|
|
164
|
+
```mermaid
|
|
165
|
+
graph LR
|
|
166
|
+
A[Source Data] --> B[Parser]
|
|
167
|
+
B --> C[JavaScript Object]
|
|
168
|
+
C --> D[UniversalMapper]
|
|
169
|
+
D --> E[Field Mapping]
|
|
170
|
+
E --> F[Resolver Transform]
|
|
171
|
+
F --> G[Validation]
|
|
172
|
+
G --> H[Output Data]
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
**Example Flow:**
|
|
176
|
+
|
|
177
|
+
1. **Source**: CSV file with columns `sku`, `qty`, `warehouse`
|
|
178
|
+
2. **Parser**: `CSVParserService` converts CSV → JavaScript array:
|
|
179
|
+
```javascript
|
|
180
|
+
[
|
|
181
|
+
{ sku: 'SKU-001', qty: '100', warehouse: 'WH-001' },
|
|
182
|
+
{ sku: 'SKU-002', qty: '200', warehouse: 'WH-002' }
|
|
183
|
+
]
|
|
184
|
+
```
|
|
185
|
+
3. **Mapper**: `UniversalMapper` transforms using config:
|
|
186
|
+
```typescript
|
|
187
|
+
{
|
|
188
|
+
fields: {
|
|
189
|
+
productRef: { source: 'sku' },
|
|
190
|
+
quantity: { source: 'qty', resolver: 'sdk.parseInt' },
|
|
191
|
+
locationRef: { source: 'warehouse' }
|
|
192
|
+
}
|
|
193
|
+
}
|
|
194
|
+
```
|
|
195
|
+
4. **Output**: Transformed data:
|
|
196
|
+
```javascript
|
|
197
|
+
[
|
|
198
|
+
{ productRef: 'SKU-001', quantity: 100, locationRef: 'WH-001' },
|
|
199
|
+
{ productRef: 'SKU-002', quantity: 200, locationRef: 'WH-002' }
|
|
200
|
+
]
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
---
|
|
204
|
+
|
|
205
|
+
## Mapping Configuration Format
|
|
206
|
+
|
|
207
|
+
### Basic Structure
|
|
208
|
+
|
|
209
|
+
```typescript
|
|
210
|
+
{
|
|
211
|
+
fields: {
|
|
212
|
+
targetField1: {
|
|
213
|
+
source: 'sourcePath1',
|
|
214
|
+
resolver: 'sdk.resolverName', // Optional
|
|
215
|
+
required: true, // Optional
|
|
216
|
+
defaultValue: 'default' // Optional
|
|
217
|
+
},
|
|
218
|
+
targetField2: {
|
|
219
|
+
source: 'sourcePath2'
|
|
220
|
+
}
|
|
221
|
+
}
|
|
222
|
+
}
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
### External JSON Configuration (Recommended)
|
|
226
|
+
|
|
227
|
+
For production, store mapping configs in external JSON files:
|
|
228
|
+
|
|
229
|
+
```json
|
|
230
|
+
// config/inventory-mapping.json
|
|
231
|
+
{
|
|
232
|
+
"fields": {
|
|
233
|
+
"ref": {
|
|
234
|
+
"source": "sku",
|
|
235
|
+
"required": true
|
|
236
|
+
},
|
|
237
|
+
"productRef": {
|
|
238
|
+
"source": "sku",
|
|
239
|
+
"required": true
|
|
240
|
+
},
|
|
241
|
+
"locationRef": {
|
|
242
|
+
"source": "warehouse",
|
|
243
|
+
"required": true
|
|
244
|
+
},
|
|
245
|
+
"qty": {
|
|
246
|
+
"source": "quantity",
|
|
247
|
+
"resolver": "sdk.parseInt",
|
|
248
|
+
"required": true
|
|
249
|
+
}
|
|
250
|
+
}
|
|
251
|
+
}
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
```typescript
|
|
255
|
+
// Load and use
|
|
256
|
+
import mappingConfig from './config/inventory-mapping.json' with { type: 'json' };
|
|
257
|
+
const mapper = new UniversalMapper(mappingConfig);
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
**Benefits of External JSON:**
|
|
261
|
+
- ✅ Environment-specific configurations
|
|
262
|
+
- ✅ No code changes for field updates
|
|
263
|
+
- ✅ Easier testing and validation
|
|
264
|
+
- ✅ Better separation of concerns
|
|
265
|
+
|
|
266
|
+
---
|
|
267
|
+
|
|
268
|
+
## When to Use UniversalMapper
|
|
269
|
+
|
|
270
|
+
### ✅ Use UniversalMapper For:
|
|
271
|
+
|
|
272
|
+
- **CSV → Batch API** - Inventory ingestion from CSV files
|
|
273
|
+
- **JSON → Batch API** - API responses to Fluent Commerce
|
|
274
|
+
- **GraphQL → Parquet** - Data extraction and export
|
|
275
|
+
- **XML → JSON** - Format conversion
|
|
276
|
+
- **Any data transformation** - When you need field mapping
|
|
277
|
+
|
|
278
|
+
### ❌ Don't Use UniversalMapper For:
|
|
279
|
+
|
|
280
|
+
- **XML → GraphQL Mutations** - Use `GraphQLMutationMapper` instead (specialized tool)
|
|
281
|
+
- **Simple field copying** - If no transformation needed, direct assignment might be simpler
|
|
282
|
+
- **Complex business logic** - Use custom resolvers or separate transformation functions
|
|
283
|
+
|
|
284
|
+
**See:** [Mapper Comparison Guide](../mapping-mapper-comparison-guide.md) for detailed decision matrix.
|
|
285
|
+
|
|
286
|
+
---
|
|
287
|
+
|
|
288
|
+
## Quick Example
|
|
289
|
+
|
|
290
|
+
Here's a complete, working example:
|
|
291
|
+
|
|
292
|
+
```typescript
|
|
293
|
+
import { UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
|
|
294
|
+
|
|
295
|
+
// Define mapping configuration
|
|
296
|
+
const mappingConfig = {
|
|
297
|
+
fields: {
|
|
298
|
+
sku: {
|
|
299
|
+
source: 'productRef',
|
|
300
|
+
required: true
|
|
301
|
+
},
|
|
302
|
+
quantity: {
|
|
303
|
+
source: 'qty',
|
|
304
|
+
resolver: 'sdk.parseInt',
|
|
305
|
+
required: true
|
|
306
|
+
},
|
|
307
|
+
warehouse: {
|
|
308
|
+
source: 'locationRef',
|
|
309
|
+
defaultValue: 'DEFAULT-WH'
|
|
310
|
+
},
|
|
311
|
+
status: {
|
|
312
|
+
source: 'status',
|
|
313
|
+
resolver: 'sdk.uppercase',
|
|
314
|
+
defaultValue: 'AVAILABLE'
|
|
315
|
+
}
|
|
316
|
+
}
|
|
317
|
+
};
|
|
318
|
+
|
|
319
|
+
// Create mapper
|
|
320
|
+
const mapper = new UniversalMapper(mappingConfig);
|
|
321
|
+
|
|
322
|
+
// Source data (could be from CSV, JSON, XML, etc.)
|
|
323
|
+
const sourceData = [
|
|
324
|
+
{ productRef: 'SKU-001', qty: '100', locationRef: 'WH-001', status: 'available' },
|
|
325
|
+
{ productRef: 'SKU-002', qty: '200', status: 'in_stock' } // Missing locationRef
|
|
326
|
+
];
|
|
327
|
+
|
|
328
|
+
// Map data
|
|
329
|
+
const result = await mapper.map(sourceData);
|
|
330
|
+
|
|
331
|
+
if (result.success) {
|
|
332
|
+
console.log('Mapped data:', result.data);
|
|
333
|
+
// [
|
|
334
|
+
// { sku: 'SKU-001', quantity: 100, warehouse: 'WH-001', status: 'AVAILABLE' },
|
|
335
|
+
// { sku: 'SKU-002', quantity: 200, warehouse: 'DEFAULT-WH', status: 'IN_STOCK' }
|
|
336
|
+
// ]
|
|
337
|
+
} else {
|
|
338
|
+
console.error('Mapping errors:', result.errors);
|
|
339
|
+
}
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
---
|
|
343
|
+
|
|
344
|
+
## Common Patterns
|
|
345
|
+
|
|
346
|
+
### Pattern 1: Simple Field Mapping
|
|
347
|
+
|
|
348
|
+
```typescript
|
|
349
|
+
{
|
|
350
|
+
fields: {
|
|
351
|
+
targetField: { source: 'sourceField' }
|
|
352
|
+
}
|
|
353
|
+
}
|
|
354
|
+
```
|
|
355
|
+
|
|
356
|
+
### Pattern 2: With Type Conversion
|
|
357
|
+
|
|
358
|
+
```typescript
|
|
359
|
+
{
|
|
360
|
+
fields: {
|
|
361
|
+
quantity: {
|
|
362
|
+
source: 'qty',
|
|
363
|
+
resolver: 'sdk.parseInt' // String → Number
|
|
364
|
+
}
|
|
365
|
+
}
|
|
366
|
+
}
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
### Pattern 3: With Default Value
|
|
370
|
+
|
|
371
|
+
```typescript
|
|
372
|
+
{
|
|
373
|
+
fields: {
|
|
374
|
+
status: {
|
|
375
|
+
source: 'status',
|
|
376
|
+
defaultValue: 'AVAILABLE' // Use if source is missing
|
|
377
|
+
}
|
|
378
|
+
}
|
|
379
|
+
}
|
|
380
|
+
```
|
|
381
|
+
|
|
382
|
+
### Pattern 4: Required Field
|
|
383
|
+
|
|
384
|
+
```typescript
|
|
385
|
+
{
|
|
386
|
+
fields: {
|
|
387
|
+
sku: {
|
|
388
|
+
source: 'productRef',
|
|
389
|
+
required: true // Validation: must be present
|
|
390
|
+
}
|
|
391
|
+
}
|
|
392
|
+
}
|
|
393
|
+
```
|
|
394
|
+
|
|
395
|
+
### Pattern 5: Nested Object Access
|
|
396
|
+
|
|
397
|
+
```typescript
|
|
398
|
+
{
|
|
399
|
+
fields: {
|
|
400
|
+
city: {
|
|
401
|
+
source: 'address.city' // Access nested property
|
|
402
|
+
}
|
|
403
|
+
}
|
|
404
|
+
}
|
|
405
|
+
```
|
|
406
|
+
|
|
407
|
+
---
|
|
408
|
+
|
|
409
|
+
## Next Steps
|
|
410
|
+
|
|
411
|
+
Now that you understand the foundations:
|
|
412
|
+
|
|
413
|
+
1. **Module 2: Quick Start** - Build your first mapping workflow
|
|
414
|
+
2. **Module 3: Schema Validation** - Validate mappings against Fluent Commerce schema
|
|
415
|
+
3. **Module 4: Use Cases** - Real-world mapping scenarios
|
|
416
|
+
4. **Module 5: Advanced Patterns** - Arrays, nested objects, conditional mapping
|
|
417
|
+
|
|
418
|
+
---
|
|
419
|
+
|
|
420
|
+
## Key Takeaways
|
|
421
|
+
|
|
422
|
+
✅ **Universal mapping** = One pattern for all data transformations
|
|
423
|
+
|
|
424
|
+
✅ **Source paths** = JavaScript object notation (not JSONPath/XPath)
|
|
425
|
+
|
|
426
|
+
✅ **Resolvers** = Transform values before mapping (built-in + custom)
|
|
427
|
+
|
|
428
|
+
✅ **Validation** = Automatic validation of required fields and types
|
|
429
|
+
|
|
430
|
+
✅ **External JSON** = Recommended for production configurations
|
|
431
|
+
|
|
432
|
+
✅ **Works with any format** = CSV, JSON, XML, Parquet, GraphQL
|
|
433
|
+
|
|
434
|
+
---
|
|
435
|
+
|
|
436
|
+
**Ready to build your first mapping?** → [Module 2: Quick Start](./mapping-02-quick-start.md)
|
|
437
|
+
|
|
438
|
+
|
|
439
|
+
|
|
440
|
+
|
|
441
|
+
|
|
442
|
+
|
|
443
|
+
|
|
444
|
+
|
|
445
|
+
|
|
446
|
+
|
|
447
|
+
|
|
448
|
+
|
|
449
|
+
|
|
450
|
+
|