@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.56
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/README.md +11 -0
- package/dist/cjs/clients/fluent-client.js +13 -6
- package/dist/cjs/utils/pagination-helpers.js +38 -2
- package/dist/cjs/versori/fluent-versori-client.js +11 -5
- package/dist/esm/clients/fluent-client.js +13 -6
- package/dist/esm/utils/pagination-helpers.js +38 -2
- package/dist/esm/versori/fluent-versori-client.js +11 -5
- package/dist/tsconfig.esm.tsbuildinfo +1 -1
- package/dist/tsconfig.tsbuildinfo +1 -1
- package/dist/tsconfig.types.tsbuildinfo +1 -1
- package/docs/00-START-HERE/EXPORT-VALIDATION.md +158 -158
- package/docs/00-START-HERE/cli-analyze-source-structure-guide.md +655 -655
- package/docs/00-START-HERE/cli-documentation-index.md +202 -202
- package/docs/00-START-HERE/cli-quick-reference.md +252 -252
- package/docs/00-START-HERE/decision-tree.md +552 -552
- package/docs/00-START-HERE/getting-started.md +1070 -1070
- package/docs/00-START-HERE/mapper-quick-decision-guide.md +235 -235
- package/docs/00-START-HERE/readme.md +237 -237
- package/docs/00-START-HERE/retailerid-configuration.md +404 -404
- package/docs/00-START-HERE/sdk-philosophy.md +794 -794
- package/docs/00-START-HERE/troubleshooting-quick-reference.md +1086 -1086
- package/docs/01-TEMPLATES/faq.md +686 -686
- package/docs/01-TEMPLATES/patterns/pattern-templates-guide.md +68 -68
- package/docs/01-TEMPLATES/patterns/patterns-csv-schema-validation-and-rejection-report.md +233 -233
- package/docs/01-TEMPLATES/patterns/patterns-custom-resolvers.md +407 -407
- package/docs/01-TEMPLATES/patterns/patterns-error-handling-retry.md +511 -511
- package/docs/01-TEMPLATES/patterns/patterns-field-mapping-universal.md +701 -701
- package/docs/01-TEMPLATES/patterns/patterns-large-file-splitting.md +1430 -1430
- package/docs/01-TEMPLATES/patterns/patterns-master-data-etl.md +2399 -2399
- package/docs/01-TEMPLATES/patterns/patterns-pagination-streaming.md +447 -447
- package/docs/01-TEMPLATES/patterns/patterns-state-duplicate-prevention.md +385 -385
- package/docs/01-TEMPLATES/readme.md +957 -957
- package/docs/01-TEMPLATES/standalone/standalone-asn-inbound-processing.md +1209 -1209
- package/docs/01-TEMPLATES/standalone/standalone-graphql-query-export.md +1140 -1140
- package/docs/01-TEMPLATES/standalone/standalone-graphql-to-parquet-partitioned-s3.md +432 -432
- package/docs/01-TEMPLATES/standalone/standalone-multi-channel-inventory-sync.md +1185 -1185
- package/docs/01-TEMPLATES/standalone/standalone-multi-source-aggregation.md +1462 -1462
- package/docs/01-TEMPLATES/standalone/standalone-s3-csv-batch-api.md +1390 -1390
- package/docs/01-TEMPLATES/standalone/standalone-s3-csv-inventory-to-batch.md +330 -330
- package/docs/01-TEMPLATES/standalone/standalone-scripts-guide.md +87 -87
- package/docs/01-TEMPLATES/standalone/standalone-sftp-xml-graphql.md +1444 -1444
- package/docs/01-TEMPLATES/standalone/standalone-webhook-payload-processing.md +688 -688
- package/docs/01-TEMPLATES/versori/business-examples/business-examples-dropship-order-routing.md +193 -193
- package/docs/01-TEMPLATES/versori/business-examples/business-examples-graphql-parquet-extraction.md +518 -518
- package/docs/01-TEMPLATES/versori/business-examples/business-examples-inter-location-transfers.md +2162 -2162
- package/docs/01-TEMPLATES/versori/business-examples/business-examples-pre-order-allocation.md +2226 -2226
- package/docs/01-TEMPLATES/versori/business-examples/business-scenarios-guide.md +87 -87
- package/docs/01-TEMPLATES/versori/patterns/versori-patterns-connection-validation-pattern.md +656 -656
- package/docs/01-TEMPLATES/versori/patterns/versori-patterns-dual-workflow-connector.md +835 -835
- package/docs/01-TEMPLATES/versori/patterns/versori-patterns-guide.md +108 -108
- package/docs/01-TEMPLATES/versori/patterns/versori-patterns-kv-state-management.md +1533 -1533
- package/docs/01-TEMPLATES/versori/patterns/versori-patterns-xml-response-patterns.md +1160 -1160
- package/docs/01-TEMPLATES/versori/versori-platform-guide.md +201 -201
- package/docs/01-TEMPLATES/versori/webhooks/template-webhook-asn-purchase-order.md +1906 -1906
- package/docs/01-TEMPLATES/versori/webhooks/template-webhook-dropship-routing.md +1074 -1074
- package/docs/01-TEMPLATES/versori/webhooks/template-webhook-flash-sale-reserve.md +1395 -1395
- package/docs/01-TEMPLATES/versori/webhooks/template-webhook-generic-xml-order.md +888 -888
- package/docs/01-TEMPLATES/versori/webhooks/template-webhook-payment-gateway-integration.md +2478 -2478
- package/docs/01-TEMPLATES/versori/webhooks/template-webhook-rma-returns-comprehensive.md +2240 -2240
- package/docs/01-TEMPLATES/versori/webhooks/template-webhook-xml-order-ingestion.md +2029 -2029
- package/docs/01-TEMPLATES/versori/webhooks/webhook-templates-guide.md +140 -140
- package/docs/01-TEMPLATES/versori/workflows/_examples/sample-data/inventory-mapping.json +20 -20
- package/docs/01-TEMPLATES/versori/workflows/_examples/sample-data/products_2025-01-22.csv +11 -11
- package/docs/01-TEMPLATES/versori/workflows/_examples/sample-data/sample-data-guide.md +34 -34
- package/docs/01-TEMPLATES/versori/workflows/_examples/workflow-examples-guide.md +36 -36
- package/docs/01-TEMPLATES/versori/workflows/extraction/extraction-modes-guide.md +1038 -1038
- package/docs/01-TEMPLATES/versori/workflows/extraction/extraction-workflows-guide.md +138 -138
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/graphql-extraction-guide.md +63 -63
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-fulfillments-to-sftp-csv.md +2062 -2062
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-fulfillments-to-sftp-xml.md +2294 -2294
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-inventory-positions-to-s3-csv.md +2461 -2461
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-inventory-positions-to-sftp-xml.md +2529 -2529
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-inventory-quantities-to-s3-csv.md +2464 -2464
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-inventory-quantities-to-s3-json.md +1959 -1959
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-orders-to-s3-csv.md +1953 -1953
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-orders-to-sftp-xml.md +2541 -2541
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-products-to-s3-json.md +2384 -2384
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-products-to-sftp-xml.md +2445 -2445
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-virtual-positions-to-s3-csv.md +2355 -2355
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-virtual-positions-to-s3-json.md +2042 -2042
- package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-virtual-positions-to-sftp-xml.md +2726 -2726
- package/docs/01-TEMPLATES/versori/workflows/ingestion/batch-api/batch-api-guide.md +206 -206
- package/docs/01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-cycle-count-reconciliation.md +2030 -2030
- package/docs/01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-multi-channel-inventory-sync.md +1882 -1882
- package/docs/01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-s3-csv-inventory-batch.md +2827 -2827
- package/docs/01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-s3-json-inventory-batch.md +1952 -1952
- package/docs/01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-s3-xml-inventory-batch.md +3289 -3289
- package/docs/01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-sftp-csv-inventory-batch.md +3064 -3064
- package/docs/01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-sftp-json-inventory-batch.md +3238 -3238
- package/docs/01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-sftp-xml-inventory-batch.md +2977 -2977
- package/docs/01-TEMPLATES/versori/workflows/ingestion/event-api/event-api-guide.md +321 -321
- package/docs/01-TEMPLATES/versori/workflows/ingestion/event-api/template-ingestion-payload-json-order-cancel-event.md +959 -959
- package/docs/01-TEMPLATES/versori/workflows/ingestion/event-api/template-ingestion-payload-xml-order-cancel-event.md +1170 -1170
- package/docs/01-TEMPLATES/versori/workflows/ingestion/event-api/template-ingestion-s3-csv-product-event.md +2312 -2312
- package/docs/01-TEMPLATES/versori/workflows/ingestion/event-api/template-ingestion-s3-json-product-event.md +2999 -2999
- package/docs/01-TEMPLATES/versori/workflows/ingestion/event-api/template-ingestion-s3-parquet-product-event.md +2836 -2836
- package/docs/01-TEMPLATES/versori/workflows/ingestion/event-api/template-ingestion-s3-xml-product-event.md +2395 -2395
- package/docs/01-TEMPLATES/versori/workflows/ingestion/event-api/template-ingestion-sftp-csv-product-event.md +2295 -2295
- package/docs/01-TEMPLATES/versori/workflows/ingestion/event-api/template-ingestion-sftp-json-product-event.md +2602 -2602
- package/docs/01-TEMPLATES/versori/workflows/ingestion/event-api/template-ingestion-sftp-parquet-product-event.md +2589 -2589
- package/docs/01-TEMPLATES/versori/workflows/ingestion/event-api/template-ingestion-sftp-xml-product-event.md +3578 -3578
- package/docs/01-TEMPLATES/versori/workflows/ingestion/graphql-mutations/graphql-mutations-guide.md +93 -93
- package/docs/01-TEMPLATES/versori/workflows/ingestion/graphql-mutations/template-ingestion-payload-json-order-update-graphql.md +1260 -1260
- package/docs/01-TEMPLATES/versori/workflows/ingestion/graphql-mutations/template-ingestion-payload-xml-order-update-graphql.md +1472 -1472
- package/docs/01-TEMPLATES/versori/workflows/ingestion/graphql-mutations/template-ingestion-s3-csv-control-graphql.md +2417 -2417
- package/docs/01-TEMPLATES/versori/workflows/ingestion/graphql-mutations/template-ingestion-s3-csv-location-graphql.md +2811 -2811
- package/docs/01-TEMPLATES/versori/workflows/ingestion/graphql-mutations/template-ingestion-s3-csv-price-graphql.md +2619 -2619
- package/docs/01-TEMPLATES/versori/workflows/ingestion/graphql-mutations/template-ingestion-s3-json-location-graphql.md +2807 -2807
- package/docs/01-TEMPLATES/versori/workflows/ingestion/graphql-mutations/template-ingestion-s3-xml-location-graphql.md +2373 -2373
- package/docs/01-TEMPLATES/versori/workflows/ingestion/graphql-mutations/template-ingestion-sftp-csv-control-graphql.md +2740 -2740
- package/docs/01-TEMPLATES/versori/workflows/ingestion/graphql-mutations/template-ingestion-sftp-csv-location-graphql.md +2760 -2760
- package/docs/01-TEMPLATES/versori/workflows/ingestion/graphql-mutations/template-ingestion-sftp-json-location-graphql.md +1710 -1710
- package/docs/01-TEMPLATES/versori/workflows/ingestion/ingestion-workflows-guide.md +136 -136
- package/docs/01-TEMPLATES/versori/workflows/rubix-webhooks/rubix-webhooks-guide.md +520 -520
- package/docs/01-TEMPLATES/versori/workflows/rubix-webhooks/template-webhook-rubix-fulfilment-to-sftp-xml-inline.md +1418 -1418
- package/docs/01-TEMPLATES/versori/workflows/rubix-webhooks/template-webhook-rubix-fulfilment-to-sftp-xml-universal-mapper.md +1785 -1785
- package/docs/01-TEMPLATES/versori/workflows/rubix-webhooks/template-webhook-rubix-order-attribute-update.md +824 -824
- package/docs/01-TEMPLATES/versori/workflows/workflows-overview-guide.md +646 -646
- package/docs/02-CORE-GUIDES/advanced-services/advanced-services-batch-archival.md +724 -724
- package/docs/02-CORE-GUIDES/advanced-services/advanced-services-job-tracker.md +627 -627
- package/docs/02-CORE-GUIDES/advanced-services/advanced-services-partial-batch-recovery.md +561 -561
- package/docs/02-CORE-GUIDES/advanced-services/advanced-services-quick-reference.md +367 -367
- package/docs/02-CORE-GUIDES/advanced-services/advanced-services-readme.md +407 -407
- package/docs/02-CORE-GUIDES/advanced-services/readme.md +49 -49
- package/docs/02-CORE-GUIDES/api-reference/api-reference-quick-reference.md +548 -548
- package/docs/02-CORE-GUIDES/api-reference/event-api-input-output-reference.md +702 -1171
- package/docs/02-CORE-GUIDES/api-reference/examples/client-initialization.ts +286 -286
- package/docs/02-CORE-GUIDES/api-reference/graphql-error-classification.md +337 -337
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-01-client-api.md +399 -520
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-03-authentication.md +199 -199
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-04-graphql-mapping.md +925 -925
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-05-services.md +1198 -1198
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-06-data-sources.md +1083 -1083
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-07-parsers.md +1097 -1097
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-08-pagination.md +513 -513
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-08-types.md +545 -597
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-09-error-handling.md +527 -527
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-09-webhook-validation.md +514 -514
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-10-extraction.md +557 -557
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-10-utilities.md +412 -412
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-11-cli-tools.md +423 -423
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-11-error-handling.md +716 -716
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-12-analyze-source-structure.md +518 -518
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-12-partial-responses.md +212 -212
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-12-testing.md +300 -300
- package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-13-resolver-builder.md +322 -322
- package/docs/02-CORE-GUIDES/api-reference/readme.md +279 -279
- package/docs/02-CORE-GUIDES/auto-pagination/auto-pagination-quick-reference.md +351 -351
- package/docs/02-CORE-GUIDES/auto-pagination/auto-pagination-readme.md +277 -277
- package/docs/02-CORE-GUIDES/auto-pagination/examples/auto-pagination-readme.md +178 -178
- package/docs/02-CORE-GUIDES/auto-pagination/examples/common-patterns.ts +351 -351
- package/docs/02-CORE-GUIDES/auto-pagination/examples/paginate-products.ts +384 -384
- package/docs/02-CORE-GUIDES/auto-pagination/examples/paginate-virtual-positions.ts +308 -308
- package/docs/02-CORE-GUIDES/auto-pagination/modules/auto-pagination-01-foundations.md +470 -470
- package/docs/02-CORE-GUIDES/auto-pagination/modules/auto-pagination-02-quick-start.md +713 -713
- package/docs/02-CORE-GUIDES/auto-pagination/modules/auto-pagination-03-configuration.md +754 -754
- package/docs/02-CORE-GUIDES/auto-pagination/modules/auto-pagination-04-advanced-patterns.md +732 -732
- package/docs/02-CORE-GUIDES/auto-pagination/modules/auto-pagination-05-sdk-integration.md +847 -847
- package/docs/02-CORE-GUIDES/auto-pagination/modules/auto-pagination-06-troubleshooting.md +359 -359
- package/docs/02-CORE-GUIDES/auto-pagination/modules/auto-pagination-07-api-reference.md +462 -462
- package/docs/02-CORE-GUIDES/auto-pagination/readme.md +54 -54
- package/docs/02-CORE-GUIDES/data-sources/data-sources-file-operations-error-handling.md +1487 -1487
- package/docs/02-CORE-GUIDES/data-sources/data-sources-quick-reference.md +836 -836
- package/docs/02-CORE-GUIDES/data-sources/data-sources-readme.md +276 -276
- package/docs/02-CORE-GUIDES/data-sources/data-sources-sftp-credential-access-security.md +553 -553
- package/docs/02-CORE-GUIDES/data-sources/examples/common-patterns.ts +409 -409
- package/docs/02-CORE-GUIDES/data-sources/examples/data-sources-readme.md +178 -178
- package/docs/02-CORE-GUIDES/data-sources/examples/s3-operations.ts +308 -308
- package/docs/02-CORE-GUIDES/data-sources/examples/sftp-operations.ts +371 -371
- package/docs/02-CORE-GUIDES/data-sources/modules/data-sources-01-foundations.md +735 -735
- package/docs/02-CORE-GUIDES/data-sources/modules/data-sources-02-s3-operations.md +1302 -1302
- package/docs/02-CORE-GUIDES/data-sources/modules/data-sources-03-sftp-operations.md +1379 -1379
- package/docs/02-CORE-GUIDES/data-sources/modules/data-sources-04-file-patterns.md +941 -941
- package/docs/02-CORE-GUIDES/data-sources/modules/data-sources-05-advanced-topics.md +813 -813
- package/docs/02-CORE-GUIDES/data-sources/modules/data-sources-06-integration-patterns.md +486 -486
- package/docs/02-CORE-GUIDES/data-sources/modules/data-sources-07-troubleshooting.md +387 -387
- package/docs/02-CORE-GUIDES/data-sources/modules/data-sources-08-api-reference.md +417 -417
- package/docs/02-CORE-GUIDES/data-sources/readme.md +77 -77
- package/docs/02-CORE-GUIDES/error-handling-guide.md +936 -936
- package/docs/02-CORE-GUIDES/extraction/examples/02-core-guides-extraction-readme.md +116 -116
- package/docs/02-CORE-GUIDES/extraction/examples/common-patterns.ts +428 -428
- package/docs/02-CORE-GUIDES/extraction/examples/extract-inventory-basic.ts +187 -187
- package/docs/02-CORE-GUIDES/extraction/extraction-quick-reference.md +596 -596
- package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-01-foundations.md +514 -514
- package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-02-basic-extraction.md +823 -823
- package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-03-parquet-processing.md +507 -507
- package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-04-data-enrichment.md +546 -546
- package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-05-transformation.md +494 -494
- package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-06-export-formats.md +458 -458
- package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-06-performance.md +138 -138
- package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-07-api-reference.md +148 -148
- package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-07-optimization.md +692 -692
- package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-08-extraction-orchestrator.md +1008 -1008
- package/docs/02-CORE-GUIDES/extraction/readme.md +151 -151
- package/docs/02-CORE-GUIDES/ingestion/examples/_simple-kv-store.ts +40 -40
- package/docs/02-CORE-GUIDES/ingestion/examples/error-recovery.ts +728 -728
- package/docs/02-CORE-GUIDES/ingestion/examples/event-driven.ts +501 -501
- package/docs/02-CORE-GUIDES/ingestion/examples/local-file-ingestion.ts +88 -88
- package/docs/02-CORE-GUIDES/ingestion/examples/parquet-ingestion.ts +117 -117
- package/docs/02-CORE-GUIDES/ingestion/examples/performance-optimized.ts +647 -647
- package/docs/02-CORE-GUIDES/ingestion/examples/s3-csv-ingestion.ts +169 -169
- package/docs/02-CORE-GUIDES/ingestion/examples/sftp-csv-ingestion.ts +134 -134
- package/docs/02-CORE-GUIDES/ingestion/ingestion-quick-reference.md +546 -546
- package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-01-introduction.md +626 -626
- package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-02-quick-start.md +658 -658
- package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-03-data-sources.md +1052 -1052
- package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-04-field-mapping.md +763 -763
- package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-05-advanced-parsers.md +676 -676
- package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-06-batch-api.md +1295 -1295
- package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-07-api-reference.md +138 -138
- package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-07-state-management.md +1037 -1037
- package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-08-performance-optimization.md +1349 -1349
- package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-09-best-practices.md +1893 -1893
- package/docs/02-CORE-GUIDES/ingestion/readme.md +160 -160
- package/docs/02-CORE-GUIDES/logging-guide.md +585 -585
- package/docs/02-CORE-GUIDES/mapping/error-handling-patterns.md +401 -401
- package/docs/02-CORE-GUIDES/mapping/examples/02-core-guides-mapping-readme.md +128 -128
- package/docs/02-CORE-GUIDES/mapping/examples/common-patterns.ts +273 -273
- package/docs/02-CORE-GUIDES/mapping/examples/csv-location-ingestion.json +36 -36
- package/docs/02-CORE-GUIDES/mapping/examples/csv-mapping.ts +242 -242
- package/docs/02-CORE-GUIDES/mapping/examples/graphql-to-parquet-extraction.json +36 -36
- package/docs/02-CORE-GUIDES/mapping/examples/json-mapping.ts +213 -213
- package/docs/02-CORE-GUIDES/mapping/examples/json-product-to-mutation.json +48 -48
- package/docs/02-CORE-GUIDES/mapping/examples/xml-mapping.ts +291 -291
- package/docs/02-CORE-GUIDES/mapping/examples/xml-order-to-mutation.json +45 -45
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/graphql-mutation-mapping-quick-reference.md +463 -463
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/graphql-mutation-mapping-readme.md +227 -227
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-01-introduction.md +222 -222
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-02-quick-start.md +351 -351
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-03-schema-validation.md +569 -569
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-04-mapping-patterns.md +471 -471
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-05-configuration-reference.md +611 -611
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-06-advanced-xpath.md +148 -148
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-06-path-syntax.md +464 -464
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-07-api-reference.md +94 -94
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-07-array-handling.md +307 -307
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-08-custom-resolvers.md +544 -544
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-09-advanced-patterns.md +427 -427
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-10-hooks-and-variables.md +336 -336
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-11-error-handling.md +488 -488
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-12-arguments-vs-nodes.md +383 -383
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/modules/graphql-mutation-mapping-13-best-practices.md +477 -477
- package/docs/02-CORE-GUIDES/mapping/graphql-mutation-mapping/readme.md +62 -62
- package/docs/02-CORE-GUIDES/mapping/mapping-format-decision-tree.md +480 -480
- package/docs/02-CORE-GUIDES/mapping/mapping-graphql-alias-batching-guide.md +820 -820
- package/docs/02-CORE-GUIDES/mapping/mapping-javascript-objects.md +2369 -2369
- package/docs/02-CORE-GUIDES/mapping/mapping-mapper-comparison-guide.md +682 -682
- package/docs/02-CORE-GUIDES/mapping/modules/02-core-guides-mapping-07-api-reference.md +1327 -1327
- package/docs/02-CORE-GUIDES/mapping/modules/02-core-guides-mapping-08-error-handling.md +1142 -1142
- package/docs/02-CORE-GUIDES/mapping/modules/mapping-04-use-cases.md +891 -891
- package/docs/02-CORE-GUIDES/mapping/modules/mapping-06-helpers-resolvers.md +1126 -1126
- package/docs/02-CORE-GUIDES/mapping/modules/mapping-06-sdk-resolvers.md +199 -199
- package/docs/02-CORE-GUIDES/mapping/modules/mapping-07-api-reference.md +1319 -1319
- package/docs/02-CORE-GUIDES/mapping/readme.md +178 -178
- package/docs/02-CORE-GUIDES/mapping/resolver-registration.md +410 -410
- package/docs/02-CORE-GUIDES/mapping/resolvers/examples/common-patterns.ts +226 -226
- package/docs/02-CORE-GUIDES/mapping/resolvers/examples/custom-resolvers.ts +227 -227
- package/docs/02-CORE-GUIDES/mapping/resolvers/examples/sdk-resolvers-usage.ts +203 -203
- package/docs/02-CORE-GUIDES/mapping/resolvers/mapping-resolvers-readme.md +274 -274
- package/docs/02-CORE-GUIDES/mapping/resolvers/mapping-resolvers-resolver-api-reference.md +679 -679
- package/docs/02-CORE-GUIDES/mapping/resolvers/mapping-resolvers-resolver-cookbook.md +826 -826
- package/docs/02-CORE-GUIDES/mapping/resolvers/mapping-resolvers-resolver-guide.md +1330 -1330
- package/docs/02-CORE-GUIDES/mapping/resolvers/mapping-resolvers-resolver-helpers-reference.md +1437 -1437
- package/docs/02-CORE-GUIDES/mapping/resolvers/mapping-resolvers-resolver-parameters-reference.md +553 -553
- package/docs/02-CORE-GUIDES/mapping/resolvers/mapping-resolvers-resolver-troubleshooting.md +854 -854
- package/docs/02-CORE-GUIDES/mapping/resolvers/readme.md +75 -75
- package/docs/02-CORE-GUIDES/parsers/examples/02-core-guides-parsers-readme.md +161 -161
- package/docs/02-CORE-GUIDES/parsers/examples/csv-parser-examples.ts +110 -110
- package/docs/02-CORE-GUIDES/parsers/examples/json-parser-examples.ts +33 -33
- package/docs/02-CORE-GUIDES/parsers/examples/parquet-parser-examples.ts +47 -47
- package/docs/02-CORE-GUIDES/parsers/examples/xml-parser-examples.ts +38 -38
- package/docs/02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-01-foundations.md +355 -355
- package/docs/02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-02-csv-parser.md +772 -772
- package/docs/02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-03-json-parser.md +789 -789
- package/docs/02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-04-xml-parser.md +857 -857
- package/docs/02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-05-parquet-parser.md +603 -603
- package/docs/02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-06-integration-patterns.md +702 -702
- package/docs/02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-06-streaming.md +121 -121
- package/docs/02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-07-api-reference.md +89 -89
- package/docs/02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-07-troubleshooting.md +727 -727
- package/docs/02-CORE-GUIDES/parsers/parsers-quick-reference.md +482 -482
- package/docs/02-CORE-GUIDES/parsers/parsers-readme.md +258 -258
- package/docs/02-CORE-GUIDES/parsers/readme.md +65 -65
- package/docs/02-CORE-GUIDES/readme.md +194 -194
- package/docs/02-CORE-GUIDES/webhook-validation/examples/basic-validation.ts +108 -108
- package/docs/02-CORE-GUIDES/webhook-validation/examples/common-patterns.ts +316 -316
- package/docs/02-CORE-GUIDES/webhook-validation/examples/webhook-validation-readme.md +61 -61
- package/docs/02-CORE-GUIDES/webhook-validation/modules/webhook-validation-01-foundations.md +440 -440
- package/docs/02-CORE-GUIDES/webhook-validation/modules/webhook-validation-02-quick-start.md +525 -525
- package/docs/02-CORE-GUIDES/webhook-validation/modules/webhook-validation-03-versori-integration.md +741 -741
- package/docs/02-CORE-GUIDES/webhook-validation/modules/webhook-validation-04-platform-integration.md +629 -629
- package/docs/02-CORE-GUIDES/webhook-validation/modules/webhook-validation-05-configuration.md +535 -535
- package/docs/02-CORE-GUIDES/webhook-validation/modules/webhook-validation-06-error-handling.md +611 -611
- package/docs/02-CORE-GUIDES/webhook-validation/modules/webhook-validation-06-troubleshooting.md +124 -124
- package/docs/02-CORE-GUIDES/webhook-validation/modules/webhook-validation-07-api-reference.md +511 -511
- package/docs/02-CORE-GUIDES/webhook-validation/modules/webhook-validation-08-rubix-webhooks.md +590 -590
- package/docs/02-CORE-GUIDES/webhook-validation/modules/webhook-validation-09-rubix-event-vs-http-call.md +432 -432
- package/docs/02-CORE-GUIDES/webhook-validation/readme.md +239 -239
- package/docs/02-CORE-GUIDES/webhook-validation/webhook-validation-quick-reference.md +392 -392
- package/docs/03-PATTERN-GUIDES/connector-scenarios/connector-scenarios-quick-reference.md +498 -498
- package/docs/03-PATTERN-GUIDES/connector-scenarios/connector-scenarios-readme.md +313 -313
- package/docs/03-PATTERN-GUIDES/connector-scenarios/examples/common-patterns.ts +612 -612
- package/docs/03-PATTERN-GUIDES/connector-scenarios/examples/connector-scenarios-readme.md +253 -253
- package/docs/03-PATTERN-GUIDES/connector-scenarios/modules/connector-scenarios-01-foundations.md +452 -452
- package/docs/03-PATTERN-GUIDES/connector-scenarios/modules/connector-scenarios-02-simple-scenarios.md +681 -681
- package/docs/03-PATTERN-GUIDES/connector-scenarios/modules/connector-scenarios-03-intermediate-scenarios.md +637 -637
- package/docs/03-PATTERN-GUIDES/connector-scenarios/modules/connector-scenarios-04-advanced-scenarios.md +650 -650
- package/docs/03-PATTERN-GUIDES/connector-scenarios/modules/connector-scenarios-05-bidirectional-sync.md +233 -233
- package/docs/03-PATTERN-GUIDES/connector-scenarios/modules/connector-scenarios-06-production-patterns.md +442 -442
- package/docs/03-PATTERN-GUIDES/connector-scenarios/modules/connector-scenarios-07-reference.md +445 -445
- package/docs/03-PATTERN-GUIDES/connector-scenarios/readme.md +31 -31
- package/docs/03-PATTERN-GUIDES/enterprise-integration-patterns.md +1528 -1528
- package/docs/03-PATTERN-GUIDES/error-handling/comprehensive-error-handling-guide.md +1437 -1437
- package/docs/03-PATTERN-GUIDES/error-handling/error-handling-quick-reference.md +390 -390
- package/docs/03-PATTERN-GUIDES/error-handling/examples/common-patterns.ts +438 -438
- package/docs/03-PATTERN-GUIDES/error-handling/modules/error-handling-01-foundations.md +362 -362
- package/docs/03-PATTERN-GUIDES/error-handling/modules/error-handling-02-error-types.md +850 -850
- package/docs/03-PATTERN-GUIDES/error-handling/modules/error-handling-03-utf8-handling.md +456 -456
- package/docs/03-PATTERN-GUIDES/error-handling/modules/error-handling-04-error-scenarios.md +658 -658
- package/docs/03-PATTERN-GUIDES/error-handling/modules/error-handling-05-calling-patterns.md +671 -671
- package/docs/03-PATTERN-GUIDES/error-handling/modules/error-handling-06-retry-strategies.md +1034 -1034
- package/docs/03-PATTERN-GUIDES/error-handling/modules/error-handling-07-monitoring.md +653 -653
- package/docs/03-PATTERN-GUIDES/error-handling/modules/error-handling-08-api-reference.md +847 -847
- package/docs/03-PATTERN-GUIDES/error-handling/readme.md +36 -36
- package/docs/03-PATTERN-GUIDES/examples/__tests__/readme.md +40 -40
- package/docs/03-PATTERN-GUIDES/examples/__tests__/resolver-examples.test.js +282 -282
- package/docs/03-PATTERN-GUIDES/examples/test-data/03-pattern-guides-readme.md +110 -110
- package/docs/03-PATTERN-GUIDES/examples/test-data/canonical-inventory.json +123 -123
- package/docs/03-PATTERN-GUIDES/examples/test-data/canonical-order.json +171 -171
- package/docs/03-PATTERN-GUIDES/examples/test-data/readme.md +28 -28
- package/docs/03-PATTERN-GUIDES/extraction/extraction-readme.md +15 -15
- package/docs/03-PATTERN-GUIDES/extraction/readme.md +25 -25
- package/docs/03-PATTERN-GUIDES/file-operations/examples/common-patterns.ts +407 -407
- package/docs/03-PATTERN-GUIDES/file-operations/examples/file-operations-readme.md +142 -142
- package/docs/03-PATTERN-GUIDES/file-operations/file-operations-quick-reference.md +462 -462
- package/docs/03-PATTERN-GUIDES/file-operations/file-operations-readme.md +379 -379
- package/docs/03-PATTERN-GUIDES/file-operations/modules/file-operations-01-foundations.md +430 -430
- package/docs/03-PATTERN-GUIDES/file-operations/modules/file-operations-02-quick-start.md +484 -484
- package/docs/03-PATTERN-GUIDES/file-operations/modules/file-operations-03-s3-operations.md +507 -507
- package/docs/03-PATTERN-GUIDES/file-operations/modules/file-operations-04-sftp-operations.md +963 -963
- package/docs/03-PATTERN-GUIDES/file-operations/modules/file-operations-05-streaming-performance.md +503 -503
- package/docs/03-PATTERN-GUIDES/file-operations/modules/file-operations-06-archive-patterns.md +386 -386
- package/docs/03-PATTERN-GUIDES/file-operations/modules/file-operations-06-error-handling.md +117 -117
- package/docs/03-PATTERN-GUIDES/file-operations/modules/file-operations-07-api-reference.md +78 -78
- package/docs/03-PATTERN-GUIDES/file-operations/modules/file-operations-07-testing-troubleshooting.md +567 -567
- package/docs/03-PATTERN-GUIDES/file-operations/modules/file-operations-08-api-reference.md +1055 -1055
- package/docs/03-PATTERN-GUIDES/file-operations/readme.md +32 -32
- package/docs/03-PATTERN-GUIDES/ingestion/ingestion-readme.md +15 -15
- package/docs/03-PATTERN-GUIDES/ingestion/readme.md +25 -25
- package/docs/03-PATTERN-GUIDES/integration-patterns/examples/batch-processing.ts +130 -130
- package/docs/03-PATTERN-GUIDES/integration-patterns/examples/common-patterns.ts +360 -360
- package/docs/03-PATTERN-GUIDES/integration-patterns/examples/delta-sync.ts +130 -130
- package/docs/03-PATTERN-GUIDES/integration-patterns/examples/integration-patterns-readme.md +100 -100
- package/docs/03-PATTERN-GUIDES/integration-patterns/examples/real-time-webhook.ts +398 -398
- package/docs/03-PATTERN-GUIDES/integration-patterns/integration-patterns-quick-reference.md +962 -962
- package/docs/03-PATTERN-GUIDES/integration-patterns/integration-patterns-readme.md +134 -134
- package/docs/03-PATTERN-GUIDES/integration-patterns/modules/integration-patterns-01-real-time-processing.md +991 -991
- package/docs/03-PATTERN-GUIDES/integration-patterns/modules/integration-patterns-02-batch-processing.md +1547 -1547
- package/docs/03-PATTERN-GUIDES/integration-patterns/modules/integration-patterns-03-delta-sync.md +1108 -1108
- package/docs/03-PATTERN-GUIDES/integration-patterns/modules/integration-patterns-04-webhook-patterns.md +1181 -1181
- package/docs/03-PATTERN-GUIDES/integration-patterns/modules/integration-patterns-05-error-handling.md +1061 -1061
- package/docs/03-PATTERN-GUIDES/integration-patterns/modules/integration-patterns-06-advanced-integration-services.md +1547 -1547
- package/docs/03-PATTERN-GUIDES/integration-patterns/modules/integration-patterns-06-performance.md +109 -109
- package/docs/03-PATTERN-GUIDES/integration-patterns/modules/integration-patterns-07-api-reference.md +34 -34
- package/docs/03-PATTERN-GUIDES/integration-patterns/readme.md +30 -30
- package/docs/03-PATTERN-GUIDES/logging-minimal-mode.md +128 -128
- package/docs/03-PATTERN-GUIDES/multiple-connections/examples/common-patterns.ts +380 -380
- package/docs/03-PATTERN-GUIDES/multiple-connections/examples/multiple-connections-readme.md +139 -139
- package/docs/03-PATTERN-GUIDES/multiple-connections/examples/parallel-root-connections.ts +149 -149
- package/docs/03-PATTERN-GUIDES/multiple-connections/examples/real-world-scenarios.ts +405 -405
- package/docs/03-PATTERN-GUIDES/multiple-connections/modules/multiple-connections-01-foundations.md +378 -378
- package/docs/03-PATTERN-GUIDES/multiple-connections/modules/multiple-connections-02-quick-start.md +566 -566
- package/docs/03-PATTERN-GUIDES/multiple-connections/modules/multiple-connections-03-targeting-connections.md +659 -659
- package/docs/03-PATTERN-GUIDES/multiple-connections/modules/multiple-connections-04-parallel-queries.md +656 -656
- package/docs/03-PATTERN-GUIDES/multiple-connections/modules/multiple-connections-05-best-practices.md +624 -624
- package/docs/03-PATTERN-GUIDES/multiple-connections/modules/multiple-connections-06-api-reference.md +824 -824
- package/docs/03-PATTERN-GUIDES/multiple-connections/modules/multiple-connections-06-versori.md +119 -119
- package/docs/03-PATTERN-GUIDES/multiple-connections/modules/multiple-connections-07-api-reference.md +87 -87
- package/docs/03-PATTERN-GUIDES/multiple-connections/multiple-connections-quick-reference.md +353 -353
- package/docs/03-PATTERN-GUIDES/multiple-connections/multiple-connections-readme.md +270 -270
- package/docs/03-PATTERN-GUIDES/multiple-connections/readme.md +30 -30
- package/docs/03-PATTERN-GUIDES/pagination/pagination-readme.md +14 -14
- package/docs/03-PATTERN-GUIDES/pagination/readme.md +24 -24
- package/docs/03-PATTERN-GUIDES/parquet/examples/common-patterns.ts +180 -180
- package/docs/03-PATTERN-GUIDES/parquet/examples/read-parquet.ts +48 -48
- package/docs/03-PATTERN-GUIDES/parquet/examples/write-parquet.ts +65 -65
- package/docs/03-PATTERN-GUIDES/parquet/modules/03-pattern-guides-parquet-01-introduction.md +393 -393
- package/docs/03-PATTERN-GUIDES/parquet/modules/03-pattern-guides-parquet-02-quick-start.md +572 -572
- package/docs/03-PATTERN-GUIDES/parquet/modules/03-pattern-guides-parquet-03-reading-parquet.md +525 -525
- package/docs/03-PATTERN-GUIDES/parquet/modules/03-pattern-guides-parquet-04-writing-parquet.md +554 -554
- package/docs/03-PATTERN-GUIDES/parquet/modules/03-pattern-guides-parquet-05-graphql-extraction.md +405 -405
- package/docs/03-PATTERN-GUIDES/parquet/modules/03-pattern-guides-parquet-06-performance.md +104 -104
- package/docs/03-PATTERN-GUIDES/parquet/modules/03-pattern-guides-parquet-06-s3-integration.md +511 -511
- package/docs/03-PATTERN-GUIDES/parquet/modules/03-pattern-guides-parquet-07-api-reference.md +90 -90
- package/docs/03-PATTERN-GUIDES/parquet/modules/03-pattern-guides-parquet-07-performance-optimization.md +525 -525
- package/docs/03-PATTERN-GUIDES/parquet/modules/03-pattern-guides-parquet-08-best-practices.md +712 -712
- package/docs/03-PATTERN-GUIDES/parquet/parquet-quick-reference.md +683 -683
- package/docs/03-PATTERN-GUIDES/parquet/parquet-readme.md +248 -248
- package/docs/03-PATTERN-GUIDES/parquet/readme.md +32 -32
- package/docs/03-PATTERN-GUIDES/parsers/parsers-readme.md +12 -12
- package/docs/03-PATTERN-GUIDES/parsers/readme.md +24 -24
- package/docs/03-PATTERN-GUIDES/readme.md +159 -159
- package/docs/03-PATTERN-GUIDES/webhooks/readme.md +24 -24
- package/docs/03-PATTERN-GUIDES/webhooks/webhooks-readme.md +8 -8
- package/docs/04-REFERENCE/architecture/architecture-01-overview.md +427 -427
- package/docs/04-REFERENCE/architecture/architecture-02-client-architecture.md +424 -424
- package/docs/04-REFERENCE/architecture/architecture-03-data-flow.md +690 -690
- package/docs/04-REFERENCE/architecture/architecture-04-service-layer.md +834 -834
- package/docs/04-REFERENCE/architecture/architecture-05-integration-architecture.md +655 -655
- package/docs/04-REFERENCE/architecture/architecture-06-state-management.md +653 -653
- package/docs/04-REFERENCE/architecture/architecture-adding-new-data-sources.md +686 -686
- package/docs/04-REFERENCE/architecture/readme.md +279 -279
- package/docs/04-REFERENCE/platforms/deno/readme.md +117 -117
- package/docs/04-REFERENCE/platforms/nodejs/readme.md +146 -146
- package/docs/04-REFERENCE/platforms/readme.md +135 -135
- package/docs/04-REFERENCE/platforms/versori/modules/platforms-versori-01-introduction.md +398 -398
- package/docs/04-REFERENCE/platforms/versori/modules/platforms-versori-02-quick-start.md +560 -560
- package/docs/04-REFERENCE/platforms/versori/modules/platforms-versori-03-authentication.md +757 -757
- package/docs/04-REFERENCE/platforms/versori/modules/platforms-versori-04-workflows.md +2476 -2476
- package/docs/04-REFERENCE/platforms/versori/modules/platforms-versori-05-connections.md +1167 -1167
- package/docs/04-REFERENCE/platforms/versori/modules/platforms-versori-06-kv-storage.md +990 -990
- package/docs/04-REFERENCE/platforms/versori/modules/platforms-versori-06-state-management.md +121 -121
- package/docs/04-REFERENCE/platforms/versori/modules/platforms-versori-07-api-reference.md +68 -68
- package/docs/04-REFERENCE/platforms/versori/modules/platforms-versori-07-deployment.md +731 -731
- package/docs/04-REFERENCE/platforms/versori/modules/platforms-versori-08-best-practices.md +1111 -1111
- package/docs/04-REFERENCE/platforms/versori/modules/platforms-versori-09-signature-reference.md +766 -766
- package/docs/04-REFERENCE/platforms/versori/platforms-versori-readme.md +299 -299
- package/docs/04-REFERENCE/platforms/versori/platforms-versori-s3-sftp-configuration-guide.md +1425 -1425
- package/docs/04-REFERENCE/platforms/versori/platforms-versori-webhook-api-key-security.md +816 -816
- package/docs/04-REFERENCE/platforms/versori/platforms-versori-webhook-connection-security.md +681 -681
- package/docs/04-REFERENCE/platforms/versori/platforms-versori-workflow-task-types.md +708 -708
- package/docs/04-REFERENCE/platforms/versori/readme.md +108 -108
- package/docs/04-REFERENCE/readme.md +148 -148
- package/docs/04-REFERENCE/resolver-signature/examples/advanced-resolvers.ts +482 -482
- package/docs/04-REFERENCE/resolver-signature/examples/async-resolvers.ts +496 -496
- package/docs/04-REFERENCE/resolver-signature/examples/basic-resolvers.ts +343 -343
- package/docs/04-REFERENCE/resolver-signature/examples/resolver-signature-readme.md +188 -188
- package/docs/04-REFERENCE/resolver-signature/examples/testing-resolvers.ts +463 -463
- package/docs/04-REFERENCE/resolver-signature/modules/resolver-signature-01-foundations.md +286 -286
- package/docs/04-REFERENCE/resolver-signature/modules/resolver-signature-02-parameter-reference.md +643 -643
- package/docs/04-REFERENCE/resolver-signature/modules/resolver-signature-03-basic-examples.md +521 -521
- package/docs/04-REFERENCE/resolver-signature/modules/resolver-signature-04-advanced-patterns.md +739 -739
- package/docs/04-REFERENCE/resolver-signature/modules/resolver-signature-05-sdk-resolvers.md +531 -531
- package/docs/04-REFERENCE/resolver-signature/modules/resolver-signature-06-migration-guide.md +650 -650
- package/docs/04-REFERENCE/resolver-signature/modules/resolver-signature-06-testing.md +125 -125
- package/docs/04-REFERENCE/resolver-signature/modules/resolver-signature-07-api-reference.md +794 -794
- package/docs/04-REFERENCE/resolver-signature/readme.md +64 -64
- package/docs/04-REFERENCE/resolver-signature/resolver-signature-quick-reference.md +270 -270
- package/docs/04-REFERENCE/resolver-signature/resolver-signature-readme.md +351 -351
- package/docs/04-REFERENCE/schema/fluent-commerce-schema.json +764 -764
- package/docs/04-REFERENCE/schema/readme.md +141 -141
- package/docs/04-REFERENCE/testing/examples/04-reference-testing-readme.md +158 -158
- package/docs/04-REFERENCE/testing/examples/fluent-testing.ts +62 -62
- package/docs/04-REFERENCE/testing/examples/health-check.ts +155 -155
- package/docs/04-REFERENCE/testing/examples/integration-test.ts +119 -119
- package/docs/04-REFERENCE/testing/examples/performance-test.ts +183 -183
- package/docs/04-REFERENCE/testing/examples/s3-testing.ts +127 -127
- package/docs/04-REFERENCE/testing/modules/04-reference-testing-01-foundations.md +267 -267
- package/docs/04-REFERENCE/testing/modules/04-reference-testing-02-s3-testing.md +599 -599
- package/docs/04-REFERENCE/testing/modules/04-reference-testing-03-fluent-testing.md +589 -589
- package/docs/04-REFERENCE/testing/modules/04-reference-testing-04-integration-testing.md +699 -699
- package/docs/04-REFERENCE/testing/modules/04-reference-testing-05-debugging.md +478 -478
- package/docs/04-REFERENCE/testing/modules/04-reference-testing-06-cicd-integration.md +463 -463
- package/docs/04-REFERENCE/testing/modules/04-reference-testing-06-preflight-validation.md +131 -131
- package/docs/04-REFERENCE/testing/modules/04-reference-testing-07-best-practices.md +499 -499
- package/docs/04-REFERENCE/testing/modules/04-reference-testing-07-coverage-ci.md +165 -165
- package/docs/04-REFERENCE/testing/modules/04-reference-testing-08-api-reference.md +634 -634
- package/docs/04-REFERENCE/testing/readme.md +86 -86
- package/docs/04-REFERENCE/testing/testing-quick-reference.md +667 -667
- package/docs/04-REFERENCE/testing/testing-readme.md +286 -286
- package/docs/04-REFERENCE/troubleshooting/readme.md +144 -144
- package/docs/04-REFERENCE/troubleshooting/troubleshooting-deno-sftp-compatibility.md +392 -392
- package/docs/template-loading-matrix.md +242 -242
- package/package.json +5 -3
- package/docs/02-CORE-GUIDES/api-reference/cli-profile-integration.md +0 -377
|
@@ -1,1528 +1,1528 @@
|
|
|
1
|
-
# Enterprise Integration Patterns with Fluent Connect SDK
|
|
2
|
-
|
|
3
|
-
**Status:** Production Ready
|
|
4
|
-
**Last Updated:** 2025-11-05
|
|
5
|
-
**SDK Version:** 0.1.39
|
|
6
|
-
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
## Table of Contents
|
|
10
|
-
|
|
11
|
-
1. [Introduction](#introduction)
|
|
12
|
-
2. [Understanding the SDK as a Toolkit](#understanding-the-sdk-as-a-toolkit)
|
|
13
|
-
3. [Pattern Categories](#pattern-categories)
|
|
14
|
-
4. [Routing Patterns](#routing-patterns)
|
|
15
|
-
5. [Transformation Patterns](#transformation-patterns)
|
|
16
|
-
6. [Endpoint Patterns](#endpoint-patterns)
|
|
17
|
-
7. [System Management Patterns](#system-management-patterns)
|
|
18
|
-
8. [Pattern Summary Matrix](#pattern-summary-matrix)
|
|
19
|
-
9. [Additional Resources](#additional-resources)
|
|
20
|
-
|
|
21
|
-
---
|
|
22
|
-
|
|
23
|
-
## Introduction
|
|
24
|
-
|
|
25
|
-
### What is Enterprise Integration Patterns (EIP)?
|
|
26
|
-
|
|
27
|
-
**Enterprise Integration Patterns (EIP)** are a collection of 65 proven design patterns for building enterprise application integration solutions. Originally documented by Gregor Hohpe and Bobby Woolf in their book ["Enterprise Integration Patterns"](https://www.enterpriseintegrationpatterns.com/), these patterns provide reusable solutions to common integration challenges.
|
|
28
|
-
|
|
29
|
-
**What EIP Means in TypeScript Context:**
|
|
30
|
-
|
|
31
|
-
In TypeScript/JavaScript applications, EIP patterns translate to:
|
|
32
|
-
- **Composable functions and services** - Building blocks you combine to solve integration problems
|
|
33
|
-
- **Message transformation** - Converting data between different formats (XML, JSON, CSV)
|
|
34
|
-
- **Routing logic** - Directing messages to appropriate destinations based on content
|
|
35
|
-
- **Error handling** - Managing failures and retries in distributed systems
|
|
36
|
-
- **State management** - Tracking processed messages and maintaining idempotency
|
|
37
|
-
|
|
38
|
-
**Key Concepts:**
|
|
39
|
-
- **Patterns are solutions**, not libraries - You implement them using your toolkit
|
|
40
|
-
- **SDK provides building blocks** - You compose them to create patterns
|
|
41
|
-
- **TypeScript enables composition** - Functions, classes, and async/await make patterns natural
|
|
42
|
-
|
|
43
|
-
**Learn More:**
|
|
44
|
-
- 📚 [Enterprise Integration Patterns Book](https://www.enterpriseintegrationpatterns.com/)
|
|
45
|
-
- 🌐 [EIP Pattern Catalog](https://www.enterpriseintegrationpatterns.com/patterns/messaging/)
|
|
46
|
-
- 📖 [EIP Wikipedia](https://en.wikipedia.org/wiki/Enterprise_Integration_Patterns)
|
|
47
|
-
|
|
48
|
-
---
|
|
49
|
-
|
|
50
|
-
**Enterprise Integration Patterns (EIP)** are proven solutions to common integration challenges. This guide shows how to build these patterns using the Fluent Connect SDK's composable building blocks.
|
|
51
|
-
|
|
52
|
-
### What This Guide Covers
|
|
53
|
-
|
|
54
|
-
- **10 Essential EIP Patterns** with real Fluent Commerce use cases
|
|
55
|
-
- **Practical SDK Implementation** using actual client methods and services
|
|
56
|
-
- **Order & Inventory Domain Examples** - the two primary Fluent Commerce domains
|
|
57
|
-
- **Support Status** - Whether each pattern can be built with SDK building blocks
|
|
58
|
-
- **Use-Case-Driven Approach** - Real-world scenarios for each pattern
|
|
59
|
-
|
|
60
|
-
### Prerequisites
|
|
61
|
-
|
|
62
|
-
- Understanding of SDK architecture (see `docs/00-START-HERE/SDK-PHILOSOPHY.md`)
|
|
63
|
-
- Familiarity with Fluent Commerce entities (Order, Fulfilment, Inventory)
|
|
64
|
-
- Basic knowledge of GraphQL and Batch API concepts
|
|
65
|
-
|
|
66
|
-
---
|
|
67
|
-
|
|
68
|
-
## Understanding the SDK as a Toolkit
|
|
69
|
-
|
|
70
|
-
> **The SDK provides services with methods. YOU build the workflow.**
|
|
71
|
-
|
|
72
|
-
The FC Connect SDK is **not an opinionated framework** - it's a **toolkit of composable primitives**. Enterprise Integration Patterns are YOUR orchestration logic built on top of SDK services.
|
|
73
|
-
|
|
74
|
-
### Core Building Blocks
|
|
75
|
-
|
|
76
|
-
```
|
|
77
|
-
Data Sources Parsers Transformation
|
|
78
|
-
├── S3DataSource ├── CSVParserService ├── UniversalMapper
|
|
79
|
-
├── SftpDataSource ├── XMLParserService ├── GraphQLMutationMapper
|
|
80
|
-
└── InventoryDataSource ├── JSONParserService └── sdkResolvers
|
|
81
|
-
└── ParquetParserService
|
|
82
|
-
|
|
83
|
-
Client Services State Management
|
|
84
|
-
├── FluentClient ├── BatchAPIClient ├── StateService
|
|
85
|
-
├── createClient() ├── S3PresignService ├── VersoriKVAdapter
|
|
86
|
-
├── graphql() ├── WebhookValidationService
|
|
87
|
-
├── sendEvent() └── SchemaValidationService
|
|
88
|
-
├── createJob()
|
|
89
|
-
├── sendBatch()
|
|
90
|
-
└── getJobStatus()
|
|
91
|
-
```
|
|
92
|
-
|
|
93
|
-
### The Fundamental Pattern
|
|
94
|
-
|
|
95
|
-
```
|
|
96
|
-
Read → Parse → Map → YOUR LOGIC → Send/Archive
|
|
97
|
-
```
|
|
98
|
-
|
|
99
|
-
Enterprise Integration Patterns sit in the **"YOUR LOGIC"** layer - they define HOW you orchestrate these primitives.
|
|
100
|
-
|
|
101
|
-
---
|
|
102
|
-
|
|
103
|
-
## Pattern Categories
|
|
104
|
-
|
|
105
|
-
We organize EIP patterns into four categories based on their primary purpose:
|
|
106
|
-
|
|
107
|
-
1. **Routing Patterns** - Direct messages to appropriate destinations
|
|
108
|
-
2. **Transformation Patterns** - Convert data between formats
|
|
109
|
-
3. **Endpoint Patterns** - Connect to external systems
|
|
110
|
-
4. **System Management Patterns** - Handle reliability and state
|
|
111
|
-
|
|
112
|
-
---
|
|
113
|
-
|
|
114
|
-
## Routing Patterns
|
|
115
|
-
|
|
116
|
-
### 1. Message Router
|
|
117
|
-
|
|
118
|
-
**Pattern:** Route messages to different destinations based on message content or type.
|
|
119
|
-
|
|
120
|
-
**Support Status:** ✅ **Fully Supported** - Can be built using SDK client methods
|
|
121
|
-
|
|
122
|
-
**SDK Building Blocks:**
|
|
123
|
-
- `FluentClient.sendEvent()` - Send to Event API
|
|
124
|
-
- `FluentClient.createJob()` + `sendBatch()` - Send to Batch API
|
|
125
|
-
- `FluentClient.graphql()` - Send to GraphQL API
|
|
126
|
-
|
|
127
|
-
#### Use Case: Order Routing Based on Type
|
|
128
|
-
|
|
129
|
-
**Business Problem:** Different order types require different processing workflows. Click & Collect orders need immediate workflow triggers, while Home Delivery orders can be batched for efficiency.
|
|
130
|
-
|
|
131
|
-
Route orders to different workflows based on order type:
|
|
132
|
-
- Click & Collect → Event API (trigger Rubix workflow)
|
|
133
|
-
- Home Delivery → Batch API (bulk processing)
|
|
134
|
-
- Store Pickup → GraphQL mutation (immediate processing)
|
|
135
|
-
|
|
136
|
-
```typescript
|
|
137
|
-
import { createClient, FluentEvent, FluentBatchPayload } from '@fluentcommerce/fc-connect-sdk';
|
|
138
|
-
|
|
139
|
-
async function routeOrders(orders: any[], ctx: any, log: any) {
|
|
140
|
-
const client = await createClient({ ...ctx, log });
|
|
141
|
-
|
|
142
|
-
for (const order of orders) {
|
|
143
|
-
const orderType = order.attributes?.find((a: any) => a.name === 'type')?.value;
|
|
144
|
-
|
|
145
|
-
// ROUTING LOGIC
|
|
146
|
-
switch (orderType) {
|
|
147
|
-
case 'CLICK_AND_COLLECT':
|
|
148
|
-
// Route to Event API → Triggers Rubix workflow
|
|
149
|
-
const event: FluentEvent = {
|
|
150
|
-
name: 'order.created',
|
|
151
|
-
entityType: 'ORDER',
|
|
152
|
-
entityRef: order.ref,
|
|
153
|
-
retailerId: order.retailerId,
|
|
154
|
-
attributes: order.attributes
|
|
155
|
-
};
|
|
156
|
-
await client.sendEvent(event, 'async');
|
|
157
|
-
log.info('Order routed to Event API', { ref: order.ref, type: orderType });
|
|
158
|
-
break;
|
|
159
|
-
|
|
160
|
-
case 'HOME_DELIVERY':
|
|
161
|
-
// Route to Event API → Trigger workflow (Batch API only supports INVENTORY)
|
|
162
|
-
const event: FluentEvent = {
|
|
163
|
-
name: 'order.created',
|
|
164
|
-
entityType: 'ORDER',
|
|
165
|
-
entityRef: order.ref,
|
|
166
|
-
retailerId: order.retailerId,
|
|
167
|
-
attributes: order.attributes
|
|
168
|
-
};
|
|
169
|
-
await client.sendEvent(event, 'async');
|
|
170
|
-
log.info('Order routed to Event API', { ref: order.ref, type: orderType });
|
|
171
|
-
break;
|
|
172
|
-
|
|
173
|
-
case 'STORE_PICKUP':
|
|
174
|
-
// Route to GraphQL → Immediate processing
|
|
175
|
-
const mutation = `
|
|
176
|
-
mutation CreateOrder($input: CreateOrderInput!) {
|
|
177
|
-
createOrder(input: $input) {
|
|
178
|
-
id
|
|
179
|
-
ref
|
|
180
|
-
status
|
|
181
|
-
}
|
|
182
|
-
}
|
|
183
|
-
`;
|
|
184
|
-
|
|
185
|
-
await client.mutate(mutation, { input: order });
|
|
186
|
-
log.info('Order routed to GraphQL', { ref: order.ref });
|
|
187
|
-
break;
|
|
188
|
-
|
|
189
|
-
default:
|
|
190
|
-
log.warn('Unknown order type - routing to default Event API', { ref: order.ref, type: orderType });
|
|
191
|
-
// Default route to Event API (Batch API only supports INVENTORY)
|
|
192
|
-
const defaultEvent: FluentEvent = {
|
|
193
|
-
name: 'order.created',
|
|
194
|
-
entityType: 'ORDER',
|
|
195
|
-
entityRef: order.ref,
|
|
196
|
-
retailerId: order.retailerId,
|
|
197
|
-
attributes: order.attributes
|
|
198
|
-
};
|
|
199
|
-
await client.sendEvent(defaultEvent, 'async');
|
|
200
|
-
}
|
|
201
|
-
}
|
|
202
|
-
}
|
|
203
|
-
```
|
|
204
|
-
|
|
205
|
-
**When to Use:**
|
|
206
|
-
- Multiple destination APIs for different message types
|
|
207
|
-
- Order type dictates processing workflow (express, standard, backorder)
|
|
208
|
-
- Inventory updates need different handling by location type (store, warehouse, virtual)
|
|
209
|
-
|
|
210
|
-
---
|
|
211
|
-
|
|
212
|
-
### 2. Content-Based Router
|
|
213
|
-
|
|
214
|
-
**Pattern:** Route messages based on content inspection (attributes, fields, computed values).
|
|
215
|
-
|
|
216
|
-
**Support Status:** ✅ **Fully Supported** - Can be built using `UniversalMapper` and application logic
|
|
217
|
-
|
|
218
|
-
**SDK Building Blocks:**
|
|
219
|
-
- `UniversalMapper` - Extract and transform routing criteria
|
|
220
|
-
- Conditional logic based on extracted values
|
|
221
|
-
|
|
222
|
-
#### Use Case: Inventory Distribution by Quantity
|
|
223
|
-
|
|
224
|
-
**Business Problem:** High-volume inventory updates should use Batch API for efficiency, while low-quantity alerts need immediate processing via GraphQL to trigger restocking workflows.
|
|
225
|
-
|
|
226
|
-
Route inventory updates based on available quantity:
|
|
227
|
-
- High quantity (>1000) → Batch API (efficient bulk updates)
|
|
228
|
-
- Medium quantity (100-1000) → Event API (standard processing)
|
|
229
|
-
- Low quantity (<100) → GraphQL + Event (immediate alert + workflow trigger)
|
|
230
|
-
|
|
231
|
-
```typescript
|
|
232
|
-
import { createClient, UniversalMapper, FluentBatchPayload } from '@fluentcommerce/fc-connect-sdk';
|
|
233
|
-
|
|
234
|
-
async function routeInventoryUpdates(inventoryData: any[], ctx: any, log: any) {
|
|
235
|
-
const client = await createClient({ ...ctx, log });
|
|
236
|
-
|
|
237
|
-
// Extract routing criteria using UniversalMapper
|
|
238
|
-
const mapper = new UniversalMapper({
|
|
239
|
-
fields: {
|
|
240
|
-
skuRef: { source: 'sku', required: true },
|
|
241
|
-
locationRef: { source: 'location', required: true },
|
|
242
|
-
qty: { source: 'quantity', resolver: 'sdk.parseInt', required: true },
|
|
243
|
-
availableQty: { source: 'available', resolver: 'sdk.parseInt' }
|
|
244
|
-
}
|
|
245
|
-
});
|
|
246
|
-
|
|
247
|
-
const highQtyBatch: any[] = [];
|
|
248
|
-
const mediumQtyEvents: any[] = [];
|
|
249
|
-
const lowQtyAlerts: any[] = [];
|
|
250
|
-
|
|
251
|
-
// Content-based routing logic
|
|
252
|
-
for (const item of inventoryData) {
|
|
253
|
-
const mapped = await mapper.map(item);
|
|
254
|
-
if (!mapped.success) {
|
|
255
|
-
log.error('Mapping failed', { item, errors: mapped.errors });
|
|
256
|
-
continue;
|
|
257
|
-
}
|
|
258
|
-
|
|
259
|
-
const qty = mapped.data.qty;
|
|
260
|
-
|
|
261
|
-
if (qty > 1000) {
|
|
262
|
-
// HIGH QUANTITY → Batch API
|
|
263
|
-
highQtyBatch.push({
|
|
264
|
-
ref: `${mapped.data.locationRef}:${mapped.data.skuRef}`,
|
|
265
|
-
type: 'INVENTORY',
|
|
266
|
-
locationRef: mapped.data.locationRef,
|
|
267
|
-
skuRef: mapped.data.skuRef,
|
|
268
|
-
qty,
|
|
269
|
-
status: 'ACTIVE'
|
|
270
|
-
});
|
|
271
|
-
} else if (qty >= 100) {
|
|
272
|
-
// MEDIUM QUANTITY → Event API
|
|
273
|
-
mediumQtyEvents.push({
|
|
274
|
-
name: 'inventory.updated',
|
|
275
|
-
entityType: 'INVENTORY_QUANTITY',
|
|
276
|
-
entityRef: `${mapped.data.locationRef}:${mapped.data.skuRef}`,
|
|
277
|
-
retailerId: ctx.activation.getVariable('fluentRetailerId'),
|
|
278
|
-
attributes: [
|
|
279
|
-
{ name: 'qty', type: 'INTEGER', value: qty },
|
|
280
|
-
{ name: 'locationRef', type: 'STRING', value: mapped.data.locationRef }
|
|
281
|
-
]
|
|
282
|
-
});
|
|
283
|
-
} else {
|
|
284
|
-
// LOW QUANTITY → GraphQL + Event (alert)
|
|
285
|
-
lowQtyAlerts.push(mapped.data);
|
|
286
|
-
}
|
|
287
|
-
}
|
|
288
|
-
|
|
289
|
-
// Execute routing destinations
|
|
290
|
-
if (highQtyBatch.length > 0) {
|
|
291
|
-
const job = await client.createJob({
|
|
292
|
-
name: `high-qty-inventory-${Date.now()}`,
|
|
293
|
-
retailerId: ctx.activation.getVariable('fluentRetailerId')
|
|
294
|
-
});
|
|
295
|
-
|
|
296
|
-
await client.sendBatch(job.id, {
|
|
297
|
-
entityType: 'INVENTORY',
|
|
298
|
-
entities: highQtyBatch,
|
|
299
|
-
action: 'UPSERT'
|
|
300
|
-
});
|
|
301
|
-
|
|
302
|
-
log.info('High quantity inventory routed to Batch API', { count: highQtyBatch.length });
|
|
303
|
-
}
|
|
304
|
-
|
|
305
|
-
if (mediumQtyEvents.length > 0) {
|
|
306
|
-
for (const event of mediumQtyEvents) {
|
|
307
|
-
await client.sendEvent(event, 'async');
|
|
308
|
-
}
|
|
309
|
-
log.info('Medium quantity inventory routed to Event API', { count: mediumQtyEvents.length });
|
|
310
|
-
}
|
|
311
|
-
|
|
312
|
-
if (lowQtyAlerts.length > 0) {
|
|
313
|
-
// Send low stock alert via GraphQL + Event
|
|
314
|
-
for (const alert of lowQtyAlerts) {
|
|
315
|
-
const mutation = `
|
|
316
|
-
mutation TriggerLowStockAlert($locationRef: String!, $skuRef: String!, $qty: Int!) {
|
|
317
|
-
createEvent(input: {
|
|
318
|
-
name: "inventory.low_stock_alert"
|
|
319
|
-
entityType: INVENTORY_QUANTITY
|
|
320
|
-
entityRef: $locationRef
|
|
321
|
-
attributes: [
|
|
322
|
-
{ name: "skuRef", type: STRING, value: $skuRef }
|
|
323
|
-
{ name: "qty", type: INTEGER, value: $qty }
|
|
324
|
-
]
|
|
325
|
-
}) {
|
|
326
|
-
id
|
|
327
|
-
}
|
|
328
|
-
}
|
|
329
|
-
`;
|
|
330
|
-
|
|
331
|
-
await client.graphql({
|
|
332
|
-
query: mutation,
|
|
333
|
-
variables: {
|
|
334
|
-
locationRef: alert.locationRef,
|
|
335
|
-
skuRef: alert.skuRef,
|
|
336
|
-
qty: alert.qty
|
|
337
|
-
}
|
|
338
|
-
});
|
|
339
|
-
}
|
|
340
|
-
log.warn('Low quantity alerts sent', { count: lowQtyAlerts.length });
|
|
341
|
-
}
|
|
342
|
-
}
|
|
343
|
-
```
|
|
344
|
-
|
|
345
|
-
**When to Use:**
|
|
346
|
-
- Routing depends on data values (quantity thresholds, priority levels)
|
|
347
|
-
- Different SLAs based on order value or customer tier
|
|
348
|
-
- Inventory allocation strategies based on location characteristics
|
|
349
|
-
|
|
350
|
-
---
|
|
351
|
-
|
|
352
|
-
## Transformation Patterns
|
|
353
|
-
|
|
354
|
-
### 3. Content Enricher
|
|
355
|
-
|
|
356
|
-
**Pattern:** Enhance message with additional data from external sources.
|
|
357
|
-
|
|
358
|
-
**Support Status:** ✅ **Fully Supported** - Can be built using `FluentClient.graphql()` and `UniversalMapper` with custom resolvers
|
|
359
|
-
|
|
360
|
-
**SDK Building Blocks:**
|
|
361
|
-
- `FluentClient.graphql()` - Fetch enrichment data
|
|
362
|
-
- `UniversalMapper` - Merge enriched data
|
|
363
|
-
- Custom resolvers - Complex enrichment logic
|
|
364
|
-
|
|
365
|
-
#### Use Case: Enrich Order Data with Product Details
|
|
366
|
-
|
|
367
|
-
**Business Problem:** Order webhooks from external systems contain only SKU references. Fulfilment workflows need product names, weights, and categories for shipping calculations and packaging decisions.
|
|
368
|
-
|
|
369
|
-
Enrich order line items with product information (name, category, weight) fetched from Fluent GraphQL.
|
|
370
|
-
|
|
371
|
-
```typescript
|
|
372
|
-
import { createClient, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
|
|
373
|
-
|
|
374
|
-
async function enrichOrdersWithProductData(orders: any[], ctx: any, log: any) {
|
|
375
|
-
const client = await createClient({ ...ctx, log });
|
|
376
|
-
|
|
377
|
-
// STEP 1: Fetch all product data in one GraphQL query
|
|
378
|
-
const skuRefs = [...new Set(
|
|
379
|
-
orders.flatMap(o => o.items?.map((i: any) => i.skuRef) || [])
|
|
380
|
-
)];
|
|
381
|
-
|
|
382
|
-
const query = `
|
|
383
|
-
query GetProducts($skuRefs: [String!]!) {
|
|
384
|
-
products(skuRefs: $skuRefs) {
|
|
385
|
-
edges {
|
|
386
|
-
node {
|
|
387
|
-
ref
|
|
388
|
-
name
|
|
389
|
-
attributes {
|
|
390
|
-
name
|
|
391
|
-
type
|
|
392
|
-
value
|
|
393
|
-
}
|
|
394
|
-
}
|
|
395
|
-
}
|
|
396
|
-
}
|
|
397
|
-
}
|
|
398
|
-
`;
|
|
399
|
-
|
|
400
|
-
const response = await client.graphql({ query, variables: { skuRefs } });
|
|
401
|
-
|
|
402
|
-
// Build lookup map for fast enrichment
|
|
403
|
-
const productMap = new Map();
|
|
404
|
-
response.data?.products?.edges?.forEach((edge: any) => {
|
|
405
|
-
const product = edge.node;
|
|
406
|
-
productMap.set(product.ref, {
|
|
407
|
-
name: product.name,
|
|
408
|
-
category: product.attributes?.find((a: any) => a.name === 'category')?.value || 'UNCATEGORIZED',
|
|
409
|
-
weight: parseFloat(product.attributes?.find((a: any) => a.name === 'weight')?.value || '0')
|
|
410
|
-
});
|
|
411
|
-
});
|
|
412
|
-
|
|
413
|
-
// STEP 2: Enrich order items with product data
|
|
414
|
-
const enrichedOrders = orders.map(order => {
|
|
415
|
-
const enrichedItems = order.items?.map((item: any) => {
|
|
416
|
-
const productData = productMap.get(item.skuRef);
|
|
417
|
-
|
|
418
|
-
return {
|
|
419
|
-
...item,
|
|
420
|
-
// ENRICHMENT: Add product details
|
|
421
|
-
productName: productData?.name || 'Unknown Product',
|
|
422
|
-
productCategory: productData?.category || 'UNCATEGORIZED',
|
|
423
|
-
productWeight: productData?.weight || 0,
|
|
424
|
-
// Computed fields based on enriched data
|
|
425
|
-
lineWeight: (productData?.weight || 0) * (item.quantity || 1),
|
|
426
|
-
isHeavyItem: (productData?.weight || 0) > 10 // kg
|
|
427
|
-
};
|
|
428
|
-
});
|
|
429
|
-
|
|
430
|
-
return {
|
|
431
|
-
...order,
|
|
432
|
-
items: enrichedItems,
|
|
433
|
-
// Order-level computed fields
|
|
434
|
-
totalWeight: enrichedItems.reduce((sum: number, item: any) => sum + (item.lineWeight || 0), 0),
|
|
435
|
-
hasHeavyItems: enrichedItems.some((item: any) => item.isHeavyItem)
|
|
436
|
-
};
|
|
437
|
-
});
|
|
438
|
-
|
|
439
|
-
log.info('Orders enriched with product data', {
|
|
440
|
-
orderCount: orders.length,
|
|
441
|
-
productCount: productMap.size
|
|
442
|
-
});
|
|
443
|
-
|
|
444
|
-
return enrichedOrders;
|
|
445
|
-
}
|
|
446
|
-
```
|
|
447
|
-
|
|
448
|
-
**When to Use:**
|
|
449
|
-
- Order processing needs product details (dimensions, weight, hazmat flags)
|
|
450
|
-
- Inventory updates require location metadata (type, capacity, zone)
|
|
451
|
-
- Fulfilment workflows need customer preferences or address validation
|
|
452
|
-
|
|
453
|
-
---
|
|
454
|
-
|
|
455
|
-
### 4. Message Translator
|
|
456
|
-
|
|
457
|
-
**Pattern:** Convert message format from one schema to another.
|
|
458
|
-
|
|
459
|
-
**Support Status:** ✅ **Fully Supported** - Core SDK capability via Parsers + `UniversalMapper` + Builders
|
|
460
|
-
|
|
461
|
-
**SDK Building Blocks:**
|
|
462
|
-
- `XMLParserService` - Parse XML input
|
|
463
|
-
- `CSVParserService` - Parse CSV input
|
|
464
|
-
- `JSONParserService` - Parse JSON input
|
|
465
|
-
- `UniversalMapper` - Transform to Fluent schema
|
|
466
|
-
- `XMLBuilder` / `CSVBuilder` - Build output format
|
|
467
|
-
|
|
468
|
-
#### Use Case: Translate External Inventory CSV to Fluent Batch Format
|
|
469
|
-
|
|
470
|
-
**Business Problem:** Supplier systems export inventory data in custom CSV formats with different field names (e.g., `supplier_sku`, `warehouse_code`, `on_hand`). Fluent Commerce requires standard field names (`skuRef`, `locationRef`, `qty`).
|
|
471
|
-
|
|
472
|
-
Translate supplier CSV format to Fluent Batch API format.
|
|
473
|
-
|
|
474
|
-
```typescript
|
|
475
|
-
import {
|
|
476
|
-
SftpDataSource,
|
|
477
|
-
CSVParserService,
|
|
478
|
-
UniversalMapper,
|
|
479
|
-
createClient,
|
|
480
|
-
FluentBatchPayload
|
|
481
|
-
} from '@fluentcommerce/fc-connect-sdk';
|
|
482
|
-
|
|
483
|
-
async function translateInventoryCsvToBatch(ctx: any, log: any) {
|
|
484
|
-
const client = await createClient({ ...ctx, log });
|
|
485
|
-
|
|
486
|
-
// STEP 1: Read source data (supplier CSV format)
|
|
487
|
-
const sftp = new SftpDataSource({
|
|
488
|
-
type: 'SFTP_CSV',
|
|
489
|
-
settings: {
|
|
490
|
-
host: ctx.activation.getVariable('sftpHost'),
|
|
491
|
-
username: ctx.activation.getVariable('sftpUsername'),
|
|
492
|
-
password: ctx.activation.getVariable('sftpPassword'),
|
|
493
|
-
remotePath: '/inbound/inventory',
|
|
494
|
-
filePattern: '*.csv'
|
|
495
|
-
}
|
|
496
|
-
}, log);
|
|
497
|
-
|
|
498
|
-
try {
|
|
499
|
-
const files = await sftp.listFiles();
|
|
500
|
-
if (files.length === 0) {
|
|
501
|
-
log.info('No files to process');
|
|
502
|
-
return;
|
|
503
|
-
}
|
|
504
|
-
|
|
505
|
-
const csvContent = await sftp.downloadFile(files[0].name, { encoding: 'utf8' }) as string;
|
|
506
|
-
|
|
507
|
-
// STEP 2: Parse source format
|
|
508
|
-
const csvParser = new CSVParserService(log);
|
|
509
|
-
const parsedData = await csvParser.parse(csvContent, {
|
|
510
|
-
delimiter: ',',
|
|
511
|
-
columns: true,
|
|
512
|
-
skip_empty_lines: true
|
|
513
|
-
});
|
|
514
|
-
|
|
515
|
-
// Source format: supplier_sku, warehouse_code, on_hand, available, reserved
|
|
516
|
-
log.info('Parsed supplier CSV', { recordCount: parsedData.length });
|
|
517
|
-
|
|
518
|
-
// STEP 3: Translate to Fluent schema using UniversalMapper
|
|
519
|
-
const mapper = new UniversalMapper({
|
|
520
|
-
fields: {
|
|
521
|
-
// TRANSLATION RULES
|
|
522
|
-
ref: {
|
|
523
|
-
resolver: 'custom.buildCompositeKey',
|
|
524
|
-
required: true
|
|
525
|
-
},
|
|
526
|
-
type: {
|
|
527
|
-
value: 'INVENTORY_QUANTITY', // Static value for Batch API
|
|
528
|
-
required: true
|
|
529
|
-
},
|
|
530
|
-
locationRef: {
|
|
531
|
-
source: 'warehouse_code',
|
|
532
|
-
resolver: 'sdk.uppercase', // Normalize location ref
|
|
533
|
-
required: true
|
|
534
|
-
},
|
|
535
|
-
skuRef: {
|
|
536
|
-
source: 'supplier_sku',
|
|
537
|
-
resolver: 'sdk.trim', // Clean whitespace
|
|
538
|
-
required: true
|
|
539
|
-
},
|
|
540
|
-
qty: {
|
|
541
|
-
source: 'available', // Map 'available' → 'qty'
|
|
542
|
-
resolver: 'sdk.parseInt',
|
|
543
|
-
required: true
|
|
544
|
-
},
|
|
545
|
-
onHand: {
|
|
546
|
-
source: 'on_hand',
|
|
547
|
-
resolver: 'sdk.parseInt'
|
|
548
|
-
},
|
|
549
|
-
reserved: {
|
|
550
|
-
source: 'reserved',
|
|
551
|
-
resolver: 'sdk.parseInt',
|
|
552
|
-
defaultValue: 0 // Default if missing
|
|
553
|
-
}
|
|
554
|
-
}
|
|
555
|
-
}, {
|
|
556
|
-
customResolvers: {
|
|
557
|
-
// Custom resolver to build composite key
|
|
558
|
-
'custom.buildCompositeKey': (value: any, context: any) => {
|
|
559
|
-
return `${context.warehouse_code}:${context.supplier_sku}`;
|
|
560
|
-
}
|
|
561
|
-
}
|
|
562
|
-
});
|
|
563
|
-
|
|
564
|
-
const translatedEntities = [];
|
|
565
|
-
for (const record of parsedData) {
|
|
566
|
-
const result = await mapper.map(record);
|
|
567
|
-
if (result.success) {
|
|
568
|
-
translatedEntities.push(result.data);
|
|
569
|
-
} else {
|
|
570
|
-
log.error('Translation failed', { record, errors: result.errors });
|
|
571
|
-
}
|
|
572
|
-
}
|
|
573
|
-
|
|
574
|
-
// STEP 4: Send translated data to Fluent Batch API
|
|
575
|
-
if (translatedEntities.length > 0) {
|
|
576
|
-
const job = await client.createJob({
|
|
577
|
-
name: `inventory-translation-${Date.now()}`,
|
|
578
|
-
retailerId: ctx.activation.getVariable('fluentRetailerId')
|
|
579
|
-
});
|
|
580
|
-
|
|
581
|
-
const batch: FluentBatchPayload = {
|
|
582
|
-
entityType: 'INVENTORY',
|
|
583
|
-
entities: translatedEntities,
|
|
584
|
-
action: 'UPSERT'
|
|
585
|
-
};
|
|
586
|
-
|
|
587
|
-
await client.sendBatch(job.id, batch);
|
|
588
|
-
|
|
589
|
-
log.info('Inventory translated and sent to Batch API', {
|
|
590
|
-
sourceRecords: parsedData.length,
|
|
591
|
-
translatedEntities: translatedEntities.length,
|
|
592
|
-
jobId: job.id
|
|
593
|
-
});
|
|
594
|
-
}
|
|
595
|
-
} finally {
|
|
596
|
-
await sftp.dispose();
|
|
597
|
-
}
|
|
598
|
-
}
|
|
599
|
-
```
|
|
600
|
-
|
|
601
|
-
**When to Use:**
|
|
602
|
-
- Supplier data formats differ from Fluent schema
|
|
603
|
-
- Multiple source systems with different CSV/XML structures
|
|
604
|
-
- Legacy system integration with non-standard formats
|
|
605
|
-
|
|
606
|
-
---
|
|
607
|
-
|
|
608
|
-
### 5. Splitter
|
|
609
|
-
|
|
610
|
-
**Pattern:** Break a single message into multiple messages for parallel processing.
|
|
611
|
-
|
|
612
|
-
**Support Status:** ✅ **Fully Supported** - Can be built using Parsers (automatic splitting) + `Promise.all()` for parallel processing
|
|
613
|
-
|
|
614
|
-
**SDK Building Blocks:**
|
|
615
|
-
- `FluentClient.sendBatch()` - Split into multiple batches
|
|
616
|
-
- `Promise.all()` - Parallel execution
|
|
617
|
-
- `UniversalMapper` with array handling
|
|
618
|
-
- Parsers automatically split files into records
|
|
619
|
-
|
|
620
|
-
#### Use Case: Split Large Inventory File into Parallel Batches
|
|
621
|
-
|
|
622
|
-
**Business Problem:** Daily inventory export files contain 50,000+ inventory positions. Processing sequentially would take hours. Need to split into chunks and process in parallel to meet SLA requirements.
|
|
623
|
-
|
|
624
|
-
**Note:** Batch API only supports `INVENTORY` entities. For orders, use GraphQL mutations with parallel processing instead.
|
|
625
|
-
|
|
626
|
-
Process a large inventory file by splitting into chunks and sending parallel batches.
|
|
627
|
-
|
|
628
|
-
```typescript
|
|
629
|
-
import {
|
|
630
|
-
S3DataSource,
|
|
631
|
-
XMLParserService,
|
|
632
|
-
UniversalMapper,
|
|
633
|
-
createClient,
|
|
634
|
-
FluentBatchPayload
|
|
635
|
-
} from '@fluentcommerce/fc-connect-sdk';
|
|
636
|
-
|
|
637
|
-
async function splitOrdersIntoParallelBatches(ctx: any, log: any) {
|
|
638
|
-
const client = await createClient({ ...ctx, log });
|
|
639
|
-
|
|
640
|
-
// STEP 1: Read large inventory file from S3
|
|
641
|
-
const s3 = new S3DataSource({
|
|
642
|
-
type: 'S3_XML',
|
|
643
|
-
s3Config: {
|
|
644
|
-
bucket: ctx.activation.getVariable('s3Bucket'),
|
|
645
|
-
region: ctx.activation.getVariable('awsRegion'),
|
|
646
|
-
accessKeyId: ctx.activation.getVariable('awsAccessKeyId'),
|
|
647
|
-
secretAccessKey: ctx.activation.getVariable('awsSecretAccessKey')
|
|
648
|
-
}
|
|
649
|
-
}, log);
|
|
650
|
-
|
|
651
|
-
const xmlContent = await s3.downloadFile('inventory/inventory-batch-large.xml', { encoding: 'utf8' }) as string;
|
|
652
|
-
|
|
653
|
-
// STEP 2: Parse XML
|
|
654
|
-
const xmlParser = new XMLParserService(log);
|
|
655
|
-
const parsed = await xmlParser.parse(xmlContent);
|
|
656
|
-
|
|
657
|
-
const rawInventory = Array.isArray(parsed.inventory?.position)
|
|
658
|
-
? parsed.inventory.position
|
|
659
|
-
: [parsed.inventory?.position];
|
|
660
|
-
|
|
661
|
-
log.info('Parsed large inventory file', { totalPositions: rawInventory.length });
|
|
662
|
-
|
|
663
|
-
// STEP 3: Map inventory positions to Fluent format
|
|
664
|
-
const mapper = new UniversalMapper({
|
|
665
|
-
fields: {
|
|
666
|
-
ref: {
|
|
667
|
-
resolver: 'custom.buildRef',
|
|
668
|
-
required: true
|
|
669
|
-
},
|
|
670
|
-
locationRef: { source: '@location', required: true },
|
|
671
|
-
skuRef: { source: 'sku', required: true },
|
|
672
|
-
qty: { source: 'quantity', resolver: 'sdk.parseInt', required: true },
|
|
673
|
-
status: { source: 'status', defaultValue: 'ACTIVE' }
|
|
674
|
-
}
|
|
675
|
-
}, {
|
|
676
|
-
customResolvers: {
|
|
677
|
-
// Custom resolver to build composite key from location and SKU
|
|
678
|
-
'custom.buildRef': (value: any, context: any) => {
|
|
679
|
-
const location = context['@location'] || context.location;
|
|
680
|
-
const sku = context.sku;
|
|
681
|
-
return `${location}:${sku}`;
|
|
682
|
-
}
|
|
683
|
-
}
|
|
684
|
-
});
|
|
685
|
-
|
|
686
|
-
const mappedInventory = [];
|
|
687
|
-
for (const position of rawInventory) {
|
|
688
|
-
const result = await mapper.map(position);
|
|
689
|
-
if (result.success) {
|
|
690
|
-
mappedInventory.push(result.data);
|
|
691
|
-
}
|
|
692
|
-
}
|
|
693
|
-
|
|
694
|
-
// STEP 4: SPLIT into chunks (batches of 500)
|
|
695
|
-
const BATCH_SIZE = 500;
|
|
696
|
-
const chunks = [];
|
|
697
|
-
for (let i = 0; i < mappedInventory.length; i += BATCH_SIZE) {
|
|
698
|
-
chunks.push(mappedInventory.slice(i, i + BATCH_SIZE));
|
|
699
|
-
}
|
|
700
|
-
|
|
701
|
-
log.info('Split inventory into chunks', {
|
|
702
|
-
totalPositions: mappedInventory.length,
|
|
703
|
-
chunkCount: chunks.length,
|
|
704
|
-
batchSize: BATCH_SIZE
|
|
705
|
-
});
|
|
706
|
-
|
|
707
|
-
// STEP 5: Create job for all batches
|
|
708
|
-
// Note: Batch API only supports INVENTORY entities, not ORDER
|
|
709
|
-
// For orders, use GraphQL mutations instead (see below)
|
|
710
|
-
// This example shows splitting for inventory entities
|
|
711
|
-
const job = await client.createJob({
|
|
712
|
-
name: `inventory-split-${Date.now()}`,
|
|
713
|
-
retailerId: ctx.activation.getVariable('fluentRetailerId')
|
|
714
|
-
});
|
|
715
|
-
|
|
716
|
-
// STEP 6: Send chunks in parallel
|
|
717
|
-
const batchPromises = chunks.map(async (chunk, index) => {
|
|
718
|
-
const batch: FluentBatchPayload = {
|
|
719
|
-
entityType: 'INVENTORY',
|
|
720
|
-
entities: chunk,
|
|
721
|
-
action: 'UPSERT'
|
|
722
|
-
};
|
|
723
|
-
|
|
724
|
-
const batchResponse = await client.sendBatch(job.id, batch);
|
|
725
|
-
log.info(`Batch ${index + 1}/${chunks.length} sent`, {
|
|
726
|
-
batchId: batchResponse.id,
|
|
727
|
-
entityCount: chunk.length
|
|
728
|
-
});
|
|
729
|
-
|
|
730
|
-
return batchResponse;
|
|
731
|
-
});
|
|
732
|
-
|
|
733
|
-
// Wait for all batches to complete
|
|
734
|
-
const batchResponses = await Promise.all(batchPromises);
|
|
735
|
-
|
|
736
|
-
log.info('All inventory batches sent successfully', {
|
|
737
|
-
jobId: job.id,
|
|
738
|
-
totalBatches: batchResponses.length,
|
|
739
|
-
totalPositions: mappedInventory.length
|
|
740
|
-
});
|
|
741
|
-
|
|
742
|
-
return { jobId: job.id, batchCount: batchResponses.length };
|
|
743
|
-
}
|
|
744
|
-
```
|
|
745
|
-
|
|
746
|
-
**When to Use:**
|
|
747
|
-
- Large inventory files exceed single batch limits (>10K records)
|
|
748
|
-
- Parallel processing improves throughput
|
|
749
|
-
- Different chunks need different processing strategies
|
|
750
|
-
|
|
751
|
-
**Note:** For orders, use GraphQL mutations with `Promise.all()` for parallel processing instead of Batch API.
|
|
752
|
-
|
|
753
|
-
---
|
|
754
|
-
|
|
755
|
-
### 6. Aggregator
|
|
756
|
-
|
|
757
|
-
**Pattern:** Combine multiple related messages into a single message.
|
|
758
|
-
|
|
759
|
-
**Support Status:** ✅ **Fully Supported** - Can be built using `FluentClient.graphql()` with pagination + application aggregation logic
|
|
760
|
-
|
|
761
|
-
**SDK Building Blocks:**
|
|
762
|
-
- `FluentClient.graphql()` with pagination - Fetch multiple pages
|
|
763
|
-
- `StateService` - Track aggregation state
|
|
764
|
-
- Array aggregation logic
|
|
765
|
-
|
|
766
|
-
#### Use Case: Aggregate Virtual Position Data from Multiple Locations
|
|
767
|
-
|
|
768
|
-
**Business Problem:** Inventory positions are stored per location (warehouse, store, virtual). Reporting dashboard needs aggregated view showing total quantity across all locations for each SKU.
|
|
769
|
-
|
|
770
|
-
Aggregate inventory positions across all locations for a single SKU view.
|
|
771
|
-
|
|
772
|
-
```typescript
|
|
773
|
-
import { createClient, StateService, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
|
|
774
|
-
|
|
775
|
-
async function aggregateVirtualPositions(skuRefs: string[], ctx: any, log: any) {
|
|
776
|
-
const client = await createClient({ ...ctx, log });
|
|
777
|
-
|
|
778
|
-
// STEP 1: Query virtual positions across all locations (with pagination)
|
|
779
|
-
const query = `
|
|
780
|
-
query GetVirtualPositions($skuRefs: [String!]!, $first: Int!, $after: String) {
|
|
781
|
-
virtualPositions(skuRefs: $skuRefs, first: $first, after: $after) {
|
|
782
|
-
edges {
|
|
783
|
-
node {
|
|
784
|
-
ref
|
|
785
|
-
quantity
|
|
786
|
-
groupRef
|
|
787
|
-
productRef
|
|
788
|
-
virtualCatalogueRef
|
|
789
|
-
}
|
|
790
|
-
cursor
|
|
791
|
-
}
|
|
792
|
-
pageInfo {
|
|
793
|
-
hasNextPage
|
|
794
|
-
endCursor
|
|
795
|
-
}
|
|
796
|
-
}
|
|
797
|
-
}
|
|
798
|
-
`;
|
|
799
|
-
|
|
800
|
-
const response = await client.graphql({
|
|
801
|
-
query,
|
|
802
|
-
variables: { skuRefs, first: 1000 },
|
|
803
|
-
pagination: {
|
|
804
|
-
enabled: true,
|
|
805
|
-
maxPages: 10,
|
|
806
|
-
direction: 'forward'
|
|
807
|
-
}
|
|
808
|
-
});
|
|
809
|
-
|
|
810
|
-
const allPositions = response.data?.virtualPositions?.edges?.map((e: any) => e.node) || [];
|
|
811
|
-
|
|
812
|
-
log.info('Fetched virtual positions', {
|
|
813
|
-
totalPositions: allPositions.length,
|
|
814
|
-
skuCount: skuRefs.length,
|
|
815
|
-
paginationStats: response.extensions?.autoPagination
|
|
816
|
-
});
|
|
817
|
-
|
|
818
|
-
// STEP 2: AGGREGATE positions by SKU
|
|
819
|
-
const aggregatedBySku = new Map<string, {
|
|
820
|
-
skuRef: string;
|
|
821
|
-
totalQuantity: number;
|
|
822
|
-
locations: string[];
|
|
823
|
-
catalogues: string[];
|
|
824
|
-
}>();
|
|
825
|
-
|
|
826
|
-
for (const position of allPositions) {
|
|
827
|
-
const skuRef = position.productRef;
|
|
828
|
-
|
|
829
|
-
if (!aggregatedBySku.has(skuRef)) {
|
|
830
|
-
aggregatedBySku.set(skuRef, {
|
|
831
|
-
skuRef,
|
|
832
|
-
totalQuantity: 0,
|
|
833
|
-
locations: [],
|
|
834
|
-
catalogues: []
|
|
835
|
-
});
|
|
836
|
-
}
|
|
837
|
-
|
|
838
|
-
const agg = aggregatedBySku.get(skuRef)!;
|
|
839
|
-
agg.totalQuantity += position.quantity || 0;
|
|
840
|
-
|
|
841
|
-
if (position.groupRef && !agg.locations.includes(position.groupRef)) {
|
|
842
|
-
agg.locations.push(position.groupRef);
|
|
843
|
-
}
|
|
844
|
-
|
|
845
|
-
if (position.virtualCatalogueRef && !agg.catalogues.includes(position.virtualCatalogueRef)) {
|
|
846
|
-
agg.catalogues.push(position.virtualCatalogueRef);
|
|
847
|
-
}
|
|
848
|
-
}
|
|
849
|
-
|
|
850
|
-
// STEP 3: Convert to array and compute metrics
|
|
851
|
-
const aggregatedResults = Array.from(aggregatedBySku.values()).map(agg => ({
|
|
852
|
-
...agg,
|
|
853
|
-
locationCount: agg.locations.length,
|
|
854
|
-
catalogueCount: agg.catalogues.length,
|
|
855
|
-
avgQuantityPerLocation: agg.totalQuantity / agg.locations.length
|
|
856
|
-
}));
|
|
857
|
-
|
|
858
|
-
log.info('Aggregation complete', {
|
|
859
|
-
totalSKUs: aggregatedResults.length,
|
|
860
|
-
totalQuantity: aggregatedResults.reduce((sum, r) => sum + r.totalQuantity, 0)
|
|
861
|
-
});
|
|
862
|
-
|
|
863
|
-
// STEP 4: Store aggregated results in KV for later retrieval
|
|
864
|
-
const kv = ctx.openKv(':project:');
|
|
865
|
-
const stateService = new StateService(log);
|
|
866
|
-
|
|
867
|
-
for (const result of aggregatedResults) {
|
|
868
|
-
await kv.set(['aggregated_inventory', result.skuRef], result);
|
|
869
|
-
}
|
|
870
|
-
|
|
871
|
-
log.info('Aggregated results stored in KV', { skuCount: aggregatedResults.length });
|
|
872
|
-
|
|
873
|
-
return aggregatedResults;
|
|
874
|
-
}
|
|
875
|
-
```
|
|
876
|
-
|
|
877
|
-
**When to Use:**
|
|
878
|
-
- Inventory views need multi-location aggregation
|
|
879
|
-
- Order totals calculated from multiple line items
|
|
880
|
-
- Reporting requires data from multiple API calls
|
|
881
|
-
|
|
882
|
-
---
|
|
883
|
-
|
|
884
|
-
## Endpoint Patterns
|
|
885
|
-
|
|
886
|
-
### 7. Polling Consumer
|
|
887
|
-
|
|
888
|
-
**Pattern:** Periodically poll an endpoint for new messages.
|
|
889
|
-
|
|
890
|
-
**Support Status:** ✅ **Fully Supported** - Can be built using Versori `schedule()` + `Data Sources` + `StateService` for tracking
|
|
891
|
-
|
|
892
|
-
**SDK Building Blocks:**
|
|
893
|
-
- `schedule()` from `@versori/run` - Scheduled execution
|
|
894
|
-
- `StateService` - Track last poll time
|
|
895
|
-
- `SftpDataSource.listFiles()` - Poll for new files
|
|
896
|
-
- `S3DataSource.listFiles()` - Poll S3 buckets
|
|
897
|
-
|
|
898
|
-
#### Use Case: Poll SFTP for New Inventory Files
|
|
899
|
-
|
|
900
|
-
**Business Problem:** Supplier systems don't support webhooks. They drop CSV files to SFTP every 2 hours. Need to check for new files periodically and process them without duplicate processing.
|
|
901
|
-
|
|
902
|
-
Check SFTP every 2 hours for new inventory files, process only new files.
|
|
903
|
-
|
|
904
|
-
```typescript
|
|
905
|
-
import { schedule } from '@versori/run';
|
|
906
|
-
import {
|
|
907
|
-
SftpDataSource,
|
|
908
|
-
CSVParserService,
|
|
909
|
-
UniversalMapper,
|
|
910
|
-
createClient,
|
|
911
|
-
StateService,
|
|
912
|
-
FluentBatchPayload
|
|
913
|
-
} from '@fluentcommerce/fc-connect-sdk';
|
|
914
|
-
|
|
915
|
-
export const pollSftpInventory = schedule('poll-sftp-inventory', '0 */2 * * *', async (ctx) => {
|
|
916
|
-
const { log, openKv, activation } = ctx;
|
|
917
|
-
const client = await createClient({ ...ctx, log });
|
|
918
|
-
|
|
919
|
-
// STEP 1: Connect to SFTP
|
|
920
|
-
const sftp = new SftpDataSource({
|
|
921
|
-
type: 'SFTP_CSV',
|
|
922
|
-
settings: {
|
|
923
|
-
host: activation.getVariable('sftpHost'),
|
|
924
|
-
username: activation.getVariable('sftpUsername'),
|
|
925
|
-
password: activation.getVariable('sftpPassword'),
|
|
926
|
-
remotePath: '/outbound/inventory',
|
|
927
|
-
filePattern: 'inventory_*.csv'
|
|
928
|
-
}
|
|
929
|
-
}, log);
|
|
930
|
-
|
|
931
|
-
try {
|
|
932
|
-
// STEP 2: Poll for files
|
|
933
|
-
const files = await sftp.listFiles();
|
|
934
|
-
log.info('Polled SFTP for inventory files', { fileCount: files.length });
|
|
935
|
-
|
|
936
|
-
if (files.length === 0) {
|
|
937
|
-
log.info('No new files found');
|
|
938
|
-
return { status: 'no_files' };
|
|
939
|
-
}
|
|
940
|
-
|
|
941
|
-
// STEP 3: Check state to identify new files
|
|
942
|
-
const kv = openKv(':project:');
|
|
943
|
-
const stateService = new StateService(log);
|
|
944
|
-
|
|
945
|
-
const newFiles = [];
|
|
946
|
-
for (const file of files) {
|
|
947
|
-
const isProcessed = await kv.get(['processed_files', file.name]);
|
|
948
|
-
if (!isProcessed) {
|
|
949
|
-
newFiles.push(file);
|
|
950
|
-
}
|
|
951
|
-
}
|
|
952
|
-
|
|
953
|
-
log.info('Identified new files', { newFileCount: newFiles.length });
|
|
954
|
-
|
|
955
|
-
if (newFiles.length === 0) {
|
|
956
|
-
return { status: 'no_new_files' };
|
|
957
|
-
}
|
|
958
|
-
|
|
959
|
-
// STEP 4: Process new files
|
|
960
|
-
const csvParser = new CSVParserService(log);
|
|
961
|
-
const mapper = new UniversalMapper({
|
|
962
|
-
fields: {
|
|
963
|
-
ref: { source: 'location_sku', required: true },
|
|
964
|
-
type: { value: 'INVENTORY_QUANTITY', required: true },
|
|
965
|
-
locationRef: { source: 'location', required: true },
|
|
966
|
-
skuRef: { source: 'sku', required: true },
|
|
967
|
-
qty: { source: 'quantity', resolver: 'sdk.parseInt', required: true }
|
|
968
|
-
}
|
|
969
|
-
});
|
|
970
|
-
|
|
971
|
-
let totalEntities = 0;
|
|
972
|
-
|
|
973
|
-
for (const file of newFiles) {
|
|
974
|
-
try {
|
|
975
|
-
const csvContent = await sftp.downloadFile(file.name, { encoding: 'utf8' }) as string;
|
|
976
|
-
const parsed = await csvParser.parse(csvContent, { columns: true });
|
|
977
|
-
|
|
978
|
-
const entities = [];
|
|
979
|
-
for (const record of parsed) {
|
|
980
|
-
const result = await mapper.map(record);
|
|
981
|
-
if (result.success) {
|
|
982
|
-
entities.push(result.data);
|
|
983
|
-
}
|
|
984
|
-
}
|
|
985
|
-
|
|
986
|
-
if (entities.length > 0) {
|
|
987
|
-
const job = await client.createJob({
|
|
988
|
-
name: `inventory-poll-${file.name}-${Date.now()}`,
|
|
989
|
-
retailerId: activation.getVariable('fluentRetailerId')
|
|
990
|
-
});
|
|
991
|
-
|
|
992
|
-
await client.sendBatch(job.id, {
|
|
993
|
-
entityType: 'INVENTORY',
|
|
994
|
-
entities,
|
|
995
|
-
action: 'UPSERT'
|
|
996
|
-
});
|
|
997
|
-
|
|
998
|
-
totalEntities += entities.length;
|
|
999
|
-
}
|
|
1000
|
-
|
|
1001
|
-
// Mark file as processed
|
|
1002
|
-
await kv.set(['processed_files', file.name], {
|
|
1003
|
-
processedAt: new Date().toISOString(),
|
|
1004
|
-
entityCount: entities.length
|
|
1005
|
-
});
|
|
1006
|
-
|
|
1007
|
-
log.info('File processed successfully', { file: file.name, entities: entities.length });
|
|
1008
|
-
} catch (error) {
|
|
1009
|
-
log.error(`Failed to process file: ${file.name}`, error as Error);
|
|
1010
|
-
}
|
|
1011
|
-
}
|
|
1012
|
-
|
|
1013
|
-
return {
|
|
1014
|
-
status: 'success',
|
|
1015
|
-
filesProcessed: newFiles.length,
|
|
1016
|
-
totalEntities
|
|
1017
|
-
};
|
|
1018
|
-
} finally {
|
|
1019
|
-
await sftp.dispose();
|
|
1020
|
-
}
|
|
1021
|
-
});
|
|
1022
|
-
```
|
|
1023
|
-
|
|
1024
|
-
**When to Use:**
|
|
1025
|
-
- Supplier systems don't support webhooks (push notifications)
|
|
1026
|
-
- Scheduled file drops to SFTP/S3 (daily inventory snapshots)
|
|
1027
|
-
- Periodic API polling for new orders/fulfilments
|
|
1028
|
-
|
|
1029
|
-
---
|
|
1030
|
-
|
|
1031
|
-
### 8. Idempotent Receiver
|
|
1032
|
-
|
|
1033
|
-
**Pattern:** Ensure messages are processed exactly once, even if received multiple times.
|
|
1034
|
-
|
|
1035
|
-
**Support Status:** ✅ **Fully Supported** - Can be built using KV Store + `StateService` for idempotency tracking
|
|
1036
|
-
|
|
1037
|
-
**SDK Building Blocks:**
|
|
1038
|
-
- `StateService` - Track processed message IDs
|
|
1039
|
-
- `VersoriKVAdapter` - Distributed state storage
|
|
1040
|
-
- KV Store (`openKv`) - Persistent state storage
|
|
1041
|
-
- Atomic operations for race-free checks
|
|
1042
|
-
|
|
1043
|
-
#### Use Case: Prevent Duplicate Order Processing
|
|
1044
|
-
|
|
1045
|
-
**Business Problem:** Webhook endpoints may receive duplicate deliveries due to network retries or webhook provider retry logic. Need to ensure each order is processed exactly once, even if webhook fires multiple times.
|
|
1046
|
-
|
|
1047
|
-
Ensure each order is processed only once, even with webhook retries.
|
|
1048
|
-
|
|
1049
|
-
```typescript
|
|
1050
|
-
import { webhook } from '@versori/run';
|
|
1051
|
-
import {
|
|
1052
|
-
createClient,
|
|
1053
|
-
StateService,
|
|
1054
|
-
parseWebhookRequest,
|
|
1055
|
-
FluentEvent
|
|
1056
|
-
} from '@fluentcommerce/fc-connect-sdk';
|
|
1057
|
-
|
|
1058
|
-
export const processOrderWebhook = webhook('process-order', {
|
|
1059
|
-
response: { mode: 'sync' }
|
|
1060
|
-
}, async (ctx) => {
|
|
1061
|
-
const { log, openKv, request } = ctx;
|
|
1062
|
-
const client = await createClient({ ...ctx, log });
|
|
1063
|
-
|
|
1064
|
-
// STEP 1: Parse webhook payload
|
|
1065
|
-
const webhookPayload = await parseWebhookRequest(request);
|
|
1066
|
-
const orderRef = webhookPayload.entityRef;
|
|
1067
|
-
const eventId = webhookPayload.id;
|
|
1068
|
-
|
|
1069
|
-
log.info('Received order webhook', { orderRef, eventId });
|
|
1070
|
-
|
|
1071
|
-
// STEP 2: IDEMPOTENCY CHECK - Has this event been processed?
|
|
1072
|
-
const kv = openKv(':project:');
|
|
1073
|
-
const processingKey = ['processed_events', eventId];
|
|
1074
|
-
|
|
1075
|
-
const existingRecord = await kv.get(processingKey);
|
|
1076
|
-
if (existingRecord?.value) {
|
|
1077
|
-
log.warn('Event already processed - skipping', {
|
|
1078
|
-
eventId,
|
|
1079
|
-
orderRef,
|
|
1080
|
-
processedAt: (existingRecord.value as any).processedAt
|
|
1081
|
-
});
|
|
1082
|
-
|
|
1083
|
-
return new Response(JSON.stringify({
|
|
1084
|
-
status: 'already_processed',
|
|
1085
|
-
eventId,
|
|
1086
|
-
processedAt: (existingRecord.value as any).processedAt
|
|
1087
|
-
}), {
|
|
1088
|
-
status: 200,
|
|
1089
|
-
headers: { 'Content-Type': 'application/json' }
|
|
1090
|
-
});
|
|
1091
|
-
}
|
|
1092
|
-
|
|
1093
|
-
// STEP 3: Process order (business logic)
|
|
1094
|
-
try {
|
|
1095
|
-
// Fetch order details
|
|
1096
|
-
const query = `
|
|
1097
|
-
query GetOrder($ref: String!) {
|
|
1098
|
-
order(ref: $ref) {
|
|
1099
|
-
id
|
|
1100
|
-
ref
|
|
1101
|
-
status
|
|
1102
|
-
totalPrice
|
|
1103
|
-
items {
|
|
1104
|
-
edges {
|
|
1105
|
-
node {
|
|
1106
|
-
ref
|
|
1107
|
-
quantity
|
|
1108
|
-
product {
|
|
1109
|
-
ref
|
|
1110
|
-
name
|
|
1111
|
-
}
|
|
1112
|
-
}
|
|
1113
|
-
}
|
|
1114
|
-
}
|
|
1115
|
-
}
|
|
1116
|
-
}
|
|
1117
|
-
`;
|
|
1118
|
-
|
|
1119
|
-
const response = await client.graphql({
|
|
1120
|
-
query,
|
|
1121
|
-
variables: { ref: orderRef }
|
|
1122
|
-
});
|
|
1123
|
-
|
|
1124
|
-
const order = response.data?.order;
|
|
1125
|
-
|
|
1126
|
-
if (!order) {
|
|
1127
|
-
throw new Error(`Order not found: ${orderRef}`);
|
|
1128
|
-
}
|
|
1129
|
-
|
|
1130
|
-
// Business logic: Send fulfilment event if order is confirmed
|
|
1131
|
-
if (order.status === 'CONFIRMED') {
|
|
1132
|
-
const event: FluentEvent = {
|
|
1133
|
-
name: 'order.ready_for_fulfilment',
|
|
1134
|
-
entityType: 'ORDER',
|
|
1135
|
-
entityRef: orderRef,
|
|
1136
|
-
retailerId: webhookPayload.retailerId,
|
|
1137
|
-
attributes: [
|
|
1138
|
-
{ name: 'totalPrice', type: 'FLOAT', value: order.totalPrice },
|
|
1139
|
-
{ name: 'itemCount', type: 'INTEGER', value: order.items?.edges?.length || 0 }
|
|
1140
|
-
]
|
|
1141
|
-
};
|
|
1142
|
-
|
|
1143
|
-
await client.sendEvent(event, 'async');
|
|
1144
|
-
}
|
|
1145
|
-
|
|
1146
|
-
// STEP 4: Mark event as processed (IDEMPOTENCY RECORD)
|
|
1147
|
-
await kv.set(processingKey, {
|
|
1148
|
-
processedAt: new Date().toISOString(),
|
|
1149
|
-
orderRef,
|
|
1150
|
-
eventId,
|
|
1151
|
-
status: 'success'
|
|
1152
|
-
});
|
|
1153
|
-
|
|
1154
|
-
log.info('Order processed successfully', { orderRef, eventId });
|
|
1155
|
-
|
|
1156
|
-
return new Response(JSON.stringify({
|
|
1157
|
-
status: 'processed',
|
|
1158
|
-
orderRef,
|
|
1159
|
-
eventId
|
|
1160
|
-
}), {
|
|
1161
|
-
status: 200,
|
|
1162
|
-
headers: { 'Content-Type': 'application/json' }
|
|
1163
|
-
});
|
|
1164
|
-
} catch (error) {
|
|
1165
|
-
log.error('Order processing failed', error as Error, { orderRef, eventId });
|
|
1166
|
-
|
|
1167
|
-
// Don't mark as processed on failure - allow retry
|
|
1168
|
-
return new Response(JSON.stringify({
|
|
1169
|
-
status: 'error',
|
|
1170
|
-
message: (error as Error).message
|
|
1171
|
-
}), {
|
|
1172
|
-
status: 500,
|
|
1173
|
-
headers: { 'Content-Type': 'application/json' }
|
|
1174
|
-
});
|
|
1175
|
-
}
|
|
1176
|
-
});
|
|
1177
|
-
```
|
|
1178
|
-
|
|
1179
|
-
**When to Use:**
|
|
1180
|
-
- Webhook endpoints that may receive duplicate deliveries
|
|
1181
|
-
- File processing where same file might be reprocessed
|
|
1182
|
-
- Distributed workflows with at-least-once delivery guarantees
|
|
1183
|
-
|
|
1184
|
-
---
|
|
1185
|
-
|
|
1186
|
-
## System Management Patterns
|
|
1187
|
-
|
|
1188
|
-
### 9. Dead Letter Channel
|
|
1189
|
-
|
|
1190
|
-
**Pattern:** Route failed messages to a separate channel for inspection and reprocessing.
|
|
1191
|
-
|
|
1192
|
-
**Support Status:** ✅ **Fully Supported** - Can be built using `S3DataSource` or `SftpDataSource` for failed message storage
|
|
1193
|
-
|
|
1194
|
-
**SDK Building Blocks:**
|
|
1195
|
-
- `S3DataSource.uploadFile()` - Write failed messages to S3
|
|
1196
|
-
- `SftpDataSource.writeFile()` - Write failed messages to SFTP
|
|
1197
|
-
- `StateService` - Track failure counts
|
|
1198
|
-
- Error handling with try/catch
|
|
1199
|
-
|
|
1200
|
-
#### Use Case: Route Failed Inventory Updates to Dead Letter S3 Bucket
|
|
1201
|
-
|
|
1202
|
-
**Business Problem:** Some inventory records fail validation (missing SKU, invalid quantity, etc.). Need to capture these failures for manual review and potential reprocessing without blocking successful records.
|
|
1203
|
-
|
|
1204
|
-
Capture failed inventory records for manual review and reprocessing.
|
|
1205
|
-
|
|
1206
|
-
```typescript
|
|
1207
|
-
import {
|
|
1208
|
-
SftpDataSource,
|
|
1209
|
-
CSVParserService,
|
|
1210
|
-
UniversalMapper,
|
|
1211
|
-
S3DataSource,
|
|
1212
|
-
createClient,
|
|
1213
|
-
FluentBatchPayload
|
|
1214
|
-
} from '@fluentcommerce/fc-connect-sdk';
|
|
1215
|
-
import { Buffer } from 'node:buffer';
|
|
1216
|
-
|
|
1217
|
-
async function processInventoryWithDeadLetterChannel(ctx: any, log: any) {
|
|
1218
|
-
const client = await createClient({ ...ctx, log });
|
|
1219
|
-
|
|
1220
|
-
// Setup: SFTP source, S3 dead letter destination
|
|
1221
|
-
const sftp = new SftpDataSource({
|
|
1222
|
-
type: 'SFTP_CSV',
|
|
1223
|
-
settings: {
|
|
1224
|
-
host: ctx.activation.getVariable('sftpHost'),
|
|
1225
|
-
username: ctx.activation.getVariable('sftpUsername'),
|
|
1226
|
-
password: ctx.activation.getVariable('sftpPassword'),
|
|
1227
|
-
remotePath: '/inbound/inventory',
|
|
1228
|
-
filePattern: '*.csv'
|
|
1229
|
-
}
|
|
1230
|
-
}, log);
|
|
1231
|
-
|
|
1232
|
-
const deadLetterS3 = new S3DataSource({
|
|
1233
|
-
type: 'S3_JSON',
|
|
1234
|
-
s3Config: {
|
|
1235
|
-
bucket: ctx.activation.getVariable('deadLetterBucket'),
|
|
1236
|
-
region: ctx.activation.getVariable('awsRegion'),
|
|
1237
|
-
accessKeyId: ctx.activation.getVariable('awsAccessKeyId'),
|
|
1238
|
-
secretAccessKey: ctx.activation.getVariable('awsSecretAccessKey')
|
|
1239
|
-
}
|
|
1240
|
-
}, log);
|
|
1241
|
-
|
|
1242
|
-
try {
|
|
1243
|
-
const files = await sftp.listFiles();
|
|
1244
|
-
if (files.length === 0) {
|
|
1245
|
-
log.info('No files to process');
|
|
1246
|
-
return;
|
|
1247
|
-
}
|
|
1248
|
-
|
|
1249
|
-
const csvContent = await sftp.downloadFile(files[0].name, { encoding: 'utf8' }) as string;
|
|
1250
|
-
const csvParser = new CSVParserService(log);
|
|
1251
|
-
const parsed = await csvParser.parse(csvContent, { columns: true });
|
|
1252
|
-
|
|
1253
|
-
// Map inventory records
|
|
1254
|
-
const mapper = new UniversalMapper({
|
|
1255
|
-
fields: {
|
|
1256
|
-
ref: { source: 'location_sku', required: true },
|
|
1257
|
-
type: { value: 'INVENTORY_QUANTITY', required: true },
|
|
1258
|
-
locationRef: { source: 'location', required: true },
|
|
1259
|
-
skuRef: { source: 'sku', required: true },
|
|
1260
|
-
qty: { source: 'quantity', resolver: 'sdk.parseInt', required: true }
|
|
1261
|
-
}
|
|
1262
|
-
});
|
|
1263
|
-
|
|
1264
|
-
const successfulEntities = [];
|
|
1265
|
-
const failedRecords = [];
|
|
1266
|
-
|
|
1267
|
-
// Process records with error tracking
|
|
1268
|
-
for (const record of parsed) {
|
|
1269
|
-
try {
|
|
1270
|
-
const result = await mapper.map(record);
|
|
1271
|
-
if (result.success) {
|
|
1272
|
-
successfulEntities.push(result.data);
|
|
1273
|
-
} else {
|
|
1274
|
-
// DEAD LETTER: Mapping validation failed
|
|
1275
|
-
failedRecords.push({
|
|
1276
|
-
sourceRecord: record,
|
|
1277
|
-
errors: result.errors,
|
|
1278
|
-
failureReason: 'mapping_validation_failed',
|
|
1279
|
-
failedAt: new Date().toISOString()
|
|
1280
|
-
});
|
|
1281
|
-
}
|
|
1282
|
-
} catch (error) {
|
|
1283
|
-
// DEAD LETTER: Unexpected error during mapping
|
|
1284
|
-
failedRecords.push({
|
|
1285
|
-
sourceRecord: record,
|
|
1286
|
-
errors: [(error as Error).message],
|
|
1287
|
-
failureReason: 'mapping_error',
|
|
1288
|
-
failedAt: new Date().toISOString(),
|
|
1289
|
-
errorStack: (error as Error).stack
|
|
1290
|
-
});
|
|
1291
|
-
}
|
|
1292
|
-
}
|
|
1293
|
-
|
|
1294
|
-
// Send successful records to Batch API
|
|
1295
|
-
if (successfulEntities.length > 0) {
|
|
1296
|
-
const job = await client.createJob({
|
|
1297
|
-
name: `inventory-with-dlq-${Date.now()}`,
|
|
1298
|
-
retailerId: ctx.activation.getVariable('fluentRetailerId')
|
|
1299
|
-
});
|
|
1300
|
-
|
|
1301
|
-
await client.sendBatch(job.id, {
|
|
1302
|
-
entityType: 'INVENTORY',
|
|
1303
|
-
entities: successfulEntities,
|
|
1304
|
-
action: 'UPSERT'
|
|
1305
|
-
});
|
|
1306
|
-
|
|
1307
|
-
log.info('Successful entities sent to Batch API', {
|
|
1308
|
-
count: successfulEntities.length,
|
|
1309
|
-
jobId: job.id
|
|
1310
|
-
});
|
|
1311
|
-
}
|
|
1312
|
-
|
|
1313
|
-
// DEAD LETTER CHANNEL: Write failed records to S3
|
|
1314
|
-
if (failedRecords.length > 0) {
|
|
1315
|
-
const deadLetterKey = `failed-inventory/${files[0].name}-failures-${Date.now()}.json`;
|
|
1316
|
-
const deadLetterContent = JSON.stringify({
|
|
1317
|
-
sourceFile: files[0].name,
|
|
1318
|
-
failedAt: new Date().toISOString(),
|
|
1319
|
-
totalFailed: failedRecords.length,
|
|
1320
|
-
records: failedRecords
|
|
1321
|
-
}, null, 2);
|
|
1322
|
-
|
|
1323
|
-
await deadLetterS3.uploadFile(
|
|
1324
|
-
deadLetterKey,
|
|
1325
|
-
Buffer.from(deadLetterContent, 'utf-8')
|
|
1326
|
-
);
|
|
1327
|
-
|
|
1328
|
-
log.warn('Failed records written to Dead Letter S3', {
|
|
1329
|
-
count: failedRecords.length,
|
|
1330
|
-
deadLetterKey
|
|
1331
|
-
});
|
|
1332
|
-
}
|
|
1333
|
-
|
|
1334
|
-
return {
|
|
1335
|
-
successful: successfulEntities.length,
|
|
1336
|
-
failed: failedRecords.length,
|
|
1337
|
-
deadLetterKey: failedRecords.length > 0 ? `failed-inventory/${files[0].name}-failures-${Date.now()}.json` : null
|
|
1338
|
-
};
|
|
1339
|
-
} finally {
|
|
1340
|
-
await sftp.dispose();
|
|
1341
|
-
}
|
|
1342
|
-
}
|
|
1343
|
-
```
|
|
1344
|
-
|
|
1345
|
-
**When to Use:**
|
|
1346
|
-
- Data validation failures need manual review
|
|
1347
|
-
- Partial batch failures require reprocessing
|
|
1348
|
-
- Audit trail for failed messages is required
|
|
1349
|
-
|
|
1350
|
-
---
|
|
1351
|
-
|
|
1352
|
-
### 10. Wire Tap
|
|
1353
|
-
|
|
1354
|
-
**Pattern:** Inspect messages passing through without altering them (audit logging).
|
|
1355
|
-
|
|
1356
|
-
**Support Status:** ✅ **Fully Supported** - Can be built using `S3DataSource` or logging for audit trail
|
|
1357
|
-
|
|
1358
|
-
**SDK Building Blocks:**
|
|
1359
|
-
- `S3DataSource.uploadFile()` - Archive message copies
|
|
1360
|
-
- Logging for message inspection
|
|
1361
|
-
- No modification to message flow
|
|
1362
|
-
|
|
1363
|
-
#### Use Case: Audit All Order Events Sent to Event API
|
|
1364
|
-
|
|
1365
|
-
**Business Problem:** Compliance requirements mandate audit trail of all order events sent to Fluent Commerce. Need to log message contents without affecting processing performance or modifying message flow.
|
|
1366
|
-
|
|
1367
|
-
Log all order events to S3 for compliance auditing.
|
|
1368
|
-
|
|
1369
|
-
```typescript
|
|
1370
|
-
import {
|
|
1371
|
-
createClient,
|
|
1372
|
-
FluentEvent,
|
|
1373
|
-
S3DataSource
|
|
1374
|
-
} from '@fluentcommerce/fc-connect-sdk';
|
|
1375
|
-
import { Buffer } from 'node:buffer';
|
|
1376
|
-
|
|
1377
|
-
async function sendOrderEventWithAudit(orderEvent: FluentEvent, ctx: any, log: any) {
|
|
1378
|
-
const client = await createClient({ ...ctx, log });
|
|
1379
|
-
|
|
1380
|
-
// Setup audit S3 bucket
|
|
1381
|
-
const auditS3 = new S3DataSource({
|
|
1382
|
-
type: 'S3_JSON',
|
|
1383
|
-
s3Config: {
|
|
1384
|
-
bucket: ctx.activation.getVariable('auditBucket'),
|
|
1385
|
-
region: ctx.activation.getVariable('awsRegion'),
|
|
1386
|
-
accessKeyId: ctx.activation.getVariable('awsAccessKeyId'),
|
|
1387
|
-
secretAccessKey: ctx.activation.getVariable('awsSecretAccessKey')
|
|
1388
|
-
}
|
|
1389
|
-
}, log);
|
|
1390
|
-
|
|
1391
|
-
try {
|
|
1392
|
-
// WIRE TAP: Capture message before sending (NO MODIFICATION)
|
|
1393
|
-
const auditRecord = {
|
|
1394
|
-
timestamp: new Date().toISOString(),
|
|
1395
|
-
eventType: 'order_event',
|
|
1396
|
-
direction: 'outbound',
|
|
1397
|
-
destination: 'fluent_event_api',
|
|
1398
|
-
payload: orderEvent,
|
|
1399
|
-
metadata: {
|
|
1400
|
-
entityRef: orderEvent.entityRef,
|
|
1401
|
-
eventName: orderEvent.name,
|
|
1402
|
-
retailerId: orderEvent.retailerId
|
|
1403
|
-
}
|
|
1404
|
-
};
|
|
1405
|
-
|
|
1406
|
-
// Write audit record to S3 (asynchronously, don't block main flow)
|
|
1407
|
-
const auditKey = `audit/order-events/${orderEvent.entityRef}-${Date.now()}.json`;
|
|
1408
|
-
const auditPromise = auditS3.uploadFile(
|
|
1409
|
-
auditKey,
|
|
1410
|
-
Buffer.from(JSON.stringify(auditRecord, null, 2), 'utf-8')
|
|
1411
|
-
).catch((error) => {
|
|
1412
|
-
// Audit failure should NOT break main flow
|
|
1413
|
-
log.error('Audit logging failed', error as Error, { auditKey });
|
|
1414
|
-
});
|
|
1415
|
-
|
|
1416
|
-
// Send event to Fluent (main flow continues)
|
|
1417
|
-
const eventResponse = await client.sendEvent(orderEvent, 'async');
|
|
1418
|
-
|
|
1419
|
-
// Wait for audit to complete (optional - for testing)
|
|
1420
|
-
await auditPromise;
|
|
1421
|
-
|
|
1422
|
-
log.info('Order event sent with audit', {
|
|
1423
|
-
entityRef: orderEvent.entityRef,
|
|
1424
|
-
eventId: eventResponse,
|
|
1425
|
-
auditKey
|
|
1426
|
-
});
|
|
1427
|
-
|
|
1428
|
-
return eventResponse;
|
|
1429
|
-
} finally {
|
|
1430
|
-
// No disposal needed - audit is fire-and-forget
|
|
1431
|
-
}
|
|
1432
|
-
}
|
|
1433
|
-
|
|
1434
|
-
// Usage example
|
|
1435
|
-
async function processOrder(order: any, ctx: any, log: any) {
|
|
1436
|
-
const orderEvent: FluentEvent = {
|
|
1437
|
-
name: 'order.created',
|
|
1438
|
-
entityType: 'ORDER',
|
|
1439
|
-
entityRef: order.ref,
|
|
1440
|
-
retailerId: order.retailerId,
|
|
1441
|
-
attributes: [
|
|
1442
|
-
{ name: 'totalPrice', type: 'FLOAT', value: order.totalPrice },
|
|
1443
|
-
{ name: 'customerEmail', type: 'STRING', value: order.customer.email }
|
|
1444
|
-
]
|
|
1445
|
-
};
|
|
1446
|
-
|
|
1447
|
-
// WIRE TAP in action - event is audited without modification
|
|
1448
|
-
await sendOrderEventWithAudit(orderEvent, ctx, log);
|
|
1449
|
-
}
|
|
1450
|
-
```
|
|
1451
|
-
|
|
1452
|
-
**When to Use:**
|
|
1453
|
-
- Compliance requires message audit trails
|
|
1454
|
-
- Debugging integration flows (inspect message contents)
|
|
1455
|
-
- Performance monitoring (message throughput, latency)
|
|
1456
|
-
|
|
1457
|
-
---
|
|
1458
|
-
|
|
1459
|
-
## Pattern Summary Matrix
|
|
1460
|
-
|
|
1461
|
-
| Pattern | Support Status | Primary Purpose | SDK Building Blocks | Fluent Use Case |
|
|
1462
|
-
|---------|---------------|----------------|-------------------|----------------|
|
|
1463
|
-
| **Message Router** | ✅ Fully Supported | Route to different APIs | `sendEvent()`, `createJob()`, `graphql()` | Route orders by type to Event/Batch/GraphQL |
|
|
1464
|
-
| **Content-Based Router** | ✅ Fully Supported | Route based on data values | `UniversalMapper`, conditional logic | Route inventory by quantity threshold |
|
|
1465
|
-
| **Content Enricher** | ✅ Fully Supported | Add data from external sources | `graphql()`, `UniversalMapper` | Enrich orders with product details |
|
|
1466
|
-
| **Message Translator** | ✅ Fully Supported | Convert between formats | `CSVParserService`, `UniversalMapper`, `XMLBuilder` | Translate supplier CSV to Fluent Batch format |
|
|
1467
|
-
| **Splitter** | ✅ Fully Supported | Break message into parts | `sendBatch()`, `Promise.all()`, Parsers | Split large order file into parallel batches |
|
|
1468
|
-
| **Aggregator** | ✅ Fully Supported | Combine multiple messages | `graphql()` with pagination, Map/Set | Aggregate inventory across locations |
|
|
1469
|
-
| **Polling Consumer** | ✅ Fully Supported | Periodic endpoint polling | `schedule()`, `SftpDataSource.listFiles()` | Poll SFTP every 2 hours for inventory files |
|
|
1470
|
-
| **Idempotent Receiver** | ✅ Fully Supported | Prevent duplicate processing | `StateService`, KV Store, atomic ops | Prevent duplicate webhook order processing |
|
|
1471
|
-
| **Dead Letter Channel** | ✅ Fully Supported | Route failed messages | `S3DataSource.uploadFile()`, error handling | Failed inventory updates to S3 for review |
|
|
1472
|
-
| **Wire Tap** | ✅ Fully Supported | Audit messages (no modification) | `S3DataSource.uploadFile()`, logging | Audit all order events to S3 for compliance |
|
|
1473
|
-
|
|
1474
|
-
**Support Status Legend:**
|
|
1475
|
-
- ✅ **Fully Supported** - Can be built using SDK building blocks with complete examples
|
|
1476
|
-
- ⚠️ **Partially Supported** - Requires additional custom code beyond SDK
|
|
1477
|
-
- ❌ **Not Supported** - Not achievable with current SDK capabilities
|
|
1478
|
-
|
|
1479
|
-
---
|
|
1480
|
-
|
|
1481
|
-
## Additional Resources
|
|
1482
|
-
|
|
1483
|
-
### SDK Documentation
|
|
1484
|
-
- **Architecture**: `docs/04-REFERENCE/architecture/readme.md` - System design and data flow
|
|
1485
|
-
- **Decision Tree**: `docs/00-START-HERE/DECISION-TREE.md` - Which approach to use?
|
|
1486
|
-
- **Troubleshooting**: `docs/00-START-HERE/troubleshooting-quick-reference.md` - Common issues
|
|
1487
|
-
|
|
1488
|
-
### Core Guides
|
|
1489
|
-
- **Ingestion**: `docs/02-CORE-GUIDES/ingestion/readme.md` - Data into Fluent
|
|
1490
|
-
- **Extraction**: `docs/02-CORE-GUIDES/extraction/readme.md` - Data from Fluent
|
|
1491
|
-
- **Mapping**: `docs/02-CORE-GUIDES/mapping/modules/` - Field transformation
|
|
1492
|
-
- **Webhook Validation**: `docs/02-CORE-GUIDES/webhook-validation/readme.md` - Signature validation
|
|
1493
|
-
|
|
1494
|
-
### Pattern Guides
|
|
1495
|
-
- **Error Handling**: `docs/03-PATTERN-GUIDES/error-handling/readme.md` - Retry, circuit breakers
|
|
1496
|
-
- **File Operations**: `docs/03-PATTERN-GUIDES/file-operations/readme.md` - S3/SFTP patterns
|
|
1497
|
-
- **Integration Patterns**: `docs/03-PATTERN-GUIDES/integration-patterns/readme.md` - Real-time, batch, delta sync
|
|
1498
|
-
|
|
1499
|
-
### Templates
|
|
1500
|
-
- **Ingestion Templates**: `docs/01-TEMPLATES/versori/scheduled/ingestion/` - Complete examples
|
|
1501
|
-
- **Extraction Templates**: `docs/01-TEMPLATES/versori/scheduled/extraction/` - GraphQL extraction patterns
|
|
1502
|
-
|
|
1503
|
-
---
|
|
1504
|
-
|
|
1505
|
-
## Summary
|
|
1506
|
-
|
|
1507
|
-
This guide demonstrated how to build 10 essential Enterprise Integration Patterns using Fluent Connect SDK building blocks:
|
|
1508
|
-
|
|
1509
|
-
**Key Takeaways:**
|
|
1510
|
-
|
|
1511
|
-
1. **SDK is a Toolkit** - Patterns are YOUR orchestration logic, not framework-imposed
|
|
1512
|
-
2. **Composable Primitives** - Combine clients, parsers, mappers, data sources to build patterns
|
|
1513
|
-
3. **Real Fluent Domains** - All examples use actual Order and Inventory entities
|
|
1514
|
-
4. **Production-Ready Code** - Every example uses real SDK methods and types
|
|
1515
|
-
|
|
1516
|
-
**Next Steps:**
|
|
1517
|
-
|
|
1518
|
-
- Review `docs/00-START-HERE/DECISION-TREE.md` for pattern selection guidance
|
|
1519
|
-
- Explore `docs/01-TEMPLATES/` for complete working implementations
|
|
1520
|
-
- See `docs/03-PATTERN-GUIDES/integration-patterns/` for additional pattern details
|
|
1521
|
-
|
|
1522
|
-
**Remember:** The SDK provides the tools. YOU define the integration patterns.
|
|
1523
|
-
|
|
1524
|
-
---
|
|
1525
|
-
|
|
1526
|
-
**Document Version:** 1.0.0
|
|
1527
|
-
**SDK Version:** 0.1.39
|
|
1528
|
-
**Last Verified:** 2025-11-05
|
|
1
|
+
# Enterprise Integration Patterns with Fluent Connect SDK
|
|
2
|
+
|
|
3
|
+
**Status:** Production Ready
|
|
4
|
+
**Last Updated:** 2025-11-05
|
|
5
|
+
**SDK Version:** 0.1.39
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Table of Contents
|
|
10
|
+
|
|
11
|
+
1. [Introduction](#introduction)
|
|
12
|
+
2. [Understanding the SDK as a Toolkit](#understanding-the-sdk-as-a-toolkit)
|
|
13
|
+
3. [Pattern Categories](#pattern-categories)
|
|
14
|
+
4. [Routing Patterns](#routing-patterns)
|
|
15
|
+
5. [Transformation Patterns](#transformation-patterns)
|
|
16
|
+
6. [Endpoint Patterns](#endpoint-patterns)
|
|
17
|
+
7. [System Management Patterns](#system-management-patterns)
|
|
18
|
+
8. [Pattern Summary Matrix](#pattern-summary-matrix)
|
|
19
|
+
9. [Additional Resources](#additional-resources)
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Introduction
|
|
24
|
+
|
|
25
|
+
### What is Enterprise Integration Patterns (EIP)?
|
|
26
|
+
|
|
27
|
+
**Enterprise Integration Patterns (EIP)** are a collection of 65 proven design patterns for building enterprise application integration solutions. Originally documented by Gregor Hohpe and Bobby Woolf in their book ["Enterprise Integration Patterns"](https://www.enterpriseintegrationpatterns.com/), these patterns provide reusable solutions to common integration challenges.
|
|
28
|
+
|
|
29
|
+
**What EIP Means in TypeScript Context:**
|
|
30
|
+
|
|
31
|
+
In TypeScript/JavaScript applications, EIP patterns translate to:
|
|
32
|
+
- **Composable functions and services** - Building blocks you combine to solve integration problems
|
|
33
|
+
- **Message transformation** - Converting data between different formats (XML, JSON, CSV)
|
|
34
|
+
- **Routing logic** - Directing messages to appropriate destinations based on content
|
|
35
|
+
- **Error handling** - Managing failures and retries in distributed systems
|
|
36
|
+
- **State management** - Tracking processed messages and maintaining idempotency
|
|
37
|
+
|
|
38
|
+
**Key Concepts:**
|
|
39
|
+
- **Patterns are solutions**, not libraries - You implement them using your toolkit
|
|
40
|
+
- **SDK provides building blocks** - You compose them to create patterns
|
|
41
|
+
- **TypeScript enables composition** - Functions, classes, and async/await make patterns natural
|
|
42
|
+
|
|
43
|
+
**Learn More:**
|
|
44
|
+
- 📚 [Enterprise Integration Patterns Book](https://www.enterpriseintegrationpatterns.com/)
|
|
45
|
+
- 🌐 [EIP Pattern Catalog](https://www.enterpriseintegrationpatterns.com/patterns/messaging/)
|
|
46
|
+
- 📖 [EIP Wikipedia](https://en.wikipedia.org/wiki/Enterprise_Integration_Patterns)
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
**Enterprise Integration Patterns (EIP)** are proven solutions to common integration challenges. This guide shows how to build these patterns using the Fluent Connect SDK's composable building blocks.
|
|
51
|
+
|
|
52
|
+
### What This Guide Covers
|
|
53
|
+
|
|
54
|
+
- **10 Essential EIP Patterns** with real Fluent Commerce use cases
|
|
55
|
+
- **Practical SDK Implementation** using actual client methods and services
|
|
56
|
+
- **Order & Inventory Domain Examples** - the two primary Fluent Commerce domains
|
|
57
|
+
- **Support Status** - Whether each pattern can be built with SDK building blocks
|
|
58
|
+
- **Use-Case-Driven Approach** - Real-world scenarios for each pattern
|
|
59
|
+
|
|
60
|
+
### Prerequisites
|
|
61
|
+
|
|
62
|
+
- Understanding of SDK architecture (see `docs/00-START-HERE/SDK-PHILOSOPHY.md`)
|
|
63
|
+
- Familiarity with Fluent Commerce entities (Order, Fulfilment, Inventory)
|
|
64
|
+
- Basic knowledge of GraphQL and Batch API concepts
|
|
65
|
+
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
## Understanding the SDK as a Toolkit
|
|
69
|
+
|
|
70
|
+
> **The SDK provides services with methods. YOU build the workflow.**
|
|
71
|
+
|
|
72
|
+
The FC Connect SDK is **not an opinionated framework** - it's a **toolkit of composable primitives**. Enterprise Integration Patterns are YOUR orchestration logic built on top of SDK services.
|
|
73
|
+
|
|
74
|
+
### Core Building Blocks
|
|
75
|
+
|
|
76
|
+
```
|
|
77
|
+
Data Sources Parsers Transformation
|
|
78
|
+
├── S3DataSource ├── CSVParserService ├── UniversalMapper
|
|
79
|
+
├── SftpDataSource ├── XMLParserService ├── GraphQLMutationMapper
|
|
80
|
+
└── InventoryDataSource ├── JSONParserService └── sdkResolvers
|
|
81
|
+
└── ParquetParserService
|
|
82
|
+
|
|
83
|
+
Client Services State Management
|
|
84
|
+
├── FluentClient ├── BatchAPIClient ├── StateService
|
|
85
|
+
├── createClient() ├── S3PresignService ├── VersoriKVAdapter
|
|
86
|
+
├── graphql() ├── WebhookValidationService
|
|
87
|
+
├── sendEvent() └── SchemaValidationService
|
|
88
|
+
├── createJob()
|
|
89
|
+
├── sendBatch()
|
|
90
|
+
└── getJobStatus()
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
### The Fundamental Pattern
|
|
94
|
+
|
|
95
|
+
```
|
|
96
|
+
Read → Parse → Map → YOUR LOGIC → Send/Archive
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Enterprise Integration Patterns sit in the **"YOUR LOGIC"** layer - they define HOW you orchestrate these primitives.
|
|
100
|
+
|
|
101
|
+
---
|
|
102
|
+
|
|
103
|
+
## Pattern Categories
|
|
104
|
+
|
|
105
|
+
We organize EIP patterns into four categories based on their primary purpose:
|
|
106
|
+
|
|
107
|
+
1. **Routing Patterns** - Direct messages to appropriate destinations
|
|
108
|
+
2. **Transformation Patterns** - Convert data between formats
|
|
109
|
+
3. **Endpoint Patterns** - Connect to external systems
|
|
110
|
+
4. **System Management Patterns** - Handle reliability and state
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## Routing Patterns
|
|
115
|
+
|
|
116
|
+
### 1. Message Router
|
|
117
|
+
|
|
118
|
+
**Pattern:** Route messages to different destinations based on message content or type.
|
|
119
|
+
|
|
120
|
+
**Support Status:** ✅ **Fully Supported** - Can be built using SDK client methods
|
|
121
|
+
|
|
122
|
+
**SDK Building Blocks:**
|
|
123
|
+
- `FluentClient.sendEvent()` - Send to Event API
|
|
124
|
+
- `FluentClient.createJob()` + `sendBatch()` - Send to Batch API
|
|
125
|
+
- `FluentClient.graphql()` - Send to GraphQL API
|
|
126
|
+
|
|
127
|
+
#### Use Case: Order Routing Based on Type
|
|
128
|
+
|
|
129
|
+
**Business Problem:** Different order types require different processing workflows. Click & Collect orders need immediate workflow triggers, while Home Delivery orders can be batched for efficiency.
|
|
130
|
+
|
|
131
|
+
Route orders to different workflows based on order type:
|
|
132
|
+
- Click & Collect → Event API (trigger Rubix workflow)
|
|
133
|
+
- Home Delivery → Batch API (bulk processing)
|
|
134
|
+
- Store Pickup → GraphQL mutation (immediate processing)
|
|
135
|
+
|
|
136
|
+
```typescript
|
|
137
|
+
import { createClient, FluentEvent, FluentBatchPayload } from '@fluentcommerce/fc-connect-sdk';
|
|
138
|
+
|
|
139
|
+
async function routeOrders(orders: any[], ctx: any, log: any) {
|
|
140
|
+
const client = await createClient({ ...ctx, log });
|
|
141
|
+
|
|
142
|
+
for (const order of orders) {
|
|
143
|
+
const orderType = order.attributes?.find((a: any) => a.name === 'type')?.value;
|
|
144
|
+
|
|
145
|
+
// ROUTING LOGIC
|
|
146
|
+
switch (orderType) {
|
|
147
|
+
case 'CLICK_AND_COLLECT':
|
|
148
|
+
// Route to Event API → Triggers Rubix workflow
|
|
149
|
+
const event: FluentEvent = {
|
|
150
|
+
name: 'order.created',
|
|
151
|
+
entityType: 'ORDER',
|
|
152
|
+
entityRef: order.ref,
|
|
153
|
+
retailerId: order.retailerId,
|
|
154
|
+
attributes: order.attributes
|
|
155
|
+
};
|
|
156
|
+
await client.sendEvent(event, 'async');
|
|
157
|
+
log.info('Order routed to Event API', { ref: order.ref, type: orderType });
|
|
158
|
+
break;
|
|
159
|
+
|
|
160
|
+
case 'HOME_DELIVERY':
|
|
161
|
+
// Route to Event API → Trigger workflow (Batch API only supports INVENTORY)
|
|
162
|
+
const event: FluentEvent = {
|
|
163
|
+
name: 'order.created',
|
|
164
|
+
entityType: 'ORDER',
|
|
165
|
+
entityRef: order.ref,
|
|
166
|
+
retailerId: order.retailerId,
|
|
167
|
+
attributes: order.attributes
|
|
168
|
+
};
|
|
169
|
+
await client.sendEvent(event, 'async');
|
|
170
|
+
log.info('Order routed to Event API', { ref: order.ref, type: orderType });
|
|
171
|
+
break;
|
|
172
|
+
|
|
173
|
+
case 'STORE_PICKUP':
|
|
174
|
+
// Route to GraphQL → Immediate processing
|
|
175
|
+
const mutation = `
|
|
176
|
+
mutation CreateOrder($input: CreateOrderInput!) {
|
|
177
|
+
createOrder(input: $input) {
|
|
178
|
+
id
|
|
179
|
+
ref
|
|
180
|
+
status
|
|
181
|
+
}
|
|
182
|
+
}
|
|
183
|
+
`;
|
|
184
|
+
|
|
185
|
+
await client.mutate(mutation, { input: order });
|
|
186
|
+
log.info('Order routed to GraphQL', { ref: order.ref });
|
|
187
|
+
break;
|
|
188
|
+
|
|
189
|
+
default:
|
|
190
|
+
log.warn('Unknown order type - routing to default Event API', { ref: order.ref, type: orderType });
|
|
191
|
+
// Default route to Event API (Batch API only supports INVENTORY)
|
|
192
|
+
const defaultEvent: FluentEvent = {
|
|
193
|
+
name: 'order.created',
|
|
194
|
+
entityType: 'ORDER',
|
|
195
|
+
entityRef: order.ref,
|
|
196
|
+
retailerId: order.retailerId,
|
|
197
|
+
attributes: order.attributes
|
|
198
|
+
};
|
|
199
|
+
await client.sendEvent(defaultEvent, 'async');
|
|
200
|
+
}
|
|
201
|
+
}
|
|
202
|
+
}
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
**When to Use:**
|
|
206
|
+
- Multiple destination APIs for different message types
|
|
207
|
+
- Order type dictates processing workflow (express, standard, backorder)
|
|
208
|
+
- Inventory updates need different handling by location type (store, warehouse, virtual)
|
|
209
|
+
|
|
210
|
+
---
|
|
211
|
+
|
|
212
|
+
### 2. Content-Based Router
|
|
213
|
+
|
|
214
|
+
**Pattern:** Route messages based on content inspection (attributes, fields, computed values).
|
|
215
|
+
|
|
216
|
+
**Support Status:** ✅ **Fully Supported** - Can be built using `UniversalMapper` and application logic
|
|
217
|
+
|
|
218
|
+
**SDK Building Blocks:**
|
|
219
|
+
- `UniversalMapper` - Extract and transform routing criteria
|
|
220
|
+
- Conditional logic based on extracted values
|
|
221
|
+
|
|
222
|
+
#### Use Case: Inventory Distribution by Quantity
|
|
223
|
+
|
|
224
|
+
**Business Problem:** High-volume inventory updates should use Batch API for efficiency, while low-quantity alerts need immediate processing via GraphQL to trigger restocking workflows.
|
|
225
|
+
|
|
226
|
+
Route inventory updates based on available quantity:
|
|
227
|
+
- High quantity (>1000) → Batch API (efficient bulk updates)
|
|
228
|
+
- Medium quantity (100-1000) → Event API (standard processing)
|
|
229
|
+
- Low quantity (<100) → GraphQL + Event (immediate alert + workflow trigger)
|
|
230
|
+
|
|
231
|
+
```typescript
|
|
232
|
+
import { createClient, UniversalMapper, FluentBatchPayload } from '@fluentcommerce/fc-connect-sdk';
|
|
233
|
+
|
|
234
|
+
async function routeInventoryUpdates(inventoryData: any[], ctx: any, log: any) {
|
|
235
|
+
const client = await createClient({ ...ctx, log });
|
|
236
|
+
|
|
237
|
+
// Extract routing criteria using UniversalMapper
|
|
238
|
+
const mapper = new UniversalMapper({
|
|
239
|
+
fields: {
|
|
240
|
+
skuRef: { source: 'sku', required: true },
|
|
241
|
+
locationRef: { source: 'location', required: true },
|
|
242
|
+
qty: { source: 'quantity', resolver: 'sdk.parseInt', required: true },
|
|
243
|
+
availableQty: { source: 'available', resolver: 'sdk.parseInt' }
|
|
244
|
+
}
|
|
245
|
+
});
|
|
246
|
+
|
|
247
|
+
const highQtyBatch: any[] = [];
|
|
248
|
+
const mediumQtyEvents: any[] = [];
|
|
249
|
+
const lowQtyAlerts: any[] = [];
|
|
250
|
+
|
|
251
|
+
// Content-based routing logic
|
|
252
|
+
for (const item of inventoryData) {
|
|
253
|
+
const mapped = await mapper.map(item);
|
|
254
|
+
if (!mapped.success) {
|
|
255
|
+
log.error('Mapping failed', { item, errors: mapped.errors });
|
|
256
|
+
continue;
|
|
257
|
+
}
|
|
258
|
+
|
|
259
|
+
const qty = mapped.data.qty;
|
|
260
|
+
|
|
261
|
+
if (qty > 1000) {
|
|
262
|
+
// HIGH QUANTITY → Batch API
|
|
263
|
+
highQtyBatch.push({
|
|
264
|
+
ref: `${mapped.data.locationRef}:${mapped.data.skuRef}`,
|
|
265
|
+
type: 'INVENTORY',
|
|
266
|
+
locationRef: mapped.data.locationRef,
|
|
267
|
+
skuRef: mapped.data.skuRef,
|
|
268
|
+
qty,
|
|
269
|
+
status: 'ACTIVE'
|
|
270
|
+
});
|
|
271
|
+
} else if (qty >= 100) {
|
|
272
|
+
// MEDIUM QUANTITY → Event API
|
|
273
|
+
mediumQtyEvents.push({
|
|
274
|
+
name: 'inventory.updated',
|
|
275
|
+
entityType: 'INVENTORY_QUANTITY',
|
|
276
|
+
entityRef: `${mapped.data.locationRef}:${mapped.data.skuRef}`,
|
|
277
|
+
retailerId: ctx.activation.getVariable('fluentRetailerId'),
|
|
278
|
+
attributes: [
|
|
279
|
+
{ name: 'qty', type: 'INTEGER', value: qty },
|
|
280
|
+
{ name: 'locationRef', type: 'STRING', value: mapped.data.locationRef }
|
|
281
|
+
]
|
|
282
|
+
});
|
|
283
|
+
} else {
|
|
284
|
+
// LOW QUANTITY → GraphQL + Event (alert)
|
|
285
|
+
lowQtyAlerts.push(mapped.data);
|
|
286
|
+
}
|
|
287
|
+
}
|
|
288
|
+
|
|
289
|
+
// Execute routing destinations
|
|
290
|
+
if (highQtyBatch.length > 0) {
|
|
291
|
+
const job = await client.createJob({
|
|
292
|
+
name: `high-qty-inventory-${Date.now()}`,
|
|
293
|
+
retailerId: ctx.activation.getVariable('fluentRetailerId')
|
|
294
|
+
});
|
|
295
|
+
|
|
296
|
+
await client.sendBatch(job.id, {
|
|
297
|
+
entityType: 'INVENTORY',
|
|
298
|
+
entities: highQtyBatch,
|
|
299
|
+
action: 'UPSERT'
|
|
300
|
+
});
|
|
301
|
+
|
|
302
|
+
log.info('High quantity inventory routed to Batch API', { count: highQtyBatch.length });
|
|
303
|
+
}
|
|
304
|
+
|
|
305
|
+
if (mediumQtyEvents.length > 0) {
|
|
306
|
+
for (const event of mediumQtyEvents) {
|
|
307
|
+
await client.sendEvent(event, 'async');
|
|
308
|
+
}
|
|
309
|
+
log.info('Medium quantity inventory routed to Event API', { count: mediumQtyEvents.length });
|
|
310
|
+
}
|
|
311
|
+
|
|
312
|
+
if (lowQtyAlerts.length > 0) {
|
|
313
|
+
// Send low stock alert via GraphQL + Event
|
|
314
|
+
for (const alert of lowQtyAlerts) {
|
|
315
|
+
const mutation = `
|
|
316
|
+
mutation TriggerLowStockAlert($locationRef: String!, $skuRef: String!, $qty: Int!) {
|
|
317
|
+
createEvent(input: {
|
|
318
|
+
name: "inventory.low_stock_alert"
|
|
319
|
+
entityType: INVENTORY_QUANTITY
|
|
320
|
+
entityRef: $locationRef
|
|
321
|
+
attributes: [
|
|
322
|
+
{ name: "skuRef", type: STRING, value: $skuRef }
|
|
323
|
+
{ name: "qty", type: INTEGER, value: $qty }
|
|
324
|
+
]
|
|
325
|
+
}) {
|
|
326
|
+
id
|
|
327
|
+
}
|
|
328
|
+
}
|
|
329
|
+
`;
|
|
330
|
+
|
|
331
|
+
await client.graphql({
|
|
332
|
+
query: mutation,
|
|
333
|
+
variables: {
|
|
334
|
+
locationRef: alert.locationRef,
|
|
335
|
+
skuRef: alert.skuRef,
|
|
336
|
+
qty: alert.qty
|
|
337
|
+
}
|
|
338
|
+
});
|
|
339
|
+
}
|
|
340
|
+
log.warn('Low quantity alerts sent', { count: lowQtyAlerts.length });
|
|
341
|
+
}
|
|
342
|
+
}
|
|
343
|
+
```
|
|
344
|
+
|
|
345
|
+
**When to Use:**
|
|
346
|
+
- Routing depends on data values (quantity thresholds, priority levels)
|
|
347
|
+
- Different SLAs based on order value or customer tier
|
|
348
|
+
- Inventory allocation strategies based on location characteristics
|
|
349
|
+
|
|
350
|
+
---
|
|
351
|
+
|
|
352
|
+
## Transformation Patterns
|
|
353
|
+
|
|
354
|
+
### 3. Content Enricher
|
|
355
|
+
|
|
356
|
+
**Pattern:** Enhance message with additional data from external sources.
|
|
357
|
+
|
|
358
|
+
**Support Status:** ✅ **Fully Supported** - Can be built using `FluentClient.graphql()` and `UniversalMapper` with custom resolvers
|
|
359
|
+
|
|
360
|
+
**SDK Building Blocks:**
|
|
361
|
+
- `FluentClient.graphql()` - Fetch enrichment data
|
|
362
|
+
- `UniversalMapper` - Merge enriched data
|
|
363
|
+
- Custom resolvers - Complex enrichment logic
|
|
364
|
+
|
|
365
|
+
#### Use Case: Enrich Order Data with Product Details
|
|
366
|
+
|
|
367
|
+
**Business Problem:** Order webhooks from external systems contain only SKU references. Fulfilment workflows need product names, weights, and categories for shipping calculations and packaging decisions.
|
|
368
|
+
|
|
369
|
+
Enrich order line items with product information (name, category, weight) fetched from Fluent GraphQL.
|
|
370
|
+
|
|
371
|
+
```typescript
|
|
372
|
+
import { createClient, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
|
|
373
|
+
|
|
374
|
+
async function enrichOrdersWithProductData(orders: any[], ctx: any, log: any) {
|
|
375
|
+
const client = await createClient({ ...ctx, log });
|
|
376
|
+
|
|
377
|
+
// STEP 1: Fetch all product data in one GraphQL query
|
|
378
|
+
const skuRefs = [...new Set(
|
|
379
|
+
orders.flatMap(o => o.items?.map((i: any) => i.skuRef) || [])
|
|
380
|
+
)];
|
|
381
|
+
|
|
382
|
+
const query = `
|
|
383
|
+
query GetProducts($skuRefs: [String!]!) {
|
|
384
|
+
products(skuRefs: $skuRefs) {
|
|
385
|
+
edges {
|
|
386
|
+
node {
|
|
387
|
+
ref
|
|
388
|
+
name
|
|
389
|
+
attributes {
|
|
390
|
+
name
|
|
391
|
+
type
|
|
392
|
+
value
|
|
393
|
+
}
|
|
394
|
+
}
|
|
395
|
+
}
|
|
396
|
+
}
|
|
397
|
+
}
|
|
398
|
+
`;
|
|
399
|
+
|
|
400
|
+
const response = await client.graphql({ query, variables: { skuRefs } });
|
|
401
|
+
|
|
402
|
+
// Build lookup map for fast enrichment
|
|
403
|
+
const productMap = new Map();
|
|
404
|
+
response.data?.products?.edges?.forEach((edge: any) => {
|
|
405
|
+
const product = edge.node;
|
|
406
|
+
productMap.set(product.ref, {
|
|
407
|
+
name: product.name,
|
|
408
|
+
category: product.attributes?.find((a: any) => a.name === 'category')?.value || 'UNCATEGORIZED',
|
|
409
|
+
weight: parseFloat(product.attributes?.find((a: any) => a.name === 'weight')?.value || '0')
|
|
410
|
+
});
|
|
411
|
+
});
|
|
412
|
+
|
|
413
|
+
// STEP 2: Enrich order items with product data
|
|
414
|
+
const enrichedOrders = orders.map(order => {
|
|
415
|
+
const enrichedItems = order.items?.map((item: any) => {
|
|
416
|
+
const productData = productMap.get(item.skuRef);
|
|
417
|
+
|
|
418
|
+
return {
|
|
419
|
+
...item,
|
|
420
|
+
// ENRICHMENT: Add product details
|
|
421
|
+
productName: productData?.name || 'Unknown Product',
|
|
422
|
+
productCategory: productData?.category || 'UNCATEGORIZED',
|
|
423
|
+
productWeight: productData?.weight || 0,
|
|
424
|
+
// Computed fields based on enriched data
|
|
425
|
+
lineWeight: (productData?.weight || 0) * (item.quantity || 1),
|
|
426
|
+
isHeavyItem: (productData?.weight || 0) > 10 // kg
|
|
427
|
+
};
|
|
428
|
+
});
|
|
429
|
+
|
|
430
|
+
return {
|
|
431
|
+
...order,
|
|
432
|
+
items: enrichedItems,
|
|
433
|
+
// Order-level computed fields
|
|
434
|
+
totalWeight: enrichedItems.reduce((sum: number, item: any) => sum + (item.lineWeight || 0), 0),
|
|
435
|
+
hasHeavyItems: enrichedItems.some((item: any) => item.isHeavyItem)
|
|
436
|
+
};
|
|
437
|
+
});
|
|
438
|
+
|
|
439
|
+
log.info('Orders enriched with product data', {
|
|
440
|
+
orderCount: orders.length,
|
|
441
|
+
productCount: productMap.size
|
|
442
|
+
});
|
|
443
|
+
|
|
444
|
+
return enrichedOrders;
|
|
445
|
+
}
|
|
446
|
+
```
|
|
447
|
+
|
|
448
|
+
**When to Use:**
|
|
449
|
+
- Order processing needs product details (dimensions, weight, hazmat flags)
|
|
450
|
+
- Inventory updates require location metadata (type, capacity, zone)
|
|
451
|
+
- Fulfilment workflows need customer preferences or address validation
|
|
452
|
+
|
|
453
|
+
---
|
|
454
|
+
|
|
455
|
+
### 4. Message Translator
|
|
456
|
+
|
|
457
|
+
**Pattern:** Convert message format from one schema to another.
|
|
458
|
+
|
|
459
|
+
**Support Status:** ✅ **Fully Supported** - Core SDK capability via Parsers + `UniversalMapper` + Builders
|
|
460
|
+
|
|
461
|
+
**SDK Building Blocks:**
|
|
462
|
+
- `XMLParserService` - Parse XML input
|
|
463
|
+
- `CSVParserService` - Parse CSV input
|
|
464
|
+
- `JSONParserService` - Parse JSON input
|
|
465
|
+
- `UniversalMapper` - Transform to Fluent schema
|
|
466
|
+
- `XMLBuilder` / `CSVBuilder` - Build output format
|
|
467
|
+
|
|
468
|
+
#### Use Case: Translate External Inventory CSV to Fluent Batch Format
|
|
469
|
+
|
|
470
|
+
**Business Problem:** Supplier systems export inventory data in custom CSV formats with different field names (e.g., `supplier_sku`, `warehouse_code`, `on_hand`). Fluent Commerce requires standard field names (`skuRef`, `locationRef`, `qty`).
|
|
471
|
+
|
|
472
|
+
Translate supplier CSV format to Fluent Batch API format.
|
|
473
|
+
|
|
474
|
+
```typescript
|
|
475
|
+
import {
|
|
476
|
+
SftpDataSource,
|
|
477
|
+
CSVParserService,
|
|
478
|
+
UniversalMapper,
|
|
479
|
+
createClient,
|
|
480
|
+
FluentBatchPayload
|
|
481
|
+
} from '@fluentcommerce/fc-connect-sdk';
|
|
482
|
+
|
|
483
|
+
async function translateInventoryCsvToBatch(ctx: any, log: any) {
|
|
484
|
+
const client = await createClient({ ...ctx, log });
|
|
485
|
+
|
|
486
|
+
// STEP 1: Read source data (supplier CSV format)
|
|
487
|
+
const sftp = new SftpDataSource({
|
|
488
|
+
type: 'SFTP_CSV',
|
|
489
|
+
settings: {
|
|
490
|
+
host: ctx.activation.getVariable('sftpHost'),
|
|
491
|
+
username: ctx.activation.getVariable('sftpUsername'),
|
|
492
|
+
password: ctx.activation.getVariable('sftpPassword'),
|
|
493
|
+
remotePath: '/inbound/inventory',
|
|
494
|
+
filePattern: '*.csv'
|
|
495
|
+
}
|
|
496
|
+
}, log);
|
|
497
|
+
|
|
498
|
+
try {
|
|
499
|
+
const files = await sftp.listFiles();
|
|
500
|
+
if (files.length === 0) {
|
|
501
|
+
log.info('No files to process');
|
|
502
|
+
return;
|
|
503
|
+
}
|
|
504
|
+
|
|
505
|
+
const csvContent = await sftp.downloadFile(files[0].name, { encoding: 'utf8' }) as string;
|
|
506
|
+
|
|
507
|
+
// STEP 2: Parse source format
|
|
508
|
+
const csvParser = new CSVParserService(log);
|
|
509
|
+
const parsedData = await csvParser.parse(csvContent, {
|
|
510
|
+
delimiter: ',',
|
|
511
|
+
columns: true,
|
|
512
|
+
skip_empty_lines: true
|
|
513
|
+
});
|
|
514
|
+
|
|
515
|
+
// Source format: supplier_sku, warehouse_code, on_hand, available, reserved
|
|
516
|
+
log.info('Parsed supplier CSV', { recordCount: parsedData.length });
|
|
517
|
+
|
|
518
|
+
// STEP 3: Translate to Fluent schema using UniversalMapper
|
|
519
|
+
const mapper = new UniversalMapper({
|
|
520
|
+
fields: {
|
|
521
|
+
// TRANSLATION RULES
|
|
522
|
+
ref: {
|
|
523
|
+
resolver: 'custom.buildCompositeKey',
|
|
524
|
+
required: true
|
|
525
|
+
},
|
|
526
|
+
type: {
|
|
527
|
+
value: 'INVENTORY_QUANTITY', // Static value for Batch API
|
|
528
|
+
required: true
|
|
529
|
+
},
|
|
530
|
+
locationRef: {
|
|
531
|
+
source: 'warehouse_code',
|
|
532
|
+
resolver: 'sdk.uppercase', // Normalize location ref
|
|
533
|
+
required: true
|
|
534
|
+
},
|
|
535
|
+
skuRef: {
|
|
536
|
+
source: 'supplier_sku',
|
|
537
|
+
resolver: 'sdk.trim', // Clean whitespace
|
|
538
|
+
required: true
|
|
539
|
+
},
|
|
540
|
+
qty: {
|
|
541
|
+
source: 'available', // Map 'available' → 'qty'
|
|
542
|
+
resolver: 'sdk.parseInt',
|
|
543
|
+
required: true
|
|
544
|
+
},
|
|
545
|
+
onHand: {
|
|
546
|
+
source: 'on_hand',
|
|
547
|
+
resolver: 'sdk.parseInt'
|
|
548
|
+
},
|
|
549
|
+
reserved: {
|
|
550
|
+
source: 'reserved',
|
|
551
|
+
resolver: 'sdk.parseInt',
|
|
552
|
+
defaultValue: 0 // Default if missing
|
|
553
|
+
}
|
|
554
|
+
}
|
|
555
|
+
}, {
|
|
556
|
+
customResolvers: {
|
|
557
|
+
// Custom resolver to build composite key
|
|
558
|
+
'custom.buildCompositeKey': (value: any, context: any) => {
|
|
559
|
+
return `${context.warehouse_code}:${context.supplier_sku}`;
|
|
560
|
+
}
|
|
561
|
+
}
|
|
562
|
+
});
|
|
563
|
+
|
|
564
|
+
const translatedEntities = [];
|
|
565
|
+
for (const record of parsedData) {
|
|
566
|
+
const result = await mapper.map(record);
|
|
567
|
+
if (result.success) {
|
|
568
|
+
translatedEntities.push(result.data);
|
|
569
|
+
} else {
|
|
570
|
+
log.error('Translation failed', { record, errors: result.errors });
|
|
571
|
+
}
|
|
572
|
+
}
|
|
573
|
+
|
|
574
|
+
// STEP 4: Send translated data to Fluent Batch API
|
|
575
|
+
if (translatedEntities.length > 0) {
|
|
576
|
+
const job = await client.createJob({
|
|
577
|
+
name: `inventory-translation-${Date.now()}`,
|
|
578
|
+
retailerId: ctx.activation.getVariable('fluentRetailerId')
|
|
579
|
+
});
|
|
580
|
+
|
|
581
|
+
const batch: FluentBatchPayload = {
|
|
582
|
+
entityType: 'INVENTORY',
|
|
583
|
+
entities: translatedEntities,
|
|
584
|
+
action: 'UPSERT'
|
|
585
|
+
};
|
|
586
|
+
|
|
587
|
+
await client.sendBatch(job.id, batch);
|
|
588
|
+
|
|
589
|
+
log.info('Inventory translated and sent to Batch API', {
|
|
590
|
+
sourceRecords: parsedData.length,
|
|
591
|
+
translatedEntities: translatedEntities.length,
|
|
592
|
+
jobId: job.id
|
|
593
|
+
});
|
|
594
|
+
}
|
|
595
|
+
} finally {
|
|
596
|
+
await sftp.dispose();
|
|
597
|
+
}
|
|
598
|
+
}
|
|
599
|
+
```
|
|
600
|
+
|
|
601
|
+
**When to Use:**
|
|
602
|
+
- Supplier data formats differ from Fluent schema
|
|
603
|
+
- Multiple source systems with different CSV/XML structures
|
|
604
|
+
- Legacy system integration with non-standard formats
|
|
605
|
+
|
|
606
|
+
---
|
|
607
|
+
|
|
608
|
+
### 5. Splitter
|
|
609
|
+
|
|
610
|
+
**Pattern:** Break a single message into multiple messages for parallel processing.
|
|
611
|
+
|
|
612
|
+
**Support Status:** ✅ **Fully Supported** - Can be built using Parsers (automatic splitting) + `Promise.all()` for parallel processing
|
|
613
|
+
|
|
614
|
+
**SDK Building Blocks:**
|
|
615
|
+
- `FluentClient.sendBatch()` - Split into multiple batches
|
|
616
|
+
- `Promise.all()` - Parallel execution
|
|
617
|
+
- `UniversalMapper` with array handling
|
|
618
|
+
- Parsers automatically split files into records
|
|
619
|
+
|
|
620
|
+
#### Use Case: Split Large Inventory File into Parallel Batches
|
|
621
|
+
|
|
622
|
+
**Business Problem:** Daily inventory export files contain 50,000+ inventory positions. Processing sequentially would take hours. Need to split into chunks and process in parallel to meet SLA requirements.
|
|
623
|
+
|
|
624
|
+
**Note:** Batch API only supports `INVENTORY` entities. For orders, use GraphQL mutations with parallel processing instead.
|
|
625
|
+
|
|
626
|
+
Process a large inventory file by splitting into chunks and sending parallel batches.
|
|
627
|
+
|
|
628
|
+
```typescript
|
|
629
|
+
import {
|
|
630
|
+
S3DataSource,
|
|
631
|
+
XMLParserService,
|
|
632
|
+
UniversalMapper,
|
|
633
|
+
createClient,
|
|
634
|
+
FluentBatchPayload
|
|
635
|
+
} from '@fluentcommerce/fc-connect-sdk';
|
|
636
|
+
|
|
637
|
+
async function splitOrdersIntoParallelBatches(ctx: any, log: any) {
|
|
638
|
+
const client = await createClient({ ...ctx, log });
|
|
639
|
+
|
|
640
|
+
// STEP 1: Read large inventory file from S3
|
|
641
|
+
const s3 = new S3DataSource({
|
|
642
|
+
type: 'S3_XML',
|
|
643
|
+
s3Config: {
|
|
644
|
+
bucket: ctx.activation.getVariable('s3Bucket'),
|
|
645
|
+
region: ctx.activation.getVariable('awsRegion'),
|
|
646
|
+
accessKeyId: ctx.activation.getVariable('awsAccessKeyId'),
|
|
647
|
+
secretAccessKey: ctx.activation.getVariable('awsSecretAccessKey')
|
|
648
|
+
}
|
|
649
|
+
}, log);
|
|
650
|
+
|
|
651
|
+
const xmlContent = await s3.downloadFile('inventory/inventory-batch-large.xml', { encoding: 'utf8' }) as string;
|
|
652
|
+
|
|
653
|
+
// STEP 2: Parse XML
|
|
654
|
+
const xmlParser = new XMLParserService(log);
|
|
655
|
+
const parsed = await xmlParser.parse(xmlContent);
|
|
656
|
+
|
|
657
|
+
const rawInventory = Array.isArray(parsed.inventory?.position)
|
|
658
|
+
? parsed.inventory.position
|
|
659
|
+
: [parsed.inventory?.position];
|
|
660
|
+
|
|
661
|
+
log.info('Parsed large inventory file', { totalPositions: rawInventory.length });
|
|
662
|
+
|
|
663
|
+
// STEP 3: Map inventory positions to Fluent format
|
|
664
|
+
const mapper = new UniversalMapper({
|
|
665
|
+
fields: {
|
|
666
|
+
ref: {
|
|
667
|
+
resolver: 'custom.buildRef',
|
|
668
|
+
required: true
|
|
669
|
+
},
|
|
670
|
+
locationRef: { source: '@location', required: true },
|
|
671
|
+
skuRef: { source: 'sku', required: true },
|
|
672
|
+
qty: { source: 'quantity', resolver: 'sdk.parseInt', required: true },
|
|
673
|
+
status: { source: 'status', defaultValue: 'ACTIVE' }
|
|
674
|
+
}
|
|
675
|
+
}, {
|
|
676
|
+
customResolvers: {
|
|
677
|
+
// Custom resolver to build composite key from location and SKU
|
|
678
|
+
'custom.buildRef': (value: any, context: any) => {
|
|
679
|
+
const location = context['@location'] || context.location;
|
|
680
|
+
const sku = context.sku;
|
|
681
|
+
return `${location}:${sku}`;
|
|
682
|
+
}
|
|
683
|
+
}
|
|
684
|
+
});
|
|
685
|
+
|
|
686
|
+
const mappedInventory = [];
|
|
687
|
+
for (const position of rawInventory) {
|
|
688
|
+
const result = await mapper.map(position);
|
|
689
|
+
if (result.success) {
|
|
690
|
+
mappedInventory.push(result.data);
|
|
691
|
+
}
|
|
692
|
+
}
|
|
693
|
+
|
|
694
|
+
// STEP 4: SPLIT into chunks (batches of 500)
|
|
695
|
+
const BATCH_SIZE = 500;
|
|
696
|
+
const chunks = [];
|
|
697
|
+
for (let i = 0; i < mappedInventory.length; i += BATCH_SIZE) {
|
|
698
|
+
chunks.push(mappedInventory.slice(i, i + BATCH_SIZE));
|
|
699
|
+
}
|
|
700
|
+
|
|
701
|
+
log.info('Split inventory into chunks', {
|
|
702
|
+
totalPositions: mappedInventory.length,
|
|
703
|
+
chunkCount: chunks.length,
|
|
704
|
+
batchSize: BATCH_SIZE
|
|
705
|
+
});
|
|
706
|
+
|
|
707
|
+
// STEP 5: Create job for all batches
|
|
708
|
+
// Note: Batch API only supports INVENTORY entities, not ORDER
|
|
709
|
+
// For orders, use GraphQL mutations instead (see below)
|
|
710
|
+
// This example shows splitting for inventory entities
|
|
711
|
+
const job = await client.createJob({
|
|
712
|
+
name: `inventory-split-${Date.now()}`,
|
|
713
|
+
retailerId: ctx.activation.getVariable('fluentRetailerId')
|
|
714
|
+
});
|
|
715
|
+
|
|
716
|
+
// STEP 6: Send chunks in parallel
|
|
717
|
+
const batchPromises = chunks.map(async (chunk, index) => {
|
|
718
|
+
const batch: FluentBatchPayload = {
|
|
719
|
+
entityType: 'INVENTORY',
|
|
720
|
+
entities: chunk,
|
|
721
|
+
action: 'UPSERT'
|
|
722
|
+
};
|
|
723
|
+
|
|
724
|
+
const batchResponse = await client.sendBatch(job.id, batch);
|
|
725
|
+
log.info(`Batch ${index + 1}/${chunks.length} sent`, {
|
|
726
|
+
batchId: batchResponse.id,
|
|
727
|
+
entityCount: chunk.length
|
|
728
|
+
});
|
|
729
|
+
|
|
730
|
+
return batchResponse;
|
|
731
|
+
});
|
|
732
|
+
|
|
733
|
+
// Wait for all batches to complete
|
|
734
|
+
const batchResponses = await Promise.all(batchPromises);
|
|
735
|
+
|
|
736
|
+
log.info('All inventory batches sent successfully', {
|
|
737
|
+
jobId: job.id,
|
|
738
|
+
totalBatches: batchResponses.length,
|
|
739
|
+
totalPositions: mappedInventory.length
|
|
740
|
+
});
|
|
741
|
+
|
|
742
|
+
return { jobId: job.id, batchCount: batchResponses.length };
|
|
743
|
+
}
|
|
744
|
+
```
|
|
745
|
+
|
|
746
|
+
**When to Use:**
|
|
747
|
+
- Large inventory files exceed single batch limits (>10K records)
|
|
748
|
+
- Parallel processing improves throughput
|
|
749
|
+
- Different chunks need different processing strategies
|
|
750
|
+
|
|
751
|
+
**Note:** For orders, use GraphQL mutations with `Promise.all()` for parallel processing instead of Batch API.
|
|
752
|
+
|
|
753
|
+
---
|
|
754
|
+
|
|
755
|
+
### 6. Aggregator
|
|
756
|
+
|
|
757
|
+
**Pattern:** Combine multiple related messages into a single message.
|
|
758
|
+
|
|
759
|
+
**Support Status:** ✅ **Fully Supported** - Can be built using `FluentClient.graphql()` with pagination + application aggregation logic
|
|
760
|
+
|
|
761
|
+
**SDK Building Blocks:**
|
|
762
|
+
- `FluentClient.graphql()` with pagination - Fetch multiple pages
|
|
763
|
+
- `StateService` - Track aggregation state
|
|
764
|
+
- Array aggregation logic
|
|
765
|
+
|
|
766
|
+
#### Use Case: Aggregate Virtual Position Data from Multiple Locations
|
|
767
|
+
|
|
768
|
+
**Business Problem:** Inventory positions are stored per location (warehouse, store, virtual). Reporting dashboard needs aggregated view showing total quantity across all locations for each SKU.
|
|
769
|
+
|
|
770
|
+
Aggregate inventory positions across all locations for a single SKU view.
|
|
771
|
+
|
|
772
|
+
```typescript
|
|
773
|
+
import { createClient, StateService, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
|
|
774
|
+
|
|
775
|
+
async function aggregateVirtualPositions(skuRefs: string[], ctx: any, log: any) {
|
|
776
|
+
const client = await createClient({ ...ctx, log });
|
|
777
|
+
|
|
778
|
+
// STEP 1: Query virtual positions across all locations (with pagination)
|
|
779
|
+
const query = `
|
|
780
|
+
query GetVirtualPositions($skuRefs: [String!]!, $first: Int!, $after: String) {
|
|
781
|
+
virtualPositions(skuRefs: $skuRefs, first: $first, after: $after) {
|
|
782
|
+
edges {
|
|
783
|
+
node {
|
|
784
|
+
ref
|
|
785
|
+
quantity
|
|
786
|
+
groupRef
|
|
787
|
+
productRef
|
|
788
|
+
virtualCatalogueRef
|
|
789
|
+
}
|
|
790
|
+
cursor
|
|
791
|
+
}
|
|
792
|
+
pageInfo {
|
|
793
|
+
hasNextPage
|
|
794
|
+
endCursor
|
|
795
|
+
}
|
|
796
|
+
}
|
|
797
|
+
}
|
|
798
|
+
`;
|
|
799
|
+
|
|
800
|
+
const response = await client.graphql({
|
|
801
|
+
query,
|
|
802
|
+
variables: { skuRefs, first: 1000 },
|
|
803
|
+
pagination: {
|
|
804
|
+
enabled: true,
|
|
805
|
+
maxPages: 10,
|
|
806
|
+
direction: 'forward'
|
|
807
|
+
}
|
|
808
|
+
});
|
|
809
|
+
|
|
810
|
+
const allPositions = response.data?.virtualPositions?.edges?.map((e: any) => e.node) || [];
|
|
811
|
+
|
|
812
|
+
log.info('Fetched virtual positions', {
|
|
813
|
+
totalPositions: allPositions.length,
|
|
814
|
+
skuCount: skuRefs.length,
|
|
815
|
+
paginationStats: response.extensions?.autoPagination
|
|
816
|
+
});
|
|
817
|
+
|
|
818
|
+
// STEP 2: AGGREGATE positions by SKU
|
|
819
|
+
const aggregatedBySku = new Map<string, {
|
|
820
|
+
skuRef: string;
|
|
821
|
+
totalQuantity: number;
|
|
822
|
+
locations: string[];
|
|
823
|
+
catalogues: string[];
|
|
824
|
+
}>();
|
|
825
|
+
|
|
826
|
+
for (const position of allPositions) {
|
|
827
|
+
const skuRef = position.productRef;
|
|
828
|
+
|
|
829
|
+
if (!aggregatedBySku.has(skuRef)) {
|
|
830
|
+
aggregatedBySku.set(skuRef, {
|
|
831
|
+
skuRef,
|
|
832
|
+
totalQuantity: 0,
|
|
833
|
+
locations: [],
|
|
834
|
+
catalogues: []
|
|
835
|
+
});
|
|
836
|
+
}
|
|
837
|
+
|
|
838
|
+
const agg = aggregatedBySku.get(skuRef)!;
|
|
839
|
+
agg.totalQuantity += position.quantity || 0;
|
|
840
|
+
|
|
841
|
+
if (position.groupRef && !agg.locations.includes(position.groupRef)) {
|
|
842
|
+
agg.locations.push(position.groupRef);
|
|
843
|
+
}
|
|
844
|
+
|
|
845
|
+
if (position.virtualCatalogueRef && !agg.catalogues.includes(position.virtualCatalogueRef)) {
|
|
846
|
+
agg.catalogues.push(position.virtualCatalogueRef);
|
|
847
|
+
}
|
|
848
|
+
}
|
|
849
|
+
|
|
850
|
+
// STEP 3: Convert to array and compute metrics
|
|
851
|
+
const aggregatedResults = Array.from(aggregatedBySku.values()).map(agg => ({
|
|
852
|
+
...agg,
|
|
853
|
+
locationCount: agg.locations.length,
|
|
854
|
+
catalogueCount: agg.catalogues.length,
|
|
855
|
+
avgQuantityPerLocation: agg.totalQuantity / agg.locations.length
|
|
856
|
+
}));
|
|
857
|
+
|
|
858
|
+
log.info('Aggregation complete', {
|
|
859
|
+
totalSKUs: aggregatedResults.length,
|
|
860
|
+
totalQuantity: aggregatedResults.reduce((sum, r) => sum + r.totalQuantity, 0)
|
|
861
|
+
});
|
|
862
|
+
|
|
863
|
+
// STEP 4: Store aggregated results in KV for later retrieval
|
|
864
|
+
const kv = ctx.openKv(':project:');
|
|
865
|
+
const stateService = new StateService(log);
|
|
866
|
+
|
|
867
|
+
for (const result of aggregatedResults) {
|
|
868
|
+
await kv.set(['aggregated_inventory', result.skuRef], result);
|
|
869
|
+
}
|
|
870
|
+
|
|
871
|
+
log.info('Aggregated results stored in KV', { skuCount: aggregatedResults.length });
|
|
872
|
+
|
|
873
|
+
return aggregatedResults;
|
|
874
|
+
}
|
|
875
|
+
```
|
|
876
|
+
|
|
877
|
+
**When to Use:**
|
|
878
|
+
- Inventory views need multi-location aggregation
|
|
879
|
+
- Order totals calculated from multiple line items
|
|
880
|
+
- Reporting requires data from multiple API calls
|
|
881
|
+
|
|
882
|
+
---
|
|
883
|
+
|
|
884
|
+
## Endpoint Patterns
|
|
885
|
+
|
|
886
|
+
### 7. Polling Consumer
|
|
887
|
+
|
|
888
|
+
**Pattern:** Periodically poll an endpoint for new messages.
|
|
889
|
+
|
|
890
|
+
**Support Status:** ✅ **Fully Supported** - Can be built using Versori `schedule()` + `Data Sources` + `StateService` for tracking
|
|
891
|
+
|
|
892
|
+
**SDK Building Blocks:**
|
|
893
|
+
- `schedule()` from `@versori/run` - Scheduled execution
|
|
894
|
+
- `StateService` - Track last poll time
|
|
895
|
+
- `SftpDataSource.listFiles()` - Poll for new files
|
|
896
|
+
- `S3DataSource.listFiles()` - Poll S3 buckets
|
|
897
|
+
|
|
898
|
+
#### Use Case: Poll SFTP for New Inventory Files
|
|
899
|
+
|
|
900
|
+
**Business Problem:** Supplier systems don't support webhooks. They drop CSV files to SFTP every 2 hours. Need to check for new files periodically and process them without duplicate processing.
|
|
901
|
+
|
|
902
|
+
Check SFTP every 2 hours for new inventory files, process only new files.
|
|
903
|
+
|
|
904
|
+
```typescript
|
|
905
|
+
import { schedule } from '@versori/run';
|
|
906
|
+
import {
|
|
907
|
+
SftpDataSource,
|
|
908
|
+
CSVParserService,
|
|
909
|
+
UniversalMapper,
|
|
910
|
+
createClient,
|
|
911
|
+
StateService,
|
|
912
|
+
FluentBatchPayload
|
|
913
|
+
} from '@fluentcommerce/fc-connect-sdk';
|
|
914
|
+
|
|
915
|
+
export const pollSftpInventory = schedule('poll-sftp-inventory', '0 */2 * * *', async (ctx) => {
|
|
916
|
+
const { log, openKv, activation } = ctx;
|
|
917
|
+
const client = await createClient({ ...ctx, log });
|
|
918
|
+
|
|
919
|
+
// STEP 1: Connect to SFTP
|
|
920
|
+
const sftp = new SftpDataSource({
|
|
921
|
+
type: 'SFTP_CSV',
|
|
922
|
+
settings: {
|
|
923
|
+
host: activation.getVariable('sftpHost'),
|
|
924
|
+
username: activation.getVariable('sftpUsername'),
|
|
925
|
+
password: activation.getVariable('sftpPassword'),
|
|
926
|
+
remotePath: '/outbound/inventory',
|
|
927
|
+
filePattern: 'inventory_*.csv'
|
|
928
|
+
}
|
|
929
|
+
}, log);
|
|
930
|
+
|
|
931
|
+
try {
|
|
932
|
+
// STEP 2: Poll for files
|
|
933
|
+
const files = await sftp.listFiles();
|
|
934
|
+
log.info('Polled SFTP for inventory files', { fileCount: files.length });
|
|
935
|
+
|
|
936
|
+
if (files.length === 0) {
|
|
937
|
+
log.info('No new files found');
|
|
938
|
+
return { status: 'no_files' };
|
|
939
|
+
}
|
|
940
|
+
|
|
941
|
+
// STEP 3: Check state to identify new files
|
|
942
|
+
const kv = openKv(':project:');
|
|
943
|
+
const stateService = new StateService(log);
|
|
944
|
+
|
|
945
|
+
const newFiles = [];
|
|
946
|
+
for (const file of files) {
|
|
947
|
+
const isProcessed = await kv.get(['processed_files', file.name]);
|
|
948
|
+
if (!isProcessed) {
|
|
949
|
+
newFiles.push(file);
|
|
950
|
+
}
|
|
951
|
+
}
|
|
952
|
+
|
|
953
|
+
log.info('Identified new files', { newFileCount: newFiles.length });
|
|
954
|
+
|
|
955
|
+
if (newFiles.length === 0) {
|
|
956
|
+
return { status: 'no_new_files' };
|
|
957
|
+
}
|
|
958
|
+
|
|
959
|
+
// STEP 4: Process new files
|
|
960
|
+
const csvParser = new CSVParserService(log);
|
|
961
|
+
const mapper = new UniversalMapper({
|
|
962
|
+
fields: {
|
|
963
|
+
ref: { source: 'location_sku', required: true },
|
|
964
|
+
type: { value: 'INVENTORY_QUANTITY', required: true },
|
|
965
|
+
locationRef: { source: 'location', required: true },
|
|
966
|
+
skuRef: { source: 'sku', required: true },
|
|
967
|
+
qty: { source: 'quantity', resolver: 'sdk.parseInt', required: true }
|
|
968
|
+
}
|
|
969
|
+
});
|
|
970
|
+
|
|
971
|
+
let totalEntities = 0;
|
|
972
|
+
|
|
973
|
+
for (const file of newFiles) {
|
|
974
|
+
try {
|
|
975
|
+
const csvContent = await sftp.downloadFile(file.name, { encoding: 'utf8' }) as string;
|
|
976
|
+
const parsed = await csvParser.parse(csvContent, { columns: true });
|
|
977
|
+
|
|
978
|
+
const entities = [];
|
|
979
|
+
for (const record of parsed) {
|
|
980
|
+
const result = await mapper.map(record);
|
|
981
|
+
if (result.success) {
|
|
982
|
+
entities.push(result.data);
|
|
983
|
+
}
|
|
984
|
+
}
|
|
985
|
+
|
|
986
|
+
if (entities.length > 0) {
|
|
987
|
+
const job = await client.createJob({
|
|
988
|
+
name: `inventory-poll-${file.name}-${Date.now()}`,
|
|
989
|
+
retailerId: activation.getVariable('fluentRetailerId')
|
|
990
|
+
});
|
|
991
|
+
|
|
992
|
+
await client.sendBatch(job.id, {
|
|
993
|
+
entityType: 'INVENTORY',
|
|
994
|
+
entities,
|
|
995
|
+
action: 'UPSERT'
|
|
996
|
+
});
|
|
997
|
+
|
|
998
|
+
totalEntities += entities.length;
|
|
999
|
+
}
|
|
1000
|
+
|
|
1001
|
+
// Mark file as processed
|
|
1002
|
+
await kv.set(['processed_files', file.name], {
|
|
1003
|
+
processedAt: new Date().toISOString(),
|
|
1004
|
+
entityCount: entities.length
|
|
1005
|
+
});
|
|
1006
|
+
|
|
1007
|
+
log.info('File processed successfully', { file: file.name, entities: entities.length });
|
|
1008
|
+
} catch (error) {
|
|
1009
|
+
log.error(`Failed to process file: ${file.name}`, error as Error);
|
|
1010
|
+
}
|
|
1011
|
+
}
|
|
1012
|
+
|
|
1013
|
+
return {
|
|
1014
|
+
status: 'success',
|
|
1015
|
+
filesProcessed: newFiles.length,
|
|
1016
|
+
totalEntities
|
|
1017
|
+
};
|
|
1018
|
+
} finally {
|
|
1019
|
+
await sftp.dispose();
|
|
1020
|
+
}
|
|
1021
|
+
});
|
|
1022
|
+
```
|
|
1023
|
+
|
|
1024
|
+
**When to Use:**
|
|
1025
|
+
- Supplier systems don't support webhooks (push notifications)
|
|
1026
|
+
- Scheduled file drops to SFTP/S3 (daily inventory snapshots)
|
|
1027
|
+
- Periodic API polling for new orders/fulfilments
|
|
1028
|
+
|
|
1029
|
+
---
|
|
1030
|
+
|
|
1031
|
+
### 8. Idempotent Receiver
|
|
1032
|
+
|
|
1033
|
+
**Pattern:** Ensure messages are processed exactly once, even if received multiple times.
|
|
1034
|
+
|
|
1035
|
+
**Support Status:** ✅ **Fully Supported** - Can be built using KV Store + `StateService` for idempotency tracking
|
|
1036
|
+
|
|
1037
|
+
**SDK Building Blocks:**
|
|
1038
|
+
- `StateService` - Track processed message IDs
|
|
1039
|
+
- `VersoriKVAdapter` - Distributed state storage
|
|
1040
|
+
- KV Store (`openKv`) - Persistent state storage
|
|
1041
|
+
- Atomic operations for race-free checks
|
|
1042
|
+
|
|
1043
|
+
#### Use Case: Prevent Duplicate Order Processing
|
|
1044
|
+
|
|
1045
|
+
**Business Problem:** Webhook endpoints may receive duplicate deliveries due to network retries or webhook provider retry logic. Need to ensure each order is processed exactly once, even if webhook fires multiple times.
|
|
1046
|
+
|
|
1047
|
+
Ensure each order is processed only once, even with webhook retries.
|
|
1048
|
+
|
|
1049
|
+
```typescript
|
|
1050
|
+
import { webhook } from '@versori/run';
|
|
1051
|
+
import {
|
|
1052
|
+
createClient,
|
|
1053
|
+
StateService,
|
|
1054
|
+
parseWebhookRequest,
|
|
1055
|
+
FluentEvent
|
|
1056
|
+
} from '@fluentcommerce/fc-connect-sdk';
|
|
1057
|
+
|
|
1058
|
+
export const processOrderWebhook = webhook('process-order', {
|
|
1059
|
+
response: { mode: 'sync' }
|
|
1060
|
+
}, async (ctx) => {
|
|
1061
|
+
const { log, openKv, request } = ctx;
|
|
1062
|
+
const client = await createClient({ ...ctx, log });
|
|
1063
|
+
|
|
1064
|
+
// STEP 1: Parse webhook payload
|
|
1065
|
+
const webhookPayload = await parseWebhookRequest(request);
|
|
1066
|
+
const orderRef = webhookPayload.entityRef;
|
|
1067
|
+
const eventId = webhookPayload.id;
|
|
1068
|
+
|
|
1069
|
+
log.info('Received order webhook', { orderRef, eventId });
|
|
1070
|
+
|
|
1071
|
+
// STEP 2: IDEMPOTENCY CHECK - Has this event been processed?
|
|
1072
|
+
const kv = openKv(':project:');
|
|
1073
|
+
const processingKey = ['processed_events', eventId];
|
|
1074
|
+
|
|
1075
|
+
const existingRecord = await kv.get(processingKey);
|
|
1076
|
+
if (existingRecord?.value) {
|
|
1077
|
+
log.warn('Event already processed - skipping', {
|
|
1078
|
+
eventId,
|
|
1079
|
+
orderRef,
|
|
1080
|
+
processedAt: (existingRecord.value as any).processedAt
|
|
1081
|
+
});
|
|
1082
|
+
|
|
1083
|
+
return new Response(JSON.stringify({
|
|
1084
|
+
status: 'already_processed',
|
|
1085
|
+
eventId,
|
|
1086
|
+
processedAt: (existingRecord.value as any).processedAt
|
|
1087
|
+
}), {
|
|
1088
|
+
status: 200,
|
|
1089
|
+
headers: { 'Content-Type': 'application/json' }
|
|
1090
|
+
});
|
|
1091
|
+
}
|
|
1092
|
+
|
|
1093
|
+
// STEP 3: Process order (business logic)
|
|
1094
|
+
try {
|
|
1095
|
+
// Fetch order details
|
|
1096
|
+
const query = `
|
|
1097
|
+
query GetOrder($ref: String!) {
|
|
1098
|
+
order(ref: $ref) {
|
|
1099
|
+
id
|
|
1100
|
+
ref
|
|
1101
|
+
status
|
|
1102
|
+
totalPrice
|
|
1103
|
+
items {
|
|
1104
|
+
edges {
|
|
1105
|
+
node {
|
|
1106
|
+
ref
|
|
1107
|
+
quantity
|
|
1108
|
+
product {
|
|
1109
|
+
ref
|
|
1110
|
+
name
|
|
1111
|
+
}
|
|
1112
|
+
}
|
|
1113
|
+
}
|
|
1114
|
+
}
|
|
1115
|
+
}
|
|
1116
|
+
}
|
|
1117
|
+
`;
|
|
1118
|
+
|
|
1119
|
+
const response = await client.graphql({
|
|
1120
|
+
query,
|
|
1121
|
+
variables: { ref: orderRef }
|
|
1122
|
+
});
|
|
1123
|
+
|
|
1124
|
+
const order = response.data?.order;
|
|
1125
|
+
|
|
1126
|
+
if (!order) {
|
|
1127
|
+
throw new Error(`Order not found: ${orderRef}`);
|
|
1128
|
+
}
|
|
1129
|
+
|
|
1130
|
+
// Business logic: Send fulfilment event if order is confirmed
|
|
1131
|
+
if (order.status === 'CONFIRMED') {
|
|
1132
|
+
const event: FluentEvent = {
|
|
1133
|
+
name: 'order.ready_for_fulfilment',
|
|
1134
|
+
entityType: 'ORDER',
|
|
1135
|
+
entityRef: orderRef,
|
|
1136
|
+
retailerId: webhookPayload.retailerId,
|
|
1137
|
+
attributes: [
|
|
1138
|
+
{ name: 'totalPrice', type: 'FLOAT', value: order.totalPrice },
|
|
1139
|
+
{ name: 'itemCount', type: 'INTEGER', value: order.items?.edges?.length || 0 }
|
|
1140
|
+
]
|
|
1141
|
+
};
|
|
1142
|
+
|
|
1143
|
+
await client.sendEvent(event, 'async');
|
|
1144
|
+
}
|
|
1145
|
+
|
|
1146
|
+
// STEP 4: Mark event as processed (IDEMPOTENCY RECORD)
|
|
1147
|
+
await kv.set(processingKey, {
|
|
1148
|
+
processedAt: new Date().toISOString(),
|
|
1149
|
+
orderRef,
|
|
1150
|
+
eventId,
|
|
1151
|
+
status: 'success'
|
|
1152
|
+
});
|
|
1153
|
+
|
|
1154
|
+
log.info('Order processed successfully', { orderRef, eventId });
|
|
1155
|
+
|
|
1156
|
+
return new Response(JSON.stringify({
|
|
1157
|
+
status: 'processed',
|
|
1158
|
+
orderRef,
|
|
1159
|
+
eventId
|
|
1160
|
+
}), {
|
|
1161
|
+
status: 200,
|
|
1162
|
+
headers: { 'Content-Type': 'application/json' }
|
|
1163
|
+
});
|
|
1164
|
+
} catch (error) {
|
|
1165
|
+
log.error('Order processing failed', error as Error, { orderRef, eventId });
|
|
1166
|
+
|
|
1167
|
+
// Don't mark as processed on failure - allow retry
|
|
1168
|
+
return new Response(JSON.stringify({
|
|
1169
|
+
status: 'error',
|
|
1170
|
+
message: (error as Error).message
|
|
1171
|
+
}), {
|
|
1172
|
+
status: 500,
|
|
1173
|
+
headers: { 'Content-Type': 'application/json' }
|
|
1174
|
+
});
|
|
1175
|
+
}
|
|
1176
|
+
});
|
|
1177
|
+
```
|
|
1178
|
+
|
|
1179
|
+
**When to Use:**
|
|
1180
|
+
- Webhook endpoints that may receive duplicate deliveries
|
|
1181
|
+
- File processing where same file might be reprocessed
|
|
1182
|
+
- Distributed workflows with at-least-once delivery guarantees
|
|
1183
|
+
|
|
1184
|
+
---
|
|
1185
|
+
|
|
1186
|
+
## System Management Patterns
|
|
1187
|
+
|
|
1188
|
+
### 9. Dead Letter Channel
|
|
1189
|
+
|
|
1190
|
+
**Pattern:** Route failed messages to a separate channel for inspection and reprocessing.
|
|
1191
|
+
|
|
1192
|
+
**Support Status:** ✅ **Fully Supported** - Can be built using `S3DataSource` or `SftpDataSource` for failed message storage
|
|
1193
|
+
|
|
1194
|
+
**SDK Building Blocks:**
|
|
1195
|
+
- `S3DataSource.uploadFile()` - Write failed messages to S3
|
|
1196
|
+
- `SftpDataSource.writeFile()` - Write failed messages to SFTP
|
|
1197
|
+
- `StateService` - Track failure counts
|
|
1198
|
+
- Error handling with try/catch
|
|
1199
|
+
|
|
1200
|
+
#### Use Case: Route Failed Inventory Updates to Dead Letter S3 Bucket
|
|
1201
|
+
|
|
1202
|
+
**Business Problem:** Some inventory records fail validation (missing SKU, invalid quantity, etc.). Need to capture these failures for manual review and potential reprocessing without blocking successful records.
|
|
1203
|
+
|
|
1204
|
+
Capture failed inventory records for manual review and reprocessing.
|
|
1205
|
+
|
|
1206
|
+
```typescript
|
|
1207
|
+
import {
|
|
1208
|
+
SftpDataSource,
|
|
1209
|
+
CSVParserService,
|
|
1210
|
+
UniversalMapper,
|
|
1211
|
+
S3DataSource,
|
|
1212
|
+
createClient,
|
|
1213
|
+
FluentBatchPayload
|
|
1214
|
+
} from '@fluentcommerce/fc-connect-sdk';
|
|
1215
|
+
import { Buffer } from 'node:buffer';
|
|
1216
|
+
|
|
1217
|
+
async function processInventoryWithDeadLetterChannel(ctx: any, log: any) {
|
|
1218
|
+
const client = await createClient({ ...ctx, log });
|
|
1219
|
+
|
|
1220
|
+
// Setup: SFTP source, S3 dead letter destination
|
|
1221
|
+
const sftp = new SftpDataSource({
|
|
1222
|
+
type: 'SFTP_CSV',
|
|
1223
|
+
settings: {
|
|
1224
|
+
host: ctx.activation.getVariable('sftpHost'),
|
|
1225
|
+
username: ctx.activation.getVariable('sftpUsername'),
|
|
1226
|
+
password: ctx.activation.getVariable('sftpPassword'),
|
|
1227
|
+
remotePath: '/inbound/inventory',
|
|
1228
|
+
filePattern: '*.csv'
|
|
1229
|
+
}
|
|
1230
|
+
}, log);
|
|
1231
|
+
|
|
1232
|
+
const deadLetterS3 = new S3DataSource({
|
|
1233
|
+
type: 'S3_JSON',
|
|
1234
|
+
s3Config: {
|
|
1235
|
+
bucket: ctx.activation.getVariable('deadLetterBucket'),
|
|
1236
|
+
region: ctx.activation.getVariable('awsRegion'),
|
|
1237
|
+
accessKeyId: ctx.activation.getVariable('awsAccessKeyId'),
|
|
1238
|
+
secretAccessKey: ctx.activation.getVariable('awsSecretAccessKey')
|
|
1239
|
+
}
|
|
1240
|
+
}, log);
|
|
1241
|
+
|
|
1242
|
+
try {
|
|
1243
|
+
const files = await sftp.listFiles();
|
|
1244
|
+
if (files.length === 0) {
|
|
1245
|
+
log.info('No files to process');
|
|
1246
|
+
return;
|
|
1247
|
+
}
|
|
1248
|
+
|
|
1249
|
+
const csvContent = await sftp.downloadFile(files[0].name, { encoding: 'utf8' }) as string;
|
|
1250
|
+
const csvParser = new CSVParserService(log);
|
|
1251
|
+
const parsed = await csvParser.parse(csvContent, { columns: true });
|
|
1252
|
+
|
|
1253
|
+
// Map inventory records
|
|
1254
|
+
const mapper = new UniversalMapper({
|
|
1255
|
+
fields: {
|
|
1256
|
+
ref: { source: 'location_sku', required: true },
|
|
1257
|
+
type: { value: 'INVENTORY_QUANTITY', required: true },
|
|
1258
|
+
locationRef: { source: 'location', required: true },
|
|
1259
|
+
skuRef: { source: 'sku', required: true },
|
|
1260
|
+
qty: { source: 'quantity', resolver: 'sdk.parseInt', required: true }
|
|
1261
|
+
}
|
|
1262
|
+
});
|
|
1263
|
+
|
|
1264
|
+
const successfulEntities = [];
|
|
1265
|
+
const failedRecords = [];
|
|
1266
|
+
|
|
1267
|
+
// Process records with error tracking
|
|
1268
|
+
for (const record of parsed) {
|
|
1269
|
+
try {
|
|
1270
|
+
const result = await mapper.map(record);
|
|
1271
|
+
if (result.success) {
|
|
1272
|
+
successfulEntities.push(result.data);
|
|
1273
|
+
} else {
|
|
1274
|
+
// DEAD LETTER: Mapping validation failed
|
|
1275
|
+
failedRecords.push({
|
|
1276
|
+
sourceRecord: record,
|
|
1277
|
+
errors: result.errors,
|
|
1278
|
+
failureReason: 'mapping_validation_failed',
|
|
1279
|
+
failedAt: new Date().toISOString()
|
|
1280
|
+
});
|
|
1281
|
+
}
|
|
1282
|
+
} catch (error) {
|
|
1283
|
+
// DEAD LETTER: Unexpected error during mapping
|
|
1284
|
+
failedRecords.push({
|
|
1285
|
+
sourceRecord: record,
|
|
1286
|
+
errors: [(error as Error).message],
|
|
1287
|
+
failureReason: 'mapping_error',
|
|
1288
|
+
failedAt: new Date().toISOString(),
|
|
1289
|
+
errorStack: (error as Error).stack
|
|
1290
|
+
});
|
|
1291
|
+
}
|
|
1292
|
+
}
|
|
1293
|
+
|
|
1294
|
+
// Send successful records to Batch API
|
|
1295
|
+
if (successfulEntities.length > 0) {
|
|
1296
|
+
const job = await client.createJob({
|
|
1297
|
+
name: `inventory-with-dlq-${Date.now()}`,
|
|
1298
|
+
retailerId: ctx.activation.getVariable('fluentRetailerId')
|
|
1299
|
+
});
|
|
1300
|
+
|
|
1301
|
+
await client.sendBatch(job.id, {
|
|
1302
|
+
entityType: 'INVENTORY',
|
|
1303
|
+
entities: successfulEntities,
|
|
1304
|
+
action: 'UPSERT'
|
|
1305
|
+
});
|
|
1306
|
+
|
|
1307
|
+
log.info('Successful entities sent to Batch API', {
|
|
1308
|
+
count: successfulEntities.length,
|
|
1309
|
+
jobId: job.id
|
|
1310
|
+
});
|
|
1311
|
+
}
|
|
1312
|
+
|
|
1313
|
+
// DEAD LETTER CHANNEL: Write failed records to S3
|
|
1314
|
+
if (failedRecords.length > 0) {
|
|
1315
|
+
const deadLetterKey = `failed-inventory/${files[0].name}-failures-${Date.now()}.json`;
|
|
1316
|
+
const deadLetterContent = JSON.stringify({
|
|
1317
|
+
sourceFile: files[0].name,
|
|
1318
|
+
failedAt: new Date().toISOString(),
|
|
1319
|
+
totalFailed: failedRecords.length,
|
|
1320
|
+
records: failedRecords
|
|
1321
|
+
}, null, 2);
|
|
1322
|
+
|
|
1323
|
+
await deadLetterS3.uploadFile(
|
|
1324
|
+
deadLetterKey,
|
|
1325
|
+
Buffer.from(deadLetterContent, 'utf-8')
|
|
1326
|
+
);
|
|
1327
|
+
|
|
1328
|
+
log.warn('Failed records written to Dead Letter S3', {
|
|
1329
|
+
count: failedRecords.length,
|
|
1330
|
+
deadLetterKey
|
|
1331
|
+
});
|
|
1332
|
+
}
|
|
1333
|
+
|
|
1334
|
+
return {
|
|
1335
|
+
successful: successfulEntities.length,
|
|
1336
|
+
failed: failedRecords.length,
|
|
1337
|
+
deadLetterKey: failedRecords.length > 0 ? `failed-inventory/${files[0].name}-failures-${Date.now()}.json` : null
|
|
1338
|
+
};
|
|
1339
|
+
} finally {
|
|
1340
|
+
await sftp.dispose();
|
|
1341
|
+
}
|
|
1342
|
+
}
|
|
1343
|
+
```
|
|
1344
|
+
|
|
1345
|
+
**When to Use:**
|
|
1346
|
+
- Data validation failures need manual review
|
|
1347
|
+
- Partial batch failures require reprocessing
|
|
1348
|
+
- Audit trail for failed messages is required
|
|
1349
|
+
|
|
1350
|
+
---
|
|
1351
|
+
|
|
1352
|
+
### 10. Wire Tap
|
|
1353
|
+
|
|
1354
|
+
**Pattern:** Inspect messages passing through without altering them (audit logging).
|
|
1355
|
+
|
|
1356
|
+
**Support Status:** ✅ **Fully Supported** - Can be built using `S3DataSource` or logging for audit trail
|
|
1357
|
+
|
|
1358
|
+
**SDK Building Blocks:**
|
|
1359
|
+
- `S3DataSource.uploadFile()` - Archive message copies
|
|
1360
|
+
- Logging for message inspection
|
|
1361
|
+
- No modification to message flow
|
|
1362
|
+
|
|
1363
|
+
#### Use Case: Audit All Order Events Sent to Event API
|
|
1364
|
+
|
|
1365
|
+
**Business Problem:** Compliance requirements mandate audit trail of all order events sent to Fluent Commerce. Need to log message contents without affecting processing performance or modifying message flow.
|
|
1366
|
+
|
|
1367
|
+
Log all order events to S3 for compliance auditing.
|
|
1368
|
+
|
|
1369
|
+
```typescript
|
|
1370
|
+
import {
|
|
1371
|
+
createClient,
|
|
1372
|
+
FluentEvent,
|
|
1373
|
+
S3DataSource
|
|
1374
|
+
} from '@fluentcommerce/fc-connect-sdk';
|
|
1375
|
+
import { Buffer } from 'node:buffer';
|
|
1376
|
+
|
|
1377
|
+
async function sendOrderEventWithAudit(orderEvent: FluentEvent, ctx: any, log: any) {
|
|
1378
|
+
const client = await createClient({ ...ctx, log });
|
|
1379
|
+
|
|
1380
|
+
// Setup audit S3 bucket
|
|
1381
|
+
const auditS3 = new S3DataSource({
|
|
1382
|
+
type: 'S3_JSON',
|
|
1383
|
+
s3Config: {
|
|
1384
|
+
bucket: ctx.activation.getVariable('auditBucket'),
|
|
1385
|
+
region: ctx.activation.getVariable('awsRegion'),
|
|
1386
|
+
accessKeyId: ctx.activation.getVariable('awsAccessKeyId'),
|
|
1387
|
+
secretAccessKey: ctx.activation.getVariable('awsSecretAccessKey')
|
|
1388
|
+
}
|
|
1389
|
+
}, log);
|
|
1390
|
+
|
|
1391
|
+
try {
|
|
1392
|
+
// WIRE TAP: Capture message before sending (NO MODIFICATION)
|
|
1393
|
+
const auditRecord = {
|
|
1394
|
+
timestamp: new Date().toISOString(),
|
|
1395
|
+
eventType: 'order_event',
|
|
1396
|
+
direction: 'outbound',
|
|
1397
|
+
destination: 'fluent_event_api',
|
|
1398
|
+
payload: orderEvent,
|
|
1399
|
+
metadata: {
|
|
1400
|
+
entityRef: orderEvent.entityRef,
|
|
1401
|
+
eventName: orderEvent.name,
|
|
1402
|
+
retailerId: orderEvent.retailerId
|
|
1403
|
+
}
|
|
1404
|
+
};
|
|
1405
|
+
|
|
1406
|
+
// Write audit record to S3 (asynchronously, don't block main flow)
|
|
1407
|
+
const auditKey = `audit/order-events/${orderEvent.entityRef}-${Date.now()}.json`;
|
|
1408
|
+
const auditPromise = auditS3.uploadFile(
|
|
1409
|
+
auditKey,
|
|
1410
|
+
Buffer.from(JSON.stringify(auditRecord, null, 2), 'utf-8')
|
|
1411
|
+
).catch((error) => {
|
|
1412
|
+
// Audit failure should NOT break main flow
|
|
1413
|
+
log.error('Audit logging failed', error as Error, { auditKey });
|
|
1414
|
+
});
|
|
1415
|
+
|
|
1416
|
+
// Send event to Fluent (main flow continues)
|
|
1417
|
+
const eventResponse = await client.sendEvent(orderEvent, 'async');
|
|
1418
|
+
|
|
1419
|
+
// Wait for audit to complete (optional - for testing)
|
|
1420
|
+
await auditPromise;
|
|
1421
|
+
|
|
1422
|
+
log.info('Order event sent with audit', {
|
|
1423
|
+
entityRef: orderEvent.entityRef,
|
|
1424
|
+
eventId: eventResponse,
|
|
1425
|
+
auditKey
|
|
1426
|
+
});
|
|
1427
|
+
|
|
1428
|
+
return eventResponse;
|
|
1429
|
+
} finally {
|
|
1430
|
+
// No disposal needed - audit is fire-and-forget
|
|
1431
|
+
}
|
|
1432
|
+
}
|
|
1433
|
+
|
|
1434
|
+
// Usage example
|
|
1435
|
+
async function processOrder(order: any, ctx: any, log: any) {
|
|
1436
|
+
const orderEvent: FluentEvent = {
|
|
1437
|
+
name: 'order.created',
|
|
1438
|
+
entityType: 'ORDER',
|
|
1439
|
+
entityRef: order.ref,
|
|
1440
|
+
retailerId: order.retailerId,
|
|
1441
|
+
attributes: [
|
|
1442
|
+
{ name: 'totalPrice', type: 'FLOAT', value: order.totalPrice },
|
|
1443
|
+
{ name: 'customerEmail', type: 'STRING', value: order.customer.email }
|
|
1444
|
+
]
|
|
1445
|
+
};
|
|
1446
|
+
|
|
1447
|
+
// WIRE TAP in action - event is audited without modification
|
|
1448
|
+
await sendOrderEventWithAudit(orderEvent, ctx, log);
|
|
1449
|
+
}
|
|
1450
|
+
```
|
|
1451
|
+
|
|
1452
|
+
**When to Use:**
|
|
1453
|
+
- Compliance requires message audit trails
|
|
1454
|
+
- Debugging integration flows (inspect message contents)
|
|
1455
|
+
- Performance monitoring (message throughput, latency)
|
|
1456
|
+
|
|
1457
|
+
---
|
|
1458
|
+
|
|
1459
|
+
## Pattern Summary Matrix
|
|
1460
|
+
|
|
1461
|
+
| Pattern | Support Status | Primary Purpose | SDK Building Blocks | Fluent Use Case |
|
|
1462
|
+
|---------|---------------|----------------|-------------------|----------------|
|
|
1463
|
+
| **Message Router** | ✅ Fully Supported | Route to different APIs | `sendEvent()`, `createJob()`, `graphql()` | Route orders by type to Event/Batch/GraphQL |
|
|
1464
|
+
| **Content-Based Router** | ✅ Fully Supported | Route based on data values | `UniversalMapper`, conditional logic | Route inventory by quantity threshold |
|
|
1465
|
+
| **Content Enricher** | ✅ Fully Supported | Add data from external sources | `graphql()`, `UniversalMapper` | Enrich orders with product details |
|
|
1466
|
+
| **Message Translator** | ✅ Fully Supported | Convert between formats | `CSVParserService`, `UniversalMapper`, `XMLBuilder` | Translate supplier CSV to Fluent Batch format |
|
|
1467
|
+
| **Splitter** | ✅ Fully Supported | Break message into parts | `sendBatch()`, `Promise.all()`, Parsers | Split large order file into parallel batches |
|
|
1468
|
+
| **Aggregator** | ✅ Fully Supported | Combine multiple messages | `graphql()` with pagination, Map/Set | Aggregate inventory across locations |
|
|
1469
|
+
| **Polling Consumer** | ✅ Fully Supported | Periodic endpoint polling | `schedule()`, `SftpDataSource.listFiles()` | Poll SFTP every 2 hours for inventory files |
|
|
1470
|
+
| **Idempotent Receiver** | ✅ Fully Supported | Prevent duplicate processing | `StateService`, KV Store, atomic ops | Prevent duplicate webhook order processing |
|
|
1471
|
+
| **Dead Letter Channel** | ✅ Fully Supported | Route failed messages | `S3DataSource.uploadFile()`, error handling | Failed inventory updates to S3 for review |
|
|
1472
|
+
| **Wire Tap** | ✅ Fully Supported | Audit messages (no modification) | `S3DataSource.uploadFile()`, logging | Audit all order events to S3 for compliance |
|
|
1473
|
+
|
|
1474
|
+
**Support Status Legend:**
|
|
1475
|
+
- ✅ **Fully Supported** - Can be built using SDK building blocks with complete examples
|
|
1476
|
+
- ⚠️ **Partially Supported** - Requires additional custom code beyond SDK
|
|
1477
|
+
- ❌ **Not Supported** - Not achievable with current SDK capabilities
|
|
1478
|
+
|
|
1479
|
+
---
|
|
1480
|
+
|
|
1481
|
+
## Additional Resources
|
|
1482
|
+
|
|
1483
|
+
### SDK Documentation
|
|
1484
|
+
- **Architecture**: `docs/04-REFERENCE/architecture/readme.md` - System design and data flow
|
|
1485
|
+
- **Decision Tree**: `docs/00-START-HERE/DECISION-TREE.md` - Which approach to use?
|
|
1486
|
+
- **Troubleshooting**: `docs/00-START-HERE/troubleshooting-quick-reference.md` - Common issues
|
|
1487
|
+
|
|
1488
|
+
### Core Guides
|
|
1489
|
+
- **Ingestion**: `docs/02-CORE-GUIDES/ingestion/readme.md` - Data into Fluent
|
|
1490
|
+
- **Extraction**: `docs/02-CORE-GUIDES/extraction/readme.md` - Data from Fluent
|
|
1491
|
+
- **Mapping**: `docs/02-CORE-GUIDES/mapping/modules/` - Field transformation
|
|
1492
|
+
- **Webhook Validation**: `docs/02-CORE-GUIDES/webhook-validation/readme.md` - Signature validation
|
|
1493
|
+
|
|
1494
|
+
### Pattern Guides
|
|
1495
|
+
- **Error Handling**: `docs/03-PATTERN-GUIDES/error-handling/readme.md` - Retry, circuit breakers
|
|
1496
|
+
- **File Operations**: `docs/03-PATTERN-GUIDES/file-operations/readme.md` - S3/SFTP patterns
|
|
1497
|
+
- **Integration Patterns**: `docs/03-PATTERN-GUIDES/integration-patterns/readme.md` - Real-time, batch, delta sync
|
|
1498
|
+
|
|
1499
|
+
### Templates
|
|
1500
|
+
- **Ingestion Templates**: `docs/01-TEMPLATES/versori/scheduled/ingestion/` - Complete examples
|
|
1501
|
+
- **Extraction Templates**: `docs/01-TEMPLATES/versori/scheduled/extraction/` - GraphQL extraction patterns
|
|
1502
|
+
|
|
1503
|
+
---
|
|
1504
|
+
|
|
1505
|
+
## Summary
|
|
1506
|
+
|
|
1507
|
+
This guide demonstrated how to build 10 essential Enterprise Integration Patterns using Fluent Connect SDK building blocks:
|
|
1508
|
+
|
|
1509
|
+
**Key Takeaways:**
|
|
1510
|
+
|
|
1511
|
+
1. **SDK is a Toolkit** - Patterns are YOUR orchestration logic, not framework-imposed
|
|
1512
|
+
2. **Composable Primitives** - Combine clients, parsers, mappers, data sources to build patterns
|
|
1513
|
+
3. **Real Fluent Domains** - All examples use actual Order and Inventory entities
|
|
1514
|
+
4. **Production-Ready Code** - Every example uses real SDK methods and types
|
|
1515
|
+
|
|
1516
|
+
**Next Steps:**
|
|
1517
|
+
|
|
1518
|
+
- Review `docs/00-START-HERE/DECISION-TREE.md` for pattern selection guidance
|
|
1519
|
+
- Explore `docs/01-TEMPLATES/` for complete working implementations
|
|
1520
|
+
- See `docs/03-PATTERN-GUIDES/integration-patterns/` for additional pattern details
|
|
1521
|
+
|
|
1522
|
+
**Remember:** The SDK provides the tools. YOU define the integration patterns.
|
|
1523
|
+
|
|
1524
|
+
---
|
|
1525
|
+
|
|
1526
|
+
**Document Version:** 1.0.0
|
|
1527
|
+
**SDK Version:** 0.1.39
|
|
1528
|
+
**Last Verified:** 2025-11-05
|