@mojaloop/central-ledger 17.7.8 → 17.8.0-orb.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (147) hide show
  1. package/.ncurc.yaml +3 -1
  2. package/.nvmrc +1 -1
  3. package/.nycrc.yml +16 -1
  4. package/Dockerfile +9 -10
  5. package/README.md +39 -9
  6. package/audit-ci.jsonc +15 -2
  7. package/config/default.json +17 -1
  8. package/docker/central-ledger/default.json +9 -0
  9. package/docker/config-modifier/configs/central-ledger.js +25 -0
  10. package/docker/env.sh +15 -0
  11. package/docker/kafka/scripts/provision.sh +4 -1
  12. package/docker/ml-api-adapter/default.json +4 -1
  13. package/docker-compose.yml +90 -2
  14. package/documentation/db/erd-transfer-timeout.png +0 -0
  15. package/documentation/db/erd-transfer-timeout.txt +81 -0
  16. package/documentation/fx-implementation/README.md +48 -0
  17. package/documentation/fx-implementation/assets/fx-position-movements.drawio.svg +4 -0
  18. package/documentation/fx-implementation/assets/test-scenario.drawio.svg +4 -0
  19. package/documentation/sequence-diagrams/Handler - FX timeout.plantuml +123 -0
  20. package/documentation/sequence-diagrams/Handler - FX timeout.png +0 -0
  21. package/documentation/sequence-diagrams/Handler - timeout.plantuml +81 -0
  22. package/documentation/sequence-diagrams/Handler - timeout.png +0 -0
  23. package/documentation/state-diagrams/transfer-ML-spec-states-diagram.png +0 -0
  24. package/documentation/state-diagrams/transfer-internal-states-diagram.png +0 -0
  25. package/documentation/state-diagrams/transfer-internal-states.plantuml +75 -0
  26. package/documentation/state-diagrams/transfer-states.plantuml +13 -0
  27. package/migrations/310204_transferParticipant-participantId.js +52 -0
  28. package/migrations/310403_participantPositionChange-participantCurrencyId.js +47 -0
  29. package/migrations/310404_participantPositionChange-change.js +46 -0
  30. package/migrations/600010_fxTransferType.js +43 -0
  31. package/migrations/600011_fxTransferType-indexes.js +38 -0
  32. package/migrations/600012_fxParticipantCurrencyType.js +43 -0
  33. package/migrations/600013_fxParticipantCurrencyType-indexes.js +38 -0
  34. package/migrations/600100_fxTransferDuplicateCheck.js +42 -0
  35. package/migrations/600110_fxTransferErrorDuplicateCheck.js.js +17 -0
  36. package/migrations/600200_fxTransfer.js +51 -0
  37. package/migrations/600201_fxTransfer-indexes.js +40 -0
  38. package/migrations/600400_fxTransferStateChange.js +46 -0
  39. package/migrations/600401_fxTransferStateChange-indexes.js +40 -0
  40. package/migrations/600501_fxWatchList.js +46 -0
  41. package/migrations/600502_fxWatchList-indexes.js +40 -0
  42. package/migrations/600600_fxTransferFulfilmentDuplicateCheck.js +43 -0
  43. package/migrations/600601_fxTransferFulfilmentDuplicateCheck-indexes.js +38 -0
  44. package/migrations/600700_fxTransferFulfilment.js +47 -0
  45. package/migrations/600701_fxTransferFulfilment-indexes.js +43 -0
  46. package/migrations/600800_fxTransferExtension.js +47 -0
  47. package/migrations/601400_fxTransferTimeout.js +43 -0
  48. package/migrations/601401_fxTransferTimeout-indexes.js +37 -0
  49. package/migrations/601500_fxTransferError.js +44 -0
  50. package/migrations/601501_fxTransferError-indexes.js +37 -0
  51. package/migrations/610200_fxTransferParticipant.js +52 -0
  52. package/migrations/610201_fxTransferParticipant-indexes.js +44 -0
  53. package/migrations/610202_fxTransferParticipant-participantId.js +52 -0
  54. package/migrations/610403_participantPositionChange-fxTransfer.js +46 -0
  55. package/migrations/910101_feature904DataMigration.js +46 -52
  56. package/migrations/910102_feature949DataMigration.js +219 -225
  57. package/migrations/950104_settlementModel-settlementAccountTypeId.js +15 -20
  58. package/migrations/950108_participantProxy.js +18 -0
  59. package/migrations/950109_fxQuote.js +19 -0
  60. package/migrations/950110_fxQuoteResponse.js +25 -0
  61. package/migrations/950111_fxQuoteError.js +23 -0
  62. package/migrations/950113_fxQuoteDuplicateCheck.js +18 -0
  63. package/migrations/950114_fxQuoteResponseDuplicateCheck.js +21 -0
  64. package/migrations/950115_fxQuoteConversionTerms.js +36 -0
  65. package/migrations/950116_fxQuoteConversionTermsExtension.js +21 -0
  66. package/migrations/950117_fxQuoteResponseConversionTerms.js +39 -0
  67. package/migrations/950118_fxQuoteResponseConversionTermsExtension.js +21 -0
  68. package/migrations/950119_fxCharge.js +27 -0
  69. package/migrations/960100_create_externalParticipant.js +47 -0
  70. package/migrations/960110_alter_transferParticipant__addFiled_externalParticipantId.js +50 -0
  71. package/migrations/960111_alter_fxTransferParticipant__addFiled_externalParticipantId.js +50 -0
  72. package/package.json +27 -20
  73. package/seeds/endpointType.js +18 -0
  74. package/seeds/fxParticipantCurrencyType.js +45 -0
  75. package/seeds/fxTransferType.js +45 -0
  76. package/seeds/participant.js +2 -1
  77. package/seeds/transferParticipantRoleType.js +9 -0
  78. package/seeds/transferState.js +10 -0
  79. package/src/api/interface/swagger.json +23 -30
  80. package/src/api/participants/handler.js +6 -2
  81. package/src/api/participants/routes.js +8 -7
  82. package/src/api/root/handler.js +15 -5
  83. package/src/domain/fx/cyril.js +466 -0
  84. package/src/domain/fx/index.js +107 -0
  85. package/src/domain/participant/index.js +108 -1
  86. package/src/domain/position/abort.js +215 -0
  87. package/src/domain/position/binProcessor.js +361 -99
  88. package/src/domain/position/fulfil.js +252 -111
  89. package/src/domain/position/fx-fulfil.js +138 -0
  90. package/src/domain/position/fx-prepare.js +280 -0
  91. package/src/domain/position/fx-timeout-reserved.js +159 -0
  92. package/src/domain/position/index.js +1 -0
  93. package/src/domain/position/prepare.js +69 -49
  94. package/src/domain/position/timeout-reserved.js +162 -0
  95. package/src/domain/timeout/index.js +26 -2
  96. package/src/domain/transfer/index.js +22 -5
  97. package/src/domain/transfer/transform.js +19 -6
  98. package/src/handlers/admin/handler.js +0 -2
  99. package/src/handlers/bulk/fulfil/handler.js +5 -5
  100. package/src/handlers/bulk/get/handler.js +5 -5
  101. package/src/handlers/bulk/prepare/handler.js +9 -9
  102. package/src/handlers/bulk/processing/handler.js +8 -7
  103. package/src/handlers/bulk/shared/validator.js +1 -1
  104. package/src/handlers/positions/handler.js +23 -10
  105. package/src/handlers/positions/handlerBatch.js +54 -26
  106. package/src/handlers/register.js +2 -1
  107. package/src/handlers/timeouts/handler.js +212 -68
  108. package/src/handlers/transfers/FxFulfilService.js +387 -0
  109. package/src/handlers/transfers/createRemittanceEntity.js +106 -0
  110. package/src/handlers/transfers/dto.js +53 -0
  111. package/src/handlers/transfers/handler.js +609 -571
  112. package/src/handlers/transfers/prepare.js +572 -0
  113. package/src/handlers/transfers/validator.js +75 -14
  114. package/src/lib/cache.js +1 -1
  115. package/src/lib/config.js +6 -3
  116. package/src/lib/healthCheck/subServiceHealth.js +12 -2
  117. package/src/lib/proxyCache.js +131 -0
  118. package/src/models/bulkTransfer/facade.js +58 -82
  119. package/src/models/fxTransfer/duplicateCheck.js +153 -0
  120. package/src/models/fxTransfer/fxTransfer.js +578 -0
  121. package/src/models/fxTransfer/fxTransferError.js +53 -0
  122. package/src/models/fxTransfer/fxTransferExtension.js +41 -0
  123. package/src/models/fxTransfer/fxTransferTimeout.js +68 -0
  124. package/src/models/fxTransfer/index.js +15 -0
  125. package/src/models/fxTransfer/stateChange.js +47 -0
  126. package/src/models/fxTransfer/watchList.js +49 -0
  127. package/src/models/ledgerAccountType/ledgerAccountType.js +8 -32
  128. package/src/models/misc/segment.js +0 -1
  129. package/src/models/participant/externalParticipant.js +96 -0
  130. package/src/models/participant/externalParticipantCached.js +148 -0
  131. package/src/models/participant/facade.js +192 -123
  132. package/src/models/participant/participant.js +2 -1
  133. package/src/models/participant/participantCurrency.js +1 -1
  134. package/src/models/participant/participantPosition.js +2 -8
  135. package/src/models/position/batch.js +90 -1
  136. package/src/models/position/facade.js +5 -5
  137. package/src/models/position/participantPositionChanges.js +68 -0
  138. package/src/models/settlement/settlementModel.js +5 -23
  139. package/src/models/transfer/facade.js +806 -447
  140. package/src/shared/constants.js +52 -0
  141. package/src/shared/fspiopErrorFactory.js +131 -0
  142. package/src/shared/logger/index.js +8 -0
  143. package/src/shared/loggingPlugin.js +43 -0
  144. package/src/shared/plugins.js +6 -0
  145. package/src/shared/setup.js +10 -0
  146. package/test-integration.Dockerfile +1 -1
  147. package/test.Dockerfile +1 -1
package/.ncurc.yaml CHANGED
@@ -9,5 +9,7 @@ reject: [
9
9
  "get-port",
10
10
  # sinon v17.0.1 causes 58 tests to fail. This will need to be resolved in a future story.
11
11
  # Issue is tracked here: https://github.com/mojaloop/project/issues/3616
12
- "sinon"
12
+ "sinon",
13
+ # glob >= 11 requires node >= 20
14
+ "glob"
13
15
  ]
package/.nvmrc CHANGED
@@ -1 +1 @@
1
- 18.17.1
1
+ 18.20.3
package/.nycrc.yml CHANGED
@@ -17,5 +17,20 @@ exclude: [
17
17
  "**/node_modules/**",
18
18
  '**/migrations/**',
19
19
  '**/ddl/**',
20
- '**/bulk*/**'
20
+ '**/bulk*/**',
21
+ 'src/shared/logger/**',
22
+ 'src/shared/loggingPlugin.js',
23
+ 'src/shared/constants.js',
24
+ 'src/domain/position/index.js',
25
+ 'src/domain/position/binProcessor.js',
26
+ 'src/handlers/positions/handler.js',
27
+ 'src/handlers/transfers/createRemittanceEntity.js',
28
+ 'src/handlers/transfers/FxFulfilService.js',
29
+ 'src/models/position/batch.js',
30
+ 'src/models/fxTransfer/**',
31
+ 'src/models/participant/externalParticipantCached.js', # todo: figure out why it shows only 50% coverage in Branch
32
+ 'src/models/transfer/facade.js', ## add more test coverage
33
+ 'src/shared/fspiopErrorFactory.js',
34
+ 'src/lib/proxyCache.js' # todo: remove this line after adding test coverage
21
35
  ]
36
+ ## todo: increase test coverage before merging feat/fx-impl to main branch
package/Dockerfile CHANGED
@@ -3,26 +3,27 @@ ARG NODE_VERSION=lts-alpine
3
3
 
4
4
  # NOTE: Ensure you set NODE_VERSION Build Argument as follows...
5
5
  #
6
- # export NODE_VERSION="$(cat .nvmrc)-alpine" \
7
- # docker build \
8
- # --build-arg NODE_VERSION=$NODE_VERSION \
9
- # -t mojaloop/central-ledger:local \
10
- # . \
6
+ # export NODE_VERSION="$(cat .nvmrc)-alpine"
7
+ # docker build \
8
+ # --build-arg NODE_VERSION=$NODE_VERSION \
9
+ # -t mojaloop/central-ledger:local \
10
+ # .
11
11
  #
12
12
 
13
13
  # Build Image
14
- FROM node:${NODE_VERSION} as builder
14
+ FROM node:${NODE_VERSION} AS builder
15
15
 
16
16
  WORKDIR /opt/app
17
17
 
18
18
  RUN apk --no-cache add git
19
- RUN apk add --no-cache -t build-dependencies make gcc g++ python3 libtool openssl-dev autoconf automake bash \
19
+ RUN apk add --no-cache -t build-dependencies make gcc g++ python3 py3-setuptools libtool openssl-dev autoconf automake bash \
20
20
  && cd $(npm root -g)/npm \
21
21
  && npm install -g node-gyp
22
22
 
23
23
  COPY package.json package-lock.json* /opt/app/
24
24
 
25
25
  RUN npm ci
26
+ RUN npm prune --omit=dev
26
27
 
27
28
  FROM node:${NODE_VERSION}
28
29
  WORKDIR /opt/app
@@ -32,7 +33,7 @@ RUN mkdir ./logs && touch ./logs/combined.log
32
33
  RUN ln -sf /dev/stdout ./logs/combined.log
33
34
 
34
35
  # Create a non-root user: ml-user
35
- RUN adduser -D ml-user
36
+ RUN adduser -D ml-user
36
37
  USER ml-user
37
38
 
38
39
  COPY --chown=ml-user --from=builder /opt/app .
@@ -43,7 +44,5 @@ COPY migrations /opt/app/migrations
43
44
  COPY seeds /opt/app/seeds
44
45
  COPY test /opt/app/test
45
46
 
46
- RUN npm prune --production
47
-
48
47
  EXPOSE 3001
49
48
  CMD ["npm", "run", "start"]
package/README.md CHANGED
@@ -56,7 +56,7 @@ Or via docker build directly:
56
56
 
57
57
  ```bash
58
58
  docker build \
59
- --build-arg NODE_VERSION="$(cat .nvmrc)-alpine" \
59
+ --build-arg NODE_VERSION="$(cat .nvmrc)-alpine3.19" \
60
60
  -t mojaloop/ml-api-adapter:local \
61
61
  .
62
62
  ```
@@ -113,12 +113,14 @@ NOTE: Only POSITION.PREPARE and POSITION.COMMIT is supported at this time, with
113
113
 
114
114
  Batch processing can be enabled in the transfer execution flow. Follow the steps below to enable batch processing for a more efficient transfer execution:
115
115
 
116
+ Note: The position messages with action 'FX_PREPARE', 'FX_COMMIT' and 'FX_TIMEOUT_RESERVED' are only supported in batch processing.
117
+
116
118
  - **Step 1:** **Create a New Kafka Topic**
117
119
 
118
120
  Create a new Kafka topic named `topic-transfer-position-batch` to handle batch processing events.
119
121
  - **Step 2:** **Configure Action Type Mapping**
120
122
 
121
- Point the prepare handler to the newly created topic for the action type `prepare` using the `KAFKA.EVENT_TYPE_ACTION_TOPIC_MAP` configuration as shown below:
123
+ Point the prepare handler to the newly created topic for the action types those are supported in batch processing using the `KAFKA.EVENT_TYPE_ACTION_TOPIC_MAP` configuration as shown below:
122
124
  ```
123
125
  "KAFKA": {
124
126
  "EVENT_TYPE_ACTION_TOPIC_MAP" : {
@@ -126,8 +128,12 @@ Batch processing can be enabled in the transfer execution flow. Follow the steps
126
128
  "PREPARE": "topic-transfer-position-batch",
127
129
  "BULK_PREPARE": "topic-transfer-position",
128
130
  "COMMIT": "topic-transfer-position-batch",
131
+ "FX_COMMIT": "topic-transfer-position-batch",
129
132
  "BULK_COMMIT": "topic-transfer-position",
130
133
  "RESERVE": "topic-transfer-position",
134
+ "FX_PREPARE": "topic-transfer-position-batch",
135
+ "TIMEOUT_RESERVED": "topic-transfer-position-batch",
136
+ "FX_TIMEOUT_RESERVED": "topic-transfer-position-batch"
131
137
  }
132
138
  }
133
139
  }
@@ -185,7 +191,8 @@ If you want to run integration tests in a repetitive manner, you can startup the
185
191
  Start containers required for Integration Tests
186
192
 
187
193
  ```bash
188
- docker-compose -f docker-compose.yml up -d mysql kafka init-kafka kafka-debug-console
194
+ source ./docker/env.sh
195
+ docker compose up -d mysql kafka init-kafka redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5
189
196
  ```
190
197
 
191
198
  Run wait script which will report once all required containers are up and running
@@ -220,7 +227,8 @@ If you want to run integration tests in a repetitive manner, you can startup the
220
227
  Start containers required for Integration Tests, including a `central-ledger` container which will be used as a proxy shell.
221
228
 
222
229
  ```bash
223
- docker-compose -f docker-compose.yml -f docker-compose.integration.yml up -d kafka mysql central-ledger
230
+ source ./docker/env.sh
231
+ docker-compose -f docker-compose.yml -f docker-compose.integration.yml up -d kafka mysql central-ledger init-kafka redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5
224
232
  ```
225
233
 
226
234
  Run the Integration Tests from the `central-ledger` container
@@ -235,24 +243,42 @@ If you want to run override position topic tests you can repeat the above and us
235
243
 
236
244
  #### For running integration tests for batch processing interactively
237
245
  - Run dependecies
238
- ```
239
- docker-compose up -d mysql kafka init-kafka kafka-debug-console
246
+ ```bash
247
+ source ./docker/env.sh
248
+ docker compose up -d mysql kafka init-kafka redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5
240
249
  npm run wait-4-docker
241
250
  ```
242
251
  - Run central-ledger services
243
252
  ```
244
253
  nvm use
245
254
  npm run migrate
246
- env "CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__PREPARE=topic-transfer-position-batch" npm start
255
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__PREPARE=topic-transfer-position-batch
256
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__COMMIT=topic-transfer-position-batch
257
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__RESERVE=topic-transfer-position-batch
258
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__TIMEOUT_RESERVED=topic-transfer-position-batch
259
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__FX_TIMEOUT_RESERVED=topic-transfer-position-batch
260
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__ABORT=topic-transfer-position-batch
261
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__FX_ABORT=topic-transfer-position-batch
262
+ npm start
247
263
  ```
248
264
  - Additionally, run position batch handler in a new terminal
249
265
  ```
266
+ nvm use
250
267
  export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__PREPARE=topic-transfer-position-batch
268
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__FX_PREPARE=topic-transfer-position-batch
251
269
  export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__COMMIT=topic-transfer-position-batch
270
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__TIMEOUT_RESERVED=topic-transfer-position-batch
271
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__FX_TIMEOUT_RESERVED=topic-transfer-position-batch
272
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__ABORT=topic-transfer-position-batch
273
+ export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__FX_ABORT=topic-transfer-position-batch
252
274
  export CLEDG_HANDLERS__API__DISABLED=true
253
275
  node src/handlers/index.js handler --positionbatch
254
276
  ```
255
- - Run tests using `npx tape 'test/integration-override/**/handlerBatch.test.js'`
277
+ - Run tests using the following commands in a new terminal
278
+ ```
279
+ nvm use
280
+ npm run test:int-override
281
+ ```
256
282
 
257
283
 
258
284
  If you want to just run all of the integration suite non-interactively then use npm run `test:integration`.
@@ -263,7 +289,11 @@ It will handle docker start up, migration, service starting and testing. Be sure
263
289
  If you want to run functional tests locally utilizing the [ml-core-test-harness](https://github.com/mojaloop/ml-core-test-harness), you can run the following commands:
264
290
 
265
291
  ```bash
266
- docker build -t mojaloop/central-ledger:local .
292
+ export NODE_VERSION="$(cat .nvmrc)-alpine"
293
+ docker build \
294
+ --build-arg NODE_VERSION=$NODE_VERSION \
295
+ -t mojaloop/central-ledger:local \
296
+ .
267
297
  ```
268
298
 
269
299
  ```bash
package/audit-ci.jsonc CHANGED
@@ -4,6 +4,19 @@
4
4
  // Only use one of ["low": true, "moderate": true, "high": true, "critical": true]
5
5
  "moderate": true,
6
6
  "allowlist": [ // NOTE: Please add as much information as possible to any items added to the allowList
7
- "GHSA-w5p7-h5w8-2hfq" // tap-spec>tap-out>trim; This has been analyzed and this is acceptable as it is used to run tests.
7
+ "GHSA-w5p7-h5w8-2hfq", // tap-spec>tap-out>trim; This has been analyzed and this is acceptable as it is used to run tests.
8
+ "GHSA-2mvq-xp48-4c77", // https://github.com/advisories/GHSA-2mvq-xp48-4c77
9
+ "GHSA-5854-jvxx-2cg9", // https://github.com/advisories/GHSA-5854-jvxx-2cg9
10
+ "GHSA-7hx8-2rxv-66xv", // https://github.com/advisories/GHSA-7hx8-2rxv-66xv
11
+ "GHSA-c429-5p7v-vgjp", // https://github.com/advisories/GHSA-c429-5p7v-vgjp
12
+ "GHSA-g64q-3vg8-8f93", // https://github.com/advisories/GHSA-g64q-3vg8-8f93
13
+ "GHSA-mg85-8mv5-ffjr", // https://github.com/advisories/GHSA-mg85-8mv5-ffjr
14
+ "GHSA-8hc4-vh64-cxmj", // https://github.com/advisories/GHSA-8hc4-vh64-cxmj
15
+ "GHSA-952p-6rrq-rcjv", // https://github.com/advisories/GHSA-952p-6rrq-rcjv
16
+ "GHSA-9wv6-86v2-598j", // https://github.com/advisories/GHSA-9wv6-86v2-598j
17
+ "GHSA-qwcr-r2fm-qrc7", // https://github.com/advisories/GHSA-qwcr-r2fm-qrc7
18
+ "GHSA-cm22-4g7w-348p", // https://github.com/advisories/GHSA-cm22-4g7w-348p
19
+ "GHSA-m6fv-jmcg-4jfg", // https://github.com/advisories/GHSA-m6fv-jmcg-4jfg
20
+ "GHSA-qw6h-vgh9-j6wx" // https://github.com/advisories/GHSA-qw6h-vgh9-j6wx
8
21
  ]
9
- }
22
+ }
@@ -78,20 +78,36 @@
78
78
  },
79
79
  "INTERNAL_TRANSFER_VALIDITY_SECONDS": "432000",
80
80
  "ENABLE_ON_US_TRANSFERS": false,
81
+ "PAYEE_PARTICIPANT_CURRENCY_VALIDATION_ENABLED": false,
81
82
  "CACHE": {
82
83
  "CACHE_ENABLED": false,
83
84
  "MAX_BYTE_SIZE": 10000000,
84
85
  "EXPIRES_IN_MS": 1000
85
86
  },
87
+ "PROXY_CACHE": {
88
+ "enabled": true,
89
+ "type": "redis-cluster",
90
+ "proxyConfig": {
91
+ "cluster": [
92
+ { "host": "localhost", "port": 6379 }
93
+ ]
94
+ }
95
+ },
86
96
  "API_DOC_ENDPOINTS_ENABLED": true,
87
97
  "KAFKA": {
88
98
  "EVENT_TYPE_ACTION_TOPIC_MAP" : {
89
99
  "POSITION":{
90
100
  "PREPARE": null,
101
+ "FX_PREPARE": "topic-transfer-position-batch",
91
102
  "BULK_PREPARE": null,
92
103
  "COMMIT": null,
93
104
  "BULK_COMMIT": null,
94
- "RESERVE": null
105
+ "RESERVE": null,
106
+ "FX_RESERVE": "topic-transfer-position-batch",
107
+ "TIMEOUT_RESERVED": null,
108
+ "FX_TIMEOUT_RESERVED": "topic-transfer-position-batch",
109
+ "ABORT": null,
110
+ "FX_ABORT": "topic-transfer-position-batch"
95
111
  }
96
112
  },
97
113
  "TOPIC_TEMPLATES": {
@@ -82,6 +82,15 @@
82
82
  "MAX_BYTE_SIZE": 10000000,
83
83
  "EXPIRES_IN_MS": 1000
84
84
  },
85
+ "PROXY_CACHE": {
86
+ "enabled": true,
87
+ "type": "redis-cluster",
88
+ "proxyConfig": {
89
+ "cluster": [
90
+ { "host": "redis-node-0", "port": 6379 }
91
+ ]
92
+ }
93
+ },
85
94
  "KAFKA": {
86
95
  "TOPIC_TEMPLATES": {
87
96
  "PARTICIPANT_TOPIC_TEMPLATE": {
@@ -12,7 +12,25 @@ module.exports = {
12
12
  PASSWORD: '',
13
13
  DATABASE: 'mlos'
14
14
  },
15
+ PROXY_CACHE: {
16
+ enabled: true,
17
+ type: 'redis',
18
+ proxyConfig: {
19
+ cluster: undefined,
20
+ host: 'redis',
21
+ port: 6379
22
+ }
23
+ },
15
24
  KAFKA: {
25
+ EVENT_TYPE_ACTION_TOPIC_MAP: {
26
+ POSITION: {
27
+ PREPARE: 'topic-transfer-position-batch',
28
+ BULK_PREPARE: null,
29
+ COMMIT: 'topic-transfer-position-batch',
30
+ BULK_COMMIT: null,
31
+ RESERVE: 'topic-transfer-position-batch'
32
+ }
33
+ },
16
34
  CONSUMER: {
17
35
  BULK: {
18
36
  PREPARE: {
@@ -72,6 +90,13 @@ module.exports = {
72
90
  'metadata.broker.list': 'kafka:29092'
73
91
  }
74
92
  }
93
+ },
94
+ POSITION_BATCH: {
95
+ config: {
96
+ rdkafkaConf: {
97
+ 'metadata.broker.list': 'kafka:29092'
98
+ }
99
+ }
75
100
  }
76
101
  },
77
102
  ADMIN: {
package/docker/env.sh ADDED
@@ -0,0 +1,15 @@
1
+ #!/bin/sh
2
+
3
+ # Retrieve the external IP address of the host machine (on macOS)
4
+ # or the IP address of the docker0 interface (on Linux)
5
+ get_external_ip() {
6
+ if [ "$(uname)" = "Linux" ]; then
7
+ echo "$(ip addr show docker0 | grep 'inet ' | awk '{print $2}' | cut -d/ -f1)"
8
+ else
9
+ # Need to find a way to support Windows here
10
+ echo "$(route get ifconfig.me | grep interface | sed -e 's/.*: //' | xargs ipconfig getifaddr)"
11
+ fi
12
+ }
13
+
14
+ # set/override dynamic variables
15
+ export REDIS_CLUSTER_ANNOUNCE_IP=$(get_external_ip)
@@ -25,8 +25,11 @@ topics=(
25
25
  "topic-bulk-prepare"
26
26
  "topic-bulk-fulfil"
27
27
  "topic-bulk-processing"
28
- "topic-bulk-get",
28
+ "topic-bulk-get"
29
29
  "topic-transfer-position-batch"
30
+ "topic-fx-quotes-post"
31
+ "topic-fx-quotes-put"
32
+ "topic-fx-quotes-get"
30
33
  )
31
34
 
32
35
  # Loop through the topics and create them using kafka-topics.sh
@@ -1,4 +1,8 @@
1
1
  {
2
+ "HUB_PARTICIPANT": {
3
+ "ID": 1,
4
+ "NAME": "Hub"
5
+ },
2
6
  "PORT": 3000,
3
7
  "HOSTNAME": "http://ml-api-adapter",
4
8
  "ENDPOINT_SOURCE_URL": "http://host.docker.internal:3001",
@@ -13,7 +17,6 @@
13
17
  },
14
18
  "JWS": {
15
19
  "JWS_SIGN": false,
16
- "FSPIOP_SOURCE_TO_SIGN": "switch",
17
20
  "JWS_SIGNING_KEY_PATH": "secrets/jwsSigningKey.key"
18
21
  }
19
22
  },
@@ -1,9 +1,22 @@
1
- version: "3.7"
2
-
3
1
  networks:
4
2
  cl-mojaloop-net:
5
3
  name: cl-mojaloop-net
6
4
 
5
+
6
+ # @see https://uninterrupted.tech/blog/hassle-free-redis-cluster-deployment-using-docker/
7
+ x-redis-node: &REDIS_NODE
8
+ image: docker.io/bitnami/redis-cluster:6.2.14
9
+ environment: &REDIS_ENVS
10
+ ALLOW_EMPTY_PASSWORD: yes
11
+ REDIS_CLUSTER_DYNAMIC_IPS: no
12
+ REDIS_CLUSTER_ANNOUNCE_IP: ${REDIS_CLUSTER_ANNOUNCE_IP}
13
+ REDIS_NODES: redis-node-0:6379 redis-node-1:9301 redis-node-2:9302 redis-node-3:9303 redis-node-4:9304 redis-node-5:9305
14
+ healthcheck:
15
+ test: [ "CMD", "redis-cli", "ping" ]
16
+ timeout: 2s
17
+ networks:
18
+ - cl-mojaloop-net
19
+
7
20
  services:
8
21
  central-ledger:
9
22
  image: mojaloop/central-ledger:local
@@ -31,10 +44,14 @@ services:
31
44
  - CLEDG_MONGODB__DISABLED=false
32
45
  networks:
33
46
  - cl-mojaloop-net
47
+ extra_hosts:
48
+ - "redis-node-0:host-gateway"
34
49
  depends_on:
35
50
  - mysql
36
51
  - kafka
37
52
  - objstore
53
+ - redis-node-0
54
+ # - redis
38
55
  healthcheck:
39
56
  test: ["CMD", "sh", "-c" ,"apk --no-cache add curl", "&&", "curl", "http://localhost:3001/health"]
40
57
  timeout: 20s
@@ -94,6 +111,77 @@ services:
94
111
  retries: 10
95
112
  start_period: 40s
96
113
  interval: 30s
114
+
115
+ redis-node-0:
116
+ <<: *REDIS_NODE
117
+ environment:
118
+ <<: *REDIS_ENVS
119
+ REDIS_CLUSTER_CREATOR: yes
120
+ REDIS_PORT_NUMBER: 6379
121
+ depends_on:
122
+ - redis-node-1
123
+ - redis-node-2
124
+ ports:
125
+ - "6379:6379"
126
+ - "16379:16379"
127
+ redis-node-1:
128
+ <<: *REDIS_NODE
129
+ environment:
130
+ <<: *REDIS_ENVS
131
+ REDIS_PORT_NUMBER: 9301
132
+ ports:
133
+ - "9301:9301"
134
+ - "19301:19301"
135
+ redis-node-2:
136
+ <<: *REDIS_NODE
137
+ environment:
138
+ <<: *REDIS_ENVS
139
+ REDIS_PORT_NUMBER: 9302
140
+ ports:
141
+ - "9302:9302"
142
+ - "19302:19302"
143
+ redis-node-3:
144
+ <<: *REDIS_NODE
145
+ environment:
146
+ <<: *REDIS_ENVS
147
+ REDIS_PORT_NUMBER: 9303
148
+ ports:
149
+ - "9303:9303"
150
+ - "19303:19303"
151
+ redis-node-4:
152
+ <<: *REDIS_NODE
153
+ environment:
154
+ <<: *REDIS_ENVS
155
+ REDIS_PORT_NUMBER: 9304
156
+ ports:
157
+ - "9304:9304"
158
+ - "19304:19304"
159
+ redis-node-5:
160
+ <<: *REDIS_NODE
161
+ environment:
162
+ <<: *REDIS_ENVS
163
+ REDIS_PORT_NUMBER: 9305
164
+ ports:
165
+ - "9305:9305"
166
+ - "19305:19305"
167
+
168
+ ## To be used with proxyCache.type === 'redis'
169
+ # redis:
170
+ # image: redis:6.2.4-alpine
171
+ # restart: "unless-stopped"
172
+ # environment:
173
+ # <<: *REDIS_ENVS
174
+ # REDIS_CLUSTER_CREATOR: yes
175
+ # depends_on:
176
+ # - redis-node-1
177
+ # - redis-node-2
178
+ # - redis-node-3
179
+ # - redis-node-4
180
+ # - redis-node-5
181
+ # ports:
182
+ # - "6379:6379"
183
+ # networks:
184
+ # - cl-mojaloop-net
97
185
 
98
186
  mockserver:
99
187
  image: jamesdbloom/mockserver
@@ -0,0 +1,81 @@
1
+ # Visualize on https://erd.surge.sh
2
+ # or https://quick-erd.surge.sh
3
+ #
4
+ # Relationship Types
5
+ # - - one to one
6
+ # -< - one to many
7
+ # >- - many to one
8
+ # >-< - many to many
9
+ # -0 - one to zero or one
10
+ # 0- - zero or one to one
11
+ # 0-0 - zero or one to zero or one
12
+ # -0< - one to zero or many
13
+ # >0- - zero or many to one
14
+ #
15
+ ////////////////////////////////////
16
+
17
+ transfer
18
+ ---------------------
19
+ transferId varchar(36) PK
20
+ amount decimal(18,4)
21
+ currencyId varchar(3) FK - currency.currencyId
22
+ ilpCondition varchar(256)
23
+ expirationDate datetime
24
+ createdDate datetime
25
+
26
+
27
+ transferStateChange__TSC
28
+ ---------------------
29
+ transferStateChangeId bigint UN AI PK
30
+ transferId varchar(36) FK >- transfer.transferId
31
+ transferStateId varchar(50) FK - transferState.transferStateId
32
+ reason varchar(512)
33
+ createdDate datetime
34
+
35
+
36
+ transferTimeout__TT
37
+ ---------------------
38
+ transferTimeoutId bigint UN AI PK
39
+ transferId varchar(36) UNIQUE FK - transfer.transferId
40
+ expirationDate datetime
41
+ createdDate datetime
42
+
43
+
44
+ transferError__TE
45
+ ---------------------
46
+ transferId varchar(36) PK
47
+ transferStateChangeId bigint UN FK - transferStateChange.transferStateChangeId
48
+ errorCode int UN
49
+ errorDescription varchar(128)
50
+ createdDate datetime
51
+
52
+
53
+ segment
54
+ ---------------------
55
+ segmentId int UN AI PK
56
+ segmentType varchar(50)
57
+ enumeration int
58
+ tableName varchar(50)
59
+ value bigint
60
+ changedDate datetime
61
+ # row example: 1, 'timeout', 0, 'transferStateChange', 255, '2024-04-24 18:07:15'
62
+
63
+
64
+ expiringTransfer
65
+ ---------------------
66
+ expiringTransferId bigint UN AI PK
67
+ transferId varchar(36) UNIQUE FK - transfer.transferId
68
+ expirationDate datetime INDEX
69
+ createdDate datetime
70
+ # todo: clarify, how we use this table
71
+
72
+
73
+
74
+ # transfer (557, 340)
75
+ # segment (348, 608)
76
+ # expiringTransfer (1033, 574)
77
+ # view: (5, -16)
78
+ # zoom: 1.089
79
+ # transferStateChange__TSC (38, 236)
80
+ # transferTimeout__TT (974, 204)
81
+ # transferError__TE (518, 34)
@@ -0,0 +1,48 @@
1
+ # FX Implementation
2
+
3
+ ## Proof of Concept (PoC) Implementation for Payer-Side Currency Conversion (Happy Path Only)
4
+
5
+ We have developed a proof of concept for foreign exchange (FX) transfer focusing on a specific scenario: Payer-side currency conversion. Please note that this PoC covers only the happy path, with no test coverage and without handling error cases.
6
+
7
+ ### Testing using ml-core-test-harness
8
+
9
+ ![Test Scenario](./assets/test-scenario.drawio.svg)
10
+
11
+ To test the functionality, you can utilize [mojaloop/ml-core-test-harness](https://github.com/mojaloop/ml-core-test-harness):
12
+
13
+ 1. Clone the repository:
14
+ ```
15
+ git clone https://github.com/mojaloop/ml-core-test-harness.git
16
+ ```
17
+ 2. Checkout to the branch `feat/fx-impl`:
18
+ ```
19
+ git checkout feat/fx-impl
20
+ ```
21
+ 3. Run the services:
22
+ ```
23
+ docker-compose --profile all-services --profile ttk-provisioning --profile ttk-tests --profile debug up -d
24
+ ```
25
+ 4. Open the testing toolkit web UI at `http://localhost:9660`.
26
+ 5. Navigate to `Test Runner`, click on `Collection Manager`, and import the folder `docker/ml-testing-toolkit/test-cases/collections`.
27
+ 6. Select the file `fxp/payer_conversion.json`.
28
+ 7. Run the test case by clicking on the `Run` button.
29
+ 8. Verify that all tests have passed.
30
+ 9. Observe the sequence of requests and responses in each item of the test case.
31
+ 10. Open the last item, `Get Accounts for FXP AFTER transfer`, and go to `Scripts->Console Logs` to observe the position movements of different participant accounts, as shown below:
32
+ ```
33
+ "Payer Position BWP : 0 -> 300 (300)"
34
+
35
+ "Payee Position TZS : 0 -> -48000 (-48000)"
36
+
37
+ "FXP Source Currency BWP : 0 -> -300 (-300)"
38
+
39
+ "FXP Target Currency TZS : 0 -> 48000 (48000)"
40
+ ```
41
+
42
+ ### Implementation
43
+
44
+ The implementation follows the information available in the repository [mojaloop/currency-conversion](https://github.com/mojaloop/currency-conversion).
45
+
46
+ The flow diagram below illustrates the transfer with payer-side currency conversion:
47
+
48
+ ![FX Position Movements](./assets/fx-position-movements.drawio.svg)