@mojaloop/central-ledger 17.7.8 → 17.8.0-orb.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.ncurc.yaml +3 -1
- package/.nvmrc +1 -1
- package/.nycrc.yml +16 -1
- package/Dockerfile +9 -10
- package/README.md +39 -9
- package/audit-ci.jsonc +15 -2
- package/config/default.json +17 -1
- package/docker/central-ledger/default.json +9 -0
- package/docker/config-modifier/configs/central-ledger.js +25 -0
- package/docker/env.sh +15 -0
- package/docker/kafka/scripts/provision.sh +4 -1
- package/docker/ml-api-adapter/default.json +4 -1
- package/docker-compose.yml +90 -2
- package/documentation/db/erd-transfer-timeout.png +0 -0
- package/documentation/db/erd-transfer-timeout.txt +81 -0
- package/documentation/fx-implementation/README.md +48 -0
- package/documentation/fx-implementation/assets/fx-position-movements.drawio.svg +4 -0
- package/documentation/fx-implementation/assets/test-scenario.drawio.svg +4 -0
- package/documentation/sequence-diagrams/Handler - FX timeout.plantuml +123 -0
- package/documentation/sequence-diagrams/Handler - FX timeout.png +0 -0
- package/documentation/sequence-diagrams/Handler - timeout.plantuml +81 -0
- package/documentation/sequence-diagrams/Handler - timeout.png +0 -0
- package/documentation/state-diagrams/transfer-ML-spec-states-diagram.png +0 -0
- package/documentation/state-diagrams/transfer-internal-states-diagram.png +0 -0
- package/documentation/state-diagrams/transfer-internal-states.plantuml +75 -0
- package/documentation/state-diagrams/transfer-states.plantuml +13 -0
- package/migrations/310204_transferParticipant-participantId.js +52 -0
- package/migrations/310403_participantPositionChange-participantCurrencyId.js +47 -0
- package/migrations/310404_participantPositionChange-change.js +46 -0
- package/migrations/600010_fxTransferType.js +43 -0
- package/migrations/600011_fxTransferType-indexes.js +38 -0
- package/migrations/600012_fxParticipantCurrencyType.js +43 -0
- package/migrations/600013_fxParticipantCurrencyType-indexes.js +38 -0
- package/migrations/600100_fxTransferDuplicateCheck.js +42 -0
- package/migrations/600110_fxTransferErrorDuplicateCheck.js.js +17 -0
- package/migrations/600200_fxTransfer.js +51 -0
- package/migrations/600201_fxTransfer-indexes.js +40 -0
- package/migrations/600400_fxTransferStateChange.js +46 -0
- package/migrations/600401_fxTransferStateChange-indexes.js +40 -0
- package/migrations/600501_fxWatchList.js +46 -0
- package/migrations/600502_fxWatchList-indexes.js +40 -0
- package/migrations/600600_fxTransferFulfilmentDuplicateCheck.js +43 -0
- package/migrations/600601_fxTransferFulfilmentDuplicateCheck-indexes.js +38 -0
- package/migrations/600700_fxTransferFulfilment.js +47 -0
- package/migrations/600701_fxTransferFulfilment-indexes.js +43 -0
- package/migrations/600800_fxTransferExtension.js +47 -0
- package/migrations/601400_fxTransferTimeout.js +43 -0
- package/migrations/601401_fxTransferTimeout-indexes.js +37 -0
- package/migrations/601500_fxTransferError.js +44 -0
- package/migrations/601501_fxTransferError-indexes.js +37 -0
- package/migrations/610200_fxTransferParticipant.js +52 -0
- package/migrations/610201_fxTransferParticipant-indexes.js +44 -0
- package/migrations/610202_fxTransferParticipant-participantId.js +52 -0
- package/migrations/610403_participantPositionChange-fxTransfer.js +46 -0
- package/migrations/910101_feature904DataMigration.js +46 -52
- package/migrations/910102_feature949DataMigration.js +219 -225
- package/migrations/950104_settlementModel-settlementAccountTypeId.js +15 -20
- package/migrations/950108_participantProxy.js +18 -0
- package/migrations/950109_fxQuote.js +19 -0
- package/migrations/950110_fxQuoteResponse.js +25 -0
- package/migrations/950111_fxQuoteError.js +23 -0
- package/migrations/950113_fxQuoteDuplicateCheck.js +18 -0
- package/migrations/950114_fxQuoteResponseDuplicateCheck.js +21 -0
- package/migrations/950115_fxQuoteConversionTerms.js +36 -0
- package/migrations/950116_fxQuoteConversionTermsExtension.js +21 -0
- package/migrations/950117_fxQuoteResponseConversionTerms.js +39 -0
- package/migrations/950118_fxQuoteResponseConversionTermsExtension.js +21 -0
- package/migrations/950119_fxCharge.js +27 -0
- package/migrations/960100_create_externalParticipant.js +47 -0
- package/migrations/960110_alter_transferParticipant__addFiled_externalParticipantId.js +50 -0
- package/migrations/960111_alter_fxTransferParticipant__addFiled_externalParticipantId.js +50 -0
- package/package.json +27 -20
- package/seeds/endpointType.js +18 -0
- package/seeds/fxParticipantCurrencyType.js +45 -0
- package/seeds/fxTransferType.js +45 -0
- package/seeds/participant.js +2 -1
- package/seeds/transferParticipantRoleType.js +9 -0
- package/seeds/transferState.js +10 -0
- package/src/api/interface/swagger.json +23 -30
- package/src/api/participants/handler.js +6 -2
- package/src/api/participants/routes.js +8 -7
- package/src/api/root/handler.js +15 -5
- package/src/domain/fx/cyril.js +466 -0
- package/src/domain/fx/index.js +107 -0
- package/src/domain/participant/index.js +108 -1
- package/src/domain/position/abort.js +215 -0
- package/src/domain/position/binProcessor.js +361 -99
- package/src/domain/position/fulfil.js +252 -111
- package/src/domain/position/fx-fulfil.js +138 -0
- package/src/domain/position/fx-prepare.js +280 -0
- package/src/domain/position/fx-timeout-reserved.js +159 -0
- package/src/domain/position/index.js +1 -0
- package/src/domain/position/prepare.js +69 -49
- package/src/domain/position/timeout-reserved.js +162 -0
- package/src/domain/timeout/index.js +26 -2
- package/src/domain/transfer/index.js +22 -5
- package/src/domain/transfer/transform.js +19 -6
- package/src/handlers/admin/handler.js +0 -2
- package/src/handlers/bulk/fulfil/handler.js +5 -5
- package/src/handlers/bulk/get/handler.js +5 -5
- package/src/handlers/bulk/prepare/handler.js +9 -9
- package/src/handlers/bulk/processing/handler.js +8 -7
- package/src/handlers/bulk/shared/validator.js +1 -1
- package/src/handlers/positions/handler.js +23 -10
- package/src/handlers/positions/handlerBatch.js +54 -26
- package/src/handlers/register.js +2 -1
- package/src/handlers/timeouts/handler.js +212 -68
- package/src/handlers/transfers/FxFulfilService.js +387 -0
- package/src/handlers/transfers/createRemittanceEntity.js +106 -0
- package/src/handlers/transfers/dto.js +53 -0
- package/src/handlers/transfers/handler.js +609 -571
- package/src/handlers/transfers/prepare.js +572 -0
- package/src/handlers/transfers/validator.js +75 -14
- package/src/lib/cache.js +1 -1
- package/src/lib/config.js +6 -3
- package/src/lib/healthCheck/subServiceHealth.js +12 -2
- package/src/lib/proxyCache.js +131 -0
- package/src/models/bulkTransfer/facade.js +58 -82
- package/src/models/fxTransfer/duplicateCheck.js +153 -0
- package/src/models/fxTransfer/fxTransfer.js +578 -0
- package/src/models/fxTransfer/fxTransferError.js +53 -0
- package/src/models/fxTransfer/fxTransferExtension.js +41 -0
- package/src/models/fxTransfer/fxTransferTimeout.js +68 -0
- package/src/models/fxTransfer/index.js +15 -0
- package/src/models/fxTransfer/stateChange.js +47 -0
- package/src/models/fxTransfer/watchList.js +49 -0
- package/src/models/ledgerAccountType/ledgerAccountType.js +8 -32
- package/src/models/misc/segment.js +0 -1
- package/src/models/participant/externalParticipant.js +96 -0
- package/src/models/participant/externalParticipantCached.js +148 -0
- package/src/models/participant/facade.js +192 -123
- package/src/models/participant/participant.js +2 -1
- package/src/models/participant/participantCurrency.js +1 -1
- package/src/models/participant/participantPosition.js +2 -8
- package/src/models/position/batch.js +90 -1
- package/src/models/position/facade.js +5 -5
- package/src/models/position/participantPositionChanges.js +68 -0
- package/src/models/settlement/settlementModel.js +5 -23
- package/src/models/transfer/facade.js +806 -447
- package/src/shared/constants.js +52 -0
- package/src/shared/fspiopErrorFactory.js +131 -0
- package/src/shared/logger/index.js +8 -0
- package/src/shared/loggingPlugin.js +43 -0
- package/src/shared/plugins.js +6 -0
- package/src/shared/setup.js +10 -0
- package/test-integration.Dockerfile +1 -1
- package/test.Dockerfile +1 -1
package/.ncurc.yaml
CHANGED
|
@@ -9,5 +9,7 @@ reject: [
|
|
|
9
9
|
"get-port",
|
|
10
10
|
# sinon v17.0.1 causes 58 tests to fail. This will need to be resolved in a future story.
|
|
11
11
|
# Issue is tracked here: https://github.com/mojaloop/project/issues/3616
|
|
12
|
-
"sinon"
|
|
12
|
+
"sinon",
|
|
13
|
+
# glob >= 11 requires node >= 20
|
|
14
|
+
"glob"
|
|
13
15
|
]
|
package/.nvmrc
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
18.
|
|
1
|
+
18.20.3
|
package/.nycrc.yml
CHANGED
|
@@ -17,5 +17,20 @@ exclude: [
|
|
|
17
17
|
"**/node_modules/**",
|
|
18
18
|
'**/migrations/**',
|
|
19
19
|
'**/ddl/**',
|
|
20
|
-
'**/bulk*/**'
|
|
20
|
+
'**/bulk*/**',
|
|
21
|
+
'src/shared/logger/**',
|
|
22
|
+
'src/shared/loggingPlugin.js',
|
|
23
|
+
'src/shared/constants.js',
|
|
24
|
+
'src/domain/position/index.js',
|
|
25
|
+
'src/domain/position/binProcessor.js',
|
|
26
|
+
'src/handlers/positions/handler.js',
|
|
27
|
+
'src/handlers/transfers/createRemittanceEntity.js',
|
|
28
|
+
'src/handlers/transfers/FxFulfilService.js',
|
|
29
|
+
'src/models/position/batch.js',
|
|
30
|
+
'src/models/fxTransfer/**',
|
|
31
|
+
'src/models/participant/externalParticipantCached.js', # todo: figure out why it shows only 50% coverage in Branch
|
|
32
|
+
'src/models/transfer/facade.js', ## add more test coverage
|
|
33
|
+
'src/shared/fspiopErrorFactory.js',
|
|
34
|
+
'src/lib/proxyCache.js' # todo: remove this line after adding test coverage
|
|
21
35
|
]
|
|
36
|
+
## todo: increase test coverage before merging feat/fx-impl to main branch
|
package/Dockerfile
CHANGED
|
@@ -3,26 +3,27 @@ ARG NODE_VERSION=lts-alpine
|
|
|
3
3
|
|
|
4
4
|
# NOTE: Ensure you set NODE_VERSION Build Argument as follows...
|
|
5
5
|
#
|
|
6
|
-
#
|
|
7
|
-
#
|
|
8
|
-
#
|
|
9
|
-
#
|
|
10
|
-
#
|
|
6
|
+
# export NODE_VERSION="$(cat .nvmrc)-alpine"
|
|
7
|
+
# docker build \
|
|
8
|
+
# --build-arg NODE_VERSION=$NODE_VERSION \
|
|
9
|
+
# -t mojaloop/central-ledger:local \
|
|
10
|
+
# .
|
|
11
11
|
#
|
|
12
12
|
|
|
13
13
|
# Build Image
|
|
14
|
-
FROM node:${NODE_VERSION}
|
|
14
|
+
FROM node:${NODE_VERSION} AS builder
|
|
15
15
|
|
|
16
16
|
WORKDIR /opt/app
|
|
17
17
|
|
|
18
18
|
RUN apk --no-cache add git
|
|
19
|
-
RUN apk add --no-cache -t build-dependencies make gcc g++ python3 libtool openssl-dev autoconf automake bash \
|
|
19
|
+
RUN apk add --no-cache -t build-dependencies make gcc g++ python3 py3-setuptools libtool openssl-dev autoconf automake bash \
|
|
20
20
|
&& cd $(npm root -g)/npm \
|
|
21
21
|
&& npm install -g node-gyp
|
|
22
22
|
|
|
23
23
|
COPY package.json package-lock.json* /opt/app/
|
|
24
24
|
|
|
25
25
|
RUN npm ci
|
|
26
|
+
RUN npm prune --omit=dev
|
|
26
27
|
|
|
27
28
|
FROM node:${NODE_VERSION}
|
|
28
29
|
WORKDIR /opt/app
|
|
@@ -32,7 +33,7 @@ RUN mkdir ./logs && touch ./logs/combined.log
|
|
|
32
33
|
RUN ln -sf /dev/stdout ./logs/combined.log
|
|
33
34
|
|
|
34
35
|
# Create a non-root user: ml-user
|
|
35
|
-
RUN adduser -D ml-user
|
|
36
|
+
RUN adduser -D ml-user
|
|
36
37
|
USER ml-user
|
|
37
38
|
|
|
38
39
|
COPY --chown=ml-user --from=builder /opt/app .
|
|
@@ -43,7 +44,5 @@ COPY migrations /opt/app/migrations
|
|
|
43
44
|
COPY seeds /opt/app/seeds
|
|
44
45
|
COPY test /opt/app/test
|
|
45
46
|
|
|
46
|
-
RUN npm prune --production
|
|
47
|
-
|
|
48
47
|
EXPOSE 3001
|
|
49
48
|
CMD ["npm", "run", "start"]
|
package/README.md
CHANGED
|
@@ -56,7 +56,7 @@ Or via docker build directly:
|
|
|
56
56
|
|
|
57
57
|
```bash
|
|
58
58
|
docker build \
|
|
59
|
-
--build-arg NODE_VERSION="$(cat .nvmrc)-
|
|
59
|
+
--build-arg NODE_VERSION="$(cat .nvmrc)-alpine3.19" \
|
|
60
60
|
-t mojaloop/ml-api-adapter:local \
|
|
61
61
|
.
|
|
62
62
|
```
|
|
@@ -113,12 +113,14 @@ NOTE: Only POSITION.PREPARE and POSITION.COMMIT is supported at this time, with
|
|
|
113
113
|
|
|
114
114
|
Batch processing can be enabled in the transfer execution flow. Follow the steps below to enable batch processing for a more efficient transfer execution:
|
|
115
115
|
|
|
116
|
+
Note: The position messages with action 'FX_PREPARE', 'FX_COMMIT' and 'FX_TIMEOUT_RESERVED' are only supported in batch processing.
|
|
117
|
+
|
|
116
118
|
- **Step 1:** **Create a New Kafka Topic**
|
|
117
119
|
|
|
118
120
|
Create a new Kafka topic named `topic-transfer-position-batch` to handle batch processing events.
|
|
119
121
|
- **Step 2:** **Configure Action Type Mapping**
|
|
120
122
|
|
|
121
|
-
Point the prepare handler to the newly created topic for the action
|
|
123
|
+
Point the prepare handler to the newly created topic for the action types those are supported in batch processing using the `KAFKA.EVENT_TYPE_ACTION_TOPIC_MAP` configuration as shown below:
|
|
122
124
|
```
|
|
123
125
|
"KAFKA": {
|
|
124
126
|
"EVENT_TYPE_ACTION_TOPIC_MAP" : {
|
|
@@ -126,8 +128,12 @@ Batch processing can be enabled in the transfer execution flow. Follow the steps
|
|
|
126
128
|
"PREPARE": "topic-transfer-position-batch",
|
|
127
129
|
"BULK_PREPARE": "topic-transfer-position",
|
|
128
130
|
"COMMIT": "topic-transfer-position-batch",
|
|
131
|
+
"FX_COMMIT": "topic-transfer-position-batch",
|
|
129
132
|
"BULK_COMMIT": "topic-transfer-position",
|
|
130
133
|
"RESERVE": "topic-transfer-position",
|
|
134
|
+
"FX_PREPARE": "topic-transfer-position-batch",
|
|
135
|
+
"TIMEOUT_RESERVED": "topic-transfer-position-batch",
|
|
136
|
+
"FX_TIMEOUT_RESERVED": "topic-transfer-position-batch"
|
|
131
137
|
}
|
|
132
138
|
}
|
|
133
139
|
}
|
|
@@ -185,7 +191,8 @@ If you want to run integration tests in a repetitive manner, you can startup the
|
|
|
185
191
|
Start containers required for Integration Tests
|
|
186
192
|
|
|
187
193
|
```bash
|
|
188
|
-
|
|
194
|
+
source ./docker/env.sh
|
|
195
|
+
docker compose up -d mysql kafka init-kafka redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5
|
|
189
196
|
```
|
|
190
197
|
|
|
191
198
|
Run wait script which will report once all required containers are up and running
|
|
@@ -220,7 +227,8 @@ If you want to run integration tests in a repetitive manner, you can startup the
|
|
|
220
227
|
Start containers required for Integration Tests, including a `central-ledger` container which will be used as a proxy shell.
|
|
221
228
|
|
|
222
229
|
```bash
|
|
223
|
-
|
|
230
|
+
source ./docker/env.sh
|
|
231
|
+
docker-compose -f docker-compose.yml -f docker-compose.integration.yml up -d kafka mysql central-ledger init-kafka redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5
|
|
224
232
|
```
|
|
225
233
|
|
|
226
234
|
Run the Integration Tests from the `central-ledger` container
|
|
@@ -235,24 +243,42 @@ If you want to run override position topic tests you can repeat the above and us
|
|
|
235
243
|
|
|
236
244
|
#### For running integration tests for batch processing interactively
|
|
237
245
|
- Run dependecies
|
|
238
|
-
```
|
|
239
|
-
docker
|
|
246
|
+
```bash
|
|
247
|
+
source ./docker/env.sh
|
|
248
|
+
docker compose up -d mysql kafka init-kafka redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5
|
|
240
249
|
npm run wait-4-docker
|
|
241
250
|
```
|
|
242
251
|
- Run central-ledger services
|
|
243
252
|
```
|
|
244
253
|
nvm use
|
|
245
254
|
npm run migrate
|
|
246
|
-
|
|
255
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__PREPARE=topic-transfer-position-batch
|
|
256
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__COMMIT=topic-transfer-position-batch
|
|
257
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__RESERVE=topic-transfer-position-batch
|
|
258
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__TIMEOUT_RESERVED=topic-transfer-position-batch
|
|
259
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__FX_TIMEOUT_RESERVED=topic-transfer-position-batch
|
|
260
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__ABORT=topic-transfer-position-batch
|
|
261
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__FX_ABORT=topic-transfer-position-batch
|
|
262
|
+
npm start
|
|
247
263
|
```
|
|
248
264
|
- Additionally, run position batch handler in a new terminal
|
|
249
265
|
```
|
|
266
|
+
nvm use
|
|
250
267
|
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__PREPARE=topic-transfer-position-batch
|
|
268
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__FX_PREPARE=topic-transfer-position-batch
|
|
251
269
|
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__COMMIT=topic-transfer-position-batch
|
|
270
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__TIMEOUT_RESERVED=topic-transfer-position-batch
|
|
271
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__FX_TIMEOUT_RESERVED=topic-transfer-position-batch
|
|
272
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__ABORT=topic-transfer-position-batch
|
|
273
|
+
export CLEDG_KAFKA__EVENT_TYPE_ACTION_TOPIC_MAP__POSITION__FX_ABORT=topic-transfer-position-batch
|
|
252
274
|
export CLEDG_HANDLERS__API__DISABLED=true
|
|
253
275
|
node src/handlers/index.js handler --positionbatch
|
|
254
276
|
```
|
|
255
|
-
- Run tests using
|
|
277
|
+
- Run tests using the following commands in a new terminal
|
|
278
|
+
```
|
|
279
|
+
nvm use
|
|
280
|
+
npm run test:int-override
|
|
281
|
+
```
|
|
256
282
|
|
|
257
283
|
|
|
258
284
|
If you want to just run all of the integration suite non-interactively then use npm run `test:integration`.
|
|
@@ -263,7 +289,11 @@ It will handle docker start up, migration, service starting and testing. Be sure
|
|
|
263
289
|
If you want to run functional tests locally utilizing the [ml-core-test-harness](https://github.com/mojaloop/ml-core-test-harness), you can run the following commands:
|
|
264
290
|
|
|
265
291
|
```bash
|
|
266
|
-
|
|
292
|
+
export NODE_VERSION="$(cat .nvmrc)-alpine"
|
|
293
|
+
docker build \
|
|
294
|
+
--build-arg NODE_VERSION=$NODE_VERSION \
|
|
295
|
+
-t mojaloop/central-ledger:local \
|
|
296
|
+
.
|
|
267
297
|
```
|
|
268
298
|
|
|
269
299
|
```bash
|
package/audit-ci.jsonc
CHANGED
|
@@ -4,6 +4,19 @@
|
|
|
4
4
|
// Only use one of ["low": true, "moderate": true, "high": true, "critical": true]
|
|
5
5
|
"moderate": true,
|
|
6
6
|
"allowlist": [ // NOTE: Please add as much information as possible to any items added to the allowList
|
|
7
|
-
|
|
7
|
+
"GHSA-w5p7-h5w8-2hfq", // tap-spec>tap-out>trim; This has been analyzed and this is acceptable as it is used to run tests.
|
|
8
|
+
"GHSA-2mvq-xp48-4c77", // https://github.com/advisories/GHSA-2mvq-xp48-4c77
|
|
9
|
+
"GHSA-5854-jvxx-2cg9", // https://github.com/advisories/GHSA-5854-jvxx-2cg9
|
|
10
|
+
"GHSA-7hx8-2rxv-66xv", // https://github.com/advisories/GHSA-7hx8-2rxv-66xv
|
|
11
|
+
"GHSA-c429-5p7v-vgjp", // https://github.com/advisories/GHSA-c429-5p7v-vgjp
|
|
12
|
+
"GHSA-g64q-3vg8-8f93", // https://github.com/advisories/GHSA-g64q-3vg8-8f93
|
|
13
|
+
"GHSA-mg85-8mv5-ffjr", // https://github.com/advisories/GHSA-mg85-8mv5-ffjr
|
|
14
|
+
"GHSA-8hc4-vh64-cxmj", // https://github.com/advisories/GHSA-8hc4-vh64-cxmj
|
|
15
|
+
"GHSA-952p-6rrq-rcjv", // https://github.com/advisories/GHSA-952p-6rrq-rcjv
|
|
16
|
+
"GHSA-9wv6-86v2-598j", // https://github.com/advisories/GHSA-9wv6-86v2-598j
|
|
17
|
+
"GHSA-qwcr-r2fm-qrc7", // https://github.com/advisories/GHSA-qwcr-r2fm-qrc7
|
|
18
|
+
"GHSA-cm22-4g7w-348p", // https://github.com/advisories/GHSA-cm22-4g7w-348p
|
|
19
|
+
"GHSA-m6fv-jmcg-4jfg", // https://github.com/advisories/GHSA-m6fv-jmcg-4jfg
|
|
20
|
+
"GHSA-qw6h-vgh9-j6wx" // https://github.com/advisories/GHSA-qw6h-vgh9-j6wx
|
|
8
21
|
]
|
|
9
|
-
}
|
|
22
|
+
}
|
package/config/default.json
CHANGED
|
@@ -78,20 +78,36 @@
|
|
|
78
78
|
},
|
|
79
79
|
"INTERNAL_TRANSFER_VALIDITY_SECONDS": "432000",
|
|
80
80
|
"ENABLE_ON_US_TRANSFERS": false,
|
|
81
|
+
"PAYEE_PARTICIPANT_CURRENCY_VALIDATION_ENABLED": false,
|
|
81
82
|
"CACHE": {
|
|
82
83
|
"CACHE_ENABLED": false,
|
|
83
84
|
"MAX_BYTE_SIZE": 10000000,
|
|
84
85
|
"EXPIRES_IN_MS": 1000
|
|
85
86
|
},
|
|
87
|
+
"PROXY_CACHE": {
|
|
88
|
+
"enabled": true,
|
|
89
|
+
"type": "redis-cluster",
|
|
90
|
+
"proxyConfig": {
|
|
91
|
+
"cluster": [
|
|
92
|
+
{ "host": "localhost", "port": 6379 }
|
|
93
|
+
]
|
|
94
|
+
}
|
|
95
|
+
},
|
|
86
96
|
"API_DOC_ENDPOINTS_ENABLED": true,
|
|
87
97
|
"KAFKA": {
|
|
88
98
|
"EVENT_TYPE_ACTION_TOPIC_MAP" : {
|
|
89
99
|
"POSITION":{
|
|
90
100
|
"PREPARE": null,
|
|
101
|
+
"FX_PREPARE": "topic-transfer-position-batch",
|
|
91
102
|
"BULK_PREPARE": null,
|
|
92
103
|
"COMMIT": null,
|
|
93
104
|
"BULK_COMMIT": null,
|
|
94
|
-
"RESERVE": null
|
|
105
|
+
"RESERVE": null,
|
|
106
|
+
"FX_RESERVE": "topic-transfer-position-batch",
|
|
107
|
+
"TIMEOUT_RESERVED": null,
|
|
108
|
+
"FX_TIMEOUT_RESERVED": "topic-transfer-position-batch",
|
|
109
|
+
"ABORT": null,
|
|
110
|
+
"FX_ABORT": "topic-transfer-position-batch"
|
|
95
111
|
}
|
|
96
112
|
},
|
|
97
113
|
"TOPIC_TEMPLATES": {
|
|
@@ -82,6 +82,15 @@
|
|
|
82
82
|
"MAX_BYTE_SIZE": 10000000,
|
|
83
83
|
"EXPIRES_IN_MS": 1000
|
|
84
84
|
},
|
|
85
|
+
"PROXY_CACHE": {
|
|
86
|
+
"enabled": true,
|
|
87
|
+
"type": "redis-cluster",
|
|
88
|
+
"proxyConfig": {
|
|
89
|
+
"cluster": [
|
|
90
|
+
{ "host": "redis-node-0", "port": 6379 }
|
|
91
|
+
]
|
|
92
|
+
}
|
|
93
|
+
},
|
|
85
94
|
"KAFKA": {
|
|
86
95
|
"TOPIC_TEMPLATES": {
|
|
87
96
|
"PARTICIPANT_TOPIC_TEMPLATE": {
|
|
@@ -12,7 +12,25 @@ module.exports = {
|
|
|
12
12
|
PASSWORD: '',
|
|
13
13
|
DATABASE: 'mlos'
|
|
14
14
|
},
|
|
15
|
+
PROXY_CACHE: {
|
|
16
|
+
enabled: true,
|
|
17
|
+
type: 'redis',
|
|
18
|
+
proxyConfig: {
|
|
19
|
+
cluster: undefined,
|
|
20
|
+
host: 'redis',
|
|
21
|
+
port: 6379
|
|
22
|
+
}
|
|
23
|
+
},
|
|
15
24
|
KAFKA: {
|
|
25
|
+
EVENT_TYPE_ACTION_TOPIC_MAP: {
|
|
26
|
+
POSITION: {
|
|
27
|
+
PREPARE: 'topic-transfer-position-batch',
|
|
28
|
+
BULK_PREPARE: null,
|
|
29
|
+
COMMIT: 'topic-transfer-position-batch',
|
|
30
|
+
BULK_COMMIT: null,
|
|
31
|
+
RESERVE: 'topic-transfer-position-batch'
|
|
32
|
+
}
|
|
33
|
+
},
|
|
16
34
|
CONSUMER: {
|
|
17
35
|
BULK: {
|
|
18
36
|
PREPARE: {
|
|
@@ -72,6 +90,13 @@ module.exports = {
|
|
|
72
90
|
'metadata.broker.list': 'kafka:29092'
|
|
73
91
|
}
|
|
74
92
|
}
|
|
93
|
+
},
|
|
94
|
+
POSITION_BATCH: {
|
|
95
|
+
config: {
|
|
96
|
+
rdkafkaConf: {
|
|
97
|
+
'metadata.broker.list': 'kafka:29092'
|
|
98
|
+
}
|
|
99
|
+
}
|
|
75
100
|
}
|
|
76
101
|
},
|
|
77
102
|
ADMIN: {
|
package/docker/env.sh
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
#!/bin/sh
|
|
2
|
+
|
|
3
|
+
# Retrieve the external IP address of the host machine (on macOS)
|
|
4
|
+
# or the IP address of the docker0 interface (on Linux)
|
|
5
|
+
get_external_ip() {
|
|
6
|
+
if [ "$(uname)" = "Linux" ]; then
|
|
7
|
+
echo "$(ip addr show docker0 | grep 'inet ' | awk '{print $2}' | cut -d/ -f1)"
|
|
8
|
+
else
|
|
9
|
+
# Need to find a way to support Windows here
|
|
10
|
+
echo "$(route get ifconfig.me | grep interface | sed -e 's/.*: //' | xargs ipconfig getifaddr)"
|
|
11
|
+
fi
|
|
12
|
+
}
|
|
13
|
+
|
|
14
|
+
# set/override dynamic variables
|
|
15
|
+
export REDIS_CLUSTER_ANNOUNCE_IP=$(get_external_ip)
|
|
@@ -25,8 +25,11 @@ topics=(
|
|
|
25
25
|
"topic-bulk-prepare"
|
|
26
26
|
"topic-bulk-fulfil"
|
|
27
27
|
"topic-bulk-processing"
|
|
28
|
-
"topic-bulk-get"
|
|
28
|
+
"topic-bulk-get"
|
|
29
29
|
"topic-transfer-position-batch"
|
|
30
|
+
"topic-fx-quotes-post"
|
|
31
|
+
"topic-fx-quotes-put"
|
|
32
|
+
"topic-fx-quotes-get"
|
|
30
33
|
)
|
|
31
34
|
|
|
32
35
|
# Loop through the topics and create them using kafka-topics.sh
|
|
@@ -1,4 +1,8 @@
|
|
|
1
1
|
{
|
|
2
|
+
"HUB_PARTICIPANT": {
|
|
3
|
+
"ID": 1,
|
|
4
|
+
"NAME": "Hub"
|
|
5
|
+
},
|
|
2
6
|
"PORT": 3000,
|
|
3
7
|
"HOSTNAME": "http://ml-api-adapter",
|
|
4
8
|
"ENDPOINT_SOURCE_URL": "http://host.docker.internal:3001",
|
|
@@ -13,7 +17,6 @@
|
|
|
13
17
|
},
|
|
14
18
|
"JWS": {
|
|
15
19
|
"JWS_SIGN": false,
|
|
16
|
-
"FSPIOP_SOURCE_TO_SIGN": "switch",
|
|
17
20
|
"JWS_SIGNING_KEY_PATH": "secrets/jwsSigningKey.key"
|
|
18
21
|
}
|
|
19
22
|
},
|
package/docker-compose.yml
CHANGED
|
@@ -1,9 +1,22 @@
|
|
|
1
|
-
version: "3.7"
|
|
2
|
-
|
|
3
1
|
networks:
|
|
4
2
|
cl-mojaloop-net:
|
|
5
3
|
name: cl-mojaloop-net
|
|
6
4
|
|
|
5
|
+
|
|
6
|
+
# @see https://uninterrupted.tech/blog/hassle-free-redis-cluster-deployment-using-docker/
|
|
7
|
+
x-redis-node: &REDIS_NODE
|
|
8
|
+
image: docker.io/bitnami/redis-cluster:6.2.14
|
|
9
|
+
environment: &REDIS_ENVS
|
|
10
|
+
ALLOW_EMPTY_PASSWORD: yes
|
|
11
|
+
REDIS_CLUSTER_DYNAMIC_IPS: no
|
|
12
|
+
REDIS_CLUSTER_ANNOUNCE_IP: ${REDIS_CLUSTER_ANNOUNCE_IP}
|
|
13
|
+
REDIS_NODES: redis-node-0:6379 redis-node-1:9301 redis-node-2:9302 redis-node-3:9303 redis-node-4:9304 redis-node-5:9305
|
|
14
|
+
healthcheck:
|
|
15
|
+
test: [ "CMD", "redis-cli", "ping" ]
|
|
16
|
+
timeout: 2s
|
|
17
|
+
networks:
|
|
18
|
+
- cl-mojaloop-net
|
|
19
|
+
|
|
7
20
|
services:
|
|
8
21
|
central-ledger:
|
|
9
22
|
image: mojaloop/central-ledger:local
|
|
@@ -31,10 +44,14 @@ services:
|
|
|
31
44
|
- CLEDG_MONGODB__DISABLED=false
|
|
32
45
|
networks:
|
|
33
46
|
- cl-mojaloop-net
|
|
47
|
+
extra_hosts:
|
|
48
|
+
- "redis-node-0:host-gateway"
|
|
34
49
|
depends_on:
|
|
35
50
|
- mysql
|
|
36
51
|
- kafka
|
|
37
52
|
- objstore
|
|
53
|
+
- redis-node-0
|
|
54
|
+
# - redis
|
|
38
55
|
healthcheck:
|
|
39
56
|
test: ["CMD", "sh", "-c" ,"apk --no-cache add curl", "&&", "curl", "http://localhost:3001/health"]
|
|
40
57
|
timeout: 20s
|
|
@@ -94,6 +111,77 @@ services:
|
|
|
94
111
|
retries: 10
|
|
95
112
|
start_period: 40s
|
|
96
113
|
interval: 30s
|
|
114
|
+
|
|
115
|
+
redis-node-0:
|
|
116
|
+
<<: *REDIS_NODE
|
|
117
|
+
environment:
|
|
118
|
+
<<: *REDIS_ENVS
|
|
119
|
+
REDIS_CLUSTER_CREATOR: yes
|
|
120
|
+
REDIS_PORT_NUMBER: 6379
|
|
121
|
+
depends_on:
|
|
122
|
+
- redis-node-1
|
|
123
|
+
- redis-node-2
|
|
124
|
+
ports:
|
|
125
|
+
- "6379:6379"
|
|
126
|
+
- "16379:16379"
|
|
127
|
+
redis-node-1:
|
|
128
|
+
<<: *REDIS_NODE
|
|
129
|
+
environment:
|
|
130
|
+
<<: *REDIS_ENVS
|
|
131
|
+
REDIS_PORT_NUMBER: 9301
|
|
132
|
+
ports:
|
|
133
|
+
- "9301:9301"
|
|
134
|
+
- "19301:19301"
|
|
135
|
+
redis-node-2:
|
|
136
|
+
<<: *REDIS_NODE
|
|
137
|
+
environment:
|
|
138
|
+
<<: *REDIS_ENVS
|
|
139
|
+
REDIS_PORT_NUMBER: 9302
|
|
140
|
+
ports:
|
|
141
|
+
- "9302:9302"
|
|
142
|
+
- "19302:19302"
|
|
143
|
+
redis-node-3:
|
|
144
|
+
<<: *REDIS_NODE
|
|
145
|
+
environment:
|
|
146
|
+
<<: *REDIS_ENVS
|
|
147
|
+
REDIS_PORT_NUMBER: 9303
|
|
148
|
+
ports:
|
|
149
|
+
- "9303:9303"
|
|
150
|
+
- "19303:19303"
|
|
151
|
+
redis-node-4:
|
|
152
|
+
<<: *REDIS_NODE
|
|
153
|
+
environment:
|
|
154
|
+
<<: *REDIS_ENVS
|
|
155
|
+
REDIS_PORT_NUMBER: 9304
|
|
156
|
+
ports:
|
|
157
|
+
- "9304:9304"
|
|
158
|
+
- "19304:19304"
|
|
159
|
+
redis-node-5:
|
|
160
|
+
<<: *REDIS_NODE
|
|
161
|
+
environment:
|
|
162
|
+
<<: *REDIS_ENVS
|
|
163
|
+
REDIS_PORT_NUMBER: 9305
|
|
164
|
+
ports:
|
|
165
|
+
- "9305:9305"
|
|
166
|
+
- "19305:19305"
|
|
167
|
+
|
|
168
|
+
## To be used with proxyCache.type === 'redis'
|
|
169
|
+
# redis:
|
|
170
|
+
# image: redis:6.2.4-alpine
|
|
171
|
+
# restart: "unless-stopped"
|
|
172
|
+
# environment:
|
|
173
|
+
# <<: *REDIS_ENVS
|
|
174
|
+
# REDIS_CLUSTER_CREATOR: yes
|
|
175
|
+
# depends_on:
|
|
176
|
+
# - redis-node-1
|
|
177
|
+
# - redis-node-2
|
|
178
|
+
# - redis-node-3
|
|
179
|
+
# - redis-node-4
|
|
180
|
+
# - redis-node-5
|
|
181
|
+
# ports:
|
|
182
|
+
# - "6379:6379"
|
|
183
|
+
# networks:
|
|
184
|
+
# - cl-mojaloop-net
|
|
97
185
|
|
|
98
186
|
mockserver:
|
|
99
187
|
image: jamesdbloom/mockserver
|
|
Binary file
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# Visualize on https://erd.surge.sh
|
|
2
|
+
# or https://quick-erd.surge.sh
|
|
3
|
+
#
|
|
4
|
+
# Relationship Types
|
|
5
|
+
# - - one to one
|
|
6
|
+
# -< - one to many
|
|
7
|
+
# >- - many to one
|
|
8
|
+
# >-< - many to many
|
|
9
|
+
# -0 - one to zero or one
|
|
10
|
+
# 0- - zero or one to one
|
|
11
|
+
# 0-0 - zero or one to zero or one
|
|
12
|
+
# -0< - one to zero or many
|
|
13
|
+
# >0- - zero or many to one
|
|
14
|
+
#
|
|
15
|
+
////////////////////////////////////
|
|
16
|
+
|
|
17
|
+
transfer
|
|
18
|
+
---------------------
|
|
19
|
+
transferId varchar(36) PK
|
|
20
|
+
amount decimal(18,4)
|
|
21
|
+
currencyId varchar(3) FK - currency.currencyId
|
|
22
|
+
ilpCondition varchar(256)
|
|
23
|
+
expirationDate datetime
|
|
24
|
+
createdDate datetime
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
transferStateChange__TSC
|
|
28
|
+
---------------------
|
|
29
|
+
transferStateChangeId bigint UN AI PK
|
|
30
|
+
transferId varchar(36) FK >- transfer.transferId
|
|
31
|
+
transferStateId varchar(50) FK - transferState.transferStateId
|
|
32
|
+
reason varchar(512)
|
|
33
|
+
createdDate datetime
|
|
34
|
+
|
|
35
|
+
|
|
36
|
+
transferTimeout__TT
|
|
37
|
+
---------------------
|
|
38
|
+
transferTimeoutId bigint UN AI PK
|
|
39
|
+
transferId varchar(36) UNIQUE FK - transfer.transferId
|
|
40
|
+
expirationDate datetime
|
|
41
|
+
createdDate datetime
|
|
42
|
+
|
|
43
|
+
|
|
44
|
+
transferError__TE
|
|
45
|
+
---------------------
|
|
46
|
+
transferId varchar(36) PK
|
|
47
|
+
transferStateChangeId bigint UN FK - transferStateChange.transferStateChangeId
|
|
48
|
+
errorCode int UN
|
|
49
|
+
errorDescription varchar(128)
|
|
50
|
+
createdDate datetime
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
segment
|
|
54
|
+
---------------------
|
|
55
|
+
segmentId int UN AI PK
|
|
56
|
+
segmentType varchar(50)
|
|
57
|
+
enumeration int
|
|
58
|
+
tableName varchar(50)
|
|
59
|
+
value bigint
|
|
60
|
+
changedDate datetime
|
|
61
|
+
# row example: 1, 'timeout', 0, 'transferStateChange', 255, '2024-04-24 18:07:15'
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
expiringTransfer
|
|
65
|
+
---------------------
|
|
66
|
+
expiringTransferId bigint UN AI PK
|
|
67
|
+
transferId varchar(36) UNIQUE FK - transfer.transferId
|
|
68
|
+
expirationDate datetime INDEX
|
|
69
|
+
createdDate datetime
|
|
70
|
+
# todo: clarify, how we use this table
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
|
|
74
|
+
# transfer (557, 340)
|
|
75
|
+
# segment (348, 608)
|
|
76
|
+
# expiringTransfer (1033, 574)
|
|
77
|
+
# view: (5, -16)
|
|
78
|
+
# zoom: 1.089
|
|
79
|
+
# transferStateChange__TSC (38, 236)
|
|
80
|
+
# transferTimeout__TT (974, 204)
|
|
81
|
+
# transferError__TE (518, 34)
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# FX Implementation
|
|
2
|
+
|
|
3
|
+
## Proof of Concept (PoC) Implementation for Payer-Side Currency Conversion (Happy Path Only)
|
|
4
|
+
|
|
5
|
+
We have developed a proof of concept for foreign exchange (FX) transfer focusing on a specific scenario: Payer-side currency conversion. Please note that this PoC covers only the happy path, with no test coverage and without handling error cases.
|
|
6
|
+
|
|
7
|
+
### Testing using ml-core-test-harness
|
|
8
|
+
|
|
9
|
+

|
|
10
|
+
|
|
11
|
+
To test the functionality, you can utilize [mojaloop/ml-core-test-harness](https://github.com/mojaloop/ml-core-test-harness):
|
|
12
|
+
|
|
13
|
+
1. Clone the repository:
|
|
14
|
+
```
|
|
15
|
+
git clone https://github.com/mojaloop/ml-core-test-harness.git
|
|
16
|
+
```
|
|
17
|
+
2. Checkout to the branch `feat/fx-impl`:
|
|
18
|
+
```
|
|
19
|
+
git checkout feat/fx-impl
|
|
20
|
+
```
|
|
21
|
+
3. Run the services:
|
|
22
|
+
```
|
|
23
|
+
docker-compose --profile all-services --profile ttk-provisioning --profile ttk-tests --profile debug up -d
|
|
24
|
+
```
|
|
25
|
+
4. Open the testing toolkit web UI at `http://localhost:9660`.
|
|
26
|
+
5. Navigate to `Test Runner`, click on `Collection Manager`, and import the folder `docker/ml-testing-toolkit/test-cases/collections`.
|
|
27
|
+
6. Select the file `fxp/payer_conversion.json`.
|
|
28
|
+
7. Run the test case by clicking on the `Run` button.
|
|
29
|
+
8. Verify that all tests have passed.
|
|
30
|
+
9. Observe the sequence of requests and responses in each item of the test case.
|
|
31
|
+
10. Open the last item, `Get Accounts for FXP AFTER transfer`, and go to `Scripts->Console Logs` to observe the position movements of different participant accounts, as shown below:
|
|
32
|
+
```
|
|
33
|
+
"Payer Position BWP : 0 -> 300 (300)"
|
|
34
|
+
|
|
35
|
+
"Payee Position TZS : 0 -> -48000 (-48000)"
|
|
36
|
+
|
|
37
|
+
"FXP Source Currency BWP : 0 -> -300 (-300)"
|
|
38
|
+
|
|
39
|
+
"FXP Target Currency TZS : 0 -> 48000 (48000)"
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
### Implementation
|
|
43
|
+
|
|
44
|
+
The implementation follows the information available in the repository [mojaloop/currency-conversion](https://github.com/mojaloop/currency-conversion).
|
|
45
|
+
|
|
46
|
+
The flow diagram below illustrates the transfer with payer-side currency conversion:
|
|
47
|
+
|
|
48
|
+

|