@rainbow-o23/n3 1.0.58 → 1.0.60

package/README.md

@@ -601,6 +601,8 @@ step set. Additionally, nested transactions are also supported, which means Tran
 | `endpoints.SYSTEM.global.headers.transparent`           | string |               | Endpoint system global request transparent-passed headers names.<br>Format follows `name1[;name2[...]]`.                                                                    |
 | `endpoints.SYSTEM.ENDPOINT.headers.transparent.omitted` | string |               | Endpoint request headers names which omitted from transparent-passed headers, use global headers if this parameter doesn't present.<br>Format follows `name1[;name2[...]]`. |
 | `endpoints.SYSTEM.global.headers.transparent.omitted`   | string |               | Endpoint system global request headers names which omitted from transparent-passed headers.<br>Format follows `name1[;name2[...]]`.                                         |
+| `endpoints.SYSTEM.ENDPOINT.trace.header.name`           | string |               | Endpoint trace header name.                                                                                                                                                 |
+| `endpoints.SYSTEM.global.trace.header.name`             | string |               | Endpoint system global trace header name.                                                                                                                                   |
 | `endpoints.SYSTEM.ENDPOINT.timeout`                     | number |               | Endpoint request timeout, in seconds, use global timeout if this parameter doesn't present.                                                                                 |
 | `endpoints.SYSTEM.global.timeout`                       | number | -1            | Endpoint system global timeout, in seconds, `-1` represents no timeout.                                                                                                     |
 
@@ -632,7 +634,7 @@ CFG_ENDPOINTS_ORDER_PAYMENT_URL=https://order.com/payment
 
 - `transparentHeaderNames` and `omittedTransparentHeaderNames`:
   Use `transparentHeaderNames` to specify the names of request headers whose values need to be transparently passed from the input
-  parameters to the downstream service. Separate the names with `;`. The names support using `.` for connection so that values from
+  parameters to the upstream service. Separate the names with `;`. The names support using `.` for connection so that values from
   multi-level objects can be directly retrieved. For example, `account.name` will retrieve the value of the `name` property from the
   `account` property of the input object. When writing the values into the header values, the following rules apply:
 	- If the value is an array, use `, ` to connect the elements. `null` and empty strings will be filtered out.
@@ -653,23 +655,29 @@ CFG_ENDPOINTS_ORDER_PAYMENT_URL=https://order.com/payment
   For example:
   If the input data contains `{account: {name: 'John', token: '******'}}` and `transparentHeaderNames` is defined as `account`, then two
   transparently passed headers will be obtained: `name=John` and `token=******`. At this time, if `omittedTransparentHeaderNames` is defined
-  as `name`, the headers that will be finally transparently passed to the downstream service are `token=******`, and `name` will be ignored.
+  as `name`, the headers that will be finally transparently passed to the upstream service are `token=******`, and `name` will be ignored.
 
 ## Request headers
 
-There are three ways to transmit request headers to downstream services. In order of priority from high to low, they are `headersGenerate`,
+There are three ways to transmit request headers to upstream services. In order of priority from high to low, they are `headersGenerate`,
 `headers.transparent`, and `headers`. If a header appears in a higher-priority method, the header with the same name generated by a
 lower-priority method will be ignored. Note that the matching is case-sensitive.
 
-Normally, if you need to transparently pass the request headers from the client to the downstream service, you should use `headers: true` in
+Normally, if you need to transparently pass the request headers from the client to the upstream service, you should use `headers: true` in
 the pipeline definition. Then you can directly use `transparentHeaderNames: headers` to obtain all the request headers, and then use
 `omittedTransparentHeaderNames` for necessary filtering.
 
-> It should be noted that since the fetch step initiates a new request to the downstream service, its request structure and data will be
+> It should be noted that since the fetch step initiates a new request to the upstream service, its request structure and data will be
 > modified or reset according to requirements. Therefore, even if you need to transparently pass the request headers, some of the headers
 > are still not applicable. So by default, the two headers `content-encoding` and `content-length` will be filtered out. No matter how the
 > request headers are generated in the above process, these two headers are always automatically generated by `node-fetch`.
 
+### Tracing in upstream service groups
+
+Use `endpoints.SYSTEM.ENDPOINT.trace.header.name` or `endpoints.SYSTEM.global.trace.header.name` to define the trace request header name for
+the upstream system. The value will be attempted to be retrieved from the response and stored in the execution context to be passed to the
+upstream system within the same `SYSTEM.ENDPOINT` or `SYSTEM`.
+
 ## Installation
 
 Note since `nestjs` only support commonjs module, which means `node-fetch` 3 series cannot be imported since it is built on esm mode.

package/index.cjs

@@ -651,23 +651,19 @@ class AbstractFragmentaryPipelineStep extends n1.AbstractPipelineStep {
     }
     async setToOutput($result, $request) {
         const $helpers = this.getHelpers();
-        return { content: await this._toResponseFunc($result, $request, $helpers, $helpers) };
+        return {
+            content: await this._toResponseFunc($result, $request, $helpers, $helpers),
+            $context: $request.$context
+        };
     }
     buildStepOptions() {
         return { config: this.getConfig(), logger: this.getLogger() };
     }
     createErrorHandleContext(request) {
-        const context = request.$context;
-        if (context == null) {
-            return {};
-        }
-        else {
-            const { authorization, traceId, ...rest } = context;
-            return { authorization, traceId, ...Utils.clone(rest) };
-        }
+        return request.$context;
     }
     async handleErrorSteps(fragment, errorCode, error, request, builders) {
-        const { $context: { authorization, traceId } = {} } = request;
+        const traceId = request.$context.traceId;
         const errorContext = this.createErrorHandleContext(request);
         const options = this.buildStepOptions();
         const steps = await Promise.all(builders.map(async (builder) => await builder.create(options)));
@@ -677,7 +673,7 @@ class AbstractFragmentaryPipelineStep extends n1.AbstractPipelineStep {
                 .execute(async () => {
                 this.traceStepIn(traceId, step, request);
                 const response = await step.perform({
-                    ...request, $context: { ...errorContext, authorization, traceId }
+                    ...request, $context: errorContext
                 });
                 this.traceStepOut(traceId, step, response);
                 return this.returnOrContinueOrClear(request, response);
@@ -773,14 +769,7 @@ class PipelineStepSets extends AbstractFragmentaryPipelineStep {
         return await Promise.all(this.getStepBuilders().map(async (builder) => await builder.create(options)));
     }
     inheritContext(request) {
-        const context = request.$context;
-        if (context == null) {
-            return {};
-        }
-        else {
-            const { authorization, traceId, ...rest } = context;
-            return { authorization, traceId, ...Utils.clone(rest) };
-        }
+        return request.$context;
     }
     async attachMineToInternalContext(inheritedContext, _request) {
         return inheritedContext;
@@ -794,7 +783,7 @@ class PipelineStepSets extends AbstractFragmentaryPipelineStep {
     }
     async doPerform(data, request) {
         return await this.performWithContext(request, async (request, context) => {
-            const { $context: { authorization, traceId } = {} } = request;
+            const traceId = context.traceId;
             const steps = await this.createSteps();
             const response = await steps.reduce(async (promise, step) => {
                 const request = await promise;
@@ -802,7 +791,7 @@ class PipelineStepSets extends AbstractFragmentaryPipelineStep {
                     .execute(async () => {
                     this.traceStepIn(traceId, step, request);
                     const response = await step.perform({
-                        ...request, $context: { ...context, authorization, traceId }
+                        ...request, $context: context
                     });
                     this.traceStepOut(traceId, step, response);
                     return this.returnOrContinueOrClear(request, response);
@@ -814,6 +803,9 @@ class PipelineStepSets extends AbstractFragmentaryPipelineStep {
 }
 
 class AsyncPipelineStepSets extends PipelineStepSets {
+    inheritContext(request) {
+        return request.$context.clone('$trans');
+    }
     async doPerform(data, request) {
         super.doPerform(data, request);
         return;
@@ -854,14 +846,14 @@ class ParallelPipelineStepSets extends PipelineStepSets {
     }
     async doPerform(data, request) {
         return await this.performWithContext(request, async (request, context) => {
-            const { $context: { authorization, traceId } = {} } = request;
+            const traceId = context.traceId;
             const steps = await this.createSteps();
             const execute = () => {
                 return steps.map(async (step) => {
                     return await this.measurePerformance(traceId, 'STEP', step.constructor.name)
                         .execute(async () => {
                         const eachData = await this.cloneDataForEach(data, request);
-                        const eachRequest = { content: eachData, $context: { ...context, authorization, traceId } };
+                        const eachRequest = { content: eachData, $context: context };
                         this.traceStepIn(traceId, step, request);
                         const response = await step.perform(eachRequest);
                         this.traceStepOut(traceId, step, response);
@@ -1226,9 +1218,8 @@ class RefPipelinePipelineStep extends AbstractFragmentaryPipelineStep {
         return { config: this.getConfig(), logger: this.getLogger() };
     }
     async doPerform(data, request) {
-        const { $context: { authorization, traceId } = {} } = request;
         const pipeline = await this.getPipelineBuilder().create(this.buildPipelineOptions());
-        const result = await pipeline.perform({ payload: data, authorization, traceId });
+        const result = await pipeline.perform({ payload: data, $context: request.$context });
         return result.payload;
     }
 }
@@ -1275,6 +1266,7 @@ const HttpAbortErrorCode = '600';
 class FetchPipelineStep extends AbstractFragmentaryPipelineStep {
     _endpointSystemCode;
     _endpointName;
+    _endpointKey;
     _endpointUrl;
     _endpointMethod;
     _endpointHeaders;
@@ -1285,6 +1277,8 @@ class FetchPipelineStep extends AbstractFragmentaryPipelineStep {
     _omittedTransparentHeaderNames;
     _headersGenerateSnippet;
     _headersGenerateFunc;
+    _endpointTraceIdHeaderName;
+    _endpointTraceIdScope;
     _bodyUsed;
     _bodyGenerateSnippet;
     _bodyGenerateFunc;
@@ -1296,13 +1290,13 @@ class FetchPipelineStep extends AbstractFragmentaryPipelineStep {
         const config = this.getConfig();
         this._endpointSystemCode = options.endpointSystemCode;
         this._endpointName = options.endpointName;
-        const endpointKey = this.getEndpointKey();
-        this._endpointUrl = config.getString(`endpoints.${endpointKey}.url`);
+        this._endpointKey = `${this._endpointSystemCode}.${this._endpointName}`;
+        this._endpointUrl = config.getString(`endpoints.${this._endpointKey}.url`);
         this._endpointMethod = (options.method ?? 'POST').trim().toLowerCase();
-        this._endpointHeaders = this.generateEndpointHeaders(config.getString(`endpoints.${endpointKey}.headers`), this.generateEndpointHeaders(config.getString(`endpoints.${this.getEndpointSystemCode()}.global.headers`)));
+        this._endpointHeaders = this.generateEndpointHeaders(config.getString(`endpoints.${this._endpointKey}.headers`), this.generateEndpointHeaders(config.getString(`endpoints.${this._endpointSystemCode}.global.headers`)));
         this._endpointTimeout = options.timeout
-            ?? config.getNumber(`endpoints.${endpointKey}.timeout`)
-            ?? config.getNumber(`endpoints.${this.getEndpointSystemCode()}.global.timeout`, -1);
+            ?? config.getNumber(`endpoints.${this._endpointKey}.timeout`)
+            ?? config.getNumber(`endpoints.${this._endpointSystemCode}.global.timeout`, -1);
         this._endpointTimeout = this._endpointTimeout > 0 ? this._endpointTimeout * 1000 : -1;
         this._urlGenerateSnippet = options.urlGenerate;
         this._urlGenerateFunc = Utils.createAsyncFunction(this.getUrlGenerateSnippet(), {
@@ -1314,9 +1308,9 @@ class FetchPipelineStep extends AbstractFragmentaryPipelineStep {
             }
         });
         this._transparentHeaderNames = options.transparentHeaderNames
-            ?? this.generateTransparentHeaderNames(config.getString(`endpoints.${endpointKey}.headers.transparent`), this.generateTransparentHeaderNames(config.getString(`endpoints.${this.getEndpointSystemCode()}.global.headers.transparent`)));
+            ?? this.generateTransparentHeaderNames(config.getString(`endpoints.${this._endpointKey}.headers.transparent`), this.generateTransparentHeaderNames(config.getString(`endpoints.${this._endpointSystemCode}.global.headers.transparent`)));
         this._omittedTransparentHeaderNames = options.omittedTransparentHeaderNames
-            ?? this.generateTransparentHeaderNames(config.getString(`endpoints.${endpointKey}.headers.transparent.omitted`), this.generateTransparentHeaderNames(config.getString(`endpoints.${this.getEndpointSystemCode()}.global.headers.transparent.omitted`)));
+            ?? this.generateTransparentHeaderNames(config.getString(`endpoints.${this._endpointKey}.headers.transparent.omitted`), this.generateTransparentHeaderNames(config.getString(`endpoints.${this._endpointSystemCode}.global.headers.transparent.omitted`)));
         this._headersGenerateSnippet = options.headersGenerate;
         this._headersGenerateFunc = Utils.createAsyncFunction(this.getHeadersGenerateSnippet(), {
             createDefault: () => async (_$factor, _$request, _$helpers, _$) => (void 0),
@@ -1326,6 +1320,22 @@ class FetchPipelineStep extends AbstractFragmentaryPipelineStep {
                 throw e;
             }
         });
+        let traceIdHeaderName = config.getString(`endpoints.${this._endpointKey}.trace.header.name`)?.trim();
+        if (traceIdHeaderName != null) {
+            this._endpointTraceIdHeaderName = traceIdHeaderName.toLowerCase();
+            this._endpointTraceIdScope = 'endpoint';
+        }
+        else {
+            traceIdHeaderName = config.getString(`endpoints.${this._endpointSystemCode}.global.trace.header.name`)?.trim();
+            if (traceIdHeaderName != null) {
+                this._endpointTraceIdHeaderName = traceIdHeaderName.toLowerCase();
+                this._endpointTraceIdScope = 'system';
+            }
+            else {
+                this._endpointTraceIdHeaderName = (void 0);
+                this._endpointTraceIdScope = (void 0);
+            }
+        }
         this._bodyUsed = options.bodyUsed;
         this._bodyGenerateSnippet = options.bodyGenerate;
         this._bodyGenerateFunc = Utils.createAsyncFunction(this.getBodyGenerateSnippet(), {
@@ -1405,7 +1415,7 @@ class FetchPipelineStep extends AbstractFragmentaryPipelineStep {
         return this._endpointName;
     }
     getEndpointKey() {
-        return `${this.getEndpointSystemCode()}.${this.getEndpointName()}`;
+        return this._endpointKey;
     }
     getEndpointUrl() {
         return this._endpointUrl;
@@ -1460,6 +1470,12 @@ class FetchPipelineStep extends AbstractFragmentaryPipelineStep {
     getHeadersGenerateVariableNames() {
         return ['$factor', '$request', ...this.getHelpersVariableNames()];
     }
+    getEndpointTraceIdHeaderName() {
+        return this._endpointTraceIdHeaderName;
+    }
+    getEndpointTraceIdScope() {
+        return this._endpointTraceIdScope;
+    }
     isBodyUsed() {
         return this._bodyUsed;
     }
@@ -1478,79 +1494,141 @@ class FetchPipelineStep extends AbstractFragmentaryPipelineStep {
     getErrorHandlerVariableName() {
         return ['$options', ...this.getHelpersVariableNames()];
     }
-    async doPerform(data, request) {
-        const $helpers = this.getHelpers();
-        let url = '';
-        try {
-            url = await this._urlGenerateFunc(this.getEndpointUrl(), data, request, $helpers, $helpers);
-            const method = this.getEndpointMethod();
-            const staticHeaders = this.getEndpointHeaders() ?? {};
-            const transparentHeaders = (this.getTransparentHeaderNames() ?? []).reduce((headers, name) => {
-                const value = Utils.getValue(data, name);
-                if (value == null) {
-                }
-                else if (Array.isArray(value)) {
-                    const headerValue = value.filter(v => v != null && `${v}`.length !== 0).join(', ');
-                    if (headerValue.length !== 0) {
-                        headers[name] = headerValue;
-                    }
-                }
-                else if (typeof value === 'object') {
-                    Object.keys(value).forEach(key => {
-                        const headerValue = value[key];
-                        if (headerValue != null) {
-                            const s = `${headerValue}`;
-                            if (s.length !== 0) {
-                                headers[key] = s;
-                            }
-                        }
-                    });
-                }
-                else {
-                    const headerValue = `${value}`;
-                    if (headerValue.length !== 0) {
-                        headers[name] = headerValue;
-                    }
-                }
-                return headers;
-            }, {});
-            (this.getOmittedTransparentHeaderNames() ?? []).forEach(name => delete transparentHeaders[name]);
-            const generatedHeaders = await this._headersGenerateFunc(data, request, $helpers, $helpers) ?? {};
-            let body;
-            const bodyUsed = this.isBodyUsed();
-            if (bodyUsed === true || (bodyUsed == null && method !== 'get')) {
-                body = await this._bodyGenerateFunc(data, request, $helpers, $helpers);
+    syncEndpointTraceId(headers, request) {
+        const endpointTraceIdHeaderName = this.getEndpointTraceIdHeaderName();
+        if (endpointTraceIdHeaderName == null) {
+            return (void 0);
+        }
+        const headerName = Object.keys(headers).find(name => name.toLowerCase() === endpointTraceIdHeaderName);
+        if (headerName != null) {
+            return (void 0);
+        }
+        let endpointTraceId = headers[headerName];
+        const endpointTraceIdScope = this.getEndpointTraceIdScope();
+        if (endpointTraceId != null) {
+            if (endpointTraceIdScope === 'system') {
+                request.$context.setScopedTraceId(this.getEndpointSystemCode(), endpointTraceIdHeaderName, endpointTraceId);
             }
             else {
-                body = (void 0);
+                request.$context.setScopedTraceId(this.getEndpointKey(), endpointTraceIdHeaderName, endpointTraceId);
             }
-            if (body != null && typeof body !== 'string') {
-                body = JSON.stringify(body);
+        }
+        else {
+            if (endpointTraceIdScope === 'system') {
+                endpointTraceId = request.$context.findScopedTraceId(this.getEndpointSystemCode())?.[1];
             }
-            const headers = { ...staticHeaders, ...transparentHeaders, ...generatedHeaders };
-            Object.keys(headers).filter(name => {
-                return ['content-encoding', 'content-length'].includes(name.toLowerCase());
-            }).forEach(name => {
-                delete headers[name];
-            });
-            const response = await fetch(url, {
-                method, headers, body,
-                signal: this.needTimeout() ? (() => {
-                    const controller = new AbortController();
-                    setTimeout(() => controller.abort(), this.getEndpointTimeout());
-                    return controller.signal;
-                })() : (void 0)
-            });
-            const status = response.status;
-            if (status >= 400) {
-                return await this._responseErrorHandleFunc({
-                    $url: url, $factor: data, $request: request, $response: response,
-                    $errorCode: `${status}`
-                }, $helpers, $helpers);
+            else {
+                endpointTraceId = request.$context.findScopedTraceId(this.getEndpointKey())?.[1];
+            }
+            if (endpointTraceId != null) {
+                headers[endpointTraceIdHeaderName] = endpointTraceId;
+            }
+        }
+        return endpointTraceId;
+    }
+    async generateRequestHeaders(data, request, $helpers) {
+        const staticHeaders = this.getEndpointHeaders() ?? {};
+        const transparentHeaders = (this.getTransparentHeaderNames() ?? []).reduce((headers, name) => {
+            const value = Utils.getValue(data, name);
+            if (value == null) ;
+            else if (Array.isArray(value)) {
+                const headerValue = value.filter(v => v != null && `${v}`.length !== 0).join(', ');
+                if (headerValue.length !== 0) {
+                    headers[name] = headerValue;
+                }
+            }
+            else if (typeof value === 'object') {
+                Object.keys(value).forEach(key => {
+                    const headerValue = value[key];
+                    if (headerValue != null) {
+                        const s = `${headerValue}`;
+                        if (s.length !== 0) {
+                            headers[key] = s;
+                        }
+                    }
+                });
             }
             else {
-                return await this._responseGenerateFunc(response, data, request, $helpers, $helpers);
+                const headerValue = `${value}`;
+                if (headerValue.length !== 0) {
+                    headers[name] = headerValue;
+                }
             }
+            return headers;
+        }, {});
+        (this.getOmittedTransparentHeaderNames() ?? []).forEach(name => delete transparentHeaders[name]);
+        const generatedHeaders = await this._headersGenerateFunc(data, request, $helpers, $helpers) ?? {};
+        const headers = { ...staticHeaders, ...transparentHeaders, ...generatedHeaders };
+        Object.keys(headers).filter(name => {
+            return ['content-encoding', 'content-length'].includes(name.toLowerCase());
+        }).forEach(name => {
+            delete headers[name];
+        });
+        return headers;
+    }
+    async generateRequestBody(method, data, request, $helpers) {
+        let body;
+        const bodyUsed = this.isBodyUsed();
+        if (bodyUsed === true || (bodyUsed == null && method !== 'get')) {
+            body = await this._bodyGenerateFunc(data, request, $helpers, $helpers);
+        }
+        else {
+            body = (void 0);
+        }
+        if (body != null && typeof body !== 'string') {
+            body = JSON.stringify(body);
+        }
+        return body;
+    }
+    tryToRetrieveTraceIdFromResponse(request, response) {
+        const endpointTraceIdHeaderName = this.getEndpointTraceIdHeaderName();
+        if (endpointTraceIdHeaderName == null) {
+            return (void 0);
+        }
+        const endpointTraceId = response.headers.get(endpointTraceIdHeaderName);
+        if (endpointTraceId == null || endpointTraceId.length === 0) {
+            return (void 0);
+        }
+        const endpointTraceIdScope = this.getEndpointTraceIdScope();
+        if (endpointTraceIdScope === 'system') {
+            request.$context.setScopedTraceId(this.getEndpointSystemCode(), endpointTraceIdHeaderName, endpointTraceId);
+        }
+        else {
+            request.$context.setScopedTraceId(this.getEndpointKey(), endpointTraceIdHeaderName, endpointTraceId);
+        }
+        return endpointTraceId;
+    }
+    async sendRequest(url, method, headers, body, data, request, $helpers) {
+        const response = await fetch(url, {
+            method, headers, body,
+            signal: this.needTimeout() ? (() => {
+                const controller = new AbortController();
+                setTimeout(() => controller.abort(), this.getEndpointTimeout());
+                return controller.signal;
+            })() : (void 0)
+        });
+        this.tryToRetrieveTraceIdFromResponse(request, response);
+        const status = response.status;
+        if (status >= 400) {
+            return await this._responseErrorHandleFunc({
+                $url: url, $factor: data, $request: request, $response: response,
+                $errorCode: `${status}`
+            }, $helpers, $helpers);
+        }
+        else {
+            return await this._responseGenerateFunc(response, data, request, $helpers, $helpers);
+        }
+    }
+    async doPerform(data, request) {
+        const $helpers = this.getHelpers();
+        let url = '';
+        try {
+            url = await this._urlGenerateFunc(this.getEndpointUrl(), data, request, $helpers, $helpers);
+            const method = this.getEndpointMethod();
+            const headers = await this.generateRequestHeaders(data, request, $helpers);
+            this.syncEndpointTraceId(headers, request);
+            const body = await this.generateRequestBody(method, data, request, $helpers);
+            return await this.sendRequest(url, method, headers, body, data, request, $helpers);
         }
         catch (e) {
             if (e instanceof DOMException || e.name === 'AbortError') {
@@ -2340,7 +2418,9 @@ class TypeOrmLoadManyBySQLUseCursorPipelineStep extends AbstractTypeOrmBySQLPipe
                             ...this.buildStepOptions(), name: this.getName(), steps: this.getStepBuilders()
                         });
                         const { content: _, $context, ...rest } = request;
-                        const contextForSub = { ...$context, $typeOrmCursorRound: cursorRound, $typeOrmCursorEnd: end };
+                        const contextForSub = $context.temporaryWith({
+                            $typeOrmCursorRound: cursorRound, $typeOrmCursorEnd: end
+                        });
                         const requestForSub = { ...rest, $context: contextForSub, content: contentForSub };
                         const result = await sets.perform(requestForSub);
                         const { content } = result;