@rainbow-o23/n1 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.babelrc +11 -0
- package/.eslintrc +23 -0
- package/README.md +233 -0
- package/index.cjs +760 -0
- package/index.d.ts +3 -0
- package/index.js +732 -0
- package/lib/pipeline/envs.d.ts +1 -0
- package/lib/pipeline/index.d.ts +6 -0
- package/lib/pipeline/pipeline-execution.d.ts +38 -0
- package/lib/pipeline/pipeline-step.d.ts +41 -0
- package/lib/pipeline/pipeline.d.ts +47 -0
- package/lib/pipeline/step-helpers-utils.d.ts +41 -0
- package/lib/pipeline/step-helpers.d.ts +31 -0
- package/lib/repo/index.d.ts +1 -0
- package/lib/repo/pipeline-repository.d.ts +11 -0
- package/lib/utils/config.d.ts +16 -0
- package/lib/utils/error.d.ts +21 -0
- package/lib/utils/index.d.ts +4 -0
- package/lib/utils/logger.d.ts +50 -0
- package/lib/utils/types.d.ts +7 -0
- package/package.json +41 -0
- package/rollup.config.base.js +30 -0
- package/rollup.config.ci.js +3 -0
- package/rollup.config.js +3 -0
- package/src/index.ts +4 -0
- package/src/lib/pipeline/envs.ts +20 -0
- package/src/lib/pipeline/index.ts +7 -0
- package/src/lib/pipeline/pipeline-execution.ts +137 -0
- package/src/lib/pipeline/pipeline-step.ts +79 -0
- package/src/lib/pipeline/pipeline.ts +148 -0
- package/src/lib/pipeline/step-helpers-utils.ts +143 -0
- package/src/lib/pipeline/step-helpers.ts +77 -0
- package/src/lib/repo/index.ts +1 -0
- package/src/lib/repo/pipeline-repository.ts +53 -0
- package/src/lib/utils/config.ts +110 -0
- package/src/lib/utils/error.ts +41 -0
- package/src/lib/utils/index.ts +6 -0
- package/src/lib/utils/logger.ts +292 -0
- package/src/lib/utils/types.ts +11 -0
- package/tsconfig.json +36 -0
package/.babelrc
ADDED
package/.eslintrc
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
{
|
|
2
|
+
"root": true,
|
|
3
|
+
"parser": "@typescript-eslint/parser",
|
|
4
|
+
"plugins": [
|
|
5
|
+
"@typescript-eslint"
|
|
6
|
+
],
|
|
7
|
+
"extends": [
|
|
8
|
+
"eslint:recommended",
|
|
9
|
+
"plugin:@typescript-eslint/eslint-recommended",
|
|
10
|
+
"plugin:@typescript-eslint/recommended"
|
|
11
|
+
],
|
|
12
|
+
"rules": {
|
|
13
|
+
"no-plusplus": 0,
|
|
14
|
+
"no-mixed-spaces-and-tabs": [
|
|
15
|
+
"error",
|
|
16
|
+
"smart-tabs"
|
|
17
|
+
],
|
|
18
|
+
"@typescript-eslint/no-empty-interface": "off",
|
|
19
|
+
"@typescript/no-non-null-assertion": "off",
|
|
20
|
+
"@typescript-eslint/adjacent-overload-signatures": "off",
|
|
21
|
+
"@typescript-eslint/ban-ts-comment": "error"
|
|
22
|
+
}
|
|
23
|
+
}
|
package/README.md
ADDED
|
@@ -0,0 +1,233 @@
|
|
|
1
|
+
# o23/n1
|
|
2
|
+
|
|
3
|
+
The `o23/n1` module provides two parts of implementation:
|
|
4
|
+
|
|
5
|
+
- The pipeline and pipeline steps, along with their unified registry.
|
|
6
|
+
- The basic building blocks, including environment variable retrieval, logging, and exception definitions.
|
|
7
|
+
|
|
8
|
+
## Pipeline and Pipeline Steps
|
|
9
|
+
|
|
10
|
+
An application may allow for multiple pipelines to exist, so each pipeline will have a globally unique code to identify it. The pipeline
|
|
11
|
+
interface also provides an execution API, which fundamentally does not care about the specific format of the incoming and outgoing data, nor
|
|
12
|
+
the internal execution logic. Therefore, the pipeline interface is a higher-order definition, and the interface itself may not even be
|
|
13
|
+
concerned with the internal execution mechanism.
|
|
14
|
+
|
|
15
|
+
> Make sure that all pipelines and pipeline steps are stateless, so that singletons can be used to save the time of creating instances.
|
|
16
|
+
|
|
17
|
+
```typescript
|
|
18
|
+
export interface PipelineRequest<C = PipelineRequestPayload> {
|
|
19
|
+
payload: C;
|
|
20
|
+
traceId?: string;
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
export interface PipelineResponse<C = PipelineResponsePayload> {
|
|
24
|
+
payload: C;
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
export interface Pipeline<In = any, Out = any> {
|
|
28
|
+
/**
|
|
29
|
+
* code should be unique globally
|
|
30
|
+
*/
|
|
31
|
+
getCode(): PipelineCode;
|
|
32
|
+
|
|
33
|
+
/**
|
|
34
|
+
* perform pipeline
|
|
35
|
+
*/
|
|
36
|
+
perform(request: PipelineRequest<In>): Promise<PipelineResponse<Out>>;
|
|
37
|
+
}
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
To facilitate tracking of the execution logic, the pipeline provides a context object that includes one `traceId` for tracing the execution
|
|
41
|
+
logs of the entire pipeline.
|
|
42
|
+
|
|
43
|
+
In fact, the execution of a pipeline depends on its pipeline steps. A pipeline can contain one or more steps. Therefore, `o23/n1` also
|
|
44
|
+
provides an implementation of AbstractPipeline. By implementing this abstract class and providing constructors for pipeline steps, you can
|
|
45
|
+
obtain a fully executable pipeline class and use it for execution.
|
|
46
|
+
|
|
47
|
+
```typescript
|
|
48
|
+
export interface PipelineStepBuilder {
|
|
49
|
+
create(options?: PipelineStepOptions): Promise<PipelineStep>;
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
export abstract class AbstractPipeline<In = any, Out = any> implements Pipeline<In, Out> {
|
|
53
|
+
// ...
|
|
54
|
+
protected abstract getStepBuilders(): Array<PipelineStepBuilder>;
|
|
55
|
+
|
|
56
|
+
// ...
|
|
57
|
+
}
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
To conveniently implement a pipeline class, `o23/n1` provides a way to construct a pipeline based on static pipeline step classes, as
|
|
61
|
+
follows:
|
|
62
|
+
|
|
63
|
+
```typescript
|
|
64
|
+
export interface PipelineStepType<S = PipelineStep> extends Function {
|
|
65
|
+
new(options?: PipelineStepOptions): S;
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
export class DefaultPipelineStepBuilder implements PipelineStepBuilder {
|
|
69
|
+
public constructor(protected readonly step: PipelineStepType) {
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
public async create(options?: PipelineStepOptions): Promise<PipelineStep> {
|
|
73
|
+
return new this.step(options);
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
export abstract class AbstractStaticPipeline<In = any, Out = any> extends AbstractPipeline<In, Out> {
|
|
78
|
+
protected abstract getStepTypes(): Array<PipelineStepType>;
|
|
79
|
+
|
|
80
|
+
protected getStepBuilders(): Array<PipelineStepBuilder> {
|
|
81
|
+
return this.getStepTypes().map(type => new DefaultPipelineStepBuilder(type));
|
|
82
|
+
}
|
|
83
|
+
}
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Here is a simple implementation of a pipeline. You can find the corresponding code in the `o23/scaffold` module.
|
|
87
|
+
|
|
88
|
+
```typescript
|
|
89
|
+
export class SimplePipelineStep1 extends AbstractPipelineStep<number, number> {
|
|
90
|
+
public perform(request: PipelineStepData<number>): Promise<PipelineStepData<number>> {
|
|
91
|
+
this.debug(() => `Perform (${request.content} + 100)`);
|
|
92
|
+
return Promise.resolve({content: request.content + 100});
|
|
93
|
+
}
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
export class SimplePipelineStep2 extends AbstractPipelineStep<number, number> {
|
|
97
|
+
public perform(request: PipelineStepData<number>): Promise<PipelineStepData<number>> {
|
|
98
|
+
this.debug(() => `Perform (${request.content} * 2)`);
|
|
99
|
+
return Promise.resolve({content: request.content * 2});
|
|
100
|
+
}
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
export class SimplePipeline extends AbstractStaticPipeline<number, number> {
|
|
104
|
+
public getCode(): PipelineCode {
|
|
105
|
+
return 'SimplePipeline';
|
|
106
|
+
}
|
|
107
|
+
|
|
108
|
+
protected getStepTypes(): Array<PipelineStepType> {
|
|
109
|
+
return [SimplePipelineStep1, SimplePipelineStep2];
|
|
110
|
+
}
|
|
111
|
+
}
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
### Function Support
|
|
115
|
+
|
|
116
|
+
Pipeline steps provide rich function support, and all the following functions or instances can be obtained from the `getHelpers` function.
|
|
117
|
+
|
|
118
|
+
| Syntax | Comments |
|
|
119
|
+
|-----------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
120
|
+
| $config | Get config instance. |
|
|
121
|
+
| $logger | Get logger instance. |
|
|
122
|
+
| $date.now() | Get current datetime, as string. |
|
|
123
|
+
| $date.dayjs | Get [Day.js](https://day.js.org/). |
|
|
124
|
+
| $nano(size?: number) | Create a nano string. |
|
|
125
|
+
| $ascii(size?: number) | Create a nano string, only contains ascii characters (0-9, a-z, A-Z, _). |
|
|
126
|
+
| $error(options: PipelineStepErrorOptions) | Throw an exposed uncatchable error. |
|
|
127
|
+
| $errors.catchable(options: Omit<PipelineStepErrorOptions, 'status'>) | Throw a catchable error. |
|
|
128
|
+
| $errors.exposed(options: PipelineStepErrorOptions) | Throw an exposed uncatchable error, same as `$helpers.error`. |
|
|
129
|
+
| $errors.catchable(uncatchable: Omit<PipelineStepErrorOptions, 'status'>) | Throw an uncatchable error. |
|
|
130
|
+
| $file(options: PipelineStepFileOptions) => PipelineStepFile | Create a file instance by given options. |
|
|
131
|
+
| $clearContextData() | If the pipeline step does not return anything or returns null or undefined, the context will continue to be used without any modifications.<br>So returning this semaphore indicates clearing the step content data. |
|
|
132
|
+
|
|
133
|
+
For example:
|
|
134
|
+
|
|
135
|
+
```typescript
|
|
136
|
+
const currentTime = this.getHelpers().$date.now();
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
## Logger
|
|
140
|
+
|
|
141
|
+
`o23/n1` provides a standard logging implementation, which by default outputs to the console. You can obtain a logging instance through the
|
|
142
|
+
following way:
|
|
143
|
+
|
|
144
|
+
```typescript
|
|
145
|
+
import {createLogger, Logger} from '@rainbow-o23/n1';
|
|
146
|
+
|
|
147
|
+
const logger = createLogger();
|
|
148
|
+
|
|
149
|
+
class CustomLogger implements Logger {
|
|
150
|
+
// your implemenation
|
|
151
|
+
}
|
|
152
|
+
|
|
153
|
+
// or use your own logger implementation, your logger should be an implemenation of Logger interface.
|
|
154
|
+
const customLogger = createLogger(new CustomLogger());
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
The log level of `o23/n1` ranges from low to high: debug, verbose, log, warn, and error, while the concept of free log classification is
|
|
158
|
+
also provided. Let's take a look at the following example:
|
|
159
|
+
|
|
160
|
+
```typescript
|
|
161
|
+
import {createLogger, Logger} from '@rainbow-o23/n1';
|
|
162
|
+
|
|
163
|
+
const logger = createLogger();
|
|
164
|
+
|
|
165
|
+
logger.debug('some log.');
|
|
166
|
+
logger.verbose('some verbose info.', {hello: 'world'});
|
|
167
|
+
logger.info('some info.', {hello: 'world'}, 'SomeCategory');
|
|
168
|
+
logger.warn('some warning.', 'SomeCategory');
|
|
169
|
+
logger.error('some error', new Error(), 'SomeCategory');
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
When the number of arguments in the log output function is more than one, and the last argument is of type string, the last argument is
|
|
173
|
+
considered as the log category. There are no restrictions on the category name. The log level and categories of the logging can be globally
|
|
174
|
+
controlled, or controlled at a more granular level with specific categories and levels. Here are some examples:
|
|
175
|
+
|
|
176
|
+
```typescript
|
|
177
|
+
import {EnhancedLogger} from '@rainbow-o23/n1';
|
|
178
|
+
|
|
179
|
+
// Note that when a log level is enabled, all log levels higher than that level will also be enabled. Conversely, when a log level is
|
|
180
|
+
// disabled, all log levels lower than that level will also be disabled.
|
|
181
|
+
EnhancedLogger.enableLevel('debug');
|
|
182
|
+
EnhancedLogger.disableLevel('info');
|
|
183
|
+
|
|
184
|
+
// Or on some category.
|
|
185
|
+
// All levels for given category are enabled.
|
|
186
|
+
EnhancedLogger.enable('SomeCategory');
|
|
187
|
+
EnhancedLogger.disable('SomeCategory');
|
|
188
|
+
// Enable or disable given category on appointed level.
|
|
189
|
+
// Note enable or disable category + level, will not impact other levels.
|
|
190
|
+
EnhancedLogger.enable('SomeCategory.debug');
|
|
191
|
+
EnhancedLogger.disable('SomeCategory.info');
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
## Config
|
|
195
|
+
|
|
196
|
+
`o23/n1` provides a Config object for reading system environment variables, as shown below:
|
|
197
|
+
|
|
198
|
+
```typescript
|
|
199
|
+
import {createConfig, createLogger} from '@rainbow-o23/n1';
|
|
200
|
+
|
|
201
|
+
let config = createConfig();
|
|
202
|
+
|
|
203
|
+
// Or use given logger
|
|
204
|
+
config = createConfig(createLogger());
|
|
205
|
+
|
|
206
|
+
config.getBoolean('pipeline.debug.log.enabled', true);
|
|
207
|
+
config.getString('app.version', 'UNDOCUMENTED');
|
|
208
|
+
config.getNumber('app.port', 3100);
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
The given environment variable name will be transformed into the following format: underscore-separated and prefixed with `CFG_`. It reads
|
|
212
|
+
from `process.env` and if not defined, it uses the provided default value (or returns `undefined` if no default value is given). Following
|
|
213
|
+
this mapping rule, the environment variable names in the example above should be as follows:
|
|
214
|
+
|
|
215
|
+
- `CFG_PIPELINE_DEBUG_LOG_ENABLED`,
|
|
216
|
+
- `CFG_APP_VERSION`,
|
|
217
|
+
- `CFG_APP_PORT`.
|
|
218
|
+
|
|
219
|
+
## Error
|
|
220
|
+
|
|
221
|
+
`o23/n1` defines three basic types of errors, namely `CatchableError`, `UncatchableError`, and `ExposedUncatchableError`. All errors will
|
|
222
|
+
have a code, and `o23`'s error code follows the format of `Oxx-xxxxx`. Here, Oxx represents the module, and the last five digits should be
|
|
223
|
+
numeric codes. For example, if the exception is defined in the `o23/n1` module, the code would be `O01`, such as `O01-00001`, `O01-99999`.
|
|
224
|
+
`ExposedUncatchableError` is a type of error specifically designed for web applications. In addition to the code and message, it also
|
|
225
|
+
includes an additional field called `status`, which represents the HTTP response status.
|
|
226
|
+
|
|
227
|
+
## Environment Parameters
|
|
228
|
+
|
|
229
|
+
| Name | Type | Default Value | Comments |
|
|
230
|
+
|------------------------------------|---------|---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
231
|
+
| `format.datetime` | string | YYYY-MM-DD HH:mm:ss | Default datetime format, follows [Day.js](https://day.js.org/) |
|
|
232
|
+
| `pipeline.debug.log.enabled` | boolean | false | Enable the pipeline debug log. |
|
|
233
|
+
| `pipeline.performance.log.enabled` | boolean | false | Enable the pipeline performance log, spent time of pipeline and pipeline step.<br>Translation: If `pipeline.debug.log.enabled` is true, this log output will also be enabled. |
|