@rainbow-o23/n1 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (40) hide show
  1. package/.babelrc +11 -0
  2. package/.eslintrc +23 -0
  3. package/README.md +233 -0
  4. package/index.cjs +760 -0
  5. package/index.d.ts +3 -0
  6. package/index.js +732 -0
  7. package/lib/pipeline/envs.d.ts +1 -0
  8. package/lib/pipeline/index.d.ts +6 -0
  9. package/lib/pipeline/pipeline-execution.d.ts +38 -0
  10. package/lib/pipeline/pipeline-step.d.ts +41 -0
  11. package/lib/pipeline/pipeline.d.ts +47 -0
  12. package/lib/pipeline/step-helpers-utils.d.ts +41 -0
  13. package/lib/pipeline/step-helpers.d.ts +31 -0
  14. package/lib/repo/index.d.ts +1 -0
  15. package/lib/repo/pipeline-repository.d.ts +11 -0
  16. package/lib/utils/config.d.ts +16 -0
  17. package/lib/utils/error.d.ts +21 -0
  18. package/lib/utils/index.d.ts +4 -0
  19. package/lib/utils/logger.d.ts +50 -0
  20. package/lib/utils/types.d.ts +7 -0
  21. package/package.json +41 -0
  22. package/rollup.config.base.js +30 -0
  23. package/rollup.config.ci.js +3 -0
  24. package/rollup.config.js +3 -0
  25. package/src/index.ts +4 -0
  26. package/src/lib/pipeline/envs.ts +20 -0
  27. package/src/lib/pipeline/index.ts +7 -0
  28. package/src/lib/pipeline/pipeline-execution.ts +137 -0
  29. package/src/lib/pipeline/pipeline-step.ts +79 -0
  30. package/src/lib/pipeline/pipeline.ts +148 -0
  31. package/src/lib/pipeline/step-helpers-utils.ts +143 -0
  32. package/src/lib/pipeline/step-helpers.ts +77 -0
  33. package/src/lib/repo/index.ts +1 -0
  34. package/src/lib/repo/pipeline-repository.ts +53 -0
  35. package/src/lib/utils/config.ts +110 -0
  36. package/src/lib/utils/error.ts +41 -0
  37. package/src/lib/utils/index.ts +6 -0
  38. package/src/lib/utils/logger.ts +292 -0
  39. package/src/lib/utils/types.ts +11 -0
  40. package/tsconfig.json +36 -0
package/.babelrc ADDED
@@ -0,0 +1,11 @@
1
+ {
2
+ "presets": [
3
+ "@babel/preset-typescript",
4
+ [
5
+ "@babel/preset-env",
6
+ {
7
+ "modules": false
8
+ }
9
+ ]
10
+ ]
11
+ }
package/.eslintrc ADDED
@@ -0,0 +1,23 @@
1
+ {
2
+ "root": true,
3
+ "parser": "@typescript-eslint/parser",
4
+ "plugins": [
5
+ "@typescript-eslint"
6
+ ],
7
+ "extends": [
8
+ "eslint:recommended",
9
+ "plugin:@typescript-eslint/eslint-recommended",
10
+ "plugin:@typescript-eslint/recommended"
11
+ ],
12
+ "rules": {
13
+ "no-plusplus": 0,
14
+ "no-mixed-spaces-and-tabs": [
15
+ "error",
16
+ "smart-tabs"
17
+ ],
18
+ "@typescript-eslint/no-empty-interface": "off",
19
+ "@typescript/no-non-null-assertion": "off",
20
+ "@typescript-eslint/adjacent-overload-signatures": "off",
21
+ "@typescript-eslint/ban-ts-comment": "error"
22
+ }
23
+ }
package/README.md ADDED
@@ -0,0 +1,233 @@
1
+ # o23/n1
2
+
3
+ The `o23/n1` module provides two parts of implementation:
4
+
5
+ - The pipeline and pipeline steps, along with their unified registry.
6
+ - The basic building blocks, including environment variable retrieval, logging, and exception definitions.
7
+
8
+ ## Pipeline and Pipeline Steps
9
+
10
+ An application may allow for multiple pipelines to exist, so each pipeline will have a globally unique code to identify it. The pipeline
11
+ interface also provides an execution API, which fundamentally does not care about the specific format of the incoming and outgoing data, nor
12
+ the internal execution logic. Therefore, the pipeline interface is a higher-order definition, and the interface itself may not even be
13
+ concerned with the internal execution mechanism.
14
+
15
+ > Make sure that all pipelines and pipeline steps are stateless, so that singletons can be used to save the time of creating instances.
16
+
17
+ ```typescript
18
+ export interface PipelineRequest<C = PipelineRequestPayload> {
19
+ payload: C;
20
+ traceId?: string;
21
+ }
22
+
23
+ export interface PipelineResponse<C = PipelineResponsePayload> {
24
+ payload: C;
25
+ }
26
+
27
+ export interface Pipeline<In = any, Out = any> {
28
+ /**
29
+ * code should be unique globally
30
+ */
31
+ getCode(): PipelineCode;
32
+
33
+ /**
34
+ * perform pipeline
35
+ */
36
+ perform(request: PipelineRequest<In>): Promise<PipelineResponse<Out>>;
37
+ }
38
+ ```
39
+
40
+ To facilitate tracking of the execution logic, the pipeline provides a context object that includes one `traceId` for tracing the execution
41
+ logs of the entire pipeline.
42
+
43
+ In fact, the execution of a pipeline depends on its pipeline steps. A pipeline can contain one or more steps. Therefore, `o23/n1` also
44
+ provides an implementation of AbstractPipeline. By implementing this abstract class and providing constructors for pipeline steps, you can
45
+ obtain a fully executable pipeline class and use it for execution.
46
+
47
+ ```typescript
48
+ export interface PipelineStepBuilder {
49
+ create(options?: PipelineStepOptions): Promise<PipelineStep>;
50
+ }
51
+
52
+ export abstract class AbstractPipeline<In = any, Out = any> implements Pipeline<In, Out> {
53
+ // ...
54
+ protected abstract getStepBuilders(): Array<PipelineStepBuilder>;
55
+
56
+ // ...
57
+ }
58
+ ```
59
+
60
+ To conveniently implement a pipeline class, `o23/n1` provides a way to construct a pipeline based on static pipeline step classes, as
61
+ follows:
62
+
63
+ ```typescript
64
+ export interface PipelineStepType<S = PipelineStep> extends Function {
65
+ new(options?: PipelineStepOptions): S;
66
+ }
67
+
68
+ export class DefaultPipelineStepBuilder implements PipelineStepBuilder {
69
+ public constructor(protected readonly step: PipelineStepType) {
70
+ }
71
+
72
+ public async create(options?: PipelineStepOptions): Promise<PipelineStep> {
73
+ return new this.step(options);
74
+ }
75
+ }
76
+
77
+ export abstract class AbstractStaticPipeline<In = any, Out = any> extends AbstractPipeline<In, Out> {
78
+ protected abstract getStepTypes(): Array<PipelineStepType>;
79
+
80
+ protected getStepBuilders(): Array<PipelineStepBuilder> {
81
+ return this.getStepTypes().map(type => new DefaultPipelineStepBuilder(type));
82
+ }
83
+ }
84
+ ```
85
+
86
+ Here is a simple implementation of a pipeline. You can find the corresponding code in the `o23/scaffold` module.
87
+
88
+ ```typescript
89
+ export class SimplePipelineStep1 extends AbstractPipelineStep<number, number> {
90
+ public perform(request: PipelineStepData<number>): Promise<PipelineStepData<number>> {
91
+ this.debug(() => `Perform (${request.content} + 100)`);
92
+ return Promise.resolve({content: request.content + 100});
93
+ }
94
+ }
95
+
96
+ export class SimplePipelineStep2 extends AbstractPipelineStep<number, number> {
97
+ public perform(request: PipelineStepData<number>): Promise<PipelineStepData<number>> {
98
+ this.debug(() => `Perform (${request.content} * 2)`);
99
+ return Promise.resolve({content: request.content * 2});
100
+ }
101
+ }
102
+
103
+ export class SimplePipeline extends AbstractStaticPipeline<number, number> {
104
+ public getCode(): PipelineCode {
105
+ return 'SimplePipeline';
106
+ }
107
+
108
+ protected getStepTypes(): Array<PipelineStepType> {
109
+ return [SimplePipelineStep1, SimplePipelineStep2];
110
+ }
111
+ }
112
+ ```
113
+
114
+ ### Function Support
115
+
116
+ Pipeline steps provide rich function support, and all the following functions or instances can be obtained from the `getHelpers` function.
117
+
118
+ | Syntax | Comments |
119
+ |-----------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
120
+ | $config | Get config instance. |
121
+ | $logger | Get logger instance. |
122
+ | $date.now() | Get current datetime, as string. |
123
+ | $date.dayjs | Get [Day.js](https://day.js.org/). |
124
+ | $nano(size?: number) | Create a nano string. |
125
+ | $ascii(size?: number) | Create a nano string, only contains ascii characters (0-9, a-z, A-Z, _). |
126
+ | $error(options: PipelineStepErrorOptions) | Throw an exposed uncatchable error. |
127
+ | $errors.catchable(options: Omit<PipelineStepErrorOptions, 'status'>) | Throw a catchable error. |
128
+ | $errors.exposed(options: PipelineStepErrorOptions) | Throw an exposed uncatchable error, same as `$helpers.error`. |
129
+ | $errors.catchable(uncatchable: Omit<PipelineStepErrorOptions, 'status'>) | Throw an uncatchable error. |
130
+ | $file(options: PipelineStepFileOptions) => PipelineStepFile | Create a file instance by given options. |
131
+ | $clearContextData() | If the pipeline step does not return anything or returns null or undefined, the context will continue to be used without any modifications.<br>So returning this semaphore indicates clearing the step content data. |
132
+
133
+ For example:
134
+
135
+ ```typescript
136
+ const currentTime = this.getHelpers().$date.now();
137
+ ```
138
+
139
+ ## Logger
140
+
141
+ `o23/n1` provides a standard logging implementation, which by default outputs to the console. You can obtain a logging instance through the
142
+ following way:
143
+
144
+ ```typescript
145
+ import {createLogger, Logger} from '@rainbow-o23/n1';
146
+
147
+ const logger = createLogger();
148
+
149
+ class CustomLogger implements Logger {
150
+ // your implemenation
151
+ }
152
+
153
+ // or use your own logger implementation, your logger should be an implemenation of Logger interface.
154
+ const customLogger = createLogger(new CustomLogger());
155
+ ```
156
+
157
+ The log level of `o23/n1` ranges from low to high: debug, verbose, log, warn, and error, while the concept of free log classification is
158
+ also provided. Let's take a look at the following example:
159
+
160
+ ```typescript
161
+ import {createLogger, Logger} from '@rainbow-o23/n1';
162
+
163
+ const logger = createLogger();
164
+
165
+ logger.debug('some log.');
166
+ logger.verbose('some verbose info.', {hello: 'world'});
167
+ logger.info('some info.', {hello: 'world'}, 'SomeCategory');
168
+ logger.warn('some warning.', 'SomeCategory');
169
+ logger.error('some error', new Error(), 'SomeCategory');
170
+ ```
171
+
172
+ When the number of arguments in the log output function is more than one, and the last argument is of type string, the last argument is
173
+ considered as the log category. There are no restrictions on the category name. The log level and categories of the logging can be globally
174
+ controlled, or controlled at a more granular level with specific categories and levels. Here are some examples:
175
+
176
+ ```typescript
177
+ import {EnhancedLogger} from '@rainbow-o23/n1';
178
+
179
+ // Note that when a log level is enabled, all log levels higher than that level will also be enabled. Conversely, when a log level is
180
+ // disabled, all log levels lower than that level will also be disabled.
181
+ EnhancedLogger.enableLevel('debug');
182
+ EnhancedLogger.disableLevel('info');
183
+
184
+ // Or on some category.
185
+ // All levels for given category are enabled.
186
+ EnhancedLogger.enable('SomeCategory');
187
+ EnhancedLogger.disable('SomeCategory');
188
+ // Enable or disable given category on appointed level.
189
+ // Note enable or disable category + level, will not impact other levels.
190
+ EnhancedLogger.enable('SomeCategory.debug');
191
+ EnhancedLogger.disable('SomeCategory.info');
192
+ ```
193
+
194
+ ## Config
195
+
196
+ `o23/n1` provides a Config object for reading system environment variables, as shown below:
197
+
198
+ ```typescript
199
+ import {createConfig, createLogger} from '@rainbow-o23/n1';
200
+
201
+ let config = createConfig();
202
+
203
+ // Or use given logger
204
+ config = createConfig(createLogger());
205
+
206
+ config.getBoolean('pipeline.debug.log.enabled', true);
207
+ config.getString('app.version', 'UNDOCUMENTED');
208
+ config.getNumber('app.port', 3100);
209
+ ```
210
+
211
+ The given environment variable name will be transformed into the following format: underscore-separated and prefixed with `CFG_`. It reads
212
+ from `process.env` and if not defined, it uses the provided default value (or returns `undefined` if no default value is given). Following
213
+ this mapping rule, the environment variable names in the example above should be as follows:
214
+
215
+ - `CFG_PIPELINE_DEBUG_LOG_ENABLED`,
216
+ - `CFG_APP_VERSION`,
217
+ - `CFG_APP_PORT`.
218
+
219
+ ## Error
220
+
221
+ `o23/n1` defines three basic types of errors, namely `CatchableError`, `UncatchableError`, and `ExposedUncatchableError`. All errors will
222
+ have a code, and `o23`'s error code follows the format of `Oxx-xxxxx`. Here, Oxx represents the module, and the last five digits should be
223
+ numeric codes. For example, if the exception is defined in the `o23/n1` module, the code would be `O01`, such as `O01-00001`, `O01-99999`.
224
+ `ExposedUncatchableError` is a type of error specifically designed for web applications. In addition to the code and message, it also
225
+ includes an additional field called `status`, which represents the HTTP response status.
226
+
227
+ ## Environment Parameters
228
+
229
+ | Name | Type | Default Value | Comments |
230
+ |------------------------------------|---------|---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
231
+ | `format.datetime` | string | YYYY-MM-DD HH:mm:ss | Default datetime format, follows [Day.js](https://day.js.org/) |
232
+ | `pipeline.debug.log.enabled` | boolean | false | Enable the pipeline debug log. |
233
+ | `pipeline.performance.log.enabled` | boolean | false | Enable the pipeline performance log, spent time of pipeline and pipeline step.<br>Translation: If `pipeline.debug.log.enabled` is true, this log output will also be enabled. |