@venizia/ignis-docs 0.2.0 → 0.2.1-0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -14
- package/content/best-practices/api-usage-examples.md +39 -19
- package/content/best-practices/architectural-patterns.md +22 -11
- package/content/best-practices/architecture-decisions.md +29 -16
- package/content/best-practices/code-style-standards/advanced-patterns.md +43 -31
- package/content/best-practices/code-style-standards/constants-configuration.md +19 -4
- package/content/best-practices/code-style-standards/control-flow.md +33 -1
- package/content/best-practices/code-style-standards/documentation.md +28 -8
- package/content/best-practices/code-style-standards/function-patterns.md +15 -4
- package/content/best-practices/code-style-standards/index.md +7 -2
- package/content/best-practices/code-style-standards/naming-conventions.md +26 -2
- package/content/best-practices/code-style-standards/route-definitions.md +9 -5
- package/content/best-practices/code-style-standards/tooling.md +14 -1
- package/content/best-practices/code-style-standards/type-safety.md +39 -6
- package/content/best-practices/common-pitfalls.md +59 -35
- package/content/best-practices/contribution-workflow.md +6 -2
- package/content/best-practices/data-modeling.md +78 -62
- package/content/best-practices/deployment-strategies.md +56 -19
- package/content/best-practices/error-handling.md +182 -93
- package/content/best-practices/index.md +6 -0
- package/content/best-practices/performance-optimization.md +26 -13
- package/content/best-practices/security-guidelines.md +35 -9
- package/content/best-practices/testing-strategies.md +7 -2
- package/content/best-practices/troubleshooting-tips.md +27 -17
- package/content/extensions/components/api-reference.md +107 -322
- package/content/extensions/components/authentication/api.md +454 -603
- package/content/extensions/components/authentication/errors.md +121 -498
- package/content/extensions/components/authentication/index.md +88 -801
- package/content/extensions/components/authentication/usage.md +207 -956
- package/content/extensions/components/authorization/api.md +736 -656
- package/content/extensions/components/authorization/errors.md +168 -206
- package/content/extensions/components/authorization/index.md +82 -797
- package/content/extensions/components/authorization/usage.md +194 -527
- package/content/extensions/components/health-check.md +71 -243
- package/content/extensions/components/mail/api.md +504 -287
- package/content/extensions/components/mail/errors.md +73 -61
- package/content/extensions/components/mail/index.md +96 -467
- package/content/extensions/components/mail/usage.md +130 -172
- package/content/extensions/components/request-tracker.md +66 -173
- package/content/extensions/components/socket-io/api.md +195 -13
- package/content/extensions/components/socket-io/errors.md +3 -3
- package/content/extensions/components/socket-io/index.md +50 -337
- package/content/extensions/components/socket-io/usage.md +143 -26
- package/content/extensions/components/static-asset/api.md +410 -142
- package/content/extensions/components/static-asset/errors.md +110 -53
- package/content/extensions/components/static-asset/index.md +79 -608
- package/content/extensions/components/static-asset/usage.md +180 -300
- package/content/extensions/components/websocket/api.md +275 -399
- package/content/extensions/components/websocket/errors.md +47 -56
- package/content/extensions/components/websocket/index.md +74 -407
- package/content/extensions/components/websocket/usage.md +110 -341
- package/content/extensions/helpers/cron/index.md +51 -160
- package/content/extensions/helpers/crypto/index.md +62 -483
- package/content/extensions/helpers/crypto/reference.md +456 -0
- package/content/extensions/helpers/env/index.md +60 -178
- package/content/extensions/helpers/error/index.md +221 -207
- package/content/extensions/helpers/inversion/index.md +65 -556
- package/content/extensions/helpers/inversion/reference.md +522 -0
- package/content/extensions/helpers/kafka/admin.md +20 -1
- package/content/extensions/helpers/kafka/compile-binary.md +41 -35
- package/content/extensions/helpers/kafka/consumer.md +54 -20
- package/content/extensions/helpers/kafka/examples.md +21 -16
- package/content/extensions/helpers/kafka/index.md +80 -610
- package/content/extensions/helpers/kafka/producer.md +134 -5
- package/content/extensions/helpers/kafka/schema-registry.md +45 -70
- package/content/extensions/helpers/logger/hf-logger.md +193 -0
- package/content/extensions/helpers/logger/index.md +64 -563
- package/content/extensions/helpers/logger/pino.md +85 -0
- package/content/extensions/helpers/logger/reference.md +746 -0
- package/content/extensions/helpers/network/api.md +241 -195
- package/content/extensions/helpers/network/index.md +72 -530
- package/content/extensions/helpers/queue/index.md +72 -900
- package/content/extensions/helpers/queue/reference.md +467 -0
- package/content/extensions/helpers/redis/index.md +73 -645
- package/content/extensions/helpers/redis/reference.md +727 -0
- package/content/extensions/helpers/secrets/index.md +66 -0
- package/content/extensions/helpers/socket-io/api.md +305 -203
- package/content/extensions/helpers/socket-io/index.md +66 -432
- package/content/extensions/helpers/storage/api.md +564 -462
- package/content/extensions/helpers/storage/index.md +77 -573
- package/content/extensions/helpers/types/index.md +66 -499
- package/content/extensions/helpers/types/reference.md +650 -0
- package/content/extensions/helpers/uid/index.md +58 -227
- package/content/extensions/helpers/websocket/api.md +329 -216
- package/content/extensions/helpers/websocket/index.md +65 -503
- package/content/extensions/helpers/worker-thread/index.md +58 -396
- package/content/extensions/helpers/worker-thread/reference.md +428 -0
- package/content/guides/core-concepts/persistent/models.md +1 -1
- package/content/guides/core-concepts/persistent/search-typesense.md +3 -3
- package/content/guides/core-concepts/persistent/transactions.md +1 -1
- package/content/guides/core-concepts/secrets-vault.md +177 -0
- package/content/guides/core-concepts/services.md +1 -1
- package/content/guides/migrations/redis-helpers-migration.md +1 -1
- package/content/guides/migrations/unified-connectors-migration.md +2 -2
- package/content/guides/tutorials/ecommerce-api.md +3 -8
- package/content/references/base/application.md +1 -1
- package/content/references/base/connectors.md +79 -136
- package/content/references/base/datasources-reference.md +599 -0
- package/content/references/base/datasources.md +84 -444
- package/content/references/base/dependency-injection.md +17 -39
- package/content/references/base/filter-system/application-usage.md +69 -121
- package/content/references/base/filter-system/array-operators.md +12 -0
- package/content/references/base/filter-system/comparison-operators.md +12 -0
- package/content/references/base/filter-system/default-filter.md +136 -348
- package/content/references/base/filter-system/fields-order-pagination.md +38 -16
- package/content/references/base/filter-system/index.md +106 -257
- package/content/references/base/filter-system/json-filtering.md +12 -2
- package/content/references/base/filter-system/list-operators.md +16 -2
- package/content/references/base/filter-system/logical-operators.md +13 -0
- package/content/references/base/filter-system/null-operators.md +13 -0
- package/content/references/base/filter-system/pattern-matching.md +12 -0
- package/content/references/base/filter-system/quick-reference.md +11 -2
- package/content/references/base/filter-system/range-operators.md +12 -0
- package/content/references/base/filter-system/tips.md +70 -133
- package/content/references/base/filter-system/use-cases.md +156 -233
- package/content/references/base/middlewares.md +35 -21
- package/content/references/base/models-reference.md +886 -0
- package/content/references/base/models.md +80 -1452
- package/content/references/base/repositories/advanced.md +156 -192
- package/content/references/base/repositories/index.md +77 -650
- package/content/references/base/repositories/mixins.md +22 -18
- package/content/references/base/repositories/relations.md +123 -171
- package/content/references/base/repositories/soft-deletable.md +58 -56
- package/content/references/base/secrets.md +263 -0
- package/content/references/base/services.md +2 -2
- package/content/references/configuration/environment-variables.md +48 -4
- package/content/references/configuration/index.md +49 -31
- package/content/references/quick-reference.md +3 -16
- package/content/references/utilities/crypto.md +35 -76
- package/content/references/utilities/date.md +33 -73
- package/content/references/utilities/index.md +1 -1
- package/content/references/utilities/jsx-reference.md +298 -0
- package/content/references/utilities/jsx.md +82 -525
- package/content/references/utilities/module.md +29 -62
- package/content/references/utilities/parse.md +34 -64
- package/content/references/utilities/performance.md +33 -58
- package/content/references/utilities/promise.md +28 -62
- package/content/references/utilities/request.md +57 -218
- package/content/references/utilities/schema.md +43 -137
- package/content/references/utilities/statuses-reference.md +361 -0
- package/content/references/utilities/statuses.md +63 -667
- package/package.json +8 -8
|
@@ -1,231 +1,80 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Worker Thread
|
|
3
|
+
description: Manage Node.js worker_threads with a pooled registry, lifecycle-event helpers, and two-way MessagePort communication
|
|
4
|
+
difficulty: intermediate
|
|
5
|
+
---
|
|
6
|
+
|
|
1
7
|
# Worker Thread
|
|
2
8
|
|
|
3
|
-
|
|
9
|
+
The worker-thread helper wraps Node's `worker_threads` with a pooled registry, lifecycle-event helpers, and a `MessagePort` bus for two-way communication between the main thread and a worker.
|
|
4
10
|
|
|
5
|
-
##
|
|
11
|
+
## In one example
|
|
6
12
|
|
|
7
|
-
|
|
8
|
-
|-------|---------|----------|
|
|
9
|
-
| `WorkerPoolHelper` | `BaseHelper` | Singleton registry that tracks and limits active worker instances |
|
|
10
|
-
| `BaseWorkerHelper<MessageType>` | `AbstractWorkerHelper<MessageType>` | Wraps a `Worker` with event lifecycle hooks (online, exit, error, message) |
|
|
11
|
-
| `BaseWorkerThreadHelper` | `AbstractWorkerThreadHelper` | Runs inside a worker thread; manages named `WorkerBus` channels |
|
|
12
|
-
| `BaseWorkerBusHelper<IC, IP>` | `AbstractWorkerBusHelper<IC, IP>` | Bidirectional `MessagePort` communication with pre/post hooks |
|
|
13
|
-
| `BaseWorkerMessageBusHandlerHelper<IC>` | `AbstractWorkerMessageBusHandlerHelper<IC>` | Defines event handlers for a worker bus (message, close, error, exit) |
|
|
13
|
+
The smallest real use: spawn a worker and register it in the singleton pool.
|
|
14
14
|
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
| **Package** | `@venizia/ignis-helpers` |
|
|
18
|
-
| **Peer Dependency** | None (uses built-in `node:worker_threads`) |
|
|
19
|
-
| **Runtimes** | Node.js (uses `node:worker_threads` and `node:os`) |
|
|
15
|
+
```typescript
|
|
16
|
+
import { WorkerPoolHelper, BaseWorkerHelper } from '@venizia/ignis-helpers';
|
|
20
17
|
|
|
21
|
-
|
|
18
|
+
const pool = WorkerPoolHelper.getInstance();
|
|
22
19
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
BaseWorkerBusHelper,
|
|
29
|
-
BaseWorkerMessageBusHandlerHelper,
|
|
30
|
-
} from '@venizia/ignis-helpers';
|
|
20
|
+
const worker = new BaseWorkerHelper<string>({
|
|
21
|
+
identifier: 'image-resizer',
|
|
22
|
+
path: './workers/image-resizer.js',
|
|
23
|
+
options: { workerData: { quality: 80 } },
|
|
24
|
+
});
|
|
31
25
|
|
|
32
|
-
|
|
33
|
-
IWorker,
|
|
34
|
-
IWorkerThread,
|
|
35
|
-
IWorkerBus,
|
|
36
|
-
IWorkerMessageBusHandler,
|
|
37
|
-
} from '@venizia/ignis-helpers';
|
|
26
|
+
pool.register({ key: 'image-resizer', worker });
|
|
38
27
|
```
|
|
39
28
|
|
|
40
|
-
|
|
29
|
+
`WorkerPoolHelper` tracks active workers so the application never spawns more threads than it has CPU cores for.
|
|
41
30
|
|
|
42
|
-
|
|
31
|
+
## How it works
|
|
43
32
|
|
|
44
|
-
`
|
|
33
|
+
- **Main thread vs worker thread.** Two class families split by side: `BaseWorkerHelper` runs on the main thread and wraps a `Worker` instance; `BaseWorkerThreadHelper` runs inside the spawned worker script and throws `[BaseWorker] Cannot start worker in MAIN_THREAD` if constructed there instead.
|
|
34
|
+
- **Pool caps concurrency.** `WorkerPoolHelper` is a lazy singleton (`getInstance()`) that limits registrations to `os.cpus().length`. Past the limit, `register()` returns `false` and logs a warning - it never throws.
|
|
35
|
+
- **Lifecycle hooks, not raw events.** `BaseWorkerHelper` binds `online`, `exit`, `error`, `message`, and `messageerror` once in its constructor. Each has a default logging behavior, overridable per instance via `eventHandlers`. A synchronous throw inside a handler is caught and logged, not left to crash the process.
|
|
36
|
+
- **Two-way messaging via buses.** Inside a worker script, `BaseWorkerThreadHelper` manages named `BaseWorkerBusHelper` instances - each wraps one `MessagePort` - so a single worker can multiplex several independent channels by key.
|
|
45
37
|
|
|
46
|
-
|
|
47
|
-
import { WorkerPoolHelper } from '@venizia/ignis-helpers';
|
|
38
|
+
## Common tasks
|
|
48
39
|
|
|
49
|
-
|
|
50
|
-
const pool = WorkerPoolHelper.getInstance();
|
|
51
|
-
```
|
|
40
|
+
### Look up and message a registered worker
|
|
52
41
|
|
|
53
|
-
|
|
42
|
+
`get()` and `has()` read the pool by key; `size()` reports how many workers are registered.
|
|
54
43
|
|
|
55
44
|
```typescript
|
|
56
|
-
const
|
|
45
|
+
const worker = pool.get<string>({ key: 'image-resizer' });
|
|
46
|
+
if (worker && pool.has({ key: 'image-resizer' })) {
|
|
47
|
+
worker.worker.postMessage('start');
|
|
48
|
+
}
|
|
57
49
|
```
|
|
58
50
|
|
|
59
|
-
|
|
51
|
+
### Unregister and terminate a worker
|
|
60
52
|
|
|
61
|
-
|
|
62
|
-
|--------|------|---------|-------------|
|
|
63
|
-
| `ignoreMaxWarning` | `boolean` | `false` | When `true`, allows registering workers beyond the CPU core count. When `false`, registration is skipped once the pool reaches the CPU core limit. |
|
|
53
|
+
`unregister()` calls `worker.terminate()` before removing the pool entry.
|
|
64
54
|
|
|
65
|
-
|
|
66
|
-
|
|
55
|
+
```typescript
|
|
56
|
+
await pool.unregister({ key: 'image-resizer' });
|
|
57
|
+
```
|
|
67
58
|
|
|
68
|
-
###
|
|
59
|
+
### Override lifecycle handlers
|
|
69
60
|
|
|
70
|
-
`
|
|
61
|
+
Pass `eventHandlers` to react to worker events instead of the default log lines.
|
|
71
62
|
|
|
72
63
|
```typescript
|
|
73
|
-
import { BaseWorkerHelper } from '@venizia/ignis-helpers';
|
|
74
|
-
|
|
75
64
|
const worker = new BaseWorkerHelper<MyMessageType>({
|
|
76
65
|
identifier: 'data-processor',
|
|
77
66
|
path: './workers/data-processor.js',
|
|
78
67
|
options: { workerData: { batchSize: 100 } },
|
|
79
68
|
eventHandlers: {
|
|
80
|
-
onMessage:
|
|
81
|
-
|
|
82
|
-
},
|
|
83
|
-
onError: (opts) => {
|
|
84
|
-
console.error('Worker error:', opts.error);
|
|
85
|
-
},
|
|
86
|
-
},
|
|
87
|
-
});
|
|
88
|
-
```
|
|
89
|
-
|
|
90
|
-
#### BaseWorkerHelper Constructor Options
|
|
91
|
-
|
|
92
|
-
| Option | Type | Default | Description |
|
|
93
|
-
|--------|------|---------|-------------|
|
|
94
|
-
| `identifier` | `string` | -- | A unique name for this worker instance, used in log output. Required. |
|
|
95
|
-
| `path` | `string \| URL` | -- | Path to the worker script file. Required. |
|
|
96
|
-
| `options` | `WorkerOptions` | -- | Node.js `WorkerOptions` passed directly to `new Worker()`. Required. Supports `workerData`, `transferList`, `env`, etc. |
|
|
97
|
-
| `scope` | `string` | `'BaseWorkerHelper'` | Accepted but currently ignored -- the constructor always sets the logger scope to `'BaseWorkerHelper'`. |
|
|
98
|
-
| `eventHandlers` | `Partial<Pick<IWorker<MessageType>, ...>>` | `undefined` | Optional overrides for lifecycle event callbacks. Any handler not provided falls back to default logging behavior. |
|
|
99
|
-
|
|
100
|
-
#### Event Handler Overrides
|
|
101
|
-
|
|
102
|
-
| Handler | Signature | Default Behavior |
|
|
103
|
-
|---------|-----------|-----------------|
|
|
104
|
-
| `onOnline` | `() => ValueOrPromise<void>` | Logs `"Worker ONLINE"` at info level |
|
|
105
|
-
| `onExit` | `(opts: { code: string \| number }) => ValueOrPromise<void>` | Logs `"Worker EXIT"` with exit code at warn level |
|
|
106
|
-
| `onError` | `(opts: { error: Error }) => ValueOrPromise<void>` | Logs `"Worker ERROR"` with error at error level |
|
|
107
|
-
| `onMessage` | `(opts: { message: MessageType }) => ValueOrPromise<void>` | Logs `"Worker MESSAGE"` with message at error level |
|
|
108
|
-
| `onMessageError` | `(opts: { error: Error }) => ValueOrPromise<void>` | Logs `"Worker MESSAGE_ERROR"` with error at error level |
|
|
109
|
-
|
|
110
|
-
### BaseWorkerThreadHelper (Inside Worker Thread)
|
|
111
|
-
|
|
112
|
-
`BaseWorkerThreadHelper` is used inside a worker script to manage named communication buses. It must be instantiated from within a worker thread -- constructing it on the main thread throws an error.
|
|
113
|
-
|
|
114
|
-
```typescript
|
|
115
|
-
// Inside worker-script.js
|
|
116
|
-
import { BaseWorkerThreadHelper } from '@venizia/ignis-helpers';
|
|
117
|
-
|
|
118
|
-
const thread = new BaseWorkerThreadHelper({ scope: 'DataProcessor' });
|
|
119
|
-
```
|
|
120
|
-
|
|
121
|
-
#### BaseWorkerThreadHelper Constructor Options
|
|
122
|
-
|
|
123
|
-
| Option | Type | Default | Description |
|
|
124
|
-
|--------|------|---------|-------------|
|
|
125
|
-
| `scope` | `string` | -- | Logger scope and identifier for this worker thread. Required. |
|
|
126
|
-
|
|
127
|
-
### BaseWorkerBusHelper (MessagePort Communication)
|
|
128
|
-
|
|
129
|
-
`BaseWorkerBusHelper` wraps a `MessagePort` to provide structured bidirectional messaging with lifecycle event handlers.
|
|
130
|
-
|
|
131
|
-
```typescript
|
|
132
|
-
import { BaseWorkerBusHelper, BaseWorkerMessageBusHandlerHelper } from '@venizia/ignis-helpers';
|
|
133
|
-
import { parentPort } from 'node:worker_threads';
|
|
134
|
-
|
|
135
|
-
const handler = new BaseWorkerMessageBusHandlerHelper<IncomingMessage>({
|
|
136
|
-
scope: 'MyBusHandler',
|
|
137
|
-
onMessage: (opts) => {
|
|
138
|
-
console.log('Received:', opts.message);
|
|
69
|
+
onMessage: opts => console.log('Received:', opts.message),
|
|
70
|
+
onError: opts => console.error('Worker error:', opts.error),
|
|
139
71
|
},
|
|
140
72
|
});
|
|
141
|
-
|
|
142
|
-
const bus = new BaseWorkerBusHelper<IncomingMessage, OutgoingMessage>({
|
|
143
|
-
scope: 'MyBus',
|
|
144
|
-
port: parentPort!,
|
|
145
|
-
busHandler: handler,
|
|
146
|
-
});
|
|
147
|
-
```
|
|
148
|
-
|
|
149
|
-
#### BaseWorkerBusHelper Constructor Options
|
|
150
|
-
|
|
151
|
-
| Option | Type | Default | Description |
|
|
152
|
-
|--------|------|---------|-------------|
|
|
153
|
-
| `scope` | `string` | -- | Logger scope and identifier. Required. |
|
|
154
|
-
| `port` | `MessagePort` | -- | The `MessagePort` to bind for communication. Required. |
|
|
155
|
-
| `busHandler` | `IWorkerMessageBusHandler<IConsumePayload>` | -- | Handler that receives incoming messages and lifecycle events. Required. |
|
|
156
|
-
|
|
157
|
-
### BaseWorkerMessageBusHandlerHelper
|
|
158
|
-
|
|
159
|
-
Defines the event callbacks for a worker bus.
|
|
160
|
-
|
|
161
|
-
```typescript
|
|
162
|
-
const handler = new BaseWorkerMessageBusHandlerHelper<MyPayload>({
|
|
163
|
-
scope: 'ProcessorHandler',
|
|
164
|
-
onMessage: (opts) => {
|
|
165
|
-
console.log('Processing:', opts.message);
|
|
166
|
-
},
|
|
167
|
-
onError: (opts) => {
|
|
168
|
-
console.error('Bus error:', opts.error);
|
|
169
|
-
},
|
|
170
|
-
});
|
|
171
|
-
```
|
|
172
|
-
|
|
173
|
-
#### BaseWorkerMessageBusHandlerHelper Constructor Options
|
|
174
|
-
|
|
175
|
-
| Option | Type | Default | Description |
|
|
176
|
-
|--------|------|---------|-------------|
|
|
177
|
-
| `scope` | `string` | -- | Logger scope and identifier. Required. |
|
|
178
|
-
| `onMessage` | `(opts: { message: IConsumePayload }) => ValueOrPromise<void>` | -- | Handler called when a message is received on the port. Required. |
|
|
179
|
-
| `onClose` | `() => ValueOrPromise<void>` | No-op `() => {}` | Handler called when the port is closed. |
|
|
180
|
-
| `onError` | `(opts: { error: Error }) => ValueOrPromise<void>` | Logs error at error level | Handler called on port errors and message deserialization errors. |
|
|
181
|
-
| `onExit` | `(opts: { exitCode: number \| string }) => ValueOrPromise<void>` | Logs exit code at warn level | Handler called when the port exits. |
|
|
182
|
-
|
|
183
|
-
## Usage
|
|
184
|
-
|
|
185
|
-
### Registering Workers in the Pool
|
|
186
|
-
|
|
187
|
-
Use `WorkerPoolHelper` to track active workers. The pool prevents over-allocation by limiting registrations to the number of CPU cores.
|
|
188
|
-
|
|
189
|
-
```typescript
|
|
190
|
-
const pool = WorkerPoolHelper.getInstance();
|
|
191
|
-
|
|
192
|
-
const worker = new BaseWorkerHelper<string>({
|
|
193
|
-
identifier: 'image-resizer',
|
|
194
|
-
path: './workers/image-resizer.js',
|
|
195
|
-
options: { workerData: { quality: 80 } },
|
|
196
|
-
});
|
|
197
|
-
|
|
198
|
-
// Register the worker in the pool
|
|
199
|
-
pool.register({ key: 'image-resizer', worker });
|
|
200
|
-
|
|
201
|
-
// Check pool state
|
|
202
|
-
pool.has({ key: 'image-resizer' }); // true
|
|
203
|
-
pool.size(); // 1
|
|
204
|
-
```
|
|
205
|
-
|
|
206
|
-
> [!WARNING]
|
|
207
|
-
> When the pool reaches the CPU core limit and `ignoreMaxWarning` is `false`, further `register()` calls are silently skipped with a warning log. No error is thrown.
|
|
208
|
-
|
|
209
|
-
### Retrieving and Unregistering Workers
|
|
210
|
-
|
|
211
|
-
```typescript
|
|
212
|
-
const pool = WorkerPoolHelper.getInstance();
|
|
213
|
-
|
|
214
|
-
// Retrieve a registered worker
|
|
215
|
-
const worker = pool.get<string>({ key: 'image-resizer' });
|
|
216
|
-
if (worker) {
|
|
217
|
-
worker.worker.postMessage('start');
|
|
218
|
-
}
|
|
219
|
-
|
|
220
|
-
// Unregister terminates the worker and removes it from the pool
|
|
221
|
-
await pool.unregister({ key: 'image-resizer' });
|
|
222
73
|
```
|
|
223
74
|
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
### Managing Worker Buses from Inside a Worker Thread
|
|
75
|
+
### Bind a MessagePort bus inside a worker script
|
|
227
76
|
|
|
228
|
-
`BaseWorkerThreadHelper`
|
|
77
|
+
`BaseWorkerThreadHelper` keys buses by name so a worker can run several channels at once.
|
|
229
78
|
|
|
230
79
|
```typescript
|
|
231
80
|
// worker-script.js
|
|
@@ -237,16 +86,11 @@ import {
|
|
|
237
86
|
} from '@venizia/ignis-helpers';
|
|
238
87
|
|
|
239
88
|
const thread = new BaseWorkerThreadHelper({ scope: 'MyWorker' });
|
|
89
|
+
const { port1 } = new MessageChannel();
|
|
240
90
|
|
|
241
|
-
// Create a message channel
|
|
242
|
-
const { port1, port2 } = new MessageChannel();
|
|
243
|
-
|
|
244
|
-
// Create a handler and bus
|
|
245
91
|
const handler = new BaseWorkerMessageBusHandlerHelper<{ task: string }>({
|
|
246
92
|
scope: 'TaskHandler',
|
|
247
|
-
onMessage:
|
|
248
|
-
console.log('Task received:', opts.message.task);
|
|
249
|
-
},
|
|
93
|
+
onMessage: opts => console.log('Task received:', opts.message.task),
|
|
250
94
|
});
|
|
251
95
|
|
|
252
96
|
const bus = new BaseWorkerBusHelper<{ task: string }, { result: string }>({
|
|
@@ -255,25 +99,14 @@ const bus = new BaseWorkerBusHelper<{ task: string }, { result: string }>({
|
|
|
255
99
|
busHandler: handler,
|
|
256
100
|
});
|
|
257
101
|
|
|
258
|
-
// Bind and retrieve buses
|
|
259
102
|
thread.bindWorkerBus({ key: 'tasks', bus });
|
|
260
|
-
const taskBus = thread.getWorkerBus<{ task: string }, { result: string }>({
|
|
261
|
-
key: 'tasks',
|
|
262
|
-
});
|
|
263
103
|
```
|
|
264
104
|
|
|
265
|
-
###
|
|
105
|
+
### Send a message with a transferable object
|
|
266
106
|
|
|
267
|
-
`
|
|
107
|
+
`postMessage()` accepts an optional `transferList` for zero-copy transfer of `ArrayBuffer` and similar objects.
|
|
268
108
|
|
|
269
109
|
```typescript
|
|
270
|
-
// Send a simple message
|
|
271
|
-
bus.postMessage({
|
|
272
|
-
message: { result: 'processed' },
|
|
273
|
-
transferList: undefined,
|
|
274
|
-
});
|
|
275
|
-
|
|
276
|
-
// Send with transferable objects (zero-copy)
|
|
277
110
|
const buffer = new ArrayBuffer(1024);
|
|
278
111
|
bus.postMessage({
|
|
279
112
|
message: { result: 'binary-data' },
|
|
@@ -281,190 +114,19 @@ bus.postMessage({
|
|
|
281
114
|
});
|
|
282
115
|
```
|
|
283
116
|
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
`BaseWorkerBusHelper` supports optional `onBeforePostMessage` and `onAfterPostMessage` hooks. These are undefined by default but can be assigned after construction.
|
|
287
|
-
|
|
288
|
-
```typescript
|
|
289
|
-
bus.onBeforePostMessage = (opts) => {
|
|
290
|
-
console.log('About to send:', opts.message);
|
|
291
|
-
};
|
|
292
|
-
|
|
293
|
-
bus.onAfterPostMessage = (opts) => {
|
|
294
|
-
console.log('Sent:', opts.message);
|
|
295
|
-
};
|
|
296
|
-
```
|
|
297
|
-
|
|
298
|
-
### Unbinding a Worker Bus
|
|
299
|
-
|
|
300
|
-
Remove a bus from the worker thread and clean up its port listeners:
|
|
301
|
-
|
|
302
|
-
```typescript
|
|
303
|
-
thread.unbindWorkerBus({ key: 'tasks' });
|
|
304
|
-
```
|
|
305
|
-
|
|
306
|
-
This calls `port.removeAllListeners()` on the bus's port before deleting it from the registry.
|
|
307
|
-
|
|
308
|
-
### Subclassing AbstractWorkerHelper
|
|
309
|
-
|
|
310
|
-
For full control, extend `AbstractWorkerHelper` and implement all lifecycle methods directly:
|
|
311
|
-
|
|
312
|
-
```typescript
|
|
313
|
-
import { AbstractWorkerHelper } from '@venizia/ignis-helpers';
|
|
314
|
-
|
|
315
|
-
class CustomWorker extends AbstractWorkerHelper<MyMessage> {
|
|
316
|
-
onOnline() {
|
|
317
|
-
// Custom online handling
|
|
318
|
-
}
|
|
319
|
-
|
|
320
|
-
onExit(opts: { code: string | number }) {
|
|
321
|
-
// Custom exit handling, e.g., restart logic
|
|
322
|
-
}
|
|
323
|
-
|
|
324
|
-
onError(opts: { error: Error }) {
|
|
325
|
-
// Custom error handling
|
|
326
|
-
}
|
|
327
|
-
|
|
328
|
-
onMessage(opts: { message: MyMessage }) {
|
|
329
|
-
// Custom message processing
|
|
330
|
-
}
|
|
331
|
-
|
|
332
|
-
onMessageError(opts: { error: Error }) {
|
|
333
|
-
// Custom message error handling
|
|
334
|
-
}
|
|
335
|
-
}
|
|
336
|
-
```
|
|
337
|
-
|
|
338
|
-
## API Summary
|
|
339
|
-
|
|
340
|
-
### WorkerPoolHelper
|
|
341
|
-
|
|
342
|
-
| Method | Signature | Description |
|
|
343
|
-
|--------|-----------|-------------|
|
|
344
|
-
| `getInstance` | `static getInstance(): WorkerPoolHelper` | Returns the singleton pool instance (creates one if needed) |
|
|
345
|
-
| `register` | `register<MessageType>(opts: { key: string; worker: IWorker<MessageType> }): void` | Adds a worker to the pool. Skipped if key exists or pool is at CPU limit |
|
|
346
|
-
| `unregister` | `async unregister(opts: { key: string }): Promise<void>` | Terminates the worker and removes it from the pool |
|
|
347
|
-
| `get` | `get<MessageType>(opts: { key: string }): IWorker<MessageType> \| undefined` | Retrieves a registered worker by key |
|
|
348
|
-
| `has` | `has(opts: { key: string }): boolean` | Checks if a worker is registered under the given key |
|
|
349
|
-
| `size` | `size(): number` | Returns the number of currently registered workers |
|
|
350
|
-
|
|
351
|
-
### BaseWorkerHelper
|
|
352
|
-
|
|
353
|
-
| Method | Signature | Description |
|
|
354
|
-
|--------|-----------|-------------|
|
|
355
|
-
| `onOnline` | `onOnline(): ValueOrPromise<void>` | Called when the worker thread comes online |
|
|
356
|
-
| `onExit` | `onExit(opts: { code: string \| number }): ValueOrPromise<void>` | Called when the worker exits |
|
|
357
|
-
| `onError` | `onError(opts: { error: Error }): ValueOrPromise<void>` | Called on worker errors |
|
|
358
|
-
| `onMessage` | `onMessage(opts: { message: MessageType }): ValueOrPromise<void>` | Called when a message is received from the worker |
|
|
359
|
-
| `onMessageError` | `onMessageError(opts: { error: Error }): ValueOrPromise<void>` | Called on message deserialization errors |
|
|
360
|
-
| `binding` | `binding(): void` | Binds all event handlers to the internal `Worker` instance. Called automatically by the constructor |
|
|
361
|
-
|
|
362
|
-
### BaseWorkerThreadHelper
|
|
363
|
-
|
|
364
|
-
| Method | Signature | Description |
|
|
365
|
-
|--------|-----------|-------------|
|
|
366
|
-
| `bindWorkerBus` | `bindWorkerBus<IC, IP>(opts: { key: string; bus: IWorkerBus<IC, IP> }): void` | Registers a bus under the given key. Skipped with warning if key already exists |
|
|
367
|
-
| `unbindWorkerBus` | `unbindWorkerBus(opts: { key: string }): void` | Removes a bus and calls `port.removeAllListeners()`. Warns if key not found |
|
|
368
|
-
| `getWorkerBus` | `getWorkerBus<IC, IP>(opts: { key: string }): IWorkerBus<IC, IP>` | Returns the bus for the given key. Throws if not found |
|
|
369
|
-
|
|
370
|
-
### BaseWorkerBusHelper
|
|
371
|
-
|
|
372
|
-
| Method | Signature | Description |
|
|
373
|
-
|--------|-----------|-------------|
|
|
374
|
-
| `postMessage` | `postMessage(opts: { message: IP; transferList: readonly Transferable[] \| undefined }): ValueOrPromise<void>` | Sends a message through the port, optionally with transferable objects |
|
|
375
|
-
| `onBeforePostMessage` | `onBeforePostMessage?(opts: { message: IP }): ValueOrPromise<void>` | Optional hook called before posting a message |
|
|
376
|
-
| `onAfterPostMessage` | `onAfterPostMessage?(opts: { message: IP }): ValueOrPromise<void>` | Optional hook called after posting a message |
|
|
377
|
-
|
|
378
|
-
## Troubleshooting
|
|
379
|
-
|
|
380
|
-
### "[BaseWorker] Cannot start worker in MAIN_THREAD"
|
|
381
|
-
|
|
382
|
-
**Cause:** `BaseWorkerThreadHelper` was instantiated on the main thread. This class is designed to run only inside a worker thread (where `isMainThread` from `node:worker_threads` is `false`).
|
|
383
|
-
|
|
384
|
-
**Fix:** Only create `BaseWorkerThreadHelper` instances inside worker scripts that are spawned via `new Worker(path)`:
|
|
385
|
-
|
|
386
|
-
```typescript
|
|
387
|
-
// worker-script.js (spawned by the main thread)
|
|
388
|
-
import { BaseWorkerThreadHelper } from '@venizia/ignis-helpers';
|
|
389
|
-
|
|
390
|
-
const thread = new BaseWorkerThreadHelper({ scope: 'MyWorker' }); // OK here
|
|
391
|
-
```
|
|
392
|
-
|
|
393
|
-
### "[binding] Invalid worker instance to bind event handlers"
|
|
394
|
-
|
|
395
|
-
**Cause:** `BaseWorkerHelper.binding()` was called but the internal `Worker` instance is null or undefined. This can occur if the worker script path is invalid and the `Worker` constructor fails.
|
|
396
|
-
|
|
397
|
-
**Fix:** Ensure the `path` passed to `BaseWorkerHelper` points to a valid, existing JavaScript file:
|
|
398
|
-
|
|
399
|
-
```typescript
|
|
400
|
-
const worker = new BaseWorkerHelper({
|
|
401
|
-
identifier: 'my-worker',
|
|
402
|
-
path: './workers/my-worker.js', // Must exist and be a valid worker script
|
|
403
|
-
options: {},
|
|
404
|
-
});
|
|
405
|
-
```
|
|
406
|
-
|
|
407
|
-
### "[register] Invalid worker registry instance"
|
|
408
|
-
|
|
409
|
-
**Cause:** `WorkerPoolHelper.register()` was called but the internal registry `Map` is null or undefined. This is a defensive check that should not occur under normal usage.
|
|
410
|
-
|
|
411
|
-
**Fix:** Ensure you are using either `WorkerPoolHelper.getInstance()` or `new WorkerPoolHelper()` which both initialize the registry correctly.
|
|
412
|
-
|
|
413
|
-
### "[getWorkerBus] Not found worker bus | key: {key}"
|
|
414
|
-
|
|
415
|
-
**Cause:** `BaseWorkerThreadHelper.getWorkerBus()` was called with a key that has not been registered via `bindWorkerBus()`.
|
|
416
|
-
|
|
417
|
-
**Fix:** Verify the bus was bound before retrieving it:
|
|
418
|
-
|
|
419
|
-
```typescript
|
|
420
|
-
thread.bindWorkerBus({ key: 'my-bus', bus: myBus });
|
|
421
|
-
|
|
422
|
-
// Now safe to retrieve
|
|
423
|
-
const bus = thread.getWorkerBus({ key: 'my-bus' });
|
|
424
|
-
```
|
|
425
|
-
|
|
426
|
-
### "Failed to post message to main | Invalid parentPort!"
|
|
427
|
-
|
|
428
|
-
**Cause:** `BaseWorkerBusHelper.postMessage()` was called but the `port` property is null or undefined. This typically means the bus was constructed with an invalid `MessagePort`. Note this is logged at error level, not thrown -- the message is silently dropped.
|
|
429
|
-
|
|
430
|
-
**Fix:** Ensure a valid `MessagePort` (e.g., `parentPort` from `node:worker_threads` or a port from `new MessageChannel()`) is passed to the constructor:
|
|
431
|
-
|
|
432
|
-
```typescript
|
|
433
|
-
import { parentPort } from 'node:worker_threads';
|
|
434
|
-
|
|
435
|
-
const bus = new BaseWorkerBusHelper({
|
|
436
|
-
scope: 'MyBus',
|
|
437
|
-
port: parentPort!, // Must be a valid MessagePort
|
|
438
|
-
busHandler: handler,
|
|
439
|
-
});
|
|
440
|
-
```
|
|
441
|
-
|
|
442
|
-
### Worker pool silently skips registration
|
|
443
|
-
|
|
444
|
-
**Cause:** The pool has reached the CPU core limit and `ignoreMaxWarning` is `false` (the default for `getInstance()`).
|
|
445
|
-
|
|
446
|
-
**Fix:** Either unregister unused workers first, or create a pool with `ignoreMaxWarning: true`:
|
|
447
|
-
|
|
448
|
-
```typescript
|
|
449
|
-
// Option 1: Free up pool slots
|
|
450
|
-
await pool.unregister({ key: 'old-worker' });
|
|
451
|
-
pool.register({ key: 'new-worker', worker: newWorker });
|
|
452
|
-
|
|
453
|
-
// Option 2: Allow exceeding the limit
|
|
454
|
-
const pool = new WorkerPoolHelper({ ignoreMaxWarning: true });
|
|
455
|
-
```
|
|
117
|
+
Every constructor option, event-handler default, pre/post message hook, and error message is in the [Full reference](/extensions/helpers/worker-thread/reference).
|
|
456
118
|
|
|
457
|
-
## See
|
|
119
|
+
## See also
|
|
458
120
|
|
|
459
|
-
-
|
|
460
|
-
|
|
461
|
-
|
|
121
|
+
- [Full reference](/extensions/helpers/worker-thread/reference) - every constructor option, method signature, and troubleshooting case
|
|
122
|
+
- [Queue Helper](/extensions/helpers/queue/) - message-queue processing as an alternative to worker threads
|
|
123
|
+
- [Services](/guides/core-concepts/services) - running background workers within services
|
|
124
|
+
- [Application](/guides/core-concepts/application/) - spawning workers during application lifecycle
|
|
125
|
+
- [Node.js Worker Threads](https://nodejs.org/api/worker_threads.html) - underlying Node.js API
|
|
462
126
|
|
|
463
|
-
|
|
464
|
-
- [Helpers Index](../index) -- All available helpers
|
|
465
|
-
- [Queue Helper](../queue/) -- Message queue processing as an alternative to worker threads
|
|
127
|
+
**Files:**
|
|
466
128
|
|
|
467
|
-
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
129
|
+
- [`packages/helpers/src/modules/worker-thread/base.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/worker-thread/base.ts) - `AbstractWorkerHelper`, `BaseWorkerHelper`, `AbstractWorkerThreadHelper`, `BaseWorkerThreadHelper`
|
|
130
|
+
- [`packages/helpers/src/modules/worker-thread/worker-bus.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/worker-thread/worker-bus.ts) - `AbstractWorkerBusHelper`, `BaseWorkerBusHelper`, `AbstractWorkerMessageBusHandlerHelper`, `BaseWorkerMessageBusHandlerHelper`
|
|
131
|
+
- [`packages/helpers/src/modules/worker-thread/worker-pool.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/worker-thread/worker-pool.ts) - `WorkerPoolHelper`
|
|
132
|
+
- [`packages/helpers/src/modules/worker-thread/types.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/worker-thread/types.ts) - `IWorker`, `IWorkerThread`, `IWorkerBus`, `IWorkerMessageBusHandler`
|