spooder 5.1.11 → 6.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -38,14 +38,16 @@ Below is a full map of the available configuration options in their default stat
38
38
  "spooder": {
39
39
 
40
40
  // see CLI > Usage
41
- "run": "bun run index.ts",
41
+ "run": "",
42
42
  "run_dev": "",
43
43
 
44
44
  // see CLI > Auto Restart
45
- "auto_restart": true,
46
- "auto_restart_max": 30000,
47
- "auto_restart_attempts": 10,
48
- "auto_restart_grace": 30000,
45
+ "auto_restart": {
46
+ "enabled": false,
47
+ "backoff_max": 300000,
48
+ "backoff_grace": 30000,
49
+ "max_attempts": -1
50
+ },
49
51
 
50
52
  // see CLI > Auto Update
51
53
  "update": [
@@ -55,6 +57,7 @@ Below is a full map of the available configuration options in their default stat
55
57
 
56
58
  // see CLI > Canary
57
59
  "canary": {
60
+ "enabled": false,
58
61
  "account": "",
59
62
  "repository": "",
60
63
  "labels": [],
@@ -79,6 +82,7 @@ The `CLI` component of `spooder` is a global command-line tool for running serve
79
82
  - [CLI > Dev Mode](#cli-dev-mode)
80
83
  - [CLI > Auto Restart](#cli-auto-restart)
81
84
  - [CLI > Auto Update](#cli-auto-update)
85
+ - [CLI > Instancing](#cli-instancing)
82
86
  - [CLI > Canary](#cli-canary)
83
87
  - [CLI > Canary > Crash](#cli-canary-crash)
84
88
  - [CLI > Canary > Sanitization](#cli-canary-sanitization)
@@ -90,6 +94,7 @@ The `CLI` component of `spooder` is a global command-line tool for running serve
90
94
 
91
95
  - [API > Cheatsheet](#api-cheatsheet)
92
96
  - [API > Logging](#api-logging)
97
+ - [API > IPC](#api-ipc)
93
98
  - [API > HTTP](#api-http)
94
99
  - [API > HTTP > Directory Serving](#api-http-directory)
95
100
  - [API > HTTP > Server-Sent Events (SSE)](#api-http-sse)
@@ -122,7 +127,7 @@ cd /var/www/my-website-about-fish.net/
122
127
  spooder
123
128
  ```
124
129
 
125
- `spooder` will launch your server either by executing the `run` command provided in the configuration, or by executing `bun run index.ts` by default.
130
+ `spooder` will launch your server either by executing the `run` command provided in the configuration. If this is not defined, an error will be thrown.
126
131
 
127
132
  ```json
128
133
  {
@@ -155,7 +160,7 @@ The following differences will be observed when running in development mode:
155
160
 
156
161
  - If `run_dev` is configured, it will be used instead of the default `run` command.
157
162
  - Update commands defined in `spooder.update` will not be executed when starting a server.
158
- - If the server crashes and `auto_restart` is enabled, the server will not be restarted, and spooder will exit with the same exit code as the server.
163
+ - If the server crashes and `auto_restart` is configured, the server will not be restarted, and spooder will exit with the same exit code as the server.
159
164
  - If canary is configured, reports will not be dispatched to GitHub and instead be printed to the console; this includes crash reports.
160
165
 
161
166
  It is possible to detect in userland if a server is running in development mode by checking the `SPOODER_ENV` environment variable.
@@ -188,27 +193,35 @@ You can configure a different command to run when in development mode using the
188
193
  > [!NOTE]
189
194
  > This feature is not enabled by default.
190
195
 
191
- In the event that the server process exits with a non-zero exit code, `spooder` can automatically restart it using an exponential backoff strategy. To enable this feature set `auto_restart` to `true` in the configuration.
196
+ In the event that the server process exits, `spooder` can automatically restart it.
197
+
198
+ If the server exits with a non-zero exit code, this will be considered an **unexpected shutdown**. The process will be restarted using an [exponential backoff strategy](https://en.wikipedia.org/wiki/Exponential_backoff).
192
199
 
193
200
  ```json
194
201
  {
195
202
  "spooder": {
196
- "auto_restart": true,
197
- "auto_restart_max": 30000,
198
- "auto_restart_attempts": 10,
199
- "auto_restart_grace": 30000
203
+ "auto_restart": {
204
+ "enabled": true,
205
+
206
+ // max restarts before giving up
207
+ "max_attempts": -1, // default (unlimited)
208
+
209
+ // max delay (ms) between restart attempts
210
+ "backoff_max": 300000, // default 5 min
211
+
212
+ // grace period after which the backoff protocol
213
+ "backoff_grace": 30000 // default 30s
214
+ }
200
215
  }
201
216
  }
202
217
  ```
203
218
 
204
- ### Configuration Options
219
+ If the server exits with a `0` exit code, this will be considered an **intentional shutdown** and `spooder` will execute the update commands before restarting the server.
205
220
 
206
- - **`auto_restart`** (boolean, default: `false`): Enable or disable the auto-restart feature
207
- - **`auto_restart_max`** (number, default: `30000`): Maximum delay in milliseconds between restart attempts
208
- - **`auto_restart_attempts`** (number, default: `-1`): Maximum number of restart attempts before giving up. Set to `-1` for unlimited attempts
209
- - **`auto_restart_grace`** (number, default: `30000`): Period of time after which the backoff protocol disables if the server remains stable.
221
+ > [!TIP]
222
+ > An **intentional shutdown** can be useful for auto-updating in response to events, such as webhooks.
210
223
 
211
- If the server exits with a zero exit code (successful termination), auto-restart will not trigger.
224
+ If the server exits with `42` (SPOODER_AUTO_RESTART), the update commands will **not** be executed before starting the server. [See Auto Update for information](#cli-auto-update).
212
225
 
213
226
  <a id="cli-auto-update"></a>
214
227
  ## CLI > Auto Update
@@ -238,22 +251,106 @@ Each command should be a separate entry in the array and will be executed in seq
238
251
 
239
252
  If a command in the sequence fails, the remaining commands will not be executed, however the server will still be started. This is preferred over entering a restart loop or failing to start the server at all.
240
253
 
241
- You can utilize this to automatically update your server in response to a webhook by exiting the process.
254
+ You can combine this with [Auto Restart](#cli-auto-restart) to automatically update your server in response to a webhook by exiting the process.
242
255
 
243
256
  ```ts
244
257
  server.webhook(process.env.WEBHOOK_SECRET, '/webhook', payload => {
245
258
  setImmediate(async () => {
246
259
  await server.stop(false);
247
- process.exit();
260
+ process.exit(0);
248
261
  });
249
262
  return HTTP_STATUS_CODE.OK_200;
250
263
  });
251
264
  ```
252
265
 
266
+ ### Multi-Instance Auto Update
267
+
268
+ See [Instancing](#cli-instancing) for instructions on how to use [Auto Update](#cli-auto-update) with multiple instances.
269
+
253
270
  ### Skip Updates
254
271
 
255
272
  In addition to being skipped in [dev mode](#cli-dev-mode), updates can also be skipped in production mode by passing the `--no-update` flag.
256
273
 
274
+ <a id="cli-instancing"></a>
275
+ ## CLI > Instancing
276
+
277
+ > [!NOTE]
278
+ > This feature is not enabled by default.
279
+
280
+ By default, `spooder` will start and manage a single process as defined by the `run` and `run_dev` configuration properties. In some scenarios, you may want multiple processes for a single codebase, such as variant sub-domains.
281
+
282
+ This can be configured in `spooder` using the `instances` array, with each entry defining a unique instance.
283
+
284
+ ```json
285
+ "spooder": {
286
+ "instances": [
287
+ {
288
+ "id": "dev01",
289
+ "run": "bun run --env-file=.env.a index.ts",
290
+ "run_dev": "bun run --env-file=.env.a.dev index.ts --inspect"
291
+ },
292
+ {
293
+ "id": "dev02",
294
+ "run": "bun run --env-file=.env.b index.ts",
295
+ "run_dev": "bun run --env-file=.env.b.dev index.ts --inspect"
296
+ }
297
+ ]
298
+ }
299
+ ```
300
+
301
+ Instances will be managed individually in the same manner that a single process would be, including auto-restarting and other functionality.
302
+
303
+ ### Instance Stagger
304
+
305
+ By default, instances are all launched instantly. This behavior can be configured with the `instance_stagger_interval` configuration property, which defines an interval between instance launches in milliseconds.
306
+
307
+ This interval effects both server start-up, auto-restarting and crash recovery. No two instances will be launched within that interval regardless of the reason.
308
+
309
+ ### Canary
310
+
311
+ The [canary](#cli-canary) feature functions the same for multiple instances as it would for a single instance with the caveat that the `instance` object as defined in the configuration is included in the crash report for diagnostics.
312
+
313
+ This allows you to define custom properties on the instance which will be included as part of the crash report.
314
+
315
+ ```json
316
+ {
317
+ "id": "dev01",
318
+ "run": "bun run --env-file=.env.a index.ts",
319
+ "sub_domain": "dev01.spooder.dev" // custom, for diagnostics
320
+ }
321
+ ```
322
+
323
+ > ![IMPORTANT]
324
+ > You should not include sensitive or confidential credentials in your instance configuration for this reason. This should always be handled using environment variables or credential storage.
325
+
326
+ ### Multi-instance Auto Restart
327
+
328
+ Combining [Auto Restart](#cli-auto-restart) and [Auto Update](#cli-auto-update), when a server process exits with a zero exit code, the update commands will be run as the server restarts. This is suitable for a single-instance setup.
329
+
330
+ In the event of multiple instances, this does not work. One server instance would receive the webhook and exit, resulting in the update commands being run and that instance being restarted, leaving the other instances still running.
331
+
332
+ A solution might be to send the web-hook to every instance, but now each instance is going to restart individually, running the update commands unnecessarily and, if at the same time, causing conflicts. In addition, the concept of multiple instances in spooder is that they operate from a single codebase, which makes sending multiple webhooks a challenge - so don't do this.
333
+
334
+ The solution is to the use the [IPC](#api-ipc) to instruct the host process to handle this.
335
+
336
+ ```ts
337
+ server.webhook(process.env.WEBHOOK_SECRET, '/webhook', payload => {
338
+ setImmediate(async () => {
339
+ ipc_send(IPC_TARGET.SPOODER, IPC_OP.CMSG_TRIGGER_UPDATE);
340
+ });
341
+ return HTTP_STATUS_CODE.OK_200;
342
+ });
343
+
344
+ ipc_register(IPC_OP.SMSG_UPDATE_READY, async () => {
345
+ await server.stop(false);
346
+ process.exit(EXIT_CODE.SPOODER_AUTO_UPDATE);
347
+ });
348
+ ```
349
+
350
+ In this scenario, we instruct the host process from one instance receiving the webhook to apply the updates. Once the update commands have been run, all instances are send the `SMSG_UPDATE_READY` event, indicating they can restart.
351
+
352
+ Exiting with the `SPOODER_AUTO_UPDATE` exit code instructs spooder that we're exiting as part of this process, and prevents auto-update from running on restart.
353
+
257
354
  <a id="cli-canary"></a>
258
355
  ## CLI > Canary
259
356
 
@@ -291,6 +388,7 @@ Each server that intends to use the canary feature will need to have the private
291
388
  ```json
292
389
  "spooder": {
293
390
  "canary": {
391
+ "enabled": true,
294
392
  "account": "<GITHUB_ACCOUNT_NAME>",
295
393
  "repository": "<GITHUB_REPOSITORY>",
296
394
  "labels": ["some-label"]
@@ -537,12 +635,30 @@ caution(err_message_or_obj: string | object, ...err: object[]): Promise<void>;
537
635
  panic(err_message_or_obj: string | object, ...err: object[]): Promise<void>;
538
636
  safe(fn: Callable): Promise<void>;
539
637
 
540
- // worker
541
- worker_event_pipe(worker: Worker, options?: WorkerEventPipeOptions): WorkerEventPipe;
542
- pipe.send(id: string, data?: object): void;
543
- pipe.on(event: string, callback: (data: object) => void | Promise<void>): void;
544
- pipe.once(event: string, callback: (data: object) => void | Promise<void>): void;
545
- pipe.off(event: string): void;
638
+ // worker (main thread)
639
+ worker_pool(options: WorkerPoolOptions): Promise<WorkerPool>;
640
+ pool.id: string;
641
+ pool.send: (peer: string, id: string, data?: WorkerMessageData) => void;
642
+ pool.broadcast: (id: string, data?: WorkerMessageData) => void;
643
+ pool.on: (event: string, callback: (message: WorkerMessage) => Promise<void> | void) => void;
644
+ pool.once: (event: string, callback: (message: WorkerMessage) => Promise<void> | void) => void;
645
+ pool.off: (event: string) => void;
646
+
647
+ type WorkerPoolOptions = {
648
+ id?: string;
649
+ worker: string | string[];
650
+ size?: number;
651
+ auto_restart?: boolean | AutoRestartConfig;
652
+ };
653
+
654
+ type AutoRestartConfig = {
655
+ backoff_max?: number; // default: 5 * 60 * 1000 (5 min)
656
+ backoff_grace?: number; // default: 30000 (30 seconds)
657
+ max_attempts?: number; // default: 5, -1 for unlimited
658
+ };
659
+
660
+ // worker (worker thread)
661
+ worker_connect(peer_id?: string): WorkerPool;
546
662
 
547
663
  // templates
548
664
  Replacements = Record<string, string | Array<string> | object | object[]> | ReplacerFn | AsyncReplaceFn;
@@ -604,10 +720,19 @@ cache.request(req: Request, cache_key: string, content_generator: () => string |
604
720
 
605
721
  // utilities
606
722
  filesize(bytes: number): string;
723
+ BiMap: class BiMap<K, V>;
724
+
725
+ // ipc
726
+ ipc_register(op: number, callback: IPC_Callback);
727
+ ipc_send(target: string, op: number, data?: object);
607
728
 
608
729
  // constants
609
730
  HTTP_STATUS_TEXT: Record<number, string>;
610
731
  HTTP_STATUS_CODE: { OK_200: 200, NotFound_404: 404, ... };
732
+ EXIT_CODE: Record<string, number>;
733
+ EXIT_CODE_NAMES: Record<number, string>;
734
+ IPC_TARGET: Record<string, string>;
735
+ IPC_OP: Record<string, number>;
611
736
  ```
612
737
 
613
738
  <a id="api-logging"></a>
@@ -667,6 +792,60 @@ log(`Fruit must be one of ${fruit.map(e => `{${e}}`).join(', ')}`);
667
792
  log(`Fruit must be one of ${log_list(fruit)}`);
668
793
  ```
669
794
 
795
+ <a id="api-ipc"></a>
796
+ ## API > IPC
797
+
798
+ `spooder` provides a way to send/receive messages between different instances via IPC. See [CLI > Instancing](#cli-instancing) for documentation on instances.
799
+
800
+ ```ts
801
+ // listen for a message
802
+ ipc_register(0x1, msg => {
803
+ // msg.peer, msg.op, msg.data
804
+ console.log(msg.data.foo); // 42
805
+ });
806
+
807
+ // send a message to dev02
808
+ ipc_send('dev02', 0x1, { foo: 42 });
809
+
810
+ // send a message to all other instances
811
+ ipc_send(IPC_TARGET.BROADCAST, 0x1, { foo: 42 });
812
+ ```
813
+
814
+ This can also be used to communicate with the host process for certain functionality, such as [auto-restarting](#cli-auto-restart).
815
+
816
+ #### OpCodes
817
+
818
+ When sending/receiving IPC messages, the message will include an opcode. When communicating with the host process, that will be one of the following:
819
+
820
+ ```ts
821
+ IPC_OP.CMSG_TRIGGER_UPDATE = -1;
822
+ IPC_OP.SMSG_UPDATE_READY = -2;
823
+ IPC_OP.CMSG_REGISTER_LISTENER = -3; // used internally by ipc_register
824
+ ```
825
+
826
+ When sending/receiving your own messages, you can define and use your own ID schema. To prevent conflict with internal opcodes, always use positive values; `spooder` internal opcodes will always be negative.
827
+
828
+ ### `ipc_register(op: number, callback: IPC_Callback)`
829
+
830
+ Register a listener for IPC events. The callback will receive an object with this structure:
831
+
832
+ ```ts
833
+ type IPC_Message = {
834
+ op: number; // opcode received
835
+ peer: string; // sender
836
+ data?: object // payload data (optional)
837
+ };
838
+ ```
839
+
840
+ ### `ipc_send(peer: string, op: number, data?: object)`
841
+
842
+ Send an IPC event. The target can either be the ID of another instance (such as the `peer` ID from an `IPC_Message`) or one of the following constants.
843
+
844
+ ```ts
845
+ IPC_TARGET.SPOODER; // communicate with the host
846
+ IPC_TARGET.BROADCAST; // broadcast to all other instances
847
+ ```
848
+
670
849
  <a id="api-http"></a>
671
850
  ## API > HTTP
672
851
 
@@ -1688,78 +1867,251 @@ await safe(() => {
1688
1867
  <a id="api-workers"></a>
1689
1868
  ## API > Workers
1690
1869
 
1691
- ### 🔧 `worker_event_pipe(worker: Worker, options?: WorkerEventPipeOptions): WorkerEventPipe`
1870
+ ### 🔧 `worker_pool(options: WorkerPoolOptions): Promise<WorkerPool>` (Main Thread)
1692
1871
 
1693
- Create an event-based communication pipe between host and worker processes. This function works both inside and outside of workers and provides a simple event system on top of the native `postMessage` API.
1872
+ Create a worker pool with an event-based communication system between the main thread and one or more workers. This provides a networked event system on top of the native `postMessage` API.
1694
1873
 
1695
1874
  ```ts
1696
- // main thread
1697
- const worker = new Worker('./some_file.ts');
1698
- const pipe = worker_event_pipe(worker);
1875
+ // with a single worker (id defaults to 'main')
1876
+ const pool = await worker_pool({
1877
+ worker: './worker.ts'
1878
+ });
1879
+
1880
+ // with multiple workers and custom ID
1881
+ const pool = await worker_pool({
1882
+ id: 'main',
1883
+ worker: ['./worker_a.ts', './worker_b.ts']
1884
+ });
1885
+
1886
+ // spawn multiple instances of the same worker
1887
+ const pool = await worker_pool({
1888
+ worker: './worker.ts',
1889
+ size: 5 // spawns 5 instances
1890
+ });
1891
+
1892
+ // with custom response timeout
1893
+ const pool = await worker_pool({
1894
+ worker: './worker.ts',
1895
+ response_timeout: 10000 // 10 seconds (default: 5000ms, use -1 to disable)
1896
+ });
1897
+
1898
+ // with auto-restart enabled (boolean)
1899
+ const pool = await worker_pool({
1900
+ worker: './worker.ts',
1901
+ auto_restart: true // uses default settings
1902
+ });
1903
+
1904
+ // with custom auto-restart configuration
1905
+ const pool = await worker_pool({
1906
+ worker: './worker.ts',
1907
+ auto_restart: {
1908
+ backoff_max: 5 * 60 * 1000, // 5 min (default)
1909
+ backoff_grace: 30000, // 30 seconds (default)
1910
+ max_attempts: 5 // -1 for unlimited (default: 5)
1911
+ }
1912
+ });
1913
+ ```
1914
+
1915
+ ### 🔧 `worker_connect(peer_id?: string, response_timeout?: number): WorkerPool` (Worker Thread)
1699
1916
 
1700
- pipe.on('bar', data => console.log('Received from worker:', data));
1701
- pipe.send('foo', { x: 42 });
1917
+ Connect a worker thread to the worker pool. This should be called from within a worker thread to establish communication with the main thread and other workers.
1702
1918
 
1919
+ **Parameters:**
1920
+ - `peer_id` - Optional worker ID (defaults to `worker-UUID`)
1921
+ - `response_timeout` - Optional timeout in milliseconds for request-response patterns (default: 5000ms, use -1 to disable)
1922
+
1923
+ ```ts
1703
1924
  // worker thread
1704
- import { worker_event_pipe } from 'spooder';
1925
+ const pool = worker_connect('my_worker'); // defaults to worker-UUID, 5000ms timeout
1926
+ pool.on('test', msg => {
1927
+ console.log(`Received ${msg.data.foo} from ${msg.peer}`);
1928
+ });
1705
1929
 
1706
- const pipe = worker_event_pipe(globalThis as unknown as Worker);
1930
+ // with custom timeout
1931
+ const pool = worker_connect('my_worker', 10000); // 10 second timeout
1932
+ const pool = worker_connect('my_worker', -1); // no timeout
1933
+ ```
1934
+
1935
+ ### Basic Usage
1707
1936
 
1708
- pipe.on('foo', data => {
1709
- console.log('Received from main:', data); // { x: 42 }
1710
- pipe.send('bar', { response: 'success' });
1937
+ ```ts
1938
+ // main thread
1939
+ const pool = await worker_pool({
1940
+ id: 'main',
1941
+ worker: './worker.ts'
1942
+ });
1943
+
1944
+ pool.send('my_worker', 'test', { foo: 42 });
1945
+
1946
+ // worker thread (worker.ts)
1947
+ const pool = worker_connect('my_worker');
1948
+ pool.on('test', msg => {
1949
+ console.log(`Received ${msg.data.foo} from ${msg.peer}`);
1950
+ // > Received 42 from main
1711
1951
  });
1712
1952
  ```
1713
1953
 
1714
- ### WorkerEventPipeOptions
1954
+ ### Cross-Worker Communication
1715
1955
 
1716
- The second parameter of `worker_event_pipe` accepts an object of options.
1956
+ ```ts
1957
+ // main thread
1958
+ const pool = await worker_pool({
1959
+ id: 'main',
1960
+ worker: ['./worker_a.ts', './worker_b.ts']
1961
+ });
1717
1962
 
1718
- Currently the only available option is `use_canary_reporting`. If enabled, the event pipe will call `caution()` when it encounters errors such as malformed payloads.
1963
+ pool.send('worker_a', 'test', { foo: 42 }); // send to just worker_a
1964
+ pool.broadcast('test', { foo: 50 } ); // send to all workers
1719
1965
 
1720
- ### 🔧 `pipe.send(id: string, data?: object): void`
1966
+ // worker_a.ts
1967
+ const pool = worker_connect('worker_a');
1968
+ // send from worker_a to worker_b
1969
+ pool.send('worker_b', 'test', { foo: 500 });
1970
+ ```
1721
1971
 
1722
- Send a message to the other side of the worker pipe with the specified event ID and optional data payload.
1972
+ ### 🔧 `pool.send(peer: string, id: string, data?: Record<string, any>, expect_response?: boolean): void | Promise<WorkerMessage>`
1973
+
1974
+ Send a message to a specific peer in the pool, which can be the main host or another worker.
1975
+
1976
+ When `expect_response` is `false` (default), the function returns `void`. When `true`, it returns a `Promise<WorkerMessage>` that resolves when the peer responds using `pool.respond()`.
1723
1977
 
1724
1978
  ```ts
1725
- pipe.send('user_update', { user_id: 123, name: 'John' });
1726
- pipe.send('simple_event'); // data defaults to {}
1979
+ // Fire-and-forget (default behavior)
1980
+ pool.send('main', 'user_update', { user_id: 123, name: 'John' });
1981
+ pool.send('worker_b', 'simple_event');
1982
+
1983
+ // Request-response pattern
1984
+ const response = await pool.send('worker_b', 'calculate', { value: 42 }, true);
1985
+ console.log('Result:', response.data);
1727
1986
  ```
1728
1987
 
1729
- ### 🔧 `pipe.on(event: string, callback: (data: object) => void | Promise<void>): void`
1988
+ > [!NOTE]
1989
+ > When using `expect_response: true`, the promise will reject with a timeout error if no response is received within the configured timeout (default: 5000ms). You can configure this timeout in `worker_pool()` options or `worker_connect()` parameters, or disable it entirely by setting it to `-1`.
1730
1990
 
1731
- Register an event handler for messages with the specified event ID. The callback can be synchronous or asynchronous.
1991
+ ### 🔧 `pool.broadcast(id: string, data?: Record<string, any>): void`
1992
+
1993
+ Broadcast a message to all peers in the pool.
1732
1994
 
1733
1995
  ```ts
1734
- pipe.on('process_data', async (data) => {
1735
- const result = await processData(data);
1736
- pipe.send('data_processed', { result });
1737
- });
1996
+ pool.broadcast('test_event', { foo: 42 });
1997
+ ```
1998
+
1999
+ ### 🔧 `pool.on(event: string, callback: (data: Record<string, any>) => void | Promise<void>): void`
2000
+
2001
+ Register an event handler for messages with the specified event ID. The callback can be synchronous or asynchronous.
1738
2002
 
1739
- pipe.on('log_message', (data) => {
1740
- console.log(data.message);
2003
+ ```ts
2004
+ pool.on('process_data', async msg => {
2005
+ // msg.peer
2006
+ // msg.id
2007
+ // msg.data
1741
2008
  });
1742
2009
  ```
1743
2010
 
1744
2011
  > [!NOTE]
1745
2012
  > There can only be one event handler for a specific event ID. Registering a new handler for an existing event ID will overwrite the previous handler.
1746
2013
 
1747
- ### 🔧 `pipe.once(event: string, callback: (data: object) => void | Promise<void>): void`
2014
+ ### 🔧 `pool.once(event: string, callback: (data: Record<string, any>) => void | Promise<void>): void`
1748
2015
 
1749
- Register an event handler for messages with the specified event ID. This is the same as `pipe.on`, except the handler is automatically removed once it is fired.
2016
+ Register an event handler for messages with the specified event ID. This is the same as `pool.on`, except the handler is automatically removed once it is fired.
1750
2017
 
1751
2018
  ```ts
1752
- pipe.once('one_time_event', async (data) => {
2019
+ pool.once('one_time_event', async msg => {
1753
2020
  // this will only fire once
1754
2021
  });
1755
2022
  ```
1756
2023
 
1757
- ### 🔧 `pipe.off(event: string): void`
2024
+ ### 🔧 `pool.off(event: string): void`
1758
2025
 
1759
2026
  Unregister an event handler for events with the specified event ID.
1760
2027
 
1761
2028
  ```ts
1762
- pipe.off('event_name');
2029
+ pool.off('event_name');
2030
+ ```
2031
+
2032
+ ### 🔧 `pool.respond(message: WorkerMessage, data?: Record<string, any>): void`
2033
+
2034
+ Respond to a message that was sent with `expect_response: true`. This allows implementing request-response patterns between peers.
2035
+
2036
+ ```ts
2037
+ pool.on('calculate', msg => {
2038
+ const result = msg.data.value * 2;
2039
+ pool.respond(msg, { result });
2040
+ });
2041
+
2042
+ const response = await pool.send('worker_a', 'calculate', { value: 42 }, true);
2043
+ console.log(response.data.result); // 84
2044
+ ```
2045
+
2046
+ **Message Structure:**
2047
+ - `message.id` - The event ID
2048
+ - `message.peer` - The sender's peer ID
2049
+ - `message.data` - The message payload
2050
+ - `message.uuid` - Unique identifier for this message
2051
+ - `message.response_to` - UUID of the message being responded to (only present in responses)
2052
+
2053
+ ### Request-Response Example
2054
+
2055
+ ```ts
2056
+ // main.ts
2057
+ const pool = await worker_pool({
2058
+ id: 'main',
2059
+ worker: './worker.ts'
2060
+ });
2061
+
2062
+ const response = await pool.send('worker_a', 'MSG_REQUEST', { value: 42 }, true);
2063
+ console.log(`Got response ${response.data.value} from ${response.peer}`);
2064
+
2065
+ // worker.ts
2066
+ const pool = worker_connect('worker_a');
2067
+
2068
+ pool.on('MSG_REQUEST', msg => {
2069
+ console.log(`Received request with value: ${msg.data.value}`);
2070
+ pool.respond(msg, { value: msg.data.value * 2 });
2071
+ });
2072
+ ```
2073
+
2074
+ ### Auto-Restart
2075
+
2076
+ The `worker_pool` function supports automatic worker restart when workers crash or close unexpectedly. This feature includes an exponential backoff protocol to prevent restart loops.
2077
+
2078
+ #### Configuration:
2079
+ - `auto_restart`: `boolean | AutoRestartConfig` - Enable auto-restart (optional)
2080
+ - If `true`, uses default settings
2081
+ - If an object, allows customization of restart behavior
2082
+
2083
+ #### AutoRestartConfig
2084
+ - `backoff_max`: `number` - Maximum delay between restart attempts in milliseconds (default: `5 * 60 * 1000` = 5 minutes)
2085
+ - `backoff_grace`: `number` - Time in milliseconds a worker must run successfully before restart attempts are reset (default: `30000` = 30 seconds)
2086
+ - `max_attempts`: `number` - Maximum number of restart attempts before giving up (default: `5`, use `-1` for unlimited)
2087
+
2088
+ #### Backoff Protocol
2089
+ 1. Initial restart delay starts at 100ms
2090
+ 2. Each subsequent restart doubles the delay
2091
+ 3. Delay is capped at `backoff_max`
2092
+ 4. If a worker runs successfully for `backoff_grace` milliseconds, the delay and attempt counter reset
2093
+ 5. After `max_attempts` failures, auto-restart stops for that worker
2094
+
2095
+ **Example:**
2096
+ ```ts
2097
+ const pool = await worker_pool({
2098
+ worker: './worker.ts',
2099
+ auto_restart: {
2100
+ backoff_max: 5 * 60 * 1000, // cap at 5 minutes
2101
+ backoff_grace: 30000, // reset after 30 seconds of successful operation
2102
+ max_attempts: 5 // give up after 5 failed attempts
2103
+ }
2104
+ });
2105
+ ```
2106
+
2107
+ #### Graceful Exit
2108
+
2109
+ Workers can exit gracefully without triggering an auto-restart by using the `WORKER_EXIT_NO_RESTART` exit code (42):
2110
+
2111
+ ```ts
2112
+ // worker thread
2113
+ import { WORKER_EXIT_NO_RESTART } from 'spooder';
2114
+ process.exit(WORKER_EXIT_NO_RESTART); // exits without auto-restart
1763
2115
  ```
1764
2116
 
1765
2117
  > [!IMPORTANT]
@@ -1784,6 +2136,10 @@ server.route('/', cache.file('./index.html'));
1784
2136
 
1785
2137
  // Use with server routes for dynamic content
1786
2138
  server.route('/dynamic', async (req) => cache.request(req, 'dynamic-page', () => 'Dynamic Content'));
2139
+
2140
+ // Disable caching (useful for development mode)
2141
+ const devCache = cache_http({ enabled: process.env.SPOODER_ENV !== 'dev' });
2142
+ server.route('/no-cache', devCache.file('./index.html')); // Always reads from disk
1787
2143
  ```
1788
2144
 
1789
2145
  The `cache_http()` function returns an object with two methods:
@@ -1827,7 +2183,8 @@ server.route('/api/stats', async (req) => {
1827
2183
  | `max_size` | `number` | `5242880` (5 MB) | Maximum total size of all cached files in bytes |
1828
2184
  | `use_etags` | `boolean` | `true` | Generate and use ETag headers for cache validation |
1829
2185
  | `headers` | `Record<string, string>` | `{}` | Additional HTTP headers to include in responses |
1830
- | `use_canary_reporting` | `boolean` | `false` | Reports faults to canary (see below).
2186
+ | `use_canary_reporting` | `boolean` | `false` | Reports faults to canary (see below) |
2187
+ | `enabled` | `boolean` | `true` | When false, content is generated but not stored
1831
2188
 
1832
2189
  #### Canary Reporting
1833
2190
 
@@ -2705,6 +3062,37 @@ filesize(1073741824); // > "1 gb"
2705
3062
  filesize(1099511627776); // > "1 tb"
2706
3063
  ```
2707
3064
 
3065
+ ### 🔧 ``BiMap<K, V>``
3066
+
3067
+ A bidirectional map that maintains a two-way relationship between keys and values, allowing efficient lookups in both directions.
3068
+
3069
+ ```ts
3070
+ const users = new BiMap<number, string>();
3071
+
3072
+ // Set key-value pairs
3073
+ users.set(1, "Alice");
3074
+ users.set(2, "Bob");
3075
+ users.set(3, "Charlie");
3076
+
3077
+ // Lookup by key
3078
+ users.getByKey(1); // > "Alice"
3079
+
3080
+ // Lookup by value
3081
+ users.getByValue("Bob"); // > 2
3082
+
3083
+ // Check existence
3084
+ users.hasKey(1); // > true
3085
+ users.hasValue("Charlie"); // > true
3086
+
3087
+ // Delete by key or value
3088
+ users.deleteByKey(1); // > true
3089
+ users.deleteByValue("Bob"); // > true
3090
+
3091
+ // Other operations
3092
+ users.size; // > 1
3093
+ users.clear();
3094
+ ```
3095
+
2708
3096
  ## Legal
2709
3097
  This software is provided as-is with no warranty or guarantee. The authors of this project are not responsible or liable for any problems caused by using this software or any part thereof. Use of this software does not entitle you to any support or assistance from the authors of this project.
2710
3098