observed-js 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License Copyright (c) 2026 Alexander Mason
2
+
3
+ Permission is hereby granted,
4
+ free of charge, to any person obtaining a copy of this software and associated
5
+ documentation files (the "Software"), to deal in the Software without
6
+ restriction, including without limitation the rights to use, copy, modify, merge,
7
+ publish, distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to the
9
+ following conditions:
10
+
11
+ The above copyright notice and this permission notice
12
+ (including the next paragraph) shall be included in all copies or substantial
13
+ portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
16
+ ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
17
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
18
+ EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
19
+ OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
20
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,589 @@
1
+ # observed-js
2
+
3
+ **Enterprise-ready action scheduling with concurrency, rate limiting, and wide event observability**
4
+
5
+ `observed-js` is a TypeScript library for controlled execution of async operations with first-class support for structured logging and observability. Built on a fluent API, it enables precise control over concurrency, rate limiting, retries, and rich event captureβ€”all with automatic type inference.
6
+
7
+ ## Features
8
+
9
+ - 🎯 **Fluent Builder API** - Chain configuration methods with full type inference
10
+ - πŸ”„ **Concurrency Control** - Limit parallel execution to manage resource usage
11
+ - ⏱️ **Rate Limiting** - Sliding window algorithm for precise throttling
12
+ - πŸ” **Automatic Retry** - Configurable backoff strategies (linear/exponential) with jitter
13
+ - πŸ“Š **Wide Events** - Capture rich, structured context for every invocation
14
+ - πŸ“¦ **Batch Operations** - Process multiple items with `invokeAll()` or stream results with `invokeStream()`
15
+ - πŸ’ͺ **Type Safe** - Full TypeScript support with automatic type inference from handlers
16
+ - πŸͺΆ **Zero Dependencies** - Lightweight and focused
17
+
18
+ ## Installation
19
+
20
+ ```bash
21
+ npm install observed-js
22
+ ```
23
+
24
+ ## Quick Start
25
+
26
+ ```typescript
27
+ import { createAction } from "observed-js";
28
+
29
+ // Create an action with automatic type inference
30
+ const fetchUser = createAction(async (userId: string) => {
31
+ const response = await fetch(`/api/users/${userId}`);
32
+ return response.json();
33
+ })
34
+ .setConcurrency(5) // Max 5 parallel requests
35
+ .setRateLimit(10) // Max 10 requests/second
36
+ .setRetry({ // Retry on failure
37
+ maxRetries: 3,
38
+ backoff: 'exponential',
39
+ baseDelay: 100
40
+ });
41
+
42
+ // Invoke the action
43
+ const { actionId, data } = fetchUser.invoke("user-123");
44
+ const user = await data; // Type-safe result
45
+ ```
46
+
47
+ ## Core Concepts
48
+
49
+ ### Actions
50
+
51
+ Actions are created using `createAction()` with an async handler function. The library automatically infers input and output types:
52
+
53
+ ```typescript
54
+ const processImage = createAction(async (imageUrl: string, width: number) => {
55
+ // Process the image...
56
+ return { url: processedUrl, dimensions: { width, height } };
57
+ });
58
+
59
+ // TypeScript knows the exact input and output types
60
+ const result = await processImage.invoke("image.jpg", 800).data;
61
+ // result is typed as { url: string, dimensions: { width: number, height: number } }
62
+ ```
63
+
64
+ ### Concurrency Control
65
+
66
+ Limit how many operations run in parallel:
67
+
68
+ ```typescript
69
+ const action = createAction(handler)
70
+ .setConcurrency(3); // Max 3 concurrent executions
71
+ ```
72
+
73
+ Default is **1** (sequential execution) for safe-by-default behavior.
74
+
75
+ ### Rate Limiting
76
+
77
+ Control execution rate using a sliding window algorithm:
78
+
79
+ ```typescript
80
+ const action = createAction(handler)
81
+ .setRateLimit(100); // Max 100 executions per second
82
+ ```
83
+
84
+ Tasks exceeding the limit are automatically queued and executed when the window allows.
85
+
86
+ ### Retry with Backoff
87
+
88
+ Automatically retry failed operations with configurable strategies:
89
+
90
+ ```typescript
91
+ const action = createAction(handler)
92
+ .setRetry({
93
+ maxRetries: 3, // Number of retry attempts
94
+ backoff: 'exponential', // 'linear' or 'exponential'
95
+ baseDelay: 100, // Base delay in ms
96
+ maxDelay: 10000, // Cap for exponential growth
97
+ jitter: true, // Add randomness to prevent thundering herd
98
+ shouldRetry: (error) => { // Optional: selective retry
99
+ return error instanceof NetworkError;
100
+ }
101
+ });
102
+ ```
103
+
104
+ **Backoff strategies:**
105
+ - **Linear**: `baseDelay * attemptNumber`
106
+ - **Exponential**: `baseDelay * 2^(attemptNumber - 1)` (capped at `maxDelay`)
107
+ - **Jitter**: Multiplies delay by random factor (0.5 to 1.0) when enabled
108
+
109
+ ### Wide Events
110
+
111
+ Capture rich, structured context for observability:
112
+
113
+ ```typescript
114
+ import { createAction, withContext } from "observed-js";
115
+
116
+ const dbQuery = createAction(
117
+ withContext(async (ctx, userId: string) => {
118
+ ctx.attach("userId", userId);
119
+ ctx.attach("operation", "SELECT");
120
+
121
+ const result = await database.query(`SELECT * FROM users WHERE id = ?`, [userId]);
122
+
123
+ ctx.attach("rowsReturned", result.length);
124
+ ctx.attach("queryDurationMs", result.duration);
125
+
126
+ return result;
127
+ })
128
+ ).onEvent((event) => {
129
+ logger.info({
130
+ actionId: event.actionId,
131
+ input: event.input,
132
+ output: event.output,
133
+ duration: event.duration,
134
+ attachments: event.attachments, // All attached context
135
+ error: event.error
136
+ });
137
+ });
138
+ ```
139
+
140
+ **Context features:**
141
+ - `.attach(key, value)` for single key-value pairs
142
+ - `.attach(object)` for bulk attachment
143
+ - Objects are deep-merged when attaching to the same key
144
+ - Primitives overwrite previous values
145
+
146
+ ### Batch Operations
147
+
148
+ Process multiple items with two different patterns:
149
+
150
+ #### `invokeAll()` - Promise.all style
151
+
152
+ Returns all results in input order (using Promise.allSettled internally):
153
+
154
+ ```typescript
155
+ const action = createAction(handler);
156
+
157
+ const results = await action.invokeAll([
158
+ ["input1"],
159
+ ["input2"],
160
+ ["input3"]
161
+ ]);
162
+
163
+ // Each result is a discriminated union
164
+ results.forEach((result, index) => {
165
+ if (result.error) {
166
+ console.error(`Item ${index} failed:`, result.error);
167
+ } else {
168
+ console.log(`Item ${index} succeeded:`, result.data);
169
+ }
170
+ });
171
+ ```
172
+
173
+ #### `invokeStream()` - AsyncGenerator
174
+
175
+ Yields results as they complete (out-of-order):
176
+
177
+ ```typescript
178
+ const action = createAction(handler).setConcurrency(5);
179
+
180
+ for await (const result of action.invokeStream([
181
+ ["input1"],
182
+ ["input2"],
183
+ ["input3"]
184
+ ])) {
185
+ console.log(`Completed item ${result.index}`);
186
+ if (result.error) {
187
+ console.error("Failed:", result.error);
188
+ } else {
189
+ console.log("Success:", result.data);
190
+ }
191
+ }
192
+ ```
193
+
194
+ Both methods:
195
+ - Work with concurrency and rate limiting
196
+ - Individual failures don't fail the entire batch
197
+ - Return discriminated unions with `data` or `error`
198
+
199
+ ## API Reference
200
+
201
+ ### `createAction(handler)`
202
+
203
+ Creates a new action with automatic type inference.
204
+
205
+ **Parameters:**
206
+ - `handler: (...args: I) => Promise<O> | O` - Async function to execute
207
+
208
+ **Returns:** `ActionBuilder<I, O>`
209
+
210
+ ```typescript
211
+ const action = createAction(async (input: string) => {
212
+ return { result: input.toUpperCase() };
213
+ });
214
+ ```
215
+
216
+ ### `withContext(handler)`
217
+
218
+ Wraps a handler to receive `InvocationContext` as the first parameter.
219
+
220
+ **Parameters:**
221
+ - `handler: (ctx: InvocationContext, ...args: I) => Promise<O> | O`
222
+
223
+ **Returns:** Handler function compatible with `createAction()`
224
+
225
+ ```typescript
226
+ const action = createAction(
227
+ withContext(async (ctx, input: string) => {
228
+ ctx.attach("inputLength", input.length);
229
+ return process(input);
230
+ })
231
+ );
232
+ ```
233
+
234
+ ### `ActionBuilder` Methods
235
+
236
+ #### `.setConcurrency(limit: number)`
237
+
238
+ Set maximum concurrent executions (default: 1).
239
+
240
+ ```typescript
241
+ action.setConcurrency(5);
242
+ ```
243
+
244
+ #### `.setRateLimit(limit: number)`
245
+
246
+ Set maximum executions per second (default: Infinity).
247
+
248
+ ```typescript
249
+ action.setRateLimit(100);
250
+ ```
251
+
252
+ #### `.setRetry(options: RetryOptions)`
253
+
254
+ Configure automatic retry behavior.
255
+
256
+ ```typescript
257
+ action.setRetry({
258
+ maxRetries: 3,
259
+ backoff: 'exponential',
260
+ baseDelay: 100,
261
+ maxDelay: 10000,
262
+ jitter: true,
263
+ shouldRetry: (error) => error instanceof TransientError
264
+ });
265
+ ```
266
+
267
+ #### `.onEvent(callback: EventCallback)`
268
+
269
+ Register a callback to receive wide events.
270
+
271
+ ```typescript
272
+ action.onEvent((event) => {
273
+ console.log({
274
+ actionId: event.actionId,
275
+ duration: event.duration,
276
+ attachments: event.attachments,
277
+ error: event.error
278
+ });
279
+ });
280
+ ```
281
+
282
+ #### `.invoke(...args: I)`
283
+
284
+ Execute the action with given arguments.
285
+
286
+ **Returns:** `{ actionId: string, data: Promise<O> }`
287
+
288
+ ```typescript
289
+ const { actionId, data } = action.invoke("input");
290
+ const result = await data;
291
+ ```
292
+
293
+ #### `.invokeAll(payloads: I[])`
294
+
295
+ Execute multiple invocations, returning all results in order.
296
+
297
+ **Returns:** `Promise<BatchResult<O>[]>`
298
+
299
+ ```typescript
300
+ const results = await action.invokeAll([
301
+ ["input1"],
302
+ ["input2"]
303
+ ]);
304
+ ```
305
+
306
+ #### `.invokeStream(payloads: I[])`
307
+
308
+ Execute multiple invocations, yielding results as they complete.
309
+
310
+ **Returns:** `AsyncGenerator<BatchResult<O> & { index: number }>`
311
+
312
+ ```typescript
313
+ for await (const result of action.invokeStream([["input1"], ["input2"]])) {
314
+ console.log(`Item ${result.index} completed`);
315
+ }
316
+ ```
317
+
318
+ ### Types
319
+
320
+ #### `InvocationContext`
321
+
322
+ Context object for attaching observability data.
323
+
324
+ ```typescript
325
+ interface InvocationContext {
326
+ readonly actionId: string;
327
+ attach(key: string, value: unknown): void;
328
+ attach(data: Record<string, unknown>): void;
329
+ }
330
+ ```
331
+
332
+ #### `WideEvent<I, O>`
333
+
334
+ Complete event record after invocation.
335
+
336
+ ```typescript
337
+ interface WideEvent<I extends any[], O> {
338
+ actionId: string;
339
+ startedAt: number;
340
+ completedAt: number;
341
+ duration: number;
342
+ input: I;
343
+ output?: O;
344
+ error?: Error;
345
+ attachments: Record<string, unknown>;
346
+ retryAttempt?: number;
347
+ totalAttempts?: number;
348
+ retryDelays?: number[];
349
+ isRetry?: boolean;
350
+ willRetry?: boolean;
351
+ }
352
+ ```
353
+
354
+ #### `RetryOptions`
355
+
356
+ Retry configuration options.
357
+
358
+ ```typescript
359
+ interface RetryOptions {
360
+ maxRetries: number;
361
+ backoff: 'linear' | 'exponential';
362
+ baseDelay?: number; // Default: 100ms
363
+ maxDelay?: number; // Default: 30000ms
364
+ jitter?: boolean; // Default: false
365
+ shouldRetry?: (error: unknown) => boolean;
366
+ }
367
+ ```
368
+
369
+ ## Examples
370
+
371
+ Check out the [examples/](examples/) directory for comprehensive examples:
372
+
373
+ - [01-basic-action.ts](examples/01-basic-action.ts) - Basic action creation and invocation
374
+ - [02-concurrency.ts](examples/02-concurrency.ts) - Concurrency control
375
+ - [03-rate-limiting.ts](examples/03-rate-limiting.ts) - Rate limiting
376
+ - [04-wide-events.ts](examples/04-wide-events.ts) - Wide events and context attachment
377
+ - [05-batch-invocation.ts](examples/05-batch-invocation.ts) - Batch processing
378
+ - [06-combining-features.ts](examples/06-combining-features.ts) - Combining multiple features
379
+ - [07-real-world-scenarios.ts](examples/07-real-world-scenarios.ts) - Production use cases
380
+ - [08-error-handling.ts](examples/08-error-handling.ts) - Error handling patterns
381
+ - [09-retry-examples.ts](examples/09-retry-examples.ts) - Retry strategies
382
+
383
+ Run examples with:
384
+
385
+ ```bash
386
+ npx tsx examples/01-basic-action.ts
387
+
388
+ # Or run all examples
389
+ npx tsx examples/run-all.ts
390
+ ```
391
+
392
+ ## Use Cases
393
+
394
+ ### API Client with Rate Limiting
395
+
396
+ ```typescript
397
+ const apiCall = createAction(async (endpoint: string) => {
398
+ return fetch(`https://api.example.com${endpoint}`).then(r => r.json());
399
+ })
400
+ .setRateLimit(50) // API allows 50 req/sec
401
+ .setConcurrency(10) // Keep 10 connections open
402
+ .setRetry({
403
+ maxRetries: 3,
404
+ backoff: 'exponential',
405
+ baseDelay: 200
406
+ });
407
+
408
+ const data = await apiCall.invoke("/users/123").data;
409
+ ```
410
+
411
+ ### Image Processing Pipeline
412
+
413
+ ```typescript
414
+ const processImage = createAction(
415
+ withContext(async (ctx, imageUrl: string) => {
416
+ ctx.attach("imageUrl", imageUrl);
417
+
418
+ const image = await downloadImage(imageUrl);
419
+ ctx.attach("originalSize", image.size);
420
+
421
+ const processed = await resize(image, 800, 600);
422
+ ctx.attach("processedSize", processed.size);
423
+
424
+ const uploaded = await uploadToStorage(processed);
425
+ ctx.attach("storageUrl", uploaded.url);
426
+
427
+ return uploaded;
428
+ })
429
+ )
430
+ .setConcurrency(3) // Process 3 images at a time
431
+ .onEvent((event) => {
432
+ metrics.record("image_processing", {
433
+ duration: event.duration,
434
+ ...event.attachments
435
+ });
436
+ });
437
+
438
+ // Process batch
439
+ const results = await processImage.invokeAll(
440
+ imageUrls.map(url => [url])
441
+ );
442
+ ```
443
+
444
+ ### Webhook Delivery with Retry
445
+
446
+ ```typescript
447
+ class NetworkError extends Error {}
448
+
449
+ const deliverWebhook = createAction(
450
+ withContext(async (ctx, payload: WebhookPayload) => {
451
+ ctx.attach("webhookId", payload.id);
452
+ ctx.attach("targetUrl", payload.url);
453
+
454
+ const response = await fetch(payload.url, {
455
+ method: 'POST',
456
+ body: JSON.stringify(payload.data),
457
+ headers: { 'Content-Type': 'application/json' }
458
+ });
459
+
460
+ if (!response.ok) throw new NetworkError(`HTTP ${response.status}`);
461
+
462
+ ctx.attach("statusCode", response.status);
463
+ return { delivered: true };
464
+ })
465
+ )
466
+ .setConcurrency(10)
467
+ .setRateLimit(100)
468
+ .setRetry({
469
+ maxRetries: 5,
470
+ backoff: 'exponential',
471
+ baseDelay: 1000,
472
+ maxDelay: 30000,
473
+ jitter: true,
474
+ shouldRetry: (err) => err instanceof NetworkError
475
+ })
476
+ .onEvent((event) => {
477
+ webhookLogger.log({
478
+ webhookId: event.attachments.webhookId,
479
+ success: !event.error,
480
+ attempts: event.totalAttempts,
481
+ duration: event.duration
482
+ });
483
+ });
484
+ ```
485
+
486
+ ## Development
487
+
488
+ ### Setup
489
+
490
+ ```bash
491
+ git clone https://github.com/AlexMason/observed-js.git
492
+ cd observed-js
493
+ npm install
494
+ ```
495
+
496
+ ### Commands
497
+
498
+ ```bash
499
+ npm run test # Run all tests
500
+ npm run test:actions # Actions tests only
501
+ npm run test:scheduler # Scheduler tests only
502
+ npm run test:wide-events # Wide events tests only
503
+ npm run test:retry # Retry tests only
504
+ npm run build # TypeScript compilation
505
+ npm run dev # Watch mode compilation
506
+ ```
507
+
508
+ ### Testing
509
+
510
+ Tests use a custom lightweight framework. See [src/tests/helpers.ts](src/tests/helpers.ts):
511
+
512
+ ```typescript
513
+ import { test, assert } from "./helpers.js";
514
+
515
+ await test("should execute actions", async () => {
516
+ const action = createAction(async (x: number) => x * 2);
517
+ const result = await action.invoke(5).data;
518
+ assert.strictEqual(result, 10);
519
+ })();
520
+ ```
521
+
522
+ ## Design Philosophy
523
+
524
+ 1. **Safe by default** - Concurrency defaults to 1 (sequential execution)
525
+ 2. **Type inference over explicit generics** - Let TypeScript infer types from handlers
526
+ 3. **Observability first** - Wide events are a core feature, not an afterthought
527
+ 4. **Isolated failures** - Errors in tasks or event callbacks don't break the queue
528
+ 5. **Composable features** - All features work together seamlessly
529
+ 6. **Zero magic** - Explicit, predictable behavior with clear semantics
530
+
531
+ ## Architecture
532
+
533
+ ```
534
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
535
+ β”‚ ActionBuilder β”‚ ← Fluent API (public interface)
536
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜
537
+ β”‚
538
+ β”œβ”€ setConcurrency()
539
+ β”œβ”€ setRateLimit()
540
+ β”œβ”€ setRetry()
541
+ β”œβ”€ onEvent()
542
+ β”‚
543
+ ↓
544
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
545
+ β”‚ ExecutionScheduler β”‚ ← Internal queue & flow control
546
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
547
+ β”‚
548
+ β”œβ”€ FIFO queue
549
+ β”œβ”€ Sliding window rate limiting
550
+ β”œβ”€ Concurrency slots
551
+ β”‚
552
+ ↓
553
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
554
+ β”‚ Handler β”‚ ← User's async function
555
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
556
+ ```
557
+
558
+ ## License
559
+
560
+ MIT Β© [Alexander Mason](https://github.com/yourusername)
561
+
562
+ ## Contributing
563
+
564
+ Contributions are welcome! Please check out the [docs/plan/](docs/plan/) directory for design docs and architectural decisions.
565
+
566
+ ### Key Rules for Contributors
567
+
568
+ - **Never rewrite existing tests** - Only add new tests
569
+ - **Prioritize observability** - Wide events should guide feature design
570
+ - **Check plan docs** - Review [docs/plan/](docs/plan/) before implementing features
571
+ - **Use `.js` extensions** - Import statements must use `.js` even for `.ts` files (ESM requirement)
572
+ - **Type inference** - Avoid explicit generics; let TypeScript infer types
573
+
574
+ ## Related Projects
575
+
576
+ - [p-limit](https://github.com/sindresorhus/p-limit) - Concurrency control
577
+ - [p-queue](https://github.com/sindresorhus/p-queue) - Promise queue with priority
578
+ - [bottleneck](https://github.com/SGrondin/bottleneck) - Rate limiting
579
+
580
+ **What makes `observed-js` different?**
581
+
582
+ - First-class wide event support for structured observability
583
+ - Unified API combining concurrency, rate limiting, and retries
584
+ - Type inference from handler functions
585
+ - Built for production observability from day one
586
+
587
+ ## Responsible AI Disclosure
588
+
589
+ This project utilizes AI-assisted development tools. **100% of all AI-generated code has been human reviewed and independently tested** to ensure correctness, security, and adherence to project standards. All tests are maintained and verified by human developers.