@ironflow/node 0.7.1 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +1455 -143
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -1,99 +1,1194 @@
1
1
  # @ironflow/node
2
2
 
3
- Node.js SDK for [Ironflow](https://github.com/sahina/ironflow), an event-driven backend platform. Provides workers, serve handlers, step execution, projections, entity streams, KV store, and config management.
3
+ Node.js SDK for [Ironflow](https://github.com/sahina/ironflow), an event-driven backend platform. Provides workers (pull mode), serve handlers (push mode), step execution, projections, entity streams, subscriptions, KV store, config management, webhooks, auth management, and testing utilities.
4
+
5
+ This is a **private** npm package (`"access": "restricted"`).
4
6
 
5
7
  ## Installation
6
8
 
7
- ```bash
8
- npm install @ironflow/node
9
+ ```bash
10
+ npm install @ironflow/node
11
+ ```
12
+
13
+ Requires Node.js 22+.
14
+
15
+ ## Table of Contents
16
+
17
+ 1. [Defining Functions](#defining-functions)
18
+ 2. [Step Primitives](#step-primitives)
19
+ 3. [Saga Compensation](#saga-compensation)
20
+ 4. [Push Mode (serve)](#push-mode-serve)
21
+ 5. [Pull Mode (createWorker)](#pull-mode-createworker)
22
+ 6. [Streaming Worker](#streaming-worker)
23
+ 7. [Server-Side Client](#server-side-client)
24
+ 8. [Entity Streams](#entity-streams)
25
+ 9. [KV Store](#kv-store)
26
+ 10. [Config Management](#config-management)
27
+ 11. [Auth Management](#auth-management)
28
+ 12. [Subscriptions](#subscriptions)
29
+ 13. [Projections](#projections)
30
+ 14. [Webhooks](#webhooks)
31
+ 15. [Event Versioning (Upcasters)](#event-versioning-upcasters)
32
+ 16. [Error Handling](#error-handling)
33
+ 17. [Environment Variables](#environment-variables)
34
+ 18. [Testing](#testing)
35
+
36
+ ---
37
+
38
+ ## Defining Functions
39
+
40
+ Use `createFunction(config, handler)` to define a workflow function. The function is triggered by events and executes durable steps.
41
+
42
+ ### Basic function (untyped)
43
+
44
+ ```typescript
45
+ import { createFunction } from '@ironflow/node';
46
+
47
+ const processOrder = createFunction(
48
+ {
49
+ id: 'process-order',
50
+ triggers: [{ event: 'order.placed' }],
51
+ },
52
+ async ({ event, step }) => {
53
+ const validated = await step.run('validate', async () => {
54
+ return validateOrder(event.data);
55
+ });
56
+ return { success: true, orderId: validated.orderId };
57
+ }
58
+ );
59
+ ```
60
+
61
+ ### With Zod schema validation
62
+
63
+ When you provide a `schema`, `event.data` is fully typed from the schema.
64
+
65
+ ```typescript
66
+ import { createFunction } from '@ironflow/node';
67
+ import { z } from 'zod';
68
+
69
+ const OrderSchema = z.object({
70
+ orderId: z.string(),
71
+ amount: z.number(),
72
+ customerId: z.string(),
73
+ });
74
+
75
+ const processOrder = createFunction(
76
+ {
77
+ id: 'process-order',
78
+ triggers: [{ event: 'order.placed' }],
79
+ schema: OrderSchema,
80
+ },
81
+ async ({ event, step }) => {
82
+ // event.data is typed as { orderId: string; amount: number; customerId: string }
83
+ const receipt = await step.run('charge', async () => {
84
+ return chargeCard(event.data.customerId, event.data.amount);
85
+ });
86
+ return { receiptId: receipt.id };
87
+ }
88
+ );
89
+ ```
90
+
91
+ ### FunctionConfig reference
92
+
93
+ All fields on the config object:
94
+
95
+ | Field | Type | Description |
96
+ |-------|------|-------------|
97
+ | `id` | `string` | **Required.** Unique function identifier. |
98
+ | `name` | `string` | Display name. Defaults to `id`. |
99
+ | `triggers` | `Trigger[]` | **Required.** Array of event triggers. |
100
+ | `retry` | `RetryConfig` | Retry policy for failed steps. |
101
+ | `timeout` | `number` | Function timeout in milliseconds (default: 600000 = 10 min). |
102
+ | `concurrency` | `ConcurrencyConfig` | Concurrency control. |
103
+ | `mode` | `"push" \| "pull"` | Preferred execution mode. |
104
+ | `actorKey` | `string` | JSON path for actor-based sticky routing (e.g., `"event.data.customerId"`). |
105
+ | `schema` | `ZodType` | Zod schema for event data validation and type inference. |
106
+ | `secrets` | `string[]` | Secret names this function requires (resolved by the engine). |
107
+ | `stepTimeout` | `string` | Default timeout for all `step.run()` calls (e.g., `"30s"`, `"5m"`). |
108
+ | `recording` | `boolean` | Enable audit recording for this function. |
109
+ | `recordingRetention` | `string` | Retention period for audit events (`"7d"`, `"30d"`, `"90d"`, `"forever"`). |
110
+
111
+ **Trigger** fields:
112
+
113
+ | Field | Type | Description |
114
+ |-------|------|-------------|
115
+ | `event` | `string` | Event name pattern (e.g., `"order.placed"`). |
116
+ | `expression` | `string` | Optional CEL expression for filtering. |
117
+ | `cron` | `string` | Cron schedule (e.g., `"0 9 * * *"` for 9am daily). |
118
+
119
+ **RetryConfig** fields:
120
+
121
+ | Field | Type | Default | Description |
122
+ |-------|------|---------|-------------|
123
+ | `maxAttempts` | `number` | `3` | Maximum retry attempts. |
124
+ | `initialDelayMs` | `number` | `1000` | Initial delay between retries. |
125
+ | `backoffFactor` | `number` | `2.0` | Backoff multiplier. |
126
+ | `maxDelayMs` | `number` | `300000` | Maximum delay between retries. |
127
+
128
+ **ConcurrencyConfig** fields:
129
+
130
+ | Field | Type | Description |
131
+ |-------|------|-------------|
132
+ | `limit` | `number` | Maximum concurrent executions. |
133
+ | `key` | `string` | JSON path for grouping (e.g., `"event.data.customerId"`). |
134
+
135
+ ### Full config example
136
+
137
+ ```typescript
138
+ const processOrder = createFunction(
139
+ {
140
+ id: 'process-order',
141
+ name: 'Process Order',
142
+ triggers: [
143
+ { event: 'order.placed' },
144
+ { event: 'order.retry', expression: 'event.data.priority > 5' },
145
+ { cron: '0 */6 * * *', event: 'scheduled.cleanup' },
146
+ ],
147
+ retry: { maxAttempts: 5, initialDelayMs: 2000, backoffFactor: 3 },
148
+ timeout: 300000,
149
+ concurrency: { limit: 10, key: 'event.data.customerId' },
150
+ mode: 'pull',
151
+ actorKey: 'event.data.customerId',
152
+ schema: OrderSchema,
153
+ secrets: ['STRIPE_SECRET_KEY', 'SENDGRID_API_KEY'],
154
+ stepTimeout: '30s',
155
+ recording: true,
156
+ recordingRetention: '30d',
157
+ },
158
+ async ({ event, step, secrets }) => {
159
+ const apiKey = secrets.get('STRIPE_SECRET_KEY');
160
+ // ...
161
+ }
162
+ );
163
+ ```
164
+
165
+ ---
166
+
167
+ ## Step Primitives
168
+
169
+ Every step is durable and memoized. If the workflow retries, previously completed steps are skipped and their stored results are returned.
170
+
171
+ ### step.run(name, fn, options?)
172
+
173
+ Execute a memoized step. Use for any non-idempotent operation (API calls, payments, emails).
174
+
175
+ ```typescript
176
+ const processOrder = createFunction(
177
+ { id: 'process-order', triggers: [{ event: 'order.placed' }] },
178
+ async ({ event, step }) => {
179
+ // Basic usage
180
+ const result = await step.run('charge-card', async () => {
181
+ return chargeCard(event.data.amount);
182
+ });
183
+
184
+ // With timeout override
185
+ const enriched = await step.run('enrich-data', async () => {
186
+ return callSlowApi(event.data.id);
187
+ }, { timeout: '60s' });
188
+
189
+ return { transactionId: result.id, enriched };
190
+ }
191
+ );
192
+ ```
193
+
194
+ **Options:**
195
+
196
+ | Field | Type | Description |
197
+ |-------|------|-------------|
198
+ | `timeout` | `string` | Step timeout (e.g., `"30s"`, `"5m"`, `"1h"`). Overrides function-level `stepTimeout`. |
199
+
200
+ ### step.sleep(name, duration)
201
+
202
+ Durable sleep that survives process restarts and server upgrades.
203
+
204
+ ```typescript
205
+ const delayedNotification = createFunction(
206
+ { id: 'delayed-notify', triggers: [{ event: 'user.signup' }] },
207
+ async ({ event, step }) => {
208
+ await step.run('send-welcome', async () => {
209
+ return sendWelcomeEmail(event.data.email);
210
+ });
211
+
212
+ // Durable sleep - workflow pauses and resumes after duration
213
+ await step.sleep('wait-for-trial', '7d');
214
+
215
+ await step.run('send-trial-ending', async () => {
216
+ return sendTrialEndingEmail(event.data.email);
217
+ });
218
+ }
219
+ );
220
+ ```
221
+
222
+ **Duration formats:** `"30s"`, `"5m"`, `"1h"`, `"7d"`, or milliseconds as a number.
223
+
224
+ ### step.sleepUntil(name, date)
225
+
226
+ Sleep until a specific point in time.
227
+
228
+ ```typescript
229
+ const scheduledTask = createFunction(
230
+ { id: 'scheduled-task', triggers: [{ event: 'task.scheduled' }] },
231
+ async ({ event, step }) => {
232
+ // Sleep until a specific Date object
233
+ await step.sleepUntil('wait-until-date', new Date('2025-12-31T00:00:00Z'));
234
+
235
+ // Or an ISO string
236
+ await step.sleepUntil('wait-until-deadline', event.data.deadline);
237
+
238
+ await step.run('execute', async () => {
239
+ return executeTask(event.data.taskId);
240
+ });
241
+ }
242
+ );
243
+ ```
244
+
245
+ ### step.waitForEvent(name, filter)
246
+
247
+ Wait for an external event to arrive. The workflow pauses durably until a matching event is emitted.
248
+
249
+ ```typescript
250
+ const approvalWorkflow = createFunction(
251
+ { id: 'approval-flow', triggers: [{ event: 'approval.requested' }] },
252
+ async ({ event, step }) => {
253
+ await step.run('notify-approver', async () => {
254
+ return sendApprovalRequest(event.data.approverId);
255
+ });
256
+
257
+ // Wait for the approval event (default timeout: 7 days)
258
+ const approval = await step.waitForEvent('wait-approval', {
259
+ event: 'approval.received',
260
+ match: 'data.requestId', // JSON path for matching
261
+ timeout: '48h',
262
+ });
263
+
264
+ if (approval.data.approved) {
265
+ await step.run('process-approved', async () => {
266
+ return processApproval(event.data.requestId);
267
+ });
268
+ }
269
+
270
+ return { approved: approval.data.approved };
271
+ }
272
+ );
273
+ ```
274
+
275
+ **EventFilter fields:**
276
+
277
+ | Field | Type | Default | Description |
278
+ |-------|------|---------|-------------|
279
+ | `event` | `string` | -- | Event name to wait for. |
280
+ | `match` | `string` | -- | JSON path for correlating events. |
281
+ | `timeout` | `Duration` | `"7d"` | How long to wait before timing out. |
282
+
283
+ ### step.invoke(functionId, input?, options?)
284
+
285
+ Call another Ironflow function and wait for its result. Creates a child run.
286
+
287
+ ```typescript
288
+ const orchestrator = createFunction(
289
+ { id: 'orchestrator', triggers: [{ event: 'workflow.start' }] },
290
+ async ({ event, step }) => {
291
+ // Invoke and wait for result (default timeout: 30s)
292
+ const result = await step.invoke('process-payment', {
293
+ orderId: event.data.orderId,
294
+ amount: event.data.amount,
295
+ });
296
+
297
+ // With custom timeout
298
+ const report = await step.invoke('generate-report', {
299
+ orderId: event.data.orderId,
300
+ }, { timeout: '5m' });
301
+
302
+ return { paymentResult: result, report };
303
+ }
304
+ );
305
+ ```
306
+
307
+ ### step.invokeAsync(functionId, input?)
308
+
309
+ Fire-and-forget: trigger another function without waiting for its result. Returns the child run ID.
310
+
311
+ ```typescript
312
+ const orderPipeline = createFunction(
313
+ { id: 'order-pipeline', triggers: [{ event: 'order.placed' }] },
314
+ async ({ event, step }) => {
315
+ // Fire and forget - does not block
316
+ const { runId } = await step.invokeAsync('send-confirmation-email', {
317
+ orderId: event.data.orderId,
318
+ email: event.data.email,
319
+ });
320
+
321
+ return { emailRunId: runId };
322
+ }
323
+ );
324
+ ```
325
+
326
+ ### step.parallel(name, branches, options?)
327
+
328
+ Execute multiple branches in parallel. Each branch receives its own scoped `step` client with isolated step IDs.
329
+
330
+ ```typescript
331
+ const enrichUser = createFunction(
332
+ { id: 'enrich-user', triggers: [{ event: 'user.created' }] },
333
+ async ({ event, step }) => {
334
+ const [profile, creditScore, preferences] = await step.parallel(
335
+ 'fetch-all',
336
+ [
337
+ async (s) => s.run('fetch-profile', () => fetchProfile(event.data.userId)),
338
+ async (s) => s.run('fetch-credit', () => fetchCreditScore(event.data.userId)),
339
+ async (s) => s.run('fetch-prefs', () => fetchPreferences(event.data.userId)),
340
+ ]
341
+ );
342
+
343
+ return { profile, creditScore, preferences };
344
+ }
345
+ );
346
+ ```
347
+
348
+ **ParallelOptions:**
349
+
350
+ | Field | Type | Default | Description |
351
+ |-------|------|---------|-------------|
352
+ | `concurrency` | `number` | unlimited | Maximum concurrent branches. |
353
+ | `onError` | `"failFast" \| "allSettled"` | `"failFast"` | `"failFast"`: first failure cancels pending branches. `"allSettled"`: all branches complete, errors in results. |
354
+
355
+ ```typescript
356
+ // With concurrency limit and allSettled error handling
357
+ const results = await step.parallel(
358
+ 'batch-process',
359
+ items.map((item) => async (s) => {
360
+ return s.run(`process-${item.id}`, () => processItem(item));
361
+ }),
362
+ { concurrency: 5, onError: 'allSettled' }
363
+ );
364
+ ```
365
+
366
+ ### step.map(name, items, fn, options?)
367
+
368
+ Parallel map over an array of items. Convenience wrapper around `step.parallel`.
369
+
370
+ ```typescript
371
+ const batchProcessor = createFunction(
372
+ { id: 'batch-process', triggers: [{ event: 'batch.ready' }] },
373
+ async ({ event, step }) => {
374
+ const results = await step.map(
375
+ 'process-items',
376
+ event.data.items,
377
+ async (item, s, index) => {
378
+ return s.run(`process-${index}`, () => processItem(item));
379
+ },
380
+ { concurrency: 3 }
381
+ );
382
+
383
+ return { processed: results.length };
384
+ }
385
+ );
386
+ ```
387
+
388
+ ### step.compensate(stepName, fn)
389
+
390
+ Register a compensation function (saga rollback) for a previously completed step. Compensations run automatically in reverse order on terminal (non-retryable) failures.
391
+
392
+ ```typescript
393
+ const transferFunds = createFunction(
394
+ { id: 'transfer-funds', triggers: [{ event: 'transfer.requested' }] },
395
+ async ({ event, step }) => {
396
+ const debit = await step.run('debit', async () => {
397
+ return debitAccount(event.data.fromAccount, event.data.amount);
398
+ });
399
+ step.compensate('debit', async () => {
400
+ await creditAccount(event.data.fromAccount, event.data.amount);
401
+ });
402
+
403
+ const credit = await step.run('credit', async () => {
404
+ return creditAccount(event.data.toAccount, event.data.amount);
405
+ });
406
+ step.compensate('credit', async () => {
407
+ await debitAccount(event.data.toAccount, event.data.amount);
408
+ });
409
+
410
+ return { debitRef: debit.ref, creditRef: credit.ref };
411
+ }
412
+ );
413
+ ```
414
+
415
+ ### step.publish(topic, data)
416
+
417
+ Publish a message to a developer pub/sub topic. The publish is memoized like any other step.
418
+
419
+ ```typescript
420
+ const orderProcessor = createFunction(
421
+ { id: 'order-processor', triggers: [{ event: 'order.placed' }] },
422
+ async ({ event, step }) => {
423
+ const result = await step.run('process', async () => {
424
+ return processOrder(event.data);
425
+ });
426
+
427
+ // Publish to a topic (does NOT trigger workflow functions)
428
+ await step.publish('order-notifications', {
429
+ orderId: event.data.orderId,
430
+ status: 'processed',
431
+ });
432
+
433
+ return result;
434
+ }
435
+ );
436
+ ```
437
+
438
+ ---
439
+
440
+ ## Saga Compensation
441
+
442
+ Compensations provide automatic rollback for distributed transactions. When a terminal (non-retryable) failure occurs, all registered compensations run in reverse order. Each compensation is itself a durable, memoized step.
443
+
444
+ ```typescript
445
+ import { createFunction, NonRetryableError } from '@ironflow/node';
446
+
447
+ const bookTrip = createFunction(
448
+ { id: 'book-trip', triggers: [{ event: 'trip.requested' }] },
449
+ async ({ event, step }) => {
450
+ // Step 1: Book flight
451
+ const flight = await step.run('book-flight', async () => {
452
+ return bookFlight(event.data.flightId);
453
+ });
454
+ step.compensate('book-flight', async () => {
455
+ await cancelFlight(flight.confirmationId);
456
+ });
457
+
458
+ // Step 2: Book hotel
459
+ const hotel = await step.run('book-hotel', async () => {
460
+ return bookHotel(event.data.hotelId, event.data.dates);
461
+ });
462
+ step.compensate('book-hotel', async () => {
463
+ await cancelHotel(hotel.confirmationId);
464
+ });
465
+
466
+ // Step 3: Book car rental (if this fails with a non-retryable error,
467
+ // both hotel and flight compensations run in reverse order)
468
+ const car = await step.run('book-car', async () => {
469
+ const result = await bookCar(event.data.carId);
470
+ if (!result.available) {
471
+ throw new NonRetryableError('Car not available');
472
+ }
473
+ return result;
474
+ });
475
+ step.compensate('book-car', async () => {
476
+ await cancelCar(car.confirmationId);
477
+ });
478
+
479
+ return { flight, hotel, car };
480
+ }
481
+ );
482
+ ```
483
+
484
+ Key behaviors:
485
+ - Compensations only run on **terminal** (non-retryable) failures.
486
+ - They run in **reverse registration order** (last registered runs first).
487
+ - Each compensation is recorded as a durable step (`compensate:<stepName>`).
488
+ - If a compensation itself fails, the error is logged but remaining compensations still run.
489
+
490
+ ---
491
+
492
+ ## Push Mode (serve)
493
+
494
+ The `serve()` function creates a universal HTTP handler for serverless deployment. It works with any framework that uses the Fetch `Request`/`Response` API or Node.js `IncomingMessage`/`ServerResponse`.
495
+
496
+ ### ServeConfig reference
497
+
498
+ | Field | Type | Description |
499
+ |-------|------|-------------|
500
+ | `functions` | `IronflowFunction[]` | **Required.** Functions to handle. |
501
+ | `projections` | `IronflowProjection[]` | Logs a warning -- use `createWorker` for projections. |
502
+ | `signingKey` | `string` | HMAC-SHA256 signing key for request verification. |
503
+ | `skipVerification` | `boolean` | Skip signature verification (dev only). |
504
+ | `logger` | `Logger \| false` | Custom logger or `false` to disable. |
505
+ | `environment` | `string` | Target environment (default: `IRONFLOW_ENV` or `"default"`). |
506
+ | `eventDefinitions` | `EventDefinitionRegistry` | Registry for automatic event upcasting. |
507
+ | `serverUrl` | `string` | Ironflow server URL (for emitting webhook events). |
508
+ | `webhooks` | `IronflowWebhook[]` | Webhook sources to handle. |
509
+
510
+ ### Next.js App Router
511
+
512
+ ```typescript
513
+ // app/api/ironflow/route.ts
514
+ import { serve } from '@ironflow/node';
515
+ import { processOrder } from '@/functions/process-order';
516
+
517
+ export const POST = serve({
518
+ functions: [processOrder],
519
+ signingKey: process.env.IRONFLOW_SIGNING_KEY,
520
+ });
521
+ ```
522
+
523
+ ### Express
524
+
525
+ ```typescript
526
+ import express from 'express';
527
+ import { serve } from '@ironflow/node';
528
+ import { processOrder } from './functions/process-order.js';
529
+
530
+ const app = express();
531
+
532
+ app.post('/api/ironflow', serve({
533
+ functions: [processOrder],
534
+ signingKey: process.env.IRONFLOW_SIGNING_KEY,
535
+ }));
536
+
537
+ app.listen(3000);
538
+ ```
539
+
540
+ ### Hono
541
+
542
+ ```typescript
543
+ import { Hono } from 'hono';
544
+ import { serve } from '@ironflow/node';
545
+ import { processOrder } from './functions/process-order.js';
546
+
547
+ const app = new Hono();
548
+
549
+ app.post('/api/ironflow', serve({
550
+ functions: [processOrder],
551
+ signingKey: process.env.IRONFLOW_SIGNING_KEY,
552
+ }));
553
+
554
+ export default app;
555
+ ```
556
+
557
+ ---
558
+
559
+ ## Pull Mode (createWorker)
560
+
561
+ Workers poll the Ironflow server for jobs via REST HTTP. Use for long-running tasks with no timeout limits.
562
+
563
+ ### WorkerConfig reference
564
+
565
+ | Field | Type | Default | Description |
566
+ |-------|------|---------|-------------|
567
+ | `serverUrl` | `string` | `http://localhost:9123` | Ironflow server URL. |
568
+ | `functions` | `IronflowFunction[]` | -- | **Required.** Functions this worker handles. |
569
+ | `projections` | `IronflowProjection[]` | -- | Projections to run alongside functions. |
570
+ | `maxConcurrentJobs` | `number` | `10` | Maximum concurrent job executions. |
571
+ | `heartbeatInterval` | `number` | `30000` | Heartbeat interval in ms. |
572
+ | `reconnectDelay` | `number` | `5000` | Reconnect delay in ms. |
573
+ | `labels` | `Record<string, string>` | -- | Worker labels for routing. |
574
+ | `transport` | `"polling" \| "streaming"` | `"polling"` | Transport type. |
575
+ | `logger` | `Logger \| false` | -- | Custom logger or `false` to disable. |
576
+ | `environment` | `string` | `IRONFLOW_ENV` or `"default"` | Target environment. |
577
+ | `eventDefinitions` | `EventDefinitionRegistry` | -- | Registry for automatic event upcasting. |
578
+ | `apiKey` | `string` | `IRONFLOW_API_KEY` env | API key for authentication. |
579
+
580
+ ### Worker interface
581
+
582
+ | Method | Description |
583
+ |--------|-------------|
584
+ | `start()` | Start the worker. Blocks until stopped. Auto-reconnects on failure. |
585
+ | `drain()` | Gracefully drain: stop accepting new jobs, wait for active jobs to complete, then stop. |
586
+ | `stop()` | Force stop immediately. Cancels all active jobs. |
587
+
588
+ ```typescript
589
+ import { createWorker } from '@ironflow/node';
590
+ import { processOrder, sendNotification } from './functions.js';
591
+ import { orderTotals } from './projections.js';
592
+
593
+ const worker = createWorker({
594
+ serverUrl: 'http://localhost:9123',
595
+ functions: [processOrder, sendNotification],
596
+ projections: [orderTotals],
597
+ maxConcurrentJobs: 20,
598
+ labels: { region: 'us-east-1' },
599
+ apiKey: process.env.IRONFLOW_API_KEY,
600
+ });
601
+
602
+ // Graceful shutdown
603
+ process.on('SIGTERM', async () => {
604
+ await worker.drain();
605
+ process.exit(0);
606
+ });
607
+
608
+ await worker.start();
609
+ ```
610
+
611
+ ---
612
+
613
+ ## Streaming Worker
614
+
615
+ For low-latency bidirectional streaming via ConnectRPC. Same `WorkerConfig` as `createWorker`, but uses gRPC bidirectional streaming instead of HTTP polling.
616
+
617
+ Import from the separate `@ironflow/node/worker-streaming` entry point to avoid loading protobuf dependencies unless needed.
618
+
619
+ ```typescript
620
+ import { createStreamingWorker } from '@ironflow/node/worker-streaming';
621
+ import { processOrder } from './functions.js';
622
+
623
+ const worker = createStreamingWorker({
624
+ serverUrl: 'http://localhost:9123',
625
+ functions: [processOrder],
626
+ maxConcurrentJobs: 10,
627
+ });
628
+
629
+ process.on('SIGTERM', async () => {
630
+ await worker.drain();
631
+ process.exit(0);
632
+ });
633
+
634
+ await worker.start();
635
+ ```
636
+
637
+ Requires optional dependencies: `@bufbuild/protobuf`, `@connectrpc/connect`, `@connectrpc/connect-node`.
638
+
639
+ ---
640
+
641
+ ## Server-Side Client
642
+
643
+ The HTTP client for interacting with the Ironflow server from your backend code.
644
+
645
+ ### IronflowClientConfig
646
+
647
+ | Field | Type | Default | Description |
648
+ |-------|------|---------|-------------|
649
+ | `serverUrl` | `string` | `http://localhost:9123` or `IRONFLOW_SERVER_URL` | Server URL. |
650
+ | `apiKey` | `string` | -- | API key for authentication. |
651
+ | `timeout` | `number` | `30000` | Request timeout in ms. |
652
+
653
+ ### Creating a client
654
+
655
+ ```typescript
656
+ import { createClient } from '@ironflow/node';
657
+
658
+ const client = createClient({
659
+ serverUrl: 'http://localhost:9123',
660
+ apiKey: process.env.IRONFLOW_API_KEY,
661
+ });
662
+ ```
663
+
664
+ ### emit(eventName, data, options?)
665
+
666
+ Emit an event to trigger workflow functions.
667
+
668
+ ```typescript
669
+ const result = await client.emit('order.placed', {
670
+ orderId: '123',
671
+ amount: 99.99,
672
+ });
673
+ console.log('Run IDs:', result.runIds);
674
+ console.log('Event ID:', result.eventId);
675
+ ```
676
+
677
+ **EmitOptions:**
678
+
679
+ | Field | Type | Description |
680
+ |-------|------|-------------|
681
+ | `version` | `number` | Event schema version (default: 1). |
682
+ | `idempotencyKey` | `string` | Prevent duplicate processing. |
683
+ | `metadata` | `Record<string, unknown>` | Additional metadata. |
684
+
685
+ ```typescript
686
+ await client.emit('order.placed', { orderId: '123' }, {
687
+ version: 2,
688
+ idempotencyKey: 'order-123-placed',
689
+ metadata: { source: 'api', traceId: 'abc' },
690
+ });
691
+ ```
692
+
693
+ ### trigger(eventName, data, options?)
694
+
695
+ Deprecated alias for `emit()`. Use `emit()` instead.
696
+
697
+ ### getRun(runId)
698
+
699
+ Get a run by its ID.
700
+
701
+ ```typescript
702
+ const run = await client.getRun('run_abc123');
703
+ console.log(run.status); // "completed" | "running" | "failed" | "cancelled" | ...
704
+ console.log(run.output); // Function return value (if completed)
705
+ console.log(run.error); // { message, code } (if failed)
706
+ ```
707
+
708
+ ### listRuns(options?)
709
+
710
+ List runs with optional filtering and pagination.
711
+
712
+ ```typescript
713
+ const result = await client.listRuns({
714
+ functionId: 'process-order',
715
+ status: 'failed',
716
+ limit: 25,
717
+ cursor: 'next_page_cursor',
718
+ });
719
+ console.log(result.runs); // Run[]
720
+ console.log(result.totalCount); // number
721
+ console.log(result.nextCursor); // string | undefined
722
+ ```
723
+
724
+ ### cancelRun(runId, reason?)
725
+
726
+ Cancel a running workflow.
727
+
728
+ ```typescript
729
+ const run = await client.cancelRun('run_abc123', 'no longer needed');
730
+ ```
731
+
732
+ ### retryRun(runId, fromStep?)
733
+
734
+ Retry a failed run, optionally from a specific step.
735
+
736
+ ```typescript
737
+ await client.retryRun('run_abc123');
738
+ await client.retryRun('run_abc123', 'validate'); // Retry from specific step
739
+ ```
740
+
741
+ ### resumeRun(runId, fromStep?)
742
+
743
+ Resume a paused or failed run.
744
+
745
+ ```typescript
746
+ await client.resumeRun('run_abc123');
747
+ await client.resumeRun('run_abc123', 'charge-card');
748
+ ```
749
+
750
+ ### patchStep(stepId, output, reason?)
751
+
752
+ Hot-patch a step's output. Useful for debugging or correcting bad data.
753
+
754
+ ```typescript
755
+ await client.patchStep('step_xyz', { correctedValue: 42 }, 'fix bad data');
756
+ ```
757
+
758
+ ### listFunctions()
759
+
760
+ List all registered functions.
761
+
762
+ ```typescript
763
+ const functions = await client.listFunctions();
764
+ ```
765
+
766
+ ### listWorkers()
767
+
768
+ List all connected workers.
769
+
770
+ ```typescript
771
+ const workers = await client.listWorkers();
772
+ ```
773
+
774
+ ### health()
775
+
776
+ Server health check. Returns the status string.
777
+
778
+ ```typescript
779
+ const status = await client.health();
780
+ console.log(status); // "ok"
781
+ ```
782
+
783
+ ### publish(topic, data, options?)
784
+
785
+ Publish a message to a developer pub/sub topic. Unlike `emit()`, this does **not** trigger workflow functions.
786
+
787
+ ```typescript
788
+ const result = await client.publish('notifications', {
789
+ userId: '123',
790
+ message: 'Hello!',
791
+ });
792
+ console.log(result.eventId); // string
793
+ console.log(result.sequence); // number
794
+ ```
795
+
796
+ **PublishOptions:**
797
+
798
+ | Field | Type | Description |
799
+ |-------|------|-------------|
800
+ | `idempotencyKey` | `string` | Prevent duplicate publishing. |
801
+
802
+ ### listTopics()
803
+
804
+ List all active developer pub/sub topics.
805
+
806
+ ```typescript
807
+ const topics = await client.listTopics();
808
+ for (const t of topics) {
809
+ console.log(t.name, t.messageCount, t.consumerCount);
810
+ }
811
+ ```
812
+
813
+ ### getTopicStats(topic)
814
+
815
+ Get detailed statistics for a specific topic.
816
+
817
+ ```typescript
818
+ const stats = await client.getTopicStats('notifications');
819
+ console.log('Messages:', stats.messageCount);
820
+ console.log('Lag:', stats.lag);
821
+ console.log('Consumers:', stats.consumerCount);
822
+ console.log('First seq:', stats.firstSeq, 'Last seq:', stats.lastSeq);
823
+ ```
824
+
825
+ ---
826
+
827
+ ## Entity Streams
828
+
829
+ Event sourcing primitives via `client.streams`. Append domain events per entity, read them back, and use optimistic concurrency.
830
+
831
+ ### streams.append(entityId, input, options?)
832
+
833
+ Append an event to an entity stream.
834
+
835
+ ```typescript
836
+ const client = createClient({ serverUrl: 'http://localhost:9123' });
837
+
838
+ const result = await client.streams.append('order-123', {
839
+ name: 'item.added',
840
+ data: { sku: 'ABC', qty: 2 },
841
+ entityType: 'order',
842
+ });
843
+ console.log(result.entityVersion); // number
844
+ console.log(result.eventId); // string
845
+ ```
846
+
847
+ ### Optimistic concurrency with expectedVersion
848
+
849
+ ```typescript
850
+ // Read current version
851
+ const info = await client.streams.getInfo('order-123');
852
+
853
+ // Append with version check (fails if another writer modified the stream)
854
+ try {
855
+ await client.streams.append('order-123', {
856
+ name: 'item.removed',
857
+ data: { sku: 'ABC' },
858
+ entityType: 'order',
859
+ }, { expectedVersion: info.version });
860
+ } catch (err) {
861
+ console.error('Concurrent modification detected');
862
+ }
9
863
  ```
10
864
 
11
- ## Quick Start
865
+ **AppendOptions:**
866
+
867
+ | Field | Type | Description |
868
+ |-------|------|-------------|
869
+ | `expectedVersion` | `number` | Optimistic concurrency check. |
870
+ | `idempotencyKey` | `string` | Prevent duplicate appends. |
871
+ | `version` | `number` | Event schema version (default: 1). |
12
872
 
13
- ### Define a Function
873
+ ### streams.read(entityId, options?)
874
+
875
+ Read events from an entity stream.
14
876
 
15
877
  ```typescript
16
- import { ironflow } from '@ironflow/node';
878
+ const { events, totalCount } = await client.streams.read('order-123', {
879
+ direction: 'forward', // "forward" | "backward"
880
+ fromVersion: 0,
881
+ limit: 50,
882
+ });
17
883
 
18
- const processOrder = ironflow.createFunction(
19
- {
20
- id: 'process-order',
21
- triggers: [{ event: 'order.placed' }],
22
- },
23
- async ({ event, step }) => {
24
- const validated = await step.run('validate', async () => {
25
- return validateOrder(event.data);
26
- });
884
+ for (const event of events) {
885
+ console.log(event.name, event.data, event.entityVersion);
886
+ }
887
+ ```
27
888
 
28
- await step.sleep('wait', '5m');
889
+ ### streams.getInfo(entityId)
29
890
 
30
- const result = await step.run('process', async () => {
31
- return processPayment(validated);
32
- });
891
+ Get metadata about an entity stream.
33
892
 
34
- return { success: true, receiptId: result.id };
35
- }
36
- );
893
+ ```typescript
894
+ const info = await client.streams.getInfo('order-123');
895
+ console.log(info.entityId); // "order-123"
896
+ console.log(info.entityType); // "order"
897
+ console.log(info.version); // current version number
898
+ console.log(info.eventCount); // total events
899
+ console.log(info.createdAt); // ISO string
900
+ console.log(info.updatedAt); // ISO string
901
+ ```
902
+
903
+ ---
904
+
905
+ ## KV Store
906
+
907
+ Distributed key-value storage with bucket management, CAS (compare-and-swap), TTL, and wildcard key listing.
908
+
909
+ ### Getting a KV client
910
+
911
+ ```typescript
912
+ const client = createClient({ serverUrl: 'http://localhost:9123' });
913
+ const kv = client.kv();
37
914
  ```
38
915
 
39
- ### Push Mode (Serverless)
916
+ ### Bucket management
40
917
 
41
918
  ```typescript
42
- // app/api/ironflow/route.ts (Next.js App Router)
43
- import { serve } from '@ironflow/node';
919
+ // Create a bucket with TTL
920
+ await kv.createBucket({ name: 'sessions', ttlSeconds: 3600 });
44
921
 
45
- export const POST = serve({
46
- functions: [processOrder],
47
- signingKey: process.env.IRONFLOW_SIGNING_KEY,
48
- });
922
+ // List all buckets
923
+ const buckets = await kv.listBuckets();
924
+
925
+ // Get bucket info
926
+ const info = await kv.getBucketInfo('sessions');
927
+
928
+ // Delete a bucket
929
+ await kv.deleteBucket('sessions');
49
930
  ```
50
931
 
51
- ### Pull Mode (Worker)
932
+ **KVBucketConfig:**
933
+
934
+ | Field | Type | Description |
935
+ |-------|------|-------------|
936
+ | `name` | `string` | Bucket name. |
937
+ | `description` | `string` | Optional description. |
938
+ | `ttlSeconds` | `number` | Time-to-live for keys in seconds. |
939
+ | `maxValueSize` | `number` | Maximum value size in bytes. |
940
+ | `maxBytes` | `number` | Maximum total bucket size. |
941
+ | `history` | `number` | Number of historical revisions to keep. |
942
+
943
+ ### Key operations
52
944
 
53
945
  ```typescript
54
- import { createWorker } from '@ironflow/node';
946
+ const bucket = kv.bucket('sessions');
55
947
 
56
- const worker = createWorker({
948
+ // Put a value (unconditional write)
949
+ const { revision } = await bucket.put('user-123', { token: 'abc', role: 'admin' });
950
+
951
+ // Get a value
952
+ const entry = await bucket.get('user-123');
953
+ console.log(entry.value); // the stored value
954
+ console.log(entry.revision); // revision number for CAS
955
+
956
+ // Create only if key doesn't exist (throws HTTP 412 on conflict)
957
+ await bucket.create('user-456', { token: 'def' });
958
+
959
+ // Compare-and-swap update (throws HTTP 412 on revision mismatch)
960
+ await bucket.update('user-123', { token: 'xyz', role: 'admin' }, entry.revision);
961
+
962
+ // Soft delete (tombstone)
963
+ await bucket.delete('user-123');
964
+
965
+ // Purge key and all history
966
+ await bucket.purge('user-123');
967
+
968
+ // List keys with optional wildcard filter
969
+ const keys = await bucket.listKeys('user-*');
970
+ const allKeys = await bucket.listKeys();
971
+ ```
972
+
973
+ ---
974
+
975
+ ## Config Management
976
+
977
+ Centralized configuration store with set, get, patch (shallow merge), list, and delete.
978
+
979
+ ```typescript
980
+ const client = createClient({ serverUrl: 'http://localhost:9123' });
981
+ const config = client.config();
982
+
983
+ // Set a config (full replacement)
984
+ await config.set('app-settings', { theme: 'dark', locale: 'en', maxRetries: 3 });
985
+
986
+ // Get a config
987
+ const settings = await config.get('app-settings');
988
+ console.log(settings.data); // { theme: 'dark', locale: 'en', maxRetries: 3 }
989
+
990
+ // Patch a config (shallow merge)
991
+ await config.patch('app-settings', { locale: 'fr' });
992
+ // Result: { theme: 'dark', locale: 'fr', maxRetries: 3 }
993
+
994
+ // List all configs
995
+ const all = await config.list();
996
+ for (const entry of all) {
997
+ console.log(entry.name, entry.data);
998
+ }
999
+
1000
+ // Delete a config (idempotent)
1001
+ await config.delete('app-settings');
1002
+ ```
1003
+
1004
+ ---
1005
+
1006
+ ## Auth Management
1007
+
1008
+ ### API Keys
1009
+
1010
+ ```typescript
1011
+ const client = createClient({
57
1012
  serverUrl: 'http://localhost:9123',
58
- functions: [processOrder],
59
- maxConcurrentJobs: 10,
1013
+ apiKey: process.env.IRONFLOW_API_KEY,
60
1014
  });
61
1015
 
62
- await worker.start();
1016
+ // Create an API key
1017
+ const newKey = await client.apiKeys.create({ name: 'ci-key', envId: 'env_default' });
1018
+ console.log(newKey.secret); // Only returned on create/rotate
1019
+
1020
+ // List all API keys
1021
+ const keys = await client.apiKeys.list();
1022
+
1023
+ // Get a specific key
1024
+ const key = await client.apiKeys.get(keys[0].id);
1025
+
1026
+ // Rotate (generates new secret)
1027
+ const rotated = await client.apiKeys.rotate(key.id);
1028
+ console.log(rotated.secret);
1029
+
1030
+ // Delete
1031
+ await client.apiKeys.delete(key.id);
63
1032
  ```
64
1033
 
65
- ### Streaming Worker (ConnectRPC)
1034
+ ### Organizations (Enterprise)
66
1035
 
67
- For low-latency bidirectional streaming via ConnectRPC:
1036
+ Requires an enterprise license.
68
1037
 
69
1038
  ```typescript
70
- import { createStreamingWorker } from '@ironflow/node/worker-streaming';
1039
+ const org = await client.orgs.create({ name: 'Acme Corp' });
1040
+ const orgs = await client.orgs.list();
1041
+ const fetched = await client.orgs.get(org.id);
1042
+ await client.orgs.update(org.id, { name: 'Acme Inc' });
1043
+ await client.orgs.delete(org.id);
1044
+ ```
71
1045
 
72
- const worker = createStreamingWorker({
1046
+ ### Roles (Enterprise)
1047
+
1048
+ ```typescript
1049
+ const role = await client.roles.create({ name: 'deployer', org_id: orgId });
1050
+ const roles = await client.roles.list(orgId); // optional org filter
1051
+ const fetched = await client.roles.get(role.id);
1052
+ await client.roles.update(role.id, { name: 'senior-deployer' });
1053
+
1054
+ // Assign/remove policies
1055
+ await client.roles.assignPolicy(role.id, policyId);
1056
+ await client.roles.removePolicy(role.id, policyId);
1057
+
1058
+ await client.roles.delete(role.id);
1059
+ ```
1060
+
1061
+ ### Policies (Enterprise)
1062
+
1063
+ ```typescript
1064
+ const policy = await client.policies.create({
1065
+ name: 'allow-emit',
1066
+ effect: 'allow',
1067
+ actions: 'emit:*',
1068
+ resources: '*',
1069
+ org_id: orgId,
1070
+ });
1071
+ const policies = await client.policies.list(orgId); // optional org filter
1072
+ const fetched = await client.policies.get(policy.id);
1073
+ await client.policies.update(policy.id, { name: 'allow-all-emit' });
1074
+ await client.policies.delete(policy.id);
1075
+ ```
1076
+
1077
+ ---
1078
+
1079
+ ## Subscriptions
1080
+
1081
+ Real-time event subscriptions via WebSocket with auto-reconnect, consumer groups, and ackable delivery.
1082
+
1083
+ ### SubscriptionClientConfig
1084
+
1085
+ | Field | Type | Default | Description |
1086
+ |-------|------|---------|-------------|
1087
+ | `serverUrl` | `string` | -- | **Required.** Server URL (e.g., `"http://localhost:9123"`). |
1088
+ | `apiKey` | `string` | -- | API key for authentication. |
1089
+ | `environment` | `string` | -- | Environment for scoped subscriptions. |
1090
+ | `autoReconnect` | `boolean` | `true` | Enable automatic reconnection. |
1091
+ | `reconnectDelay` | `number` | `1000` | Initial reconnect delay in ms. |
1092
+ | `maxReconnectDelay` | `number` | `30000` | Maximum reconnect delay in ms. |
1093
+ | `reconnectBackoff` | `number` | `1.5` | Reconnect backoff multiplier. |
1094
+ | `connectionTimeout` | `number` | `10000` | Connection timeout in ms. |
1095
+
1096
+ ### Basic subscription
1097
+
1098
+ ```typescript
1099
+ import { createSubscriptionClient } from '@ironflow/node';
1100
+
1101
+ const sub = createSubscriptionClient({
73
1102
  serverUrl: 'http://localhost:9123',
74
- functions: [processOrder],
75
- maxConcurrentJobs: 10,
1103
+ apiKey: process.env.IRONFLOW_API_KEY,
76
1104
  });
77
1105
 
78
- await worker.start();
1106
+ await sub.connect();
1107
+
1108
+ // Subscribe with callbacks
1109
+ const subscription = await sub.subscribe('events:order.*', {
1110
+ onEvent: (event) => {
1111
+ console.log('Topic:', event.topic);
1112
+ console.log('Data:', event.data);
1113
+ },
1114
+ onError: (err) => console.error(err.message),
1115
+ });
1116
+
1117
+ // Replay last N events on subscribe
1118
+ const replaySubscription = await sub.subscribe('system.run.>', {
1119
+ onEvent: (event) => console.log(event),
1120
+ replay: 100,
1121
+ includeMetadata: true,
1122
+ });
1123
+
1124
+ // Cleanup
1125
+ subscription.unsubscribe();
1126
+ sub.close();
1127
+ ```
1128
+
1129
+ ### Ackable subscriptions with consumer groups
1130
+
1131
+ For load-balanced processing with manual acknowledgment.
1132
+
1133
+ ```typescript
1134
+ const subscription = await sub.subscribe('events:order.*', {
1135
+ consumerGroup: 'order-processors',
1136
+ ackMode: 'manual',
1137
+ onEvent: async (event) => {
1138
+ try {
1139
+ await processOrder(event.data);
1140
+ await subscription.ack(event.eventId!);
1141
+ } catch (err) {
1142
+ // Negative ack with optional redeliver delay in ms
1143
+ await subscription.nak(event.eventId!, 5000);
1144
+ }
1145
+ },
1146
+ });
1147
+
1148
+ // Terminal ack - message will not be redelivered
1149
+ await subscription.term(eventId);
79
1150
  ```
80
1151
 
81
- ## Features
1152
+ ### Connection monitoring
1153
+
1154
+ ```typescript
1155
+ // Global connection state changes
1156
+ const unsubscribe = sub.onConnectionChange((state) => {
1157
+ console.log('Connection:', state);
1158
+ // state: "connecting" | "connected" | "disconnected" | "reconnecting"
1159
+ });
1160
+
1161
+ // Global error handler
1162
+ const unsubErr = sub.onError((error) => {
1163
+ console.error('Subscription error:', error.code, error.message);
1164
+ });
1165
+
1166
+ // Remove listeners
1167
+ unsubscribe();
1168
+ unsubErr();
1169
+ ```
82
1170
 
83
- - **Durable step execution** with automatic memoization
84
- - **Push mode** for serverless (Next.js, Lambda, Vercel)
85
- - **Pull mode** for long-running workers (REST polling or ConnectRPC streaming)
86
- - **Step primitives**: `run`, `sleep`, `sleepUntil`, `waitForEvent`, `parallel`, `map`
87
- - **Projections** for building read models from event streams
88
- - **Entity streams** for event sourcing with optimistic concurrency
89
- - **KV store** for distributed key-value storage with buckets, CAS (Compare-And-Swap), and TTL
90
- - **Config management** for centralized configuration
91
- - **Server-side client** for emitting events, managing runs, and inspecting workers
92
- - **Type-safe** API with full TypeScript support
1171
+ ---
93
1172
 
94
1173
  ## Projections
95
1174
 
96
- Build read models from event streams with managed (pure reducer) or external (side effects) mode:
1175
+ Projections build derived state from event streams. Two modes: **managed** (pure reducer maintaining state) and **external** (side effects).
1176
+
1177
+ ### ProjectionConfig
1178
+
1179
+ | Field | Type | Description |
1180
+ |-------|------|-------------|
1181
+ | `name` | `string` | **Required.** Unique projection name. |
1182
+ | `events` | `string[]` | **Required.** Events to subscribe to. |
1183
+ | `mode` | `"managed" \| "external"` | Auto-detected: `"managed"` if `initialState` is provided, else `"external"`. |
1184
+ | `initialState` | `() => TState` | Initial state factory (managed mode). |
1185
+ | `handler` | Function | Event handler (signature varies by mode). |
1186
+ | `maxRetries` | `number` | Maximum retries per event (default: 3). |
1187
+ | `batchSize` | `number` | Events per batch (default: 100). |
1188
+
1189
+ ### Managed projection (pure reducer)
1190
+
1191
+ The handler receives current state and an event, and returns the new state. Ironflow persists the state.
97
1192
 
98
1193
  ```typescript
99
1194
  import { createProjection, createWorker } from '@ironflow/node';
@@ -101,150 +1196,367 @@ import { createProjection, createWorker } from '@ironflow/node';
101
1196
  const orderTotals = createProjection({
102
1197
  name: 'order-totals',
103
1198
  events: ['order.placed', 'order.cancelled'],
104
- mode: 'managed',
105
1199
  initialState: () => ({ total: 0, count: 0 }),
106
1200
  handler: (state, event) => {
107
1201
  if (event.name === 'order.placed') {
108
- return { total: state.total + event.data.amount, count: state.count + 1 };
1202
+ return {
1203
+ total: state.total + event.data.amount,
1204
+ count: state.count + 1,
1205
+ };
1206
+ }
1207
+ if (event.name === 'order.cancelled') {
1208
+ return {
1209
+ total: state.total - event.data.amount,
1210
+ count: state.count - 1,
1211
+ };
109
1212
  }
110
- return { total: state.total - event.data.amount, count: state.count - 1 };
1213
+ return state;
111
1214
  },
112
1215
  });
1216
+ ```
1217
+
1218
+ ### External projection (side effects)
1219
+
1220
+ The handler receives the event and a context object. Use for sending emails, updating external databases, calling APIs.
1221
+
1222
+ ```typescript
1223
+ const emailNotifier = createProjection({
1224
+ name: 'email-notifier',
1225
+ events: ['order.completed'],
1226
+ handler: async (event, ctx) => {
1227
+ await sendEmail(event.data.email, 'Your order is complete!');
1228
+ },
1229
+ });
1230
+ ```
113
1231
 
114
- // Run projections alongside functions in a worker
1232
+ ### Running projections
1233
+
1234
+ Projections run inside a worker, not in push mode.
1235
+
1236
+ ```typescript
115
1237
  const worker = createWorker({
116
1238
  serverUrl: 'http://localhost:9123',
117
1239
  functions: [processOrder],
118
- projections: [orderTotals],
1240
+ projections: [orderTotals, emailNotifier],
119
1241
  });
120
- ```
121
1242
 
122
- ## Server-Side Client
1243
+ await worker.start();
1244
+ ```
123
1245
 
124
- Use the client to interact with Ironflow from your backend:
1246
+ ---
125
1247
 
126
- ```typescript
127
- import { createClient } from '@ironflow/node';
1248
+ ## Webhooks
128
1249
 
129
- const client = createClient({ serverUrl: 'http://localhost:9123' });
1250
+ Receive and transform external HTTP events from third-party services.
130
1251
 
131
- // Emit events
132
- await client.emit('order.placed', { orderId: '123', amount: 99.99 });
1252
+ ### WebhookConfig
133
1253
 
134
- // Trigger workflows
135
- const run = await client.trigger('process-order', { data: { orderId: '123' } });
1254
+ | Field | Type | Description |
1255
+ |-------|------|-------------|
1256
+ | `id` | `string` | Webhook source identifier (used in URL path). |
1257
+ | `verify` | `(req) => void \| Promise<void>` | Verification function. Throw to reject. |
1258
+ | `transform` | `(payload) => WebhookEvent` | Transform raw payload to an Ironflow event. |
136
1259
 
137
- // Run management
138
- const runInfo = await client.getRun(run.id);
139
- const runs = await client.listRuns({ functionId: 'process-order', status: 'failed' });
140
- await client.cancelRun(run.id, 'no longer needed');
141
- await client.retryRun(run.id);
142
- await client.resumeRun(run.id, 'validate');
1260
+ ### Stripe webhook example
143
1261
 
144
- // Hot-patch a step output (debugging)
145
- await client.patchStep(stepId, { correctedValue: 42 }, 'fix bad data');
1262
+ ```typescript
1263
+ import { createWebhook, serve } from '@ironflow/node';
1264
+ import Stripe from 'stripe';
1265
+
1266
+ const stripe = new Stripe(process.env.STRIPE_SECRET_KEY!);
1267
+
1268
+ const stripeWebhook = createWebhook({
1269
+ id: 'stripe',
1270
+ verify: async (req) => {
1271
+ const sig = req.headers['stripe-signature'];
1272
+ if (!sig) throw new Error('Missing stripe-signature header');
1273
+ // Throws if invalid
1274
+ stripe.webhooks.constructEvent(req.body, sig, process.env.STRIPE_WEBHOOK_SECRET!);
1275
+ },
1276
+ transform: (payload) => ({
1277
+ name: `stripe.${payload.type}`,
1278
+ data: payload.data.object,
1279
+ idempotencyKey: payload.id,
1280
+ }),
1281
+ });
146
1282
 
147
- // Server inspection
148
- const functions = await client.listFunctions();
149
- const workers = await client.listWorkers();
150
- const health = await client.health();
1283
+ // Webhook endpoint: POST /api/ironflow/webhooks/stripe
1284
+ export const POST = serve({
1285
+ functions: [processOrder],
1286
+ webhooks: [stripeWebhook],
1287
+ serverUrl: process.env.IRONFLOW_SERVER_URL,
1288
+ });
151
1289
  ```
152
1290
 
153
- ## Entity Streams (Event Sourcing)
1291
+ The webhook URL path is derived from the `id`: `POST /api/ironflow/webhooks/<id>`.
1292
+
1293
+ ---
1294
+
1295
+ ## Event Versioning (Upcasters)
1296
+
1297
+ Upcasters transform event data from older schema versions to newer ones. They are applied SDK-side when reading events.
1298
+
1299
+ ### defineEvent and EventDefinitionRegistry
154
1300
 
155
1301
  ```typescript
156
- const client = createClient({ serverUrl: 'http://localhost:9123' });
1302
+ import { defineEvent, createEventDefinitionRegistry } from '@ironflow/core';
157
1303
 
158
- // Append events with optimistic concurrency
159
- await client.streams.append('order-123', {
160
- eventName: 'item.added',
161
- data: { sku: 'ABC', qty: 2 },
162
- expectedVersion: 3,
1304
+ // Define event versions with upcasters
1305
+ const orderPlacedV1 = defineEvent({
1306
+ name: 'order.placed',
1307
+ version: 1,
163
1308
  });
164
1309
 
165
- // Read events from a stream
166
- const events = await client.streams.read('order-123', {
167
- direction: 'forward',
168
- limit: 50,
1310
+ const orderPlacedV2 = defineEvent({
1311
+ name: 'order.placed',
1312
+ version: 2,
1313
+ upcast: (data) => ({
1314
+ ...(data as Record<string, unknown>),
1315
+ // v2 adds a currency field with a default
1316
+ currency: (data as Record<string, unknown>).currency ?? 'USD',
1317
+ }),
169
1318
  });
170
1319
 
171
- // Get stream metadata
172
- const info = await client.streams.getInfo('order-123');
173
- ```
1320
+ const orderPlacedV3 = defineEvent({
1321
+ name: 'order.placed',
1322
+ version: 3,
1323
+ upcast: (data) => {
1324
+ const d = data as Record<string, unknown>;
1325
+ return {
1326
+ ...d,
1327
+ // v3 renames "total" to "amount"
1328
+ amount: d.total ?? d.amount,
1329
+ total: undefined,
1330
+ };
1331
+ },
1332
+ });
174
1333
 
175
- ## KV Store
1334
+ // Register all versions
1335
+ const registry = createEventDefinitionRegistry();
1336
+ registry.register(orderPlacedV1);
1337
+ registry.register(orderPlacedV2);
1338
+ registry.register(orderPlacedV3);
1339
+
1340
+ // Pass to worker or serve for automatic upcasting
1341
+ const worker = createWorker({
1342
+ serverUrl: 'http://localhost:9123',
1343
+ functions: [processOrder],
1344
+ eventDefinitions: registry,
1345
+ });
1346
+ ```
176
1347
 
177
- Distributed key-value storage with bucket management:
1348
+ ### UpcasterRegistry (lower-level)
178
1349
 
179
1350
  ```typescript
180
- const kv = client.kv();
181
-
182
- // Bucket management
183
- await kv.createBucket({ name: 'sessions', ttl: 3600 });
184
- const buckets = await kv.listBuckets();
1351
+ import { createUpcasterRegistry } from '@ironflow/core';
185
1352
 
186
- // Key operations
187
- const bucket = kv.bucket('sessions');
188
- await bucket.put('user-123', { token: 'abc', role: 'admin' });
189
- const entry = await bucket.get('user-123');
1353
+ const upcasters = createUpcasterRegistry();
190
1354
 
191
- // Compare-and-swap update
192
- await bucket.update('user-123', { token: 'xyz', role: 'admin' }, entry.revision);
1355
+ upcasters.register('order.placed', 1, 2, (data) => ({
1356
+ ...(data as Record<string, unknown>),
1357
+ currency: 'USD',
1358
+ }));
193
1359
 
194
- // Create only if key doesn't exist
195
- await bucket.create('user-456', { token: 'def' });
1360
+ upcasters.register('order.placed', 2, 3, (data) => {
1361
+ const d = data as Record<string, unknown>;
1362
+ return { ...d, amount: d.total, total: undefined };
1363
+ });
196
1364
 
197
- // List keys with wildcard filtering
198
- const keys = await bucket.listKeys({ pattern: 'user-*' });
1365
+ // Manually upcast
1366
+ const v3Data = upcasters.upcast('order.placed', v1Data, 1, 3);
199
1367
  ```
200
1368
 
201
- ## Config Management
1369
+ The upcaster chain must be complete. If v2->v3 is registered but v1->v2 is missing, upcasting from v1 to v3 throws an error.
1370
+
1371
+ ---
1372
+
1373
+ ## Error Handling
202
1374
 
203
- Centralized configuration store:
1375
+ All error classes are re-exported from `@ironflow/core`:
1376
+
1377
+ | Error Class | Description |
1378
+ |-------------|-------------|
1379
+ | `IronflowError` | Base error class. Has `code`, `retryable`, `details` properties. |
1380
+ | `StepError` | Step execution failure. Has `stepId`, `stepName`. |
1381
+ | `TimeoutError` | Request or operation timeout. |
1382
+ | `ValidationError` | Input validation failure. |
1383
+ | `SchemaValidationError` | Zod schema validation failure. |
1384
+ | `SignatureError` | Request signature verification failure. |
1385
+ | `FunctionNotFoundError` | Function not found. |
1386
+ | `RunNotFoundError` | Run not found. |
1387
+ | `NonRetryableError` | Marks an error as non-retryable (triggers compensations). |
1388
+ | `UnauthenticatedError` | Missing or invalid authentication (HTTP 401). |
1389
+ | `UnauthorizedError` | Insufficient permissions (HTTP 403). |
1390
+ | `EnterpriseRequiredError` | Feature requires enterprise license (HTTP 402). |
1391
+
1392
+ ### Utility functions
204
1393
 
205
1394
  ```typescript
206
- const config = client.config();
1395
+ import { isRetryable, isIronflowError, NonRetryableError } from '@ironflow/node';
1396
+
1397
+ // Check if an error is retryable
1398
+ try {
1399
+ await step.run('api-call', async () => { /* ... */ });
1400
+ } catch (err) {
1401
+ if (isRetryable(err)) {
1402
+ console.log('Will be retried');
1403
+ }
1404
+ }
207
1405
 
208
- await config.set('app-settings', { theme: 'dark', locale: 'en' });
209
- const settings = await config.get('app-settings');
210
- await config.patch('app-settings', { locale: 'fr' }); // Shallow merge
211
- const all = await config.list();
212
- await config.delete('app-settings');
1406
+ // Check if an error is an Ironflow error
1407
+ if (isIronflowError(err)) {
1408
+ console.log(err.code, err.retryable);
1409
+ }
1410
+
1411
+ // Mark an error as non-retryable (stops retries, triggers compensations)
1412
+ throw new NonRetryableError('Payment declined - do not retry');
213
1413
  ```
214
1414
 
215
- ## Key APIs
216
-
217
- | API | Description |
218
- |-----|-------------|
219
- | `ironflow.createFunction(config, handler)` | Define a workflow function |
220
- | `createFunction(config, handler)` | Shorthand function definition |
221
- | `serve({ functions })` | Create HTTP handler for push mode |
222
- | `createWorker({ functions, projections? })` | Create pull mode worker |
223
- | `createStreamingWorker({ functions })` | Create ConnectRPC streaming worker |
224
- | `createProjection(config)` | Define a projection |
225
- | `createClient(config)` | Create server-side client |
226
- | `step.run(id, fn)` | Execute memoized step |
227
- | `step.sleep(id, duration)` | Durable sleep |
228
- | `step.sleepUntil(id, timestamp)` | Sleep until specific time |
229
- | `step.waitForEvent(id, options)` | Wait for external event |
230
- | `step.parallel(name, branches, options?)` | Execute branches in parallel |
231
- | `step.map(name, items, fn, options?)` | Map over items in parallel |
1415
+ ---
232
1416
 
233
1417
  ## Environment Variables
234
1418
 
235
1419
  | Variable | Description | Default |
236
1420
  |----------|-------------|---------|
237
- | `IRONFLOW_SERVER_URL` | Server URL | `http://localhost:9123` |
238
- | `IRONFLOW_SIGNING_KEY` | Request signing key | - |
239
- | `IRONFLOW_LOG_LEVEL` | Log level | `info` |
1421
+ | `IRONFLOW_SERVER_URL` | Ironflow server URL | `http://localhost:9123` |
1422
+ | `IRONFLOW_SIGNING_KEY` | HMAC-SHA256 signing key for push mode verification | -- |
1423
+ | `IRONFLOW_API_KEY` | API key for authentication (worker and client) | -- |
1424
+ | `IRONFLOW_LOG_LEVEL` | Log level: `debug`, `info`, `warn`, `error` | `info` |
1425
+ | `IRONFLOW_ENV` | Target environment name | `default` |
1426
+
1427
+ ---
1428
+
1429
+ ## Testing
1430
+
1431
+ The `@ironflow/node/test` entry point provides a `createTestClient` for unit testing functions without a running server.
1432
+
1433
+ ### Import
1434
+
1435
+ ```typescript
1436
+ import { createTestClient } from '@ironflow/node/test';
1437
+ ```
1438
+
1439
+ ### TestClient interface
1440
+
1441
+ | Method | Description |
1442
+ |--------|-------------|
1443
+ | `mockStep(name, fn)` | Mock a `step.run()` call by name. |
1444
+ | `mockInvoke(functionId, fn)` | Mock a `step.invoke()` or `step.invokeAsync()` call. |
1445
+ | `sendEvent(eventName, data)` | Pre-register an event for `step.waitForEvent()`. |
1446
+ | `emit(eventName, data)` | Run the function triggered by this event. Returns `TestRun`. |
1447
+
1448
+ ### TestRun interface
1449
+
1450
+ | Property/Method | Type | Description |
1451
+ |-----------------|------|-------------|
1452
+ | `status` | `"completed" \| "failed"` | Run outcome. |
1453
+ | `output` | `unknown` | Function return value (if completed). |
1454
+ | `error` | `Error` | Error (if failed). |
1455
+ | `steps` | `TestStep[]` | All executed steps. |
1456
+ | `compensationsRan` | `string[]` | Step names whose compensations executed. |
1457
+ | `stepOutput(name)` | `unknown` | Get output of a specific step by name. |
1458
+
1459
+ ### Example
1460
+
1461
+ ```typescript
1462
+ import { describe, it, expect } from 'vitest';
1463
+ import { createFunction } from '@ironflow/node';
1464
+ import { createTestClient } from '@ironflow/node/test';
1465
+
1466
+ const processOrder = createFunction(
1467
+ { id: 'process-order', triggers: [{ event: 'order.placed' }] },
1468
+ async ({ event, step }) => {
1469
+ const validated = await step.run('validate', async () => {
1470
+ return { orderId: event.data.orderId, valid: true };
1471
+ });
1472
+
1473
+ const approval = await step.waitForEvent('wait-approval', {
1474
+ event: 'approval.received',
1475
+ });
1476
+
1477
+ const result = await step.invoke('send-confirmation', {
1478
+ orderId: validated.orderId,
1479
+ });
1480
+
1481
+ return { orderId: validated.orderId, confirmed: true };
1482
+ }
1483
+ );
1484
+
1485
+ describe('processOrder', () => {
1486
+ it('completes successfully', async () => {
1487
+ const t = createTestClient({ functions: [processOrder] });
1488
+
1489
+ // Mock steps
1490
+ t.mockStep('validate', () => ({ orderId: '123', valid: true }));
1491
+ t.mockInvoke('send-confirmation', (input) => ({ sent: true }));
1492
+
1493
+ // Pre-register events for waitForEvent
1494
+ t.sendEvent('approval.received', { approved: true });
1495
+
1496
+ // Run function
1497
+ const run = await t.emit('order.placed', { orderId: '123' });
1498
+
1499
+ expect(run.status).toBe('completed');
1500
+ expect(run.output).toEqual({ orderId: '123', confirmed: true });
1501
+ expect(run.stepOutput('validate')).toEqual({ orderId: '123', valid: true });
1502
+ });
1503
+ });
1504
+ ```
1505
+
1506
+ ### Testing compensations
1507
+
1508
+ ```typescript
1509
+ import { NonRetryableError } from '@ironflow/node';
1510
+
1511
+ const transferFunds = createFunction(
1512
+ { id: 'transfer', triggers: [{ event: 'transfer.requested' }] },
1513
+ async ({ event, step }) => {
1514
+ await step.run('debit', async () => ({ ref: 'D1' }));
1515
+ step.compensate('debit', async () => { /* refund */ });
1516
+
1517
+ await step.run('credit', async () => {
1518
+ throw new NonRetryableError('Insufficient funds');
1519
+ });
1520
+ }
1521
+ );
1522
+
1523
+ describe('transferFunds', () => {
1524
+ it('runs compensations on failure', async () => {
1525
+ const t = createTestClient({ functions: [transferFunds] });
1526
+ t.mockStep('debit', () => ({ ref: 'D1' }));
1527
+ t.mockStep('credit', () => { throw new NonRetryableError('Insufficient funds'); });
1528
+
1529
+ const run = await t.emit('transfer.requested', {});
1530
+
1531
+ expect(run.status).toBe('failed');
1532
+ expect(run.compensationsRan).toContain('debit');
1533
+ });
1534
+ });
1535
+ ```
1536
+
1537
+ **Note:** Every `step.run()` and `step.invoke()` call **must** have a corresponding mock registered via `mockStep()` or `mockInvoke()`. Unmocked steps throw with a helpful error message.
1538
+
1539
+ ---
240
1540
 
241
- ## Requirements
1541
+ ## API Summary
242
1542
 
243
- - Node.js 22+
1543
+ | Export | Description |
1544
+ |--------|-------------|
1545
+ | `createFunction(config, handler)` | Define a workflow function. |
1546
+ | `serve(config)` | Create HTTP handler for push mode. |
1547
+ | `createWorker(config)` | Create pull mode worker (REST polling). |
1548
+ | `createStreamingWorker(config)` | Create pull mode worker (ConnectRPC streaming). Import from `@ironflow/node/worker-streaming`. |
1549
+ | `createProjection(config)` | Define a projection. |
1550
+ | `createClient(config)` | Create server-side HTTP client. |
1551
+ | `createSubscriptionClient(config)` | Create WebSocket subscription client. |
1552
+ | `createWebhook(config)` | Define a webhook source. |
1553
+ | `createTestClient(config)` | Create test client. Import from `@ironflow/node/test`. |
1554
+ | `createSecretsClient(secrets)` | Create a read-only secrets accessor. |
1555
+ | `ironflow` | Singleton with `ironflow.createFunction()`. |
244
1556
 
245
1557
  ## Documentation
246
1558
 
247
- For the full API reference, see the [Node Package Documentation](https://ironflow.dev/docs/api-reference/js-sdk/node).
1559
+ Full documentation: [https://github.com/sahina/ironflow/tree/main/docs](https://github.com/sahina/ironflow/tree/main/docs)
248
1560
 
249
1561
  ## License
250
1562