@emmvish/stable-request 2.0.0 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1309 -1595
- package/dist/core/stable-function.js +1 -1
- package/dist/core/stable-function.js.map +1 -1
- package/dist/core/stable-request.js +1 -1
- package/dist/core/stable-request.js.map +1 -1
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -1,2048 +1,1762 @@
|
|
|
1
1
|
# @emmvish/stable-request
|
|
2
2
|
|
|
3
|
-
A production-grade
|
|
3
|
+
A production-grade TypeScript framework for resilient API integrations, batch processing, and orchestrating complex workflows with deterministic error handling, type safety, and comprehensive observability.
|
|
4
4
|
|
|
5
|
-
##
|
|
5
|
+
## Table of Contents
|
|
6
6
|
|
|
7
7
|
- [Overview](#overview)
|
|
8
|
-
- [
|
|
9
|
-
- [
|
|
10
|
-
- [
|
|
11
|
-
- [
|
|
12
|
-
- [
|
|
13
|
-
- [
|
|
14
|
-
- [
|
|
15
|
-
|
|
16
|
-
- [
|
|
17
|
-
- [
|
|
18
|
-
- [
|
|
19
|
-
- [
|
|
20
|
-
- [
|
|
21
|
-
|
|
22
|
-
- [
|
|
23
|
-
- [MetricsAggregator Utility](#metricsaggregator-utility)
|
|
24
|
-
- [Workflow Execution Patterns](#workflow-execution-patterns)
|
|
25
|
-
- [Sequential and Concurrent Phases](#sequential-and-concurrent-phases)
|
|
26
|
-
- [Mixed Execution Mode](#mixed-execution-mode)
|
|
8
|
+
- [Core Concepts](#core-concepts)
|
|
9
|
+
- [Core Modules](#core-modules)
|
|
10
|
+
- [stableRequest](#stablerequest)
|
|
11
|
+
- [stableFunction](#stablefunction)
|
|
12
|
+
- [stableApiGateway](#stableapigateway)
|
|
13
|
+
- [stableWorkflow](#stableworkflow)
|
|
14
|
+
- [stableWorkflowGraph](#stableworkflowgraph)
|
|
15
|
+
- [Resilience Mechanisms](#resilience-mechanisms)
|
|
16
|
+
- [Retry Strategies](#retry-strategies)
|
|
17
|
+
- [Circuit Breaker](#circuit-breaker)
|
|
18
|
+
- [Caching](#caching)
|
|
19
|
+
- [Rate Limiting](#rate-limiting)
|
|
20
|
+
- [Concurrency Limiting](#concurrency-limiting)
|
|
21
|
+
- [Workflow Patterns](#workflow-patterns)
|
|
22
|
+
- [Sequential & Concurrent Phases](#sequential--concurrent-phases)
|
|
27
23
|
- [Non-Linear Workflows](#non-linear-workflows)
|
|
28
24
|
- [Branched Workflows](#branched-workflows)
|
|
29
|
-
- [
|
|
25
|
+
- [Graph-Based Workflows](#graph-based-workflows)
|
|
26
|
+
- [Configuration & State](#configuration--state)
|
|
30
27
|
- [Config Cascading](#config-cascading)
|
|
31
|
-
- [
|
|
32
|
-
|
|
33
|
-
- [
|
|
34
|
-
- [
|
|
28
|
+
- [Shared & State Buffers](#shared--state-buffers)
|
|
29
|
+
- [Hooks & Observability](#hooks--observability)
|
|
30
|
+
- [Pre-Execution Hooks](#pre-execution-hooks)
|
|
31
|
+
- [Analysis Hooks](#analysis-hooks)
|
|
32
|
+
- [Handler Hooks](#handler-hooks)
|
|
33
|
+
- [Decision Hooks](#decision-hooks)
|
|
34
|
+
- [Metrics & Logging](#metrics--logging)
|
|
35
|
+
- [Advanced Features](#advanced-features)
|
|
35
36
|
- [Trial Mode](#trial-mode)
|
|
36
|
-
- [
|
|
37
|
-
- [
|
|
37
|
+
- [State Persistence](#state-persistence)
|
|
38
|
+
- [Mixed Request & Function Phases](#mixed-request--function-phases)
|
|
39
|
+
- [Best Practices](#best-practices)
|
|
40
|
+
|
|
41
|
+
---
|
|
38
42
|
|
|
39
43
|
## Overview
|
|
40
44
|
|
|
41
|
-
|
|
45
|
+
**@emmvish/stable-request** evolved from a focused library for resilient API calls to a comprehensive workflow execution framework. Originally addressing **API integration challenges** via `stableRequest` function, it expanded to include:
|
|
42
46
|
|
|
43
|
-
- **
|
|
44
|
-
- **
|
|
45
|
-
- **
|
|
46
|
-
- **
|
|
47
|
-
- **Performance Optimization**: Response caching, rate limiting, and concurrency control to maximize efficiency
|
|
48
|
-
- **Type Safety**: Full TypeScript support with 40+ exported types
|
|
47
|
+
- **Batch orchestration** via `stableApiGateway` for processing groups of mixed requests/functions
|
|
48
|
+
- **Phased workflows** via `stableWorkflow` for array-based multi-phase execution with dynamic control flow
|
|
49
|
+
- **Graph-based workflows** via `stableWorkflowGraph` for DAG execution with higher parallelism
|
|
50
|
+
- **Generic function execution** via `stableFunction`, inheriting all resilience guards
|
|
49
51
|
|
|
50
|
-
|
|
52
|
+
All four execution modes support the same resilience stack: retries, jitter, circuit breaking, caching, rate/concurrency limits, config cascading, shared buffers, trial mode, comprehensive hooks, and metrics. This uniformity makes it trivial to compose requests and functions in any topology.
|
|
51
53
|
|
|
52
|
-
|
|
53
|
-
- **Orchestrate complex API workflows** with dependencies between steps
|
|
54
|
-
- **Handle unreliable APIs** with intelligent retry and fallback mechanisms
|
|
55
|
-
- **Prevent cascade failures** when downstream services fail
|
|
56
|
-
- **Optimize performance** by caching responses and controlling request rates
|
|
57
|
-
- **Monitor and debug** complex request flows in production
|
|
58
|
-
- **Implement conditional logic** based on API responses (branching, looping)
|
|
54
|
+
---
|
|
59
55
|
|
|
60
|
-
|
|
56
|
+
## Core Concepts
|
|
61
57
|
|
|
62
|
-
|
|
58
|
+
### Resilience as Default
|
|
63
59
|
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
60
|
+
Every execution—whether a single request, a pure function, or an entire workflow—inherits built-in resilience:
|
|
61
|
+
|
|
62
|
+
- **Retries** with configurable backoff strategies (FIXED, LINEAR, EXPONENTIAL)
|
|
63
|
+
- **Jitter** to prevent thundering herd
|
|
64
|
+
- **Circuit breaker** to fail fast and protect downstream systems
|
|
65
|
+
- **Caching** for idempotent read operations
|
|
66
|
+
- **Rate & concurrency limits** to respect external constraints
|
|
67
|
+
|
|
68
|
+
### Type Safety
|
|
69
|
+
|
|
70
|
+
All examples in this guide use TypeScript generics for type-safe request/response data and function arguments/returns. Analyzers validate shapes at runtime; TypeScript ensures compile-time safety.
|
|
71
|
+
|
|
72
|
+
### Config Cascading
|
|
73
|
+
|
|
74
|
+
Global defaults → group overrides → phase overrides → branch overrides → item overrides. Lower levels always win, preventing repetition while maintaining expressiveness.
|
|
75
|
+
|
|
76
|
+
### Shared State
|
|
77
|
+
|
|
78
|
+
Workflows and gateways support `sharedBuffer` for passing computed state across phases/branches/items without global state.
|
|
67
79
|
|
|
68
|
-
|
|
80
|
+
---
|
|
69
81
|
|
|
70
|
-
##
|
|
82
|
+
## Core Modules
|
|
71
83
|
|
|
72
|
-
###
|
|
84
|
+
### stableRequest
|
|
73
85
|
|
|
74
|
-
|
|
86
|
+
Single API call with resilience, type-safe request and response types.
|
|
75
87
|
|
|
76
88
|
```typescript
|
|
77
|
-
import { stableRequest,
|
|
89
|
+
import { stableRequest, REQUEST_METHODS, VALID_REQUEST_PROTOCOLS } from '@emmvish/stable-request';
|
|
78
90
|
|
|
79
|
-
|
|
91
|
+
type User = { id: number; name: string };
|
|
92
|
+
|
|
93
|
+
const result = await stableRequest<unknown, User>({
|
|
80
94
|
reqData: {
|
|
95
|
+
method: REQUEST_METHODS.GET,
|
|
96
|
+
protocol: VALID_REQUEST_PROTOCOLS.HTTPS,
|
|
81
97
|
hostname: 'api.example.com',
|
|
82
|
-
path: '/users/
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
98
|
+
path: '/users/1'
|
|
99
|
+
},
|
|
100
|
+
resReq: true,
|
|
101
|
+
attempts: 3,
|
|
102
|
+
wait: 500,
|
|
103
|
+
jitter: 100,
|
|
104
|
+
cache: { enabled: true, ttl: 5000 },
|
|
105
|
+
rateLimit: { maxRequests: 10, windowMs: 1000 },
|
|
106
|
+
maxConcurrentRequests: 5,
|
|
107
|
+
responseAnalyzer: ({ data }) => {
|
|
108
|
+
return typeof data === 'object' && data !== null && 'id' in data;
|
|
109
|
+
},
|
|
110
|
+
handleSuccessfulAttemptData: ({ successfulAttemptData }) => {
|
|
111
|
+
console.log(`User loaded: ${successfulAttemptData.data.name}`);
|
|
112
|
+
}
|
|
113
|
+
});
|
|
114
|
+
|
|
115
|
+
if (result.success) {
|
|
116
|
+
console.log(result.data.name, result.metrics.totalAttempts);
|
|
117
|
+
} else {
|
|
118
|
+
console.error(result.error);
|
|
119
|
+
}
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
**Key responsibilities:**
|
|
123
|
+
- Execute a single HTTP request with automatic retry and backoff
|
|
124
|
+
- Validate response shape via analyzer; retry if invalid
|
|
125
|
+
- Cache successful responses with TTL
|
|
126
|
+
- Apply rate and concurrency limits
|
|
127
|
+
- Throw or gracefully suppress errors via finalErrorAnalyzer
|
|
128
|
+
- Collect attempt metrics and infra dashboards (circuit breaker, cache, rate limiter state)
|
|
129
|
+
|
|
130
|
+
### stableFunction
|
|
131
|
+
|
|
132
|
+
Generic async/sync function execution with identical resilience.
|
|
133
|
+
|
|
134
|
+
```typescript
|
|
135
|
+
import { stableFunction, RETRY_STRATEGIES } from '@emmvish/stable-request';
|
|
136
|
+
|
|
137
|
+
type ComputeArgs = [number, number];
|
|
138
|
+
type ComputeResult = number;
|
|
139
|
+
|
|
140
|
+
const multiply = (a: number, b: number) => a * b;
|
|
141
|
+
|
|
142
|
+
const result = await stableFunction<ComputeArgs, ComputeResult>({
|
|
143
|
+
fn: multiply,
|
|
144
|
+
args: [5, 3],
|
|
145
|
+
returnResult: true,
|
|
146
|
+
attempts: 2,
|
|
147
|
+
wait: 100,
|
|
88
148
|
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL,
|
|
89
|
-
|
|
149
|
+
responseAnalyzer: ({ data }) => data > 0,
|
|
150
|
+
cache: { enabled: true, ttl: 10000 }
|
|
90
151
|
});
|
|
91
152
|
|
|
92
|
-
|
|
153
|
+
if (result.success) {
|
|
154
|
+
console.log('Result:', result.data); // 15
|
|
155
|
+
}
|
|
93
156
|
```
|
|
94
157
|
|
|
95
|
-
|
|
158
|
+
**Key responsibilities:**
|
|
159
|
+
- Execute any async or sync function with typed arguments and return
|
|
160
|
+
- Support argument-based cache key generation
|
|
161
|
+
- Retry on error or analyzer rejection
|
|
162
|
+
- Enforce success criteria via analyzer
|
|
163
|
+
- Optionally suppress exceptions
|
|
164
|
+
|
|
165
|
+
### stableApiGateway
|
|
96
166
|
|
|
97
|
-
|
|
167
|
+
Batch orchestration of mixed requests and functions.
|
|
98
168
|
|
|
99
169
|
```typescript
|
|
100
|
-
import {
|
|
170
|
+
import {
|
|
171
|
+
stableApiGateway,
|
|
172
|
+
REQUEST_METHODS,
|
|
173
|
+
VALID_REQUEST_PROTOCOLS,
|
|
174
|
+
RequestOrFunction
|
|
175
|
+
} from '@emmvish/stable-request';
|
|
176
|
+
import type { API_GATEWAY_ITEM } from '@emmvish/stable-request';
|
|
101
177
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
178
|
+
type ApiResponse = { id: number; value: string };
|
|
179
|
+
|
|
180
|
+
const items: API_GATEWAY_ITEM[] = [
|
|
181
|
+
{
|
|
182
|
+
type: RequestOrFunction.REQUEST,
|
|
183
|
+
request: {
|
|
184
|
+
id: 'fetch-user',
|
|
185
|
+
requestOptions: {
|
|
186
|
+
reqData: {
|
|
187
|
+
method: REQUEST_METHODS.GET,
|
|
188
|
+
protocol: VALID_REQUEST_PROTOCOLS.HTTPS,
|
|
189
|
+
hostname: 'api.example.com',
|
|
190
|
+
path: '/users/1'
|
|
191
|
+
},
|
|
192
|
+
resReq: true,
|
|
193
|
+
attempts: 3
|
|
194
|
+
}
|
|
195
|
+
}
|
|
116
196
|
},
|
|
117
|
-
{
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
197
|
+
{
|
|
198
|
+
type: RequestOrFunction.FUNCTION,
|
|
199
|
+
function: {
|
|
200
|
+
id: 'transform-data',
|
|
201
|
+
functionOptions: {
|
|
202
|
+
fn: (user?: any) => ({
|
|
203
|
+
transformed: user?.name?.toUpperCase() || 'UNKNOWN'
|
|
204
|
+
}),
|
|
205
|
+
args: [],
|
|
206
|
+
returnResult: true,
|
|
207
|
+
attempts: 1
|
|
208
|
+
}
|
|
209
|
+
}
|
|
123
210
|
}
|
|
124
211
|
];
|
|
125
212
|
|
|
126
|
-
const
|
|
127
|
-
concurrentExecution: true,
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
commonWait: 500
|
|
213
|
+
const responses = await stableApiGateway<unknown, ApiResponse>(items, {
|
|
214
|
+
concurrentExecution: true,
|
|
215
|
+
stopOnFirstError: false,
|
|
216
|
+
sharedBuffer: {},
|
|
217
|
+
commonAttempts: 2,
|
|
218
|
+
commonWait: 300,
|
|
219
|
+
maxConcurrentRequests: 3
|
|
134
220
|
});
|
|
135
221
|
|
|
136
|
-
|
|
137
|
-
console.log(
|
|
222
|
+
responses.forEach((resp, i) => {
|
|
223
|
+
console.log(`Item ${i}: success=${resp.success}`);
|
|
138
224
|
});
|
|
139
225
|
```
|
|
140
226
|
|
|
141
|
-
|
|
227
|
+
**Key responsibilities:**
|
|
228
|
+
- Execute a batch of requests and functions concurrently or sequentially
|
|
229
|
+
- Apply global, group-level, and item-level config overrides
|
|
230
|
+
- Maintain shared buffer across items for state passing
|
|
231
|
+
- Stop on first error or continue despite failures
|
|
232
|
+
- Collect per-item and aggregate metrics
|
|
233
|
+
- Support request grouping with group-specific config
|
|
234
|
+
|
|
235
|
+
### stableWorkflow
|
|
142
236
|
|
|
143
|
-
|
|
237
|
+
Phased array-based workflows with sequential/concurrent phases, mixed items, and non-linear control flow.
|
|
144
238
|
|
|
145
239
|
```typescript
|
|
146
|
-
import { stableWorkflow, PHASE_DECISION_ACTIONS
|
|
240
|
+
import { stableWorkflow, PHASE_DECISION_ACTIONS } from '@emmvish/stable-request';
|
|
241
|
+
import type { STABLE_WORKFLOW_PHASE } from '@emmvish/stable-request';
|
|
147
242
|
|
|
148
|
-
const phases = [
|
|
243
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
149
244
|
{
|
|
150
|
-
id: '
|
|
245
|
+
id: 'fetch-data',
|
|
151
246
|
requests: [
|
|
152
|
-
{
|
|
153
|
-
id: '
|
|
154
|
-
requestOptions: {
|
|
155
|
-
reqData: {
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
}
|
|
247
|
+
{
|
|
248
|
+
id: 'api-call',
|
|
249
|
+
requestOptions: {
|
|
250
|
+
reqData: {
|
|
251
|
+
hostname: 'api.example.com',
|
|
252
|
+
path: '/data'
|
|
253
|
+
},
|
|
254
|
+
resReq: true,
|
|
255
|
+
attempts: 3
|
|
256
|
+
}
|
|
162
257
|
}
|
|
163
258
|
]
|
|
164
259
|
},
|
|
165
260
|
{
|
|
166
|
-
id: '
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
{
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
261
|
+
id: 'process-and-audit',
|
|
262
|
+
markConcurrentPhase: true, // Run requests concurrently within phase
|
|
263
|
+
items: [
|
|
264
|
+
{
|
|
265
|
+
type: RequestOrFunction.FUNCTION,
|
|
266
|
+
function: {
|
|
267
|
+
id: 'process',
|
|
268
|
+
functionOptions: {
|
|
269
|
+
fn: async (data?: any) => ({ processed: !!data }),
|
|
270
|
+
args: [],
|
|
271
|
+
returnResult: true
|
|
272
|
+
}
|
|
273
|
+
}
|
|
274
|
+
},
|
|
275
|
+
{
|
|
276
|
+
type: RequestOrFunction.FUNCTION,
|
|
277
|
+
function: {
|
|
278
|
+
id: 'audit-log',
|
|
279
|
+
functionOptions: {
|
|
280
|
+
fn: () => 'logged',
|
|
281
|
+
args: [],
|
|
282
|
+
returnResult: true
|
|
283
|
+
}
|
|
284
|
+
}
|
|
285
|
+
}
|
|
286
|
+
],
|
|
287
|
+
phaseDecisionHook: async ({ phaseResult, sharedBuffer }) => {
|
|
288
|
+
if (!phaseResult.success) {
|
|
289
|
+
return { action: PHASE_DECISION_ACTIONS.TERMINATE };
|
|
290
|
+
}
|
|
291
|
+
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
292
|
+
}
|
|
173
293
|
},
|
|
174
294
|
{
|
|
175
|
-
id: '
|
|
295
|
+
id: 'finalize',
|
|
176
296
|
requests: [
|
|
177
|
-
{
|
|
178
|
-
id: '
|
|
179
|
-
requestOptions: {
|
|
180
|
-
reqData: {
|
|
181
|
-
|
|
182
|
-
|
|
297
|
+
{
|
|
298
|
+
id: 'store-result',
|
|
299
|
+
requestOptions: {
|
|
300
|
+
reqData: {
|
|
301
|
+
hostname: 'api.example.com',
|
|
302
|
+
path: '/store',
|
|
303
|
+
method: 'POST'
|
|
304
|
+
},
|
|
305
|
+
resReq: false
|
|
306
|
+
}
|
|
183
307
|
}
|
|
184
308
|
]
|
|
185
309
|
}
|
|
186
310
|
];
|
|
187
311
|
|
|
188
312
|
const result = await stableWorkflow(phases, {
|
|
189
|
-
workflowId: '
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
313
|
+
workflowId: 'data-pipeline',
|
|
314
|
+
concurrentPhaseExecution: false, // Phases sequential
|
|
315
|
+
enableNonLinearExecution: true,
|
|
316
|
+
sharedBuffer: { userId: '123' },
|
|
317
|
+
commonAttempts: 2,
|
|
318
|
+
commonWait: 200,
|
|
319
|
+
handlePhaseCompletion: ({ phaseResult, workflowId }) => {
|
|
320
|
+
console.log(`Phase ${phaseResult.phaseId} complete in workflow ${workflowId}`);
|
|
321
|
+
}
|
|
194
322
|
});
|
|
195
323
|
|
|
196
|
-
console.log(`Workflow
|
|
197
|
-
console.log(`Total requests: ${result.totalRequests}`);
|
|
198
|
-
console.log(`Successful: ${result.successfulRequests}`);
|
|
199
|
-
console.log(`Failed: ${result.failedRequests}`);
|
|
200
|
-
console.log(`Execution time: ${result.executionTime}ms`);
|
|
324
|
+
console.log(`Workflow succeeded: ${result.success}, phases: ${result.totalPhases}`);
|
|
201
325
|
```
|
|
202
326
|
|
|
203
|
-
|
|
327
|
+
**Key responsibilities:**
|
|
328
|
+
- Execute phases sequentially or concurrently
|
|
329
|
+
- Support mixed requests and functions per phase
|
|
330
|
+
- Enable non-linear flow (CONTINUE, SKIP, REPLAY, JUMP, TERMINATE)
|
|
331
|
+
- Maintain shared buffer across all phases
|
|
332
|
+
- Apply phase-level and request-level config cascading
|
|
333
|
+
- Support branching with parallel/sequential branches
|
|
334
|
+
- Collect per-phase metrics and workflow aggregates
|
|
204
335
|
|
|
205
|
-
|
|
336
|
+
### stableWorkflowGraph
|
|
337
|
+
|
|
338
|
+
DAG-based execution for higher parallelism and explicit phase dependencies.
|
|
206
339
|
|
|
207
340
|
```typescript
|
|
208
|
-
import { stableWorkflowGraph, WorkflowGraphBuilder
|
|
341
|
+
import { stableWorkflowGraph, WorkflowGraphBuilder } from '@emmvish/stable-request';
|
|
209
342
|
|
|
210
343
|
const graph = new WorkflowGraphBuilder()
|
|
211
|
-
|
|
212
|
-
.addPhase('validate-order', {
|
|
344
|
+
.addPhase('fetch-posts', {
|
|
213
345
|
requests: [{
|
|
214
|
-
id: '
|
|
346
|
+
id: 'get-posts',
|
|
215
347
|
requestOptions: {
|
|
216
|
-
reqData: { hostname: 'api.example.com', path: '/
|
|
217
|
-
resReq: true
|
|
218
|
-
logAllSuccessfulAttempts: true
|
|
348
|
+
reqData: { hostname: 'api.example.com', path: '/posts' },
|
|
349
|
+
resReq: true
|
|
219
350
|
}
|
|
220
351
|
}]
|
|
221
352
|
})
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
.addPhase('check-inventory', {
|
|
232
|
-
requests: [{ id: 'inventory', requestOptions: { reqData: { path: '/inventory' }, resReq: true } }]
|
|
233
|
-
})
|
|
234
|
-
|
|
235
|
-
.addPhase('process-payment', {
|
|
236
|
-
requests: [{ id: 'payment', requestOptions: { reqData: { path: '/payment' }, resReq: true } }]
|
|
237
|
-
})
|
|
238
|
-
|
|
239
|
-
// Add merge point to synchronize parallel paths
|
|
240
|
-
.addMergePoint('processing-complete', ['check-inventory', 'process-payment'])
|
|
241
|
-
|
|
242
|
-
.addPhase('fulfill-order', {
|
|
243
|
-
requests: [{ id: 'fulfill', requestOptions: { reqData: { path: '/fulfill' }, resReq: true } }]
|
|
353
|
+
.addPhase('fetch-users', {
|
|
354
|
+
requests: [{
|
|
355
|
+
id: 'get-users',
|
|
356
|
+
requestOptions: {
|
|
357
|
+
reqData: { hostname: 'api.example.com', path: '/users' },
|
|
358
|
+
resReq: true
|
|
359
|
+
}
|
|
360
|
+
}]
|
|
244
361
|
})
|
|
245
|
-
|
|
246
|
-
.addPhase('
|
|
247
|
-
|
|
362
|
+
.addParallelGroup('fetch-all', ['fetch-posts', 'fetch-users'])
|
|
363
|
+
.addPhase('aggregate', {
|
|
364
|
+
functions: [{
|
|
365
|
+
id: 'combine',
|
|
366
|
+
functionOptions: {
|
|
367
|
+
fn: () => ({ posts: [], users: [] }),
|
|
368
|
+
args: [],
|
|
369
|
+
returnResult: true
|
|
370
|
+
}
|
|
371
|
+
}]
|
|
248
372
|
})
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
.
|
|
252
|
-
.connect('check-validation', 'process-order', { condition: { type: WorkflowEdgeConditionTypes.SUCCESS } })
|
|
253
|
-
.connect('check-validation', 'reject-order', { condition: { type: WorkflowEdgeConditionTypes.FAILURE } })
|
|
254
|
-
.connect('process-order', 'check-inventory', { condition: { type: WorkflowEdgeConditionTypes.ALWAYS } })
|
|
255
|
-
.connect('process-order', 'process-payment', { condition: { type: WorkflowEdgeConditionTypes.ALWAYS } })
|
|
256
|
-
.connect('check-inventory', 'processing-complete', { condition: { type: WorkflowEdgeConditionTypes.ALWAYS } })
|
|
257
|
-
.connect('process-payment', 'processing-complete', { condition: { type: WorkflowEdgeConditionTypes.ALWAYS } })
|
|
258
|
-
.connect('processing-complete', 'fulfill-order', { condition: { type: WorkflowEdgeConditionTypes.ALWAYS } })
|
|
259
|
-
|
|
260
|
-
.setEntryPoint('validate-order')
|
|
373
|
+
.addMergePoint('sync', ['fetch-all'])
|
|
374
|
+
.connectSequence('fetch-all', 'sync', 'aggregate')
|
|
375
|
+
.setEntryPoint('fetch-all')
|
|
261
376
|
.build();
|
|
262
377
|
|
|
263
378
|
const result = await stableWorkflowGraph(graph, {
|
|
264
|
-
workflowId: '
|
|
265
|
-
sharedBuffer: {},
|
|
266
|
-
validateGraph: true, // Enforce DAG constraints
|
|
267
|
-
logPhaseResults: true
|
|
379
|
+
workflowId: 'data-aggregation'
|
|
268
380
|
});
|
|
269
381
|
|
|
270
|
-
console.log(`Graph workflow
|
|
271
|
-
console.log(`Phases executed: ${result.completedPhases}/${result.totalPhases}`);
|
|
272
|
-
console.log(`Execution path:`, result.executionHistory.map(h => h.phaseId));
|
|
382
|
+
console.log(`Graph workflow success: ${result.success}`);
|
|
273
383
|
```
|
|
274
384
|
|
|
275
|
-
|
|
385
|
+
**Key responsibilities:**
|
|
386
|
+
- Define phases as DAG nodes with explicit dependency edges
|
|
387
|
+
- Execute independent phases in parallel automatically
|
|
388
|
+
- Support parallel groups, merge points, and conditional routing
|
|
389
|
+
- Validate graph structure (cycle detection, reachability, orphan detection)
|
|
390
|
+
- Provide deterministic execution order
|
|
391
|
+
- Offer higher parallelism than phased workflows for complex topologies
|
|
392
|
+
|
|
393
|
+
---
|
|
394
|
+
|
|
395
|
+
## Resilience Mechanisms
|
|
396
|
+
|
|
397
|
+
### Retry Strategies
|
|
398
|
+
|
|
399
|
+
When a request or function fails and is retryable, retry with configurable backoff.
|
|
276
400
|
|
|
277
|
-
|
|
401
|
+
#### FIXED Strategy
|
|
278
402
|
|
|
279
|
-
|
|
403
|
+
Constant wait between retries.
|
|
280
404
|
|
|
281
405
|
```typescript
|
|
282
406
|
import { stableRequest, RETRY_STRATEGIES } from '@emmvish/stable-request';
|
|
283
407
|
|
|
284
|
-
|
|
285
|
-
await stableRequest({
|
|
408
|
+
const result = await stableRequest({
|
|
286
409
|
reqData: { hostname: 'api.example.com', path: '/data' },
|
|
287
|
-
|
|
288
|
-
|
|
410
|
+
resReq: true,
|
|
411
|
+
attempts: 4,
|
|
412
|
+
wait: 500,
|
|
289
413
|
retryStrategy: RETRY_STRATEGIES.FIXED
|
|
414
|
+
// Retries at: 500ms, 1000ms, 1500ms
|
|
290
415
|
});
|
|
416
|
+
```
|
|
291
417
|
|
|
292
|
-
|
|
293
|
-
|
|
418
|
+
#### LINEAR Strategy
|
|
419
|
+
|
|
420
|
+
Wait increases linearly with attempt number.
|
|
421
|
+
|
|
422
|
+
```typescript
|
|
423
|
+
const result = await stableRequest({
|
|
294
424
|
reqData: { hostname: 'api.example.com', path: '/data' },
|
|
295
|
-
|
|
296
|
-
|
|
425
|
+
resReq: true,
|
|
426
|
+
attempts: 4,
|
|
427
|
+
wait: 100,
|
|
297
428
|
retryStrategy: RETRY_STRATEGIES.LINEAR
|
|
429
|
+
// Retries at: 100ms, 200ms, 300ms (wait * attempt)
|
|
298
430
|
});
|
|
431
|
+
```
|
|
299
432
|
|
|
300
|
-
|
|
301
|
-
|
|
433
|
+
#### EXPONENTIAL Strategy
|
|
434
|
+
|
|
435
|
+
Wait increases exponentially; useful for heavily loaded services.
|
|
436
|
+
|
|
437
|
+
```typescript
|
|
438
|
+
const result = await stableRequest({
|
|
302
439
|
reqData: { hostname: 'api.example.com', path: '/data' },
|
|
303
|
-
|
|
304
|
-
|
|
440
|
+
resReq: true,
|
|
441
|
+
attempts: 4,
|
|
442
|
+
wait: 100,
|
|
443
|
+
maxAllowedWait: 10000,
|
|
305
444
|
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL
|
|
445
|
+
// Retries at: 100ms, 200ms, 400ms (wait * 2^(attempt-1))
|
|
446
|
+
// Capped at maxAllowedWait
|
|
306
447
|
});
|
|
307
448
|
```
|
|
308
449
|
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
- Maximum allowed wait time to prevent excessive delays
|
|
313
|
-
- Per-request or workflow-level configuration
|
|
450
|
+
#### Jitter
|
|
451
|
+
|
|
452
|
+
Add random milliseconds to prevent synchronization.
|
|
314
453
|
|
|
315
|
-
**Custom Response Validation**:
|
|
316
454
|
```typescript
|
|
317
|
-
await stableRequest({
|
|
318
|
-
reqData: { hostname: 'api.example.com', path: '/
|
|
455
|
+
const result = await stableRequest({
|
|
456
|
+
reqData: { hostname: 'api.example.com', path: '/data' },
|
|
319
457
|
resReq: true,
|
|
320
|
-
attempts:
|
|
321
|
-
wait:
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
return data.status === 'completed';
|
|
325
|
-
}
|
|
458
|
+
attempts: 3,
|
|
459
|
+
wait: 500,
|
|
460
|
+
jitter: 200, // Add 0-200ms randomness
|
|
461
|
+
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL
|
|
326
462
|
});
|
|
327
463
|
```
|
|
328
464
|
|
|
329
|
-
|
|
465
|
+
#### Perform All Attempts
|
|
330
466
|
|
|
331
|
-
|
|
467
|
+
Collect all outcomes instead of failing on first error.
|
|
332
468
|
|
|
333
469
|
```typescript
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
reqData: { hostname: 'unreliable-api.example.com', path: '/data' },
|
|
470
|
+
const result = await stableRequest({
|
|
471
|
+
reqData: { hostname: 'api.example.com', path: '/data' },
|
|
472
|
+
resReq: true,
|
|
338
473
|
attempts: 3,
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
minimumRequests: 10, // Minimum requests before evaluation
|
|
342
|
-
recoveryTimeoutMs: 60000, // Wait 60s before trying again (half-open)
|
|
343
|
-
successThresholdPercentage: 20, // Close after 20% successes in half-open
|
|
344
|
-
trackIndividualAttempts: false // Track at request level (not attempt level)
|
|
345
|
-
}
|
|
474
|
+
performAllAttempts: true
|
|
475
|
+
// All 3 attempts execute; check result.successfulAttempts
|
|
346
476
|
});
|
|
347
477
|
```
|
|
348
478
|
|
|
349
|
-
|
|
350
|
-
- **CLOSED**: Normal operation, requests flow through
|
|
351
|
-
- **OPEN**: Too many failures, requests blocked immediately
|
|
352
|
-
- **HALF_OPEN**: Testing if service recovered, limited requests allowed
|
|
479
|
+
### Circuit Breaker
|
|
353
480
|
|
|
354
|
-
|
|
355
|
-
```typescript
|
|
356
|
-
import { CircuitBreaker } from '@emmvish/stable-request';
|
|
481
|
+
Prevent cascading failures by failing fast when a dependency becomes unhealthy.
|
|
357
482
|
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
483
|
+
```typescript
|
|
484
|
+
import { stableApiGateway, CircuitBreaker } from '@emmvish/stable-request';
|
|
485
|
+
|
|
486
|
+
const breaker = new CircuitBreaker({
|
|
487
|
+
failureThresholdPercentage: 50,
|
|
488
|
+
minimumRequests: 10,
|
|
489
|
+
recoveryTimeoutMs: 30000,
|
|
490
|
+
successThresholdPercentage: 80,
|
|
491
|
+
halfOpenMaxRequests: 5
|
|
363
492
|
});
|
|
364
493
|
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
494
|
+
const requests = [
|
|
495
|
+
{ id: 'req-1', requestOptions: { reqData: { path: '/flaky' }, resReq: true } },
|
|
496
|
+
{ id: 'req-2', requestOptions: { reqData: { path: '/flaky' }, resReq: true } }
|
|
497
|
+
];
|
|
498
|
+
|
|
499
|
+
const responses = await stableApiGateway(requests, {
|
|
500
|
+
circuitBreaker: breaker
|
|
368
501
|
});
|
|
369
502
|
|
|
370
|
-
//
|
|
371
|
-
|
|
372
|
-
//
|
|
503
|
+
// Circuit breaker states:
|
|
504
|
+
// CLOSED: Normal operation (accept all requests)
|
|
505
|
+
// OPEN: Too many failures; reject immediately
|
|
506
|
+
// HALF_OPEN: Testing recovery; allow limited requests
|
|
373
507
|
```
|
|
374
508
|
|
|
375
|
-
|
|
509
|
+
**State Transitions:**
|
|
376
510
|
|
|
377
|
-
|
|
511
|
+
- **CLOSED → OPEN:** Failure rate exceeds threshold after minimum requests
|
|
512
|
+
- **OPEN → HALF_OPEN:** Recovery timeout elapsed; attempt recovery
|
|
513
|
+
- **HALF_OPEN → CLOSED:** Success rate exceeds recovery threshold
|
|
514
|
+
- **HALF_OPEN → OPEN:** Success rate below recovery threshold; reopen
|
|
378
515
|
|
|
379
|
-
|
|
380
|
-
await stableRequest({
|
|
381
|
-
reqData: { hostname: 'api.example.com', path: '/static-data' },
|
|
382
|
-
resReq: true,
|
|
383
|
-
cache: {
|
|
384
|
-
enabled: true,
|
|
385
|
-
ttl: 300000, // Cache for 5 minutes
|
|
386
|
-
key: 'custom-cache-key' // Optional: custom cache key
|
|
387
|
-
}
|
|
388
|
-
});
|
|
516
|
+
### Caching
|
|
389
517
|
|
|
390
|
-
|
|
391
|
-
```
|
|
518
|
+
Cache responses to avoid redundant calls.
|
|
392
519
|
|
|
393
|
-
**Global Cache Management**:
|
|
394
520
|
```typescript
|
|
395
|
-
import {
|
|
521
|
+
import { stableRequest, CacheManager } from '@emmvish/stable-request';
|
|
396
522
|
|
|
397
|
-
const
|
|
523
|
+
const cache = new CacheManager({
|
|
524
|
+
enabled: true,
|
|
525
|
+
ttl: 5000 // 5 seconds
|
|
526
|
+
});
|
|
398
527
|
|
|
399
|
-
//
|
|
400
|
-
const
|
|
401
|
-
|
|
402
|
-
|
|
528
|
+
// First call: cache miss, hits API
|
|
529
|
+
const result1 = await stableRequest({
|
|
530
|
+
reqData: { hostname: 'api.example.com', path: '/user/1' },
|
|
531
|
+
resReq: true,
|
|
532
|
+
cache
|
|
533
|
+
});
|
|
403
534
|
|
|
404
|
-
//
|
|
405
|
-
|
|
535
|
+
// Second call within 5s: cache hit, returns cached response
|
|
536
|
+
const result2 = await stableRequest({
|
|
537
|
+
reqData: { hostname: 'api.example.com', path: '/user/1' },
|
|
538
|
+
resReq: true,
|
|
539
|
+
cache
|
|
540
|
+
});
|
|
406
541
|
|
|
407
|
-
//
|
|
408
|
-
|
|
542
|
+
// Respects Cache-Control headers if enabled
|
|
543
|
+
const cache2 = new CacheManager({
|
|
544
|
+
enabled: true,
|
|
545
|
+
ttl: 60000,
|
|
546
|
+
respectCacheControl: true // Uses max-age, no-cache, no-store
|
|
547
|
+
});
|
|
409
548
|
```
|
|
410
549
|
|
|
411
|
-
**
|
|
412
|
-
- Automatic request fingerprinting (method, URL, headers, body)
|
|
413
|
-
- TTL-based expiration
|
|
414
|
-
- Workflow-wide sharing across phases and branches
|
|
415
|
-
- Manual cache inspection and clearing
|
|
416
|
-
- Per-request cache configuration
|
|
417
|
-
|
|
418
|
-
### Rate Limiting and Concurrency Control
|
|
550
|
+
**Function Caching:**
|
|
419
551
|
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
```typescript
|
|
423
|
-
await stableWorkflow(phases, {
|
|
424
|
-
commonRequestData: { hostname: 'api.example.com' },
|
|
425
|
-
|
|
426
|
-
// Rate limiting (token bucket algorithm)
|
|
427
|
-
rateLimit: {
|
|
428
|
-
maxRequests: 100, // 100 requests
|
|
429
|
-
windowMs: 60000 // per 60 seconds
|
|
430
|
-
},
|
|
431
|
-
|
|
432
|
-
// Concurrency limiting
|
|
433
|
-
maxConcurrentRequests: 5 // Max 5 parallel requests
|
|
434
|
-
});
|
|
435
|
-
```
|
|
552
|
+
Arguments become cache key; identical args hit cache.
|
|
436
553
|
|
|
437
|
-
**Per-Phase Configuration**:
|
|
438
554
|
```typescript
|
|
439
|
-
|
|
440
|
-
{
|
|
441
|
-
id: 'bulk-import',
|
|
442
|
-
maxConcurrentRequests: 10, // Override workflow limit
|
|
443
|
-
rateLimit: {
|
|
444
|
-
maxRequests: 50,
|
|
445
|
-
windowMs: 10000
|
|
446
|
-
},
|
|
447
|
-
requests: [...]
|
|
448
|
-
}
|
|
449
|
-
];
|
|
450
|
-
```
|
|
555
|
+
import { stableFunction } from '@emmvish/stable-request';
|
|
451
556
|
|
|
452
|
-
|
|
453
|
-
```typescript
|
|
454
|
-
import { RateLimiter } from '@emmvish/stable-request';
|
|
557
|
+
const expensive = (x: number) => x * x * x; // Cubic calculation
|
|
455
558
|
|
|
456
|
-
const
|
|
559
|
+
const result1 = await stableFunction({
|
|
560
|
+
fn: expensive,
|
|
561
|
+
args: [5],
|
|
562
|
+
returnResult: true,
|
|
563
|
+
cache: { enabled: true, ttl: 10000 }
|
|
564
|
+
});
|
|
457
565
|
|
|
458
|
-
const
|
|
459
|
-
|
|
460
|
-
|
|
566
|
+
const result2 = await stableFunction({
|
|
567
|
+
fn: expensive,
|
|
568
|
+
args: [5], // Same args → cache hit
|
|
569
|
+
returnResult: true,
|
|
570
|
+
cache: { enabled: true, ttl: 10000 }
|
|
571
|
+
});
|
|
461
572
|
```
|
|
462
573
|
|
|
463
|
-
|
|
574
|
+
### Rate Limiting
|
|
464
575
|
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
### Request-Level Metrics
|
|
468
|
-
|
|
469
|
-
Every `stableRequest` call returns detailed metrics about the request execution:
|
|
576
|
+
Enforce max requests per time window.
|
|
470
577
|
|
|
471
578
|
```typescript
|
|
472
|
-
import {
|
|
473
|
-
|
|
474
|
-
const result = await stableRequest({
|
|
475
|
-
reqData: {
|
|
476
|
-
hostname: 'api.example.com',
|
|
477
|
-
path: '/users/123'
|
|
478
|
-
},
|
|
479
|
-
resReq: true,
|
|
480
|
-
attempts: 3,
|
|
481
|
-
wait: 1000,
|
|
482
|
-
logAllErrors: true
|
|
483
|
-
});
|
|
579
|
+
import { stableApiGateway } from '@emmvish/stable-request';
|
|
484
580
|
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
errorLogs: result.errorLogs, // All failed attempts
|
|
491
|
-
successfulAttempts: result.successfulAttempts, // All successful attempts
|
|
492
|
-
metrics: {
|
|
493
|
-
totalAttempts: result.metrics.totalAttempts, // 3
|
|
494
|
-
successfulAttempts: result.metrics.successfulAttempts, // 1
|
|
495
|
-
failedAttempts: result.metrics.failedAttempts, // 2
|
|
496
|
-
totalExecutionTime: result.metrics.totalExecutionTime, // ms
|
|
497
|
-
averageAttemptTime: result.metrics.averageAttemptTime, // ms
|
|
498
|
-
infrastructureMetrics: {
|
|
499
|
-
circuitBreaker: result.metrics.infrastructureMetrics?.circuitBreaker,
|
|
500
|
-
cache: result.metrics.infrastructureMetrics?.cache
|
|
501
|
-
}
|
|
581
|
+
const requests = Array.from({ length: 20 }, (_, i) => ({
|
|
582
|
+
id: `req-${i}`,
|
|
583
|
+
requestOptions: {
|
|
584
|
+
reqData: { path: `/item/${i}` },
|
|
585
|
+
resReq: true
|
|
502
586
|
}
|
|
503
|
-
});
|
|
504
|
-
|
|
505
|
-
// Error logs provide detailed attempt information
|
|
506
|
-
result.errorLogs?.forEach(log => {
|
|
507
|
-
console.log({
|
|
508
|
-
attempt: log.attempt, // "1/3"
|
|
509
|
-
timestamp: log.timestamp,
|
|
510
|
-
error: log.error,
|
|
511
|
-
statusCode: log.statusCode,
|
|
512
|
-
type: log.type, // "HTTP_ERROR" | "INVALID_CONTENT"
|
|
513
|
-
isRetryable: log.isRetryable,
|
|
514
|
-
executionTime: log.executionTime
|
|
515
|
-
});
|
|
516
|
-
});
|
|
587
|
+
}));
|
|
517
588
|
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
statusCode: attempt.statusCode
|
|
526
|
-
});
|
|
589
|
+
const responses = await stableApiGateway(requests, {
|
|
590
|
+
concurrentExecution: true,
|
|
591
|
+
rateLimit: {
|
|
592
|
+
maxRequests: 5,
|
|
593
|
+
windowMs: 1000 // 5 requests per second
|
|
594
|
+
}
|
|
595
|
+
// Requests queued until window allows; prevents overwhelming API
|
|
527
596
|
});
|
|
528
597
|
```
|
|
529
598
|
|
|
530
|
-
|
|
531
|
-
- `success`: Boolean indicating if request succeeded
|
|
532
|
-
- `data`: Response data (if `resReq: true`)
|
|
533
|
-
- `error`: Error message (if request failed)
|
|
534
|
-
- `errorLogs`: Array of all failed attempt details
|
|
535
|
-
- `successfulAttempts`: Array of all successful attempt details
|
|
536
|
-
- `metrics`: Computed execution metrics and infrastructure statistics
|
|
537
|
-
|
|
538
|
-
### API Gateway Metrics
|
|
599
|
+
### Concurrency Limiting
|
|
539
600
|
|
|
540
|
-
|
|
601
|
+
Limit concurrent in-flight requests.
|
|
541
602
|
|
|
542
603
|
```typescript
|
|
543
604
|
import { stableApiGateway } from '@emmvish/stable-request';
|
|
544
605
|
|
|
545
|
-
const requests =
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
606
|
+
const requests = Array.from({ length: 50 }, (_, i) => ({
|
|
607
|
+
id: `req-${i}`,
|
|
608
|
+
requestOptions: {
|
|
609
|
+
reqData: { path: `/item/${i}` },
|
|
610
|
+
resReq: true,
|
|
611
|
+
attempts: 1
|
|
612
|
+
}
|
|
613
|
+
}));
|
|
551
614
|
|
|
552
|
-
const
|
|
615
|
+
const responses = await stableApiGateway(requests, {
|
|
553
616
|
concurrentExecution: true,
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
circuitBreaker: { failureThresholdPercentage: 50, minimumRequests: 5 },
|
|
557
|
-
rateLimit: { maxRequests: 100, windowMs: 60000 },
|
|
558
|
-
maxConcurrentRequests: 5
|
|
617
|
+
maxConcurrentRequests: 5 // Only 5 requests in-flight at a time
|
|
618
|
+
// Others queued and executed as slots free
|
|
559
619
|
});
|
|
620
|
+
```
|
|
560
621
|
|
|
561
|
-
|
|
562
|
-
console.log('Gateway Metrics:', {
|
|
563
|
-
totalRequests: results.metrics.totalRequests, // 4
|
|
564
|
-
successfulRequests: results.metrics.successfulRequests, // 3
|
|
565
|
-
failedRequests: results.metrics.failedRequests, // 1
|
|
566
|
-
successRate: results.metrics.successRate, // 75%
|
|
567
|
-
failureRate: results.metrics.failureRate // 25%
|
|
568
|
-
});
|
|
622
|
+
---
|
|
569
623
|
|
|
570
|
-
|
|
571
|
-
results.metrics.requestGroups?.forEach(group => {
|
|
572
|
-
console.log(`Group ${group.groupId}:`, {
|
|
573
|
-
totalRequests: group.totalRequests,
|
|
574
|
-
successfulRequests: group.successfulRequests,
|
|
575
|
-
failedRequests: group.failedRequests,
|
|
576
|
-
successRate: group.successRate, // %
|
|
577
|
-
failureRate: group.failureRate, // %
|
|
578
|
-
requestIds: group.requestIds // Array of request IDs
|
|
579
|
-
});
|
|
580
|
-
});
|
|
624
|
+
## Workflow Patterns
|
|
581
625
|
|
|
582
|
-
|
|
583
|
-
if (results.metrics.infrastructureMetrics) {
|
|
584
|
-
const infra = results.metrics.infrastructureMetrics;
|
|
585
|
-
|
|
586
|
-
// Circuit Breaker metrics
|
|
587
|
-
if (infra.circuitBreaker) {
|
|
588
|
-
console.log('Circuit Breaker:', {
|
|
589
|
-
state: infra.circuitBreaker.state, // CLOSED | OPEN | HALF_OPEN
|
|
590
|
-
isHealthy: infra.circuitBreaker.isHealthy,
|
|
591
|
-
totalRequests: infra.circuitBreaker.totalRequests,
|
|
592
|
-
failurePercentage: infra.circuitBreaker.failurePercentage,
|
|
593
|
-
openCount: infra.circuitBreaker.openCount,
|
|
594
|
-
recoveryAttempts: infra.circuitBreaker.recoveryAttempts
|
|
595
|
-
});
|
|
596
|
-
}
|
|
597
|
-
|
|
598
|
-
// Cache metrics
|
|
599
|
-
if (infra.cache) {
|
|
600
|
-
console.log('Cache:', {
|
|
601
|
-
hitRate: infra.cache.hitRate, // %
|
|
602
|
-
currentSize: infra.cache.currentSize,
|
|
603
|
-
networkRequestsSaved: infra.cache.networkRequestsSaved,
|
|
604
|
-
cacheEfficiency: infra.cache.cacheEfficiency // %
|
|
605
|
-
});
|
|
606
|
-
}
|
|
607
|
-
|
|
608
|
-
// Rate Limiter metrics
|
|
609
|
-
if (infra.rateLimiter) {
|
|
610
|
-
console.log('Rate Limiter:', {
|
|
611
|
-
throttledRequests: infra.rateLimiter.throttledRequests,
|
|
612
|
-
throttleRate: infra.rateLimiter.throttleRate, // %
|
|
613
|
-
peakRequestRate: infra.rateLimiter.peakRequestRate
|
|
614
|
-
});
|
|
615
|
-
}
|
|
616
|
-
|
|
617
|
-
// Concurrency Limiter metrics
|
|
618
|
-
if (infra.concurrencyLimiter) {
|
|
619
|
-
console.log('Concurrency:', {
|
|
620
|
-
peakConcurrency: infra.concurrencyLimiter.peakConcurrency,
|
|
621
|
-
utilizationPercentage: infra.concurrencyLimiter.utilizationPercentage,
|
|
622
|
-
averageQueueWaitTime: infra.concurrencyLimiter.averageQueueWaitTime
|
|
623
|
-
});
|
|
624
|
-
}
|
|
625
|
-
}
|
|
626
|
-
```
|
|
626
|
+
### Sequential & Concurrent Phases
|
|
627
627
|
|
|
628
|
-
|
|
628
|
+
#### Sequential (Default)
|
|
629
629
|
|
|
630
|
-
|
|
630
|
+
Each phase waits for the previous to complete.
|
|
631
631
|
|
|
632
632
|
```typescript
|
|
633
|
-
import { stableWorkflow
|
|
633
|
+
import { stableWorkflow } from '@emmvish/stable-request';
|
|
634
|
+
import type { STABLE_WORKFLOW_PHASE } from '@emmvish/stable-request';
|
|
634
635
|
|
|
635
|
-
const phases = [
|
|
636
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
636
637
|
{
|
|
637
|
-
id: '
|
|
638
|
-
requests: [
|
|
638
|
+
id: 'phase-1',
|
|
639
|
+
requests: [{ id: 'r1', requestOptions: { reqData: { path: '/p1' }, resReq: true } }]
|
|
639
640
|
},
|
|
640
641
|
{
|
|
641
|
-
id: '
|
|
642
|
-
|
|
643
|
-
requests: [/* ... */]
|
|
642
|
+
id: 'phase-2',
|
|
643
|
+
requests: [{ id: 'r2', requestOptions: { reqData: { path: '/p2' }, resReq: true } }]
|
|
644
644
|
},
|
|
645
645
|
{
|
|
646
|
-
id: '
|
|
647
|
-
requests: [
|
|
646
|
+
id: 'phase-3',
|
|
647
|
+
requests: [{ id: 'r3', requestOptions: { reqData: { path: '/p3' }, resReq: true } }]
|
|
648
648
|
}
|
|
649
649
|
];
|
|
650
650
|
|
|
651
651
|
const result = await stableWorkflow(phases, {
|
|
652
|
-
workflowId: '
|
|
653
|
-
|
|
654
|
-
commonRequestData: { hostname: 'api.example.com' },
|
|
655
|
-
logPhaseResults: true
|
|
652
|
+
workflowId: 'sequential-phases',
|
|
653
|
+
concurrentPhaseExecution: false // Phase-1 → Phase-2 → Phase-3
|
|
656
654
|
});
|
|
655
|
+
```
|
|
657
656
|
|
|
658
|
-
|
|
659
|
-
console.log('Workflow Metrics:', {
|
|
660
|
-
workflowId: result.metrics.workflowId,
|
|
661
|
-
success: result.metrics.success,
|
|
662
|
-
executionTime: result.metrics.executionTime, // Total time in ms
|
|
663
|
-
|
|
664
|
-
// Phase statistics
|
|
665
|
-
totalPhases: result.metrics.totalPhases,
|
|
666
|
-
completedPhases: result.metrics.completedPhases,
|
|
667
|
-
skippedPhases: result.metrics.skippedPhases,
|
|
668
|
-
failedPhases: result.metrics.failedPhases,
|
|
669
|
-
phaseCompletionRate: result.metrics.phaseCompletionRate, // %
|
|
670
|
-
averagePhaseExecutionTime: result.metrics.averagePhaseExecutionTime, // ms
|
|
671
|
-
|
|
672
|
-
// Request statistics
|
|
673
|
-
totalRequests: result.metrics.totalRequests,
|
|
674
|
-
successfulRequests: result.metrics.successfulRequests,
|
|
675
|
-
failedRequests: result.metrics.failedRequests,
|
|
676
|
-
requestSuccessRate: result.metrics.requestSuccessRate, // %
|
|
677
|
-
requestFailureRate: result.metrics.requestFailureRate, // %
|
|
678
|
-
|
|
679
|
-
// Performance
|
|
680
|
-
throughput: result.metrics.throughput, // requests/second
|
|
681
|
-
totalPhaseReplays: result.metrics.totalPhaseReplays,
|
|
682
|
-
totalPhaseSkips: result.metrics.totalPhaseSkips,
|
|
683
|
-
|
|
684
|
-
// Branch metrics (if using branch execution)
|
|
685
|
-
totalBranches: result.metrics.totalBranches,
|
|
686
|
-
completedBranches: result.metrics.completedBranches,
|
|
687
|
-
failedBranches: result.metrics.failedBranches,
|
|
688
|
-
branchSuccessRate: result.metrics.branchSuccessRate // %
|
|
689
|
-
});
|
|
657
|
+
#### Concurrent Phases
|
|
690
658
|
|
|
691
|
-
|
|
692
|
-
result.requestGroupMetrics?.forEach(group => {
|
|
693
|
-
console.log(`Request Group ${group.groupId}:`, {
|
|
694
|
-
totalRequests: group.totalRequests,
|
|
695
|
-
successRate: group.successRate, // %
|
|
696
|
-
requestIds: group.requestIds
|
|
697
|
-
});
|
|
698
|
-
});
|
|
659
|
+
Multiple phases run in parallel.
|
|
699
660
|
|
|
700
|
-
|
|
701
|
-
|
|
702
|
-
|
|
703
|
-
|
|
704
|
-
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
|
|
708
|
-
|
|
709
|
-
}
|
|
710
|
-
|
|
661
|
+
```typescript
|
|
662
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
663
|
+
{
|
|
664
|
+
id: 'fetch-users',
|
|
665
|
+
requests: [{ id: 'get-users', requestOptions: { reqData: { path: '/users' }, resReq: true } }]
|
|
666
|
+
},
|
|
667
|
+
{
|
|
668
|
+
id: 'fetch-posts',
|
|
669
|
+
requests: [{ id: 'get-posts', requestOptions: { reqData: { path: '/posts' }, resReq: true } }]
|
|
670
|
+
},
|
|
671
|
+
{
|
|
672
|
+
id: 'fetch-comments',
|
|
673
|
+
requests: [{ id: 'get-comments', requestOptions: { reqData: { path: '/comments' }, resReq: true } }]
|
|
674
|
+
}
|
|
675
|
+
];
|
|
711
676
|
|
|
712
|
-
|
|
713
|
-
|
|
714
|
-
|
|
715
|
-
success: branch.metrics?.success,
|
|
716
|
-
executionTime: branch.metrics?.executionTime,
|
|
717
|
-
totalPhases: branch.metrics?.totalPhases,
|
|
718
|
-
completedPhases: branch.metrics?.completedPhases,
|
|
719
|
-
totalRequests: branch.metrics?.totalRequests,
|
|
720
|
-
requestSuccessRate: branch.metrics?.requestSuccessRate // %
|
|
721
|
-
});
|
|
677
|
+
const result = await stableWorkflow(phases, {
|
|
678
|
+
workflowId: 'parallel-phases',
|
|
679
|
+
concurrentPhaseExecution: true // All 3 phases in parallel
|
|
722
680
|
});
|
|
723
681
|
```
|
|
724
682
|
|
|
725
|
-
|
|
683
|
+
#### Mixed Phases
|
|
726
684
|
|
|
727
|
-
|
|
685
|
+
Combine sequential and concurrent phases in one workflow.
|
|
728
686
|
|
|
729
687
|
```typescript
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
|
|
734
|
-
|
|
735
|
-
|
|
736
|
-
|
|
737
|
-
|
|
738
|
-
|
|
739
|
-
|
|
740
|
-
|
|
741
|
-
|
|
742
|
-
|
|
743
|
-
|
|
744
|
-
|
|
745
|
-
|
|
746
|
-
|
|
747
|
-
|
|
748
|
-
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
const
|
|
752
|
-
|
|
753
|
-
// Extract rate limiter metrics
|
|
754
|
-
const rateLimiterMetrics = MetricsAggregator.extractRateLimiterMetrics(rateLimiter);
|
|
755
|
-
|
|
756
|
-
// Extract concurrency limiter metrics
|
|
757
|
-
const concurrencyMetrics = MetricsAggregator.extractConcurrencyLimiterMetrics(limiter);
|
|
758
|
-
|
|
759
|
-
// Aggregate all system metrics
|
|
760
|
-
const systemMetrics = MetricsAggregator.aggregateSystemMetrics(
|
|
761
|
-
workflowResult,
|
|
762
|
-
circuitBreaker,
|
|
763
|
-
cacheManager,
|
|
764
|
-
rateLimiter,
|
|
765
|
-
concurrencyLimiter
|
|
766
|
-
);
|
|
767
|
-
|
|
768
|
-
console.log('Complete System View:', {
|
|
769
|
-
workflow: systemMetrics.workflow,
|
|
770
|
-
phases: systemMetrics.phases,
|
|
771
|
-
branches: systemMetrics.branches,
|
|
772
|
-
requestGroups: systemMetrics.requestGroups,
|
|
773
|
-
requests: systemMetrics.requests,
|
|
774
|
-
circuitBreaker: systemMetrics.circuitBreaker,
|
|
775
|
-
cache: systemMetrics.cache,
|
|
776
|
-
rateLimiter: systemMetrics.rateLimiter,
|
|
777
|
-
concurrencyLimiter: systemMetrics.concurrencyLimiter
|
|
688
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
689
|
+
{
|
|
690
|
+
id: 'init', // Sequential
|
|
691
|
+
requests: [{ id: 'setup', requestOptions: { reqData: { path: '/init' }, resReq: true } }]
|
|
692
|
+
},
|
|
693
|
+
{
|
|
694
|
+
id: 'fetch-a',
|
|
695
|
+
markConcurrentPhase: true, // Concurrent with next
|
|
696
|
+
requests: [{ id: 'data-a', requestOptions: { reqData: { path: '/a' }, resReq: true } }]
|
|
697
|
+
},
|
|
698
|
+
{
|
|
699
|
+
id: 'fetch-b',
|
|
700
|
+
markConcurrentPhase: true, // Concurrent with fetch-a
|
|
701
|
+
requests: [{ id: 'data-b', requestOptions: { reqData: { path: '/b' }, resReq: true } }]
|
|
702
|
+
},
|
|
703
|
+
{
|
|
704
|
+
id: 'finalize', // Sequential after fetch-a/b complete
|
|
705
|
+
requests: [{ id: 'done', requestOptions: { reqData: { path: '/finalize' }, resReq: true } }]
|
|
706
|
+
}
|
|
707
|
+
];
|
|
708
|
+
|
|
709
|
+
const result = await stableWorkflow(phases, {
|
|
710
|
+
concurrentPhaseExecution: false // Respects markConcurrentPhase per phase
|
|
778
711
|
});
|
|
779
712
|
```
|
|
780
713
|
|
|
781
|
-
|
|
782
|
-
- `WorkflowMetrics`: Complete workflow statistics
|
|
783
|
-
- `BranchMetrics`: Branch execution metrics
|
|
784
|
-
- `PhaseMetrics`: Individual phase metrics
|
|
785
|
-
- `RequestGroupMetrics`: Grouped request statistics
|
|
786
|
-
- `RequestMetrics`: Individual request metrics
|
|
787
|
-
- `CircuitBreakerDashboardMetrics`: Circuit breaker state and performance
|
|
788
|
-
- `CacheDashboardMetrics`: Cache hit rates and efficiency
|
|
789
|
-
- `RateLimiterDashboardMetrics`: Throttling and rate limit statistics
|
|
790
|
-
- `ConcurrencyLimiterDashboardMetrics`: Concurrency and queue metrics
|
|
791
|
-
- `SystemMetrics`: Complete system-wide aggregation
|
|
714
|
+
### Non-Linear Workflows
|
|
792
715
|
|
|
793
|
-
|
|
716
|
+
Use decision hooks to dynamically control phase flow.
|
|
794
717
|
|
|
795
|
-
|
|
718
|
+
#### CONTINUE
|
|
796
719
|
|
|
797
|
-
|
|
720
|
+
Standard flow to next sequential phase.
|
|
798
721
|
|
|
799
|
-
**Sequential Phases (Default)**:
|
|
800
722
|
```typescript
|
|
801
|
-
const phases = [
|
|
802
|
-
{ id: 'step-1', requests: [...] }, // Executes first
|
|
803
|
-
{ id: 'step-2', requests: [...] }, // Then this
|
|
804
|
-
{ id: 'step-3', requests: [...] } // Finally this
|
|
805
|
-
];
|
|
806
|
-
|
|
807
|
-
await stableWorkflow(phases, {
|
|
808
|
-
commonRequestData: { hostname: 'api.example.com' }
|
|
809
|
-
});
|
|
810
|
-
```
|
|
811
|
-
|
|
812
|
-
**Concurrent Phases**:
|
|
813
|
-
```typescript
|
|
814
|
-
const phases = [
|
|
815
|
-
{ id: 'init', requests: [...] },
|
|
816
|
-
{ id: 'parallel-1', requests: [...] },
|
|
817
|
-
{ id: 'parallel-2', requests: [...] }
|
|
818
|
-
];
|
|
819
|
-
|
|
820
|
-
await stableWorkflow(phases, {
|
|
821
|
-
concurrentPhaseExecution: true, // All phases run in parallel
|
|
822
|
-
commonRequestData: { hostname: 'api.example.com' }
|
|
823
|
-
});
|
|
824
|
-
```
|
|
825
|
-
|
|
826
|
-
**Concurrent Requests Within Phase**:
|
|
827
|
-
```typescript
|
|
828
|
-
const phases = [
|
|
723
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
829
724
|
{
|
|
830
|
-
id: '
|
|
831
|
-
|
|
832
|
-
|
|
833
|
-
|
|
834
|
-
|
|
835
|
-
|
|
836
|
-
]
|
|
837
|
-
}
|
|
838
|
-
];
|
|
839
|
-
```
|
|
840
|
-
|
|
841
|
-
**Stop on First Error**:
|
|
842
|
-
```typescript
|
|
843
|
-
const phases = [
|
|
725
|
+
id: 'check-status',
|
|
726
|
+
requests: [{ id: 'api', requestOptions: { reqData: { path: '/status' }, resReq: true } }],
|
|
727
|
+
phaseDecisionHook: async ({ phaseResult, sharedBuffer }) => {
|
|
728
|
+
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
729
|
+
}
|
|
730
|
+
},
|
|
844
731
|
{
|
|
845
|
-
id: '
|
|
846
|
-
|
|
847
|
-
requests: [...]
|
|
732
|
+
id: 'process', // Executes after check-status
|
|
733
|
+
requests: [{ id: 'process-data', requestOptions: { reqData: { path: '/process' }, resReq: true } }]
|
|
848
734
|
}
|
|
849
735
|
];
|
|
850
736
|
|
|
851
|
-
await stableWorkflow(phases, {
|
|
852
|
-
|
|
853
|
-
commonRequestData: { hostname: 'api.example.com' }
|
|
737
|
+
const result = await stableWorkflow(phases, {
|
|
738
|
+
enableNonLinearExecution: true
|
|
854
739
|
});
|
|
855
740
|
```
|
|
856
741
|
|
|
857
|
-
|
|
742
|
+
#### SKIP
|
|
858
743
|
|
|
859
|
-
|
|
744
|
+
Skip the next phase; execute the one after.
|
|
860
745
|
|
|
861
746
|
```typescript
|
|
862
|
-
const phases = [
|
|
863
|
-
{
|
|
864
|
-
id: '
|
|
865
|
-
requests: [{ id: '
|
|
866
|
-
|
|
867
|
-
|
|
868
|
-
|
|
869
|
-
markConcurrentPhase: true, // This phase runs concurrently...
|
|
870
|
-
requests: [{ id: 'profile', requestOptions: {...} }]
|
|
747
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
748
|
+
{
|
|
749
|
+
id: 'phase-1',
|
|
750
|
+
requests: [{ id: 'r1', requestOptions: { reqData: { path: '/p1' }, resReq: true } }],
|
|
751
|
+
phaseDecisionHook: async () => ({
|
|
752
|
+
action: PHASE_DECISION_ACTIONS.SKIP
|
|
753
|
+
})
|
|
871
754
|
},
|
|
872
|
-
{
|
|
873
|
-
id: '
|
|
874
|
-
|
|
875
|
-
requests: [{ id: 'orders', requestOptions: {...} }]
|
|
755
|
+
{
|
|
756
|
+
id: 'phase-2', // Skipped
|
|
757
|
+
requests: [{ id: 'r2', requestOptions: { reqData: { path: '/p2' }, resReq: true } }]
|
|
876
758
|
},
|
|
877
|
-
{
|
|
878
|
-
id: '
|
|
879
|
-
requests: [{ id: '
|
|
759
|
+
{
|
|
760
|
+
id: 'phase-3', // Executes
|
|
761
|
+
requests: [{ id: 'r3', requestOptions: { reqData: { path: '/p3' }, resReq: true } }]
|
|
880
762
|
}
|
|
881
763
|
];
|
|
882
764
|
|
|
883
|
-
await stableWorkflow(phases, {
|
|
884
|
-
|
|
885
|
-
commonRequestData: { hostname: 'api.example.com' }
|
|
765
|
+
const result = await stableWorkflow(phases, {
|
|
766
|
+
enableNonLinearExecution: true
|
|
886
767
|
});
|
|
887
|
-
```
|
|
888
768
|
|
|
889
|
-
|
|
769
|
+
// Execution: phase-1 → phase-3
|
|
770
|
+
```
|
|
890
771
|
|
|
891
|
-
|
|
772
|
+
#### JUMP
|
|
892
773
|
|
|
893
|
-
|
|
774
|
+
Jump to a specific phase by ID.
|
|
894
775
|
|
|
895
776
|
```typescript
|
|
896
|
-
|
|
897
|
-
|
|
898
|
-
const phases = [
|
|
777
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
899
778
|
{
|
|
900
|
-
id: '
|
|
901
|
-
requests: [
|
|
902
|
-
|
|
903
|
-
|
|
904
|
-
|
|
905
|
-
|
|
906
|
-
|
|
907
|
-
if (isValid) {
|
|
908
|
-
// Jump directly to success phase
|
|
909
|
-
return {
|
|
910
|
-
action: PHASE_DECISION_ACTIONS.JUMP,
|
|
911
|
-
targetPhaseId: 'success-flow'
|
|
912
|
-
};
|
|
913
|
-
} else {
|
|
914
|
-
// Continue to retry logic
|
|
915
|
-
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
916
|
-
}
|
|
917
|
-
}
|
|
779
|
+
id: 'phase-1',
|
|
780
|
+
requests: [{ id: 'r1', requestOptions: { reqData: { path: '/p1' }, resReq: true } }],
|
|
781
|
+
phaseDecisionHook: async () => ({
|
|
782
|
+
action: PHASE_DECISION_ACTIONS.JUMP,
|
|
783
|
+
targetPhaseId: 'recovery'
|
|
784
|
+
})
|
|
918
785
|
},
|
|
919
786
|
{
|
|
920
|
-
id: '
|
|
921
|
-
|
|
922
|
-
maxReplayCount: 3,
|
|
923
|
-
requests: [
|
|
924
|
-
{ id: 'retry', requestOptions: { reqData: { path: '/retry-validate' }, resReq: true } }
|
|
925
|
-
],
|
|
926
|
-
phaseDecisionHook: async ({ phaseResult, executionHistory }) => {
|
|
927
|
-
const replayCount = executionHistory.filter(
|
|
928
|
-
h => h.phaseId === 'retry-validation'
|
|
929
|
-
).length;
|
|
930
|
-
|
|
931
|
-
const success = phaseResult.responses[0]?.data?.success;
|
|
932
|
-
|
|
933
|
-
if (success) {
|
|
934
|
-
return { action: PHASE_DECISION_ACTIONS.JUMP, targetPhaseId: 'success-flow' };
|
|
935
|
-
} else if (replayCount < 3) {
|
|
936
|
-
return { action: PHASE_DECISION_ACTIONS.REPLAY };
|
|
937
|
-
} else {
|
|
938
|
-
return { action: PHASE_DECISION_ACTIONS.TERMINATE, metadata: { reason: 'Max retries exceeded' } };
|
|
939
|
-
}
|
|
940
|
-
}
|
|
787
|
+
id: 'phase-2', // Skipped
|
|
788
|
+
requests: [{ id: 'r2', requestOptions: { reqData: { path: '/p2' }, resReq: true } }]
|
|
941
789
|
},
|
|
942
790
|
{
|
|
943
|
-
id: '
|
|
944
|
-
requests: [
|
|
945
|
-
{ id: 'finalize', requestOptions: { reqData: { path: '/finalize' }, resReq: true } }
|
|
946
|
-
]
|
|
791
|
+
id: 'recovery',
|
|
792
|
+
requests: [{ id: 'recover', requestOptions: { reqData: { path: '/recovery' }, resReq: true } }]
|
|
947
793
|
}
|
|
948
794
|
];
|
|
949
795
|
|
|
950
796
|
const result = await stableWorkflow(phases, {
|
|
951
|
-
enableNonLinearExecution: true
|
|
952
|
-
workflowId: 'adaptive-validation',
|
|
953
|
-
commonRequestData: { hostname: 'api.example.com' }
|
|
797
|
+
enableNonLinearExecution: true
|
|
954
798
|
});
|
|
955
799
|
|
|
956
|
-
|
|
957
|
-
// Array of execution records showing which phases ran and why
|
|
958
|
-
```
|
|
959
|
-
|
|
960
|
-
**Phase Decision Actions**:
|
|
961
|
-
- **CONTINUE**: Proceed to next sequential phase (default)
|
|
962
|
-
- **JUMP**: Skip to a specific phase by ID
|
|
963
|
-
- **SKIP**: Skip upcoming phases until a target phase (or end)
|
|
964
|
-
- **REPLAY**: Re-execute the current phase (requires `allowReplay: true`)
|
|
965
|
-
- **TERMINATE**: Stop the entire workflow immediately
|
|
966
|
-
|
|
967
|
-
**Decision Hook Context**:
|
|
968
|
-
```typescript
|
|
969
|
-
phaseDecisionHook: async ({
|
|
970
|
-
phaseResult, // Current phase execution result
|
|
971
|
-
executionHistory, // Array of all executed phases
|
|
972
|
-
sharedBuffer, // Cross-phase shared state
|
|
973
|
-
concurrentResults // Results from concurrent phases (mixed execution)
|
|
974
|
-
}) => {
|
|
975
|
-
// Your decision logic
|
|
976
|
-
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
977
|
-
}
|
|
978
|
-
```
|
|
979
|
-
|
|
980
|
-
**Replay Limits**:
|
|
981
|
-
```typescript
|
|
982
|
-
{
|
|
983
|
-
id: 'polling-phase',
|
|
984
|
-
allowReplay: true,
|
|
985
|
-
maxReplayCount: 10, // Maximum 10 replays
|
|
986
|
-
requests: [...],
|
|
987
|
-
phaseDecisionHook: async ({ phaseResult }) => {
|
|
988
|
-
if (phaseResult.responses[0]?.data?.ready) {
|
|
989
|
-
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
990
|
-
}
|
|
991
|
-
return { action: PHASE_DECISION_ACTIONS.REPLAY };
|
|
992
|
-
}
|
|
993
|
-
}
|
|
800
|
+
// Execution: phase-1 → recovery
|
|
994
801
|
```
|
|
995
802
|
|
|
996
|
-
|
|
803
|
+
#### REPLAY
|
|
997
804
|
|
|
998
|
-
|
|
805
|
+
Re-execute current phase; useful for polling.
|
|
999
806
|
|
|
1000
|
-
**Adding Phases Dynamically**:
|
|
1001
807
|
```typescript
|
|
1002
|
-
const phases = [
|
|
808
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
1003
809
|
{
|
|
1004
|
-
id: '
|
|
1005
|
-
|
|
1006
|
-
|
|
1007
|
-
|
|
1008
|
-
|
|
1009
|
-
|
|
1010
|
-
|
|
1011
|
-
|
|
1012
|
-
|
|
1013
|
-
|
|
1014
|
-
|
|
1015
|
-
|
|
1016
|
-
|
|
1017
|
-
]
|
|
1018
|
-
};
|
|
810
|
+
id: 'wait-for-job',
|
|
811
|
+
allowReplay: true,
|
|
812
|
+
maxReplayCount: 5,
|
|
813
|
+
requests: [
|
|
814
|
+
{
|
|
815
|
+
id: 'check-job',
|
|
816
|
+
requestOptions: { reqData: { path: '/job/status' }, resReq: true, attempts: 1 }
|
|
817
|
+
}
|
|
818
|
+
],
|
|
819
|
+
phaseDecisionHook: async ({ phaseResult, executionHistory }) => {
|
|
820
|
+
const lastResponse = phaseResult.responses?.[0];
|
|
821
|
+
if ((lastResponse as any)?.data?.status === 'pending' && executionHistory.length < 5) {
|
|
822
|
+
return { action: PHASE_DECISION_ACTIONS.REPLAY };
|
|
1019
823
|
}
|
|
1020
824
|
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
1021
825
|
}
|
|
826
|
+
},
|
|
827
|
+
{
|
|
828
|
+
id: 'process-result',
|
|
829
|
+
requests: [{ id: 'process', requestOptions: { reqData: { path: '/process' }, resReq: true } }]
|
|
1022
830
|
}
|
|
1023
831
|
];
|
|
1024
832
|
|
|
1025
|
-
await stableWorkflow(phases, {
|
|
833
|
+
const result = await stableWorkflow(phases, {
|
|
1026
834
|
enableNonLinearExecution: true,
|
|
1027
|
-
|
|
835
|
+
maxWorkflowIterations: 100
|
|
1028
836
|
});
|
|
837
|
+
|
|
838
|
+
// Polls up to 5 times before continuing
|
|
1029
839
|
```
|
|
1030
840
|
|
|
1031
|
-
|
|
841
|
+
#### TERMINATE
|
|
842
|
+
|
|
843
|
+
Stop workflow early.
|
|
844
|
+
|
|
1032
845
|
```typescript
|
|
1033
|
-
const
|
|
846
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
1034
847
|
{
|
|
1035
|
-
id: '
|
|
1036
|
-
|
|
1037
|
-
|
|
1038
|
-
|
|
1039
|
-
|
|
1040
|
-
if (requiresAudit) {
|
|
1041
|
-
return {
|
|
1042
|
-
action: PHASE_DECISION_ACTIONS.CONTINUE,
|
|
1043
|
-
addBranches: [
|
|
1044
|
-
{
|
|
1045
|
-
id: 'audit-branch',
|
|
1046
|
-
phases: [{ id: 'audit', requests: [...] }]
|
|
1047
|
-
}
|
|
1048
|
-
]
|
|
1049
|
-
};
|
|
848
|
+
id: 'validate',
|
|
849
|
+
requests: [{ id: 'validate-input', requestOptions: { reqData: { path: '/validate' }, resReq: true } }],
|
|
850
|
+
phaseDecisionHook: async ({ phaseResult, sharedBuffer }) => {
|
|
851
|
+
if (!phaseResult.success) {
|
|
852
|
+
return { action: PHASE_DECISION_ACTIONS.TERMINATE };
|
|
1050
853
|
}
|
|
1051
854
|
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
1052
855
|
}
|
|
856
|
+
},
|
|
857
|
+
{
|
|
858
|
+
id: 'phase-2', // Won't execute if validation fails
|
|
859
|
+
requests: [{ id: 'r2', requestOptions: { reqData: { path: '/p2' }, resReq: true } }]
|
|
1053
860
|
}
|
|
1054
861
|
];
|
|
1055
862
|
|
|
1056
|
-
await stableWorkflow(
|
|
1057
|
-
|
|
1058
|
-
branches,
|
|
1059
|
-
commonRequestData: { hostname: 'api.example.com' }
|
|
863
|
+
const result = await stableWorkflow(phases, {
|
|
864
|
+
enableNonLinearExecution: true
|
|
1060
865
|
});
|
|
1061
|
-
```
|
|
1062
866
|
|
|
1063
|
-
|
|
1064
|
-
```typescript
|
|
1065
|
-
branchDecisionHook: async ({ branchResults }) => {
|
|
1066
|
-
return {
|
|
1067
|
-
action: PHASE_DECISION_ACTIONS.CONTINUE,
|
|
1068
|
-
addPhases: [
|
|
1069
|
-
{ id: 'extra-phase', requests: [...] } // Branch re-executes with new phases
|
|
1070
|
-
]
|
|
1071
|
-
};
|
|
1072
|
-
}
|
|
867
|
+
console.log(result.terminatedEarly); // true if TERMINATE triggered
|
|
1073
868
|
```
|
|
1074
869
|
|
|
1075
870
|
### Branched Workflows
|
|
1076
871
|
|
|
1077
|
-
Execute multiple independent
|
|
872
|
+
Execute multiple independent branches with shared state.
|
|
1078
873
|
|
|
1079
874
|
```typescript
|
|
1080
|
-
|
|
1081
|
-
|
|
1082
|
-
|
|
1083
|
-
|
|
1084
|
-
phases: [
|
|
1085
|
-
{ id: 'fetch-user', requests: [...] },
|
|
1086
|
-
{ id: 'update-user', requests: [...] }
|
|
1087
|
-
]
|
|
1088
|
-
},
|
|
875
|
+
import { stableWorkflow } from '@emmvish/stable-request';
|
|
876
|
+
import type { STABLE_WORKFLOW_BRANCH } from '@emmvish/stable-request';
|
|
877
|
+
|
|
878
|
+
const branches: STABLE_WORKFLOW_BRANCH[] = [
|
|
1089
879
|
{
|
|
1090
|
-
id: '
|
|
1091
|
-
markConcurrentBranch: true, // Execute in parallel
|
|
880
|
+
id: 'branch-payment',
|
|
1092
881
|
phases: [
|
|
1093
|
-
{
|
|
1094
|
-
|
|
882
|
+
{
|
|
883
|
+
id: 'process-payment',
|
|
884
|
+
requests: [
|
|
885
|
+
{
|
|
886
|
+
id: 'charge-card',
|
|
887
|
+
requestOptions: {
|
|
888
|
+
reqData: { path: '/payment/charge' },
|
|
889
|
+
resReq: true
|
|
890
|
+
}
|
|
891
|
+
}
|
|
892
|
+
]
|
|
893
|
+
}
|
|
1095
894
|
]
|
|
1096
895
|
},
|
|
1097
896
|
{
|
|
1098
|
-
id: '
|
|
897
|
+
id: 'branch-notification',
|
|
1099
898
|
phases: [
|
|
1100
|
-
{
|
|
1101
|
-
|
|
899
|
+
{
|
|
900
|
+
id: 'send-email',
|
|
901
|
+
requests: [
|
|
902
|
+
{
|
|
903
|
+
id: 'send',
|
|
904
|
+
requestOptions: {
|
|
905
|
+
reqData: { path: '/notify/email' },
|
|
906
|
+
resReq: false
|
|
907
|
+
}
|
|
908
|
+
}
|
|
909
|
+
]
|
|
910
|
+
}
|
|
1102
911
|
]
|
|
1103
912
|
}
|
|
1104
913
|
];
|
|
1105
914
|
|
|
1106
|
-
const result = await stableWorkflow([], {
|
|
915
|
+
const result = await stableWorkflow([], {
|
|
916
|
+
workflowId: 'checkout',
|
|
1107
917
|
enableBranchExecution: true,
|
|
1108
918
|
branches,
|
|
1109
|
-
|
|
1110
|
-
|
|
919
|
+
sharedBuffer: { orderId: '12345' },
|
|
920
|
+
markConcurrentBranch: true // Branches run in parallel
|
|
1111
921
|
});
|
|
1112
922
|
|
|
1113
|
-
|
|
1114
|
-
console.log(result.branchExecutionHistory); // Branch-level execution history
|
|
923
|
+
// Both branches access/modify sharedBuffer
|
|
1115
924
|
```
|
|
1116
925
|
|
|
1117
|
-
|
|
1118
|
-
|
|
1119
|
-
|
|
1120
|
-
{
|
|
1121
|
-
id: 'high-priority-branch',
|
|
1122
|
-
markConcurrentBranch: false,
|
|
1123
|
-
commonConfig: { // Branch-level config overrides
|
|
1124
|
-
commonAttempts: 5,
|
|
1125
|
-
commonWait: 2000,
|
|
1126
|
-
commonCache: { enabled: true, ttl: 120000 }
|
|
1127
|
-
},
|
|
1128
|
-
phases: [...]
|
|
1129
|
-
}
|
|
1130
|
-
];
|
|
1131
|
-
```
|
|
926
|
+
### Graph-Based Workflows
|
|
927
|
+
|
|
928
|
+
For complex topologies with explicit dependencies, use DAG execution.
|
|
1132
929
|
|
|
1133
|
-
|
|
1134
|
-
|
|
1135
|
-
|
|
1136
|
-
- Branch decision hooks can terminate the entire workflow
|
|
1137
|
-
- Supports all execution patterns (mixed, non-linear) within branches
|
|
930
|
+
#### Parallel Groups
|
|
931
|
+
|
|
932
|
+
Execute multiple phases concurrently within a group.
|
|
1138
933
|
|
|
1139
|
-
**Branch Decision Hooks**:
|
|
1140
934
|
```typescript
|
|
1141
|
-
|
|
1142
|
-
|
|
1143
|
-
|
|
1144
|
-
|
|
1145
|
-
|
|
1146
|
-
|
|
1147
|
-
|
|
1148
|
-
|
|
1149
|
-
|
|
1150
|
-
|
|
1151
|
-
|
|
1152
|
-
|
|
1153
|
-
|
|
1154
|
-
|
|
1155
|
-
|
|
1156
|
-
|
|
935
|
+
import { stableWorkflowGraph, WorkflowGraphBuilder } from '@emmvish/stable-request';
|
|
936
|
+
|
|
937
|
+
const graph = new WorkflowGraphBuilder()
|
|
938
|
+
.addPhase('fetch-users', {
|
|
939
|
+
requests: [{
|
|
940
|
+
id: 'users',
|
|
941
|
+
requestOptions: { reqData: { path: '/users' }, resReq: true }
|
|
942
|
+
}]
|
|
943
|
+
})
|
|
944
|
+
.addPhase('fetch-posts', {
|
|
945
|
+
requests: [{
|
|
946
|
+
id: 'posts',
|
|
947
|
+
requestOptions: { reqData: { path: '/posts' }, resReq: true }
|
|
948
|
+
}]
|
|
949
|
+
})
|
|
950
|
+
.addPhase('fetch-comments', {
|
|
951
|
+
requests: [{
|
|
952
|
+
id: 'comments',
|
|
953
|
+
requestOptions: { reqData: { path: '/comments' }, resReq: true }
|
|
954
|
+
}]
|
|
955
|
+
})
|
|
956
|
+
.addParallelGroup('data-fetch', ['fetch-users', 'fetch-posts', 'fetch-comments'])
|
|
957
|
+
.setEntryPoint('data-fetch')
|
|
958
|
+
.build();
|
|
1157
959
|
|
|
1158
|
-
|
|
960
|
+
const result = await stableWorkflowGraph(graph, {
|
|
961
|
+
workflowId: 'data-aggregation'
|
|
962
|
+
});
|
|
1159
963
|
|
|
1160
|
-
|
|
964
|
+
// All 3 phases run concurrently
|
|
965
|
+
```
|
|
1161
966
|
|
|
1162
|
-
|
|
967
|
+
#### Merge Points
|
|
968
|
+
|
|
969
|
+
Synchronize multiple predecessor phases.
|
|
1163
970
|
|
|
1164
971
|
```typescript
|
|
1165
|
-
|
|
1166
|
-
|
|
1167
|
-
|
|
1168
|
-
|
|
1169
|
-
|
|
1170
|
-
|
|
1171
|
-
|
|
1172
|
-
|
|
1173
|
-
|
|
1174
|
-
|
|
1175
|
-
|
|
1176
|
-
|
|
1177
|
-
|
|
1178
|
-
|
|
1179
|
-
|
|
1180
|
-
|
|
1181
|
-
commonWait: 500
|
|
1182
|
-
},
|
|
1183
|
-
phases: [{
|
|
1184
|
-
id: 'my-phase',
|
|
1185
|
-
commonConfig: {
|
|
1186
|
-
// Phase-level config (overrides branch and workflow)
|
|
1187
|
-
commonAttempts: 1,
|
|
1188
|
-
commonCache: { enabled: false }
|
|
1189
|
-
},
|
|
1190
|
-
requests: [{
|
|
1191
|
-
id: 'my-request',
|
|
1192
|
-
requestOptions: {
|
|
1193
|
-
// Request-level config (highest priority)
|
|
1194
|
-
reqData: { path: '/critical' },
|
|
1195
|
-
attempts: 10,
|
|
1196
|
-
wait: 100,
|
|
1197
|
-
cache: { enabled: true, ttl: 300000 }
|
|
1198
|
-
}
|
|
1199
|
-
}]
|
|
972
|
+
const graph = new WorkflowGraphBuilder()
|
|
973
|
+
.addPhase('fetch-a', {
|
|
974
|
+
requests: [{ id: 'a', requestOptions: { reqData: { path: '/a' }, resReq: true } }]
|
|
975
|
+
})
|
|
976
|
+
.addPhase('fetch-b', {
|
|
977
|
+
requests: [{ id: 'b', requestOptions: { reqData: { path: '/b' }, resReq: true } }]
|
|
978
|
+
})
|
|
979
|
+
.addMergePoint('sync', ['fetch-a', 'fetch-b'])
|
|
980
|
+
.addPhase('aggregate', {
|
|
981
|
+
functions: [{
|
|
982
|
+
id: 'combine',
|
|
983
|
+
functionOptions: {
|
|
984
|
+
fn: () => 'combined',
|
|
985
|
+
args: [],
|
|
986
|
+
returnResult: true
|
|
987
|
+
}
|
|
1200
988
|
}]
|
|
1201
|
-
}
|
|
989
|
+
})
|
|
990
|
+
.connectSequence('fetch-a', 'sync')
|
|
991
|
+
.connectSequence('fetch-b', 'sync')
|
|
992
|
+
.connectSequence('sync', 'aggregate')
|
|
993
|
+
.setEntryPoint('fetch-a')
|
|
994
|
+
.build();
|
|
995
|
+
|
|
996
|
+
const result = await stableWorkflowGraph(graph, {
|
|
997
|
+
workflowId: 'parallel-sync'
|
|
1202
998
|
});
|
|
1203
|
-
```
|
|
1204
999
|
|
|
1205
|
-
|
|
1000
|
+
// fetch-a and fetch-b run in parallel
|
|
1001
|
+
// aggregate waits for both to complete
|
|
1002
|
+
```
|
|
1206
1003
|
|
|
1207
|
-
|
|
1004
|
+
#### Linear Helper
|
|
1208
1005
|
|
|
1209
|
-
|
|
1006
|
+
Convenience function for sequential phase chains.
|
|
1210
1007
|
|
|
1211
1008
|
```typescript
|
|
1212
|
-
|
|
1009
|
+
import { createLinearWorkflowGraph } from '@emmvish/stable-request';
|
|
1010
|
+
|
|
1011
|
+
const phases = [
|
|
1213
1012
|
{
|
|
1214
|
-
id: '
|
|
1215
|
-
|
|
1216
|
-
requestOptions: { reqData: { path: '/critical/1' }, resReq: true }
|
|
1013
|
+
id: 'init',
|
|
1014
|
+
requests: [{ id: 'setup', requestOptions: { reqData: { path: '/init' }, resReq: true } }]
|
|
1217
1015
|
},
|
|
1218
1016
|
{
|
|
1219
|
-
id: '
|
|
1220
|
-
|
|
1221
|
-
requestOptions: { reqData: { path: '/critical/2' }, resReq: true }
|
|
1017
|
+
id: 'process',
|
|
1018
|
+
requests: [{ id: 'do-work', requestOptions: { reqData: { path: '/work' }, resReq: true } }]
|
|
1222
1019
|
},
|
|
1223
1020
|
{
|
|
1224
|
-
id: '
|
|
1225
|
-
|
|
1226
|
-
requestOptions: { reqData: { path: '/optional/1' }, resReq: false }
|
|
1021
|
+
id: 'finalize',
|
|
1022
|
+
requests: [{ id: 'cleanup', requestOptions: { reqData: { path: '/cleanup' }, resReq: true } }]
|
|
1227
1023
|
}
|
|
1228
1024
|
];
|
|
1229
1025
|
|
|
1230
|
-
|
|
1231
|
-
|
|
1232
|
-
|
|
1233
|
-
|
|
1234
|
-
requestGroups: [
|
|
1235
|
-
{
|
|
1236
|
-
groupId: 'critical',
|
|
1237
|
-
commonAttempts: 5, // Critical requests: 5 attempts
|
|
1238
|
-
commonWait: 2000,
|
|
1239
|
-
commonRetryStrategy: RETRY_STRATEGIES.EXPONENTIAL,
|
|
1240
|
-
commonFinalErrorAnalyzer: async () => false // Never suppress errors
|
|
1241
|
-
},
|
|
1242
|
-
{
|
|
1243
|
-
groupId: 'optional',
|
|
1244
|
-
commonAttempts: 2, // Optional requests: 2 attempts
|
|
1245
|
-
commonWait: 500,
|
|
1246
|
-
commonFinalErrorAnalyzer: async () => true // Suppress errors (return false)
|
|
1247
|
-
}
|
|
1248
|
-
]
|
|
1026
|
+
const graph = createLinearWorkflowGraph(phases);
|
|
1027
|
+
|
|
1028
|
+
const result = await stableWorkflowGraph(graph, {
|
|
1029
|
+
workflowId: 'linear-workflow'
|
|
1249
1030
|
});
|
|
1250
1031
|
```
|
|
1251
1032
|
|
|
1252
|
-
|
|
1253
|
-
|
|
1254
|
-
|
|
1255
|
-
- Grouped logging and monitoring
|
|
1033
|
+
---
|
|
1034
|
+
|
|
1035
|
+
## Configuration & State
|
|
1256
1036
|
|
|
1257
|
-
###
|
|
1037
|
+
### Config Cascading
|
|
1258
1038
|
|
|
1259
|
-
|
|
1039
|
+
Define defaults globally; override at group, phase, branch, or item level.
|
|
1260
1040
|
|
|
1261
|
-
**Shared Buffer**:
|
|
1262
1041
|
```typescript
|
|
1263
|
-
|
|
1264
|
-
|
|
1265
|
-
userId: null,
|
|
1266
|
-
metrics: []
|
|
1267
|
-
};
|
|
1042
|
+
import { stableWorkflow } from '@emmvish/stable-request';
|
|
1043
|
+
import type { STABLE_WORKFLOW_PHASE } from '@emmvish/stable-request';
|
|
1268
1044
|
|
|
1269
|
-
const phases = [
|
|
1270
|
-
{
|
|
1271
|
-
id: 'auth',
|
|
1272
|
-
requests: [{
|
|
1273
|
-
id: 'login',
|
|
1274
|
-
requestOptions: {
|
|
1275
|
-
reqData: { path: '/login', method: REQUEST_METHODS.POST },
|
|
1276
|
-
resReq: true,
|
|
1277
|
-
preExecution: {
|
|
1278
|
-
preExecutionHook: ({ commonBuffer }) => {
|
|
1279
|
-
// Write to buffer after response
|
|
1280
|
-
return {};
|
|
1281
|
-
},
|
|
1282
|
-
preExecutionHookParams: {},
|
|
1283
|
-
applyPreExecutionConfigOverride: false,
|
|
1284
|
-
continueOnPreExecutionHookFailure: false
|
|
1285
|
-
}
|
|
1286
|
-
}
|
|
1287
|
-
}]
|
|
1288
|
-
},
|
|
1045
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
1289
1046
|
{
|
|
1290
|
-
id: '
|
|
1291
|
-
|
|
1292
|
-
|
|
1293
|
-
|
|
1294
|
-
|
|
1295
|
-
|
|
1296
|
-
|
|
1297
|
-
|
|
1298
|
-
|
|
1299
|
-
|
|
1300
|
-
reqData: {
|
|
1301
|
-
headers: {
|
|
1302
|
-
'Authorization': `Bearer ${commonBuffer.authToken}`
|
|
1303
|
-
}
|
|
1304
|
-
}
|
|
1305
|
-
};
|
|
1306
|
-
},
|
|
1307
|
-
applyPreExecutionConfigOverride: true // Apply returned config
|
|
1047
|
+
id: 'phase-1',
|
|
1048
|
+
attempts: 5, // Override global attempts for this phase
|
|
1049
|
+
wait: 1000,
|
|
1050
|
+
requests: [
|
|
1051
|
+
{
|
|
1052
|
+
id: 'req-1',
|
|
1053
|
+
requestOptions: {
|
|
1054
|
+
reqData: { path: '/data' },
|
|
1055
|
+
resReq: true,
|
|
1056
|
+
attempts: 2 // Override phase attempts for this item
|
|
1308
1057
|
}
|
|
1309
1058
|
}
|
|
1310
|
-
|
|
1059
|
+
]
|
|
1311
1060
|
}
|
|
1312
1061
|
];
|
|
1313
1062
|
|
|
1314
|
-
await stableWorkflow(phases, {
|
|
1315
|
-
|
|
1316
|
-
|
|
1063
|
+
const result = await stableWorkflow(phases, {
|
|
1064
|
+
workflowId: 'cascade-demo',
|
|
1065
|
+
commonAttempts: 1, // Global default
|
|
1066
|
+
commonWait: 500,
|
|
1067
|
+
retryStrategy: 'LINEAR' // Global default
|
|
1068
|
+
// Final config per item: merge common → phase → request
|
|
1317
1069
|
});
|
|
1318
|
-
|
|
1319
|
-
console.log(sharedBuffer); // Updated with data from workflow
|
|
1320
1070
|
```
|
|
1321
1071
|
|
|
1322
|
-
|
|
1323
|
-
- Dynamic header injection (auth tokens, correlation IDs)
|
|
1324
|
-
- Request payload transformation based on previous responses
|
|
1325
|
-
- Conditional request configuration (skip, modify, enhance)
|
|
1326
|
-
- Cross-phase state management
|
|
1072
|
+
Hierarchy: global → group → phase → branch → item. Lower levels override.
|
|
1327
1073
|
|
|
1328
|
-
|
|
1329
|
-
```typescript
|
|
1330
|
-
{
|
|
1331
|
-
preExecution: {
|
|
1332
|
-
preExecutionHook: async ({ commonBuffer, inputParams }) => {
|
|
1333
|
-
// May throw error
|
|
1334
|
-
const token = await fetchTokenFromExternalSource();
|
|
1335
|
-
return { reqData: { headers: { 'Authorization': token } } };
|
|
1336
|
-
},
|
|
1337
|
-
continueOnPreExecutionHookFailure: true // Continue even if hook fails
|
|
1338
|
-
}
|
|
1339
|
-
}
|
|
1340
|
-
```
|
|
1074
|
+
### Shared & State Buffers
|
|
1341
1075
|
|
|
1342
|
-
|
|
1076
|
+
Pass mutable state across phases, branches, and items.
|
|
1343
1077
|
|
|
1344
|
-
|
|
1078
|
+
#### Shared Buffer (Workflow/Gateway)
|
|
1345
1079
|
|
|
1346
1080
|
```typescript
|
|
1347
|
-
|
|
1081
|
+
import { stableWorkflow } from '@emmvish/stable-request';
|
|
1082
|
+
import type { STABLE_WORKFLOW_PHASE } from '@emmvish/stable-request';
|
|
1083
|
+
|
|
1084
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
1348
1085
|
{
|
|
1349
|
-
id: '
|
|
1350
|
-
requests: [
|
|
1351
|
-
|
|
1352
|
-
|
|
1353
|
-
|
|
1354
|
-
|
|
1086
|
+
id: 'fetch',
|
|
1087
|
+
requests: [
|
|
1088
|
+
{
|
|
1089
|
+
id: 'user-data',
|
|
1090
|
+
requestOptions: {
|
|
1091
|
+
reqData: { path: '/users/1' },
|
|
1092
|
+
resReq: true,
|
|
1093
|
+
handleSuccessfulAttemptData: ({ successfulAttemptData, stableRequestOptions }) => {
|
|
1094
|
+
// Mutate shared buffer
|
|
1095
|
+
const sharedBuffer = (stableRequestOptions as any).sharedBuffer;
|
|
1096
|
+
sharedBuffer.userId = (successfulAttemptData.data as any).id;
|
|
1097
|
+
}
|
|
1098
|
+
}
|
|
1355
1099
|
}
|
|
1356
|
-
|
|
1357
|
-
|
|
1100
|
+
]
|
|
1101
|
+
},
|
|
1102
|
+
{
|
|
1103
|
+
id: 'use-shared-data',
|
|
1104
|
+
requests: [
|
|
1105
|
+
{
|
|
1106
|
+
id: 'dependent-call',
|
|
1107
|
+
requestOptions: {
|
|
1108
|
+
reqData: { path: '/user-posts' },
|
|
1109
|
+
resReq: true,
|
|
1110
|
+
preExecution: {
|
|
1111
|
+
preExecutionHook: async ({ stableRequestOptions, commonBuffer }) => {
|
|
1112
|
+
const sharedBuffer = (stableRequestOptions as any).sharedBuffer;
|
|
1113
|
+
console.log(`Using userId: ${sharedBuffer.userId}`);
|
|
1114
|
+
}
|
|
1115
|
+
}
|
|
1116
|
+
}
|
|
1117
|
+
}
|
|
1118
|
+
]
|
|
1358
1119
|
}
|
|
1359
1120
|
];
|
|
1360
|
-
```
|
|
1361
1121
|
|
|
1362
|
-
|
|
1122
|
+
const result = await stableWorkflow(phases, {
|
|
1123
|
+
workflowId: 'shared-state-demo',
|
|
1124
|
+
sharedBuffer: {} // Mutable across phases
|
|
1125
|
+
});
|
|
1126
|
+
```
|
|
1363
1127
|
|
|
1364
|
-
|
|
1128
|
+
#### Common Buffer (Request Level)
|
|
1365
1129
|
|
|
1366
1130
|
```typescript
|
|
1367
|
-
|
|
1368
|
-
|
|
1369
|
-
|
|
1370
|
-
|
|
1371
|
-
|
|
1372
|
-
|
|
1373
|
-
|
|
1374
|
-
|
|
1375
|
-
|
|
1376
|
-
|
|
1377
|
-
|
|
1378
|
-
|
|
1379
|
-
return branch;
|
|
1131
|
+
import { stableRequest } from '@emmvish/stable-request';
|
|
1132
|
+
|
|
1133
|
+
const commonBuffer = { transactionId: null };
|
|
1134
|
+
|
|
1135
|
+
const result = await stableRequest({
|
|
1136
|
+
reqData: { path: '/transaction/start' },
|
|
1137
|
+
resReq: true,
|
|
1138
|
+
commonBuffer,
|
|
1139
|
+
preExecution: {
|
|
1140
|
+
preExecutionHook: async ({ commonBuffer, stableRequestOptions }) => {
|
|
1141
|
+
// commonBuffer writable here
|
|
1142
|
+
commonBuffer.userId = '123';
|
|
1380
1143
|
}
|
|
1144
|
+
},
|
|
1145
|
+
handleSuccessfulAttemptData: ({ successfulAttemptData }) => {
|
|
1146
|
+
// commonBuffer readable in handlers
|
|
1147
|
+
console.log(`Transaction for user ${commonBuffer.userId} done`);
|
|
1381
1148
|
}
|
|
1382
|
-
|
|
1149
|
+
});
|
|
1383
1150
|
```
|
|
1384
1151
|
|
|
1385
|
-
|
|
1152
|
+
---
|
|
1386
1153
|
|
|
1387
|
-
|
|
1154
|
+
## Hooks & Observability
|
|
1388
1155
|
|
|
1389
|
-
|
|
1390
|
-
The persistence function operates in two modes:
|
|
1391
|
-
- **LOAD Mode**: When `buffer` is empty/null, return the stored state
|
|
1392
|
-
- **STORE Mode**: When `buffer` contains data, save it to your storage
|
|
1156
|
+
### Pre-Execution Hooks
|
|
1393
1157
|
|
|
1394
|
-
|
|
1158
|
+
Modify config or state before execution.
|
|
1395
1159
|
|
|
1396
1160
|
```typescript
|
|
1397
|
-
import
|
|
1398
|
-
|
|
1399
|
-
const redis = new Redis();
|
|
1161
|
+
import { stableRequest } from '@emmvish/stable-request';
|
|
1400
1162
|
|
|
1401
|
-
const
|
|
1402
|
-
|
|
1403
|
-
|
|
1404
|
-
|
|
1405
|
-
|
|
1406
|
-
|
|
1407
|
-
|
|
1408
|
-
|
|
1409
|
-
|
|
1410
|
-
|
|
1411
|
-
|
|
1412
|
-
|
|
1413
|
-
try {
|
|
1414
|
-
if (isStoring) {
|
|
1415
|
-
// STORE MODE: Save with metadata
|
|
1416
|
-
const stateWithMeta = {
|
|
1417
|
-
...buffer,
|
|
1418
|
-
_meta: {
|
|
1419
|
-
timestamp: new Date().toISOString(),
|
|
1420
|
-
version: (buffer._meta?.version || 0) + 1
|
|
1163
|
+
const result = await stableRequest({
|
|
1164
|
+
reqData: { path: '/secure-data' },
|
|
1165
|
+
resReq: true,
|
|
1166
|
+
preExecution: {
|
|
1167
|
+
preExecutionHook: async ({ inputParams, commonBuffer, stableRequestOptions }) => {
|
|
1168
|
+
// Dynamically fetch auth token
|
|
1169
|
+
const token = await getAuthToken();
|
|
1170
|
+
|
|
1171
|
+
// Return partial config override
|
|
1172
|
+
return {
|
|
1173
|
+
reqData: {
|
|
1174
|
+
headers: { Authorization: `Bearer ${token}` }
|
|
1421
1175
|
}
|
|
1422
1176
|
};
|
|
1423
|
-
await redis.setex(stateKey, ttl, JSON.stringify(stateWithMeta));
|
|
1424
|
-
console.log(`💾 State saved (v${stateWithMeta._meta.version})`);
|
|
1425
|
-
} else {
|
|
1426
|
-
// LOAD MODE: Retrieve state
|
|
1427
|
-
const data = await redis.get(stateKey);
|
|
1428
|
-
return data ? JSON.parse(data) : {};
|
|
1429
|
-
}
|
|
1430
|
-
} finally {
|
|
1431
|
-
if (enableLocking) {
|
|
1432
|
-
await redis.del(lockKey); // Release lock
|
|
1433
|
-
}
|
|
1434
|
-
}
|
|
1435
|
-
|
|
1436
|
-
return {};
|
|
1437
|
-
};
|
|
1438
|
-
|
|
1439
|
-
// Use with workflow-level persistence (applies to all phases)
|
|
1440
|
-
await stableWorkflow(phases, {
|
|
1441
|
-
workflowId: 'distributed-job-456',
|
|
1442
|
-
commonStatePersistence: {
|
|
1443
|
-
persistenceFunction: persistToRedis,
|
|
1444
|
-
persistenceParams: {
|
|
1445
|
-
ttl: 3600,
|
|
1446
|
-
enableLocking: true // Enable distributed locking
|
|
1447
1177
|
},
|
|
1448
|
-
|
|
1449
|
-
|
|
1450
|
-
|
|
1451
|
-
|
|
1178
|
+
preExecutionHookParams: { context: 'auth-fetch' },
|
|
1179
|
+
applyPreExecutionConfigOverride: true,
|
|
1180
|
+
continueOnPreExecutionHookFailure: false
|
|
1181
|
+
}
|
|
1452
1182
|
});
|
|
1453
1183
|
```
|
|
1454
1184
|
|
|
1455
|
-
|
|
1185
|
+
### Analysis Hooks
|
|
1186
|
+
|
|
1187
|
+
Validate responses and errors.
|
|
1188
|
+
|
|
1189
|
+
#### Response Analyzer
|
|
1456
1190
|
|
|
1457
1191
|
```typescript
|
|
1458
|
-
|
|
1459
|
-
|
|
1460
|
-
|
|
1461
|
-
|
|
1462
|
-
|
|
1463
|
-
|
|
1464
|
-
|
|
1465
|
-
|
|
1466
|
-
|
|
1467
|
-
|
|
1468
|
-
|
|
1469
|
-
|
|
1470
|
-
|
|
1471
|
-
progress: buffer.progress || existing.progress || 0,
|
|
1472
|
-
lastUpdated: new Date().toISOString()
|
|
1473
|
-
};
|
|
1474
|
-
await redis.setex(checkpointKey, 7200, JSON.stringify(checkpoint));
|
|
1475
|
-
} else {
|
|
1476
|
-
// LOAD: Return checkpoint data
|
|
1477
|
-
const data = await redis.get(checkpointKey);
|
|
1478
|
-
return data ? JSON.parse(data) : { completedPhases: [] };
|
|
1192
|
+
import { stableRequest } from '@emmvish/stable-request';
|
|
1193
|
+
|
|
1194
|
+
type ApiResponse = { id: number; status: 'active' | 'inactive' };
|
|
1195
|
+
|
|
1196
|
+
const result = await stableRequest<unknown, ApiResponse>({
|
|
1197
|
+
reqData: { path: '/resource' },
|
|
1198
|
+
resReq: true,
|
|
1199
|
+
responseAnalyzer: ({ data, reqData, trialMode }) => {
|
|
1200
|
+
// Return true to accept, false to retry
|
|
1201
|
+
if (!data || typeof data !== 'object') return false;
|
|
1202
|
+
if (!('id' in data)) return false;
|
|
1203
|
+
if ((data as any).status !== 'active') return false;
|
|
1204
|
+
return true;
|
|
1479
1205
|
}
|
|
1480
|
-
|
|
1481
|
-
|
|
1206
|
+
});
|
|
1207
|
+
```
|
|
1482
1208
|
|
|
1483
|
-
|
|
1484
|
-
|
|
1485
|
-
|
|
1486
|
-
|
|
1487
|
-
|
|
1488
|
-
|
|
1489
|
-
|
|
1490
|
-
|
|
1491
|
-
|
|
1492
|
-
|
|
1493
|
-
|
|
1494
|
-
|
|
1495
|
-
|
|
1496
|
-
|
|
1497
|
-
|
|
1498
|
-
|
|
1499
|
-
...(sharedBuffer.completedPhases || []),
|
|
1500
|
-
'phase-1'
|
|
1501
|
-
];
|
|
1502
|
-
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
1503
|
-
}
|
|
1504
|
-
return { action: PHASE_DECISION_ACTIONS.TERMINATE };
|
|
1505
|
-
}
|
|
1506
|
-
},
|
|
1507
|
-
{
|
|
1508
|
-
id: 'phase-2',
|
|
1509
|
-
requests: [...],
|
|
1510
|
-
phaseDecisionHook: async ({ phaseResult, sharedBuffer }) => {
|
|
1511
|
-
if (sharedBuffer.completedPhases?.includes('phase-2')) {
|
|
1512
|
-
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
1513
|
-
}
|
|
1514
|
-
if (phaseResult.success) {
|
|
1515
|
-
sharedBuffer.completedPhases = [
|
|
1516
|
-
...(sharedBuffer.completedPhases || []),
|
|
1517
|
-
'phase-2'
|
|
1518
|
-
];
|
|
1519
|
-
}
|
|
1520
|
-
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
1209
|
+
#### Error Analyzer
|
|
1210
|
+
|
|
1211
|
+
Decide whether to suppress error gracefully.
|
|
1212
|
+
|
|
1213
|
+
```typescript
|
|
1214
|
+
import { stableRequest } from '@emmvish/stable-request';
|
|
1215
|
+
|
|
1216
|
+
const result = await stableRequest({
|
|
1217
|
+
reqData: { path: '/optional-feature' },
|
|
1218
|
+
resReq: true,
|
|
1219
|
+
finalErrorAnalyzer: ({ error, reqData, trialMode }) => {
|
|
1220
|
+
// Return true to suppress error and return failure result
|
|
1221
|
+
// Return false to throw error
|
|
1222
|
+
if (error.code === 'ECONNREFUSED') {
|
|
1223
|
+
console.warn('Service unavailable, continuing with fallback');
|
|
1224
|
+
return true; // Suppress, don't throw
|
|
1521
1225
|
}
|
|
1226
|
+
return false; // Throw
|
|
1522
1227
|
}
|
|
1523
|
-
];
|
|
1524
|
-
|
|
1525
|
-
await stableWorkflow(phases, {
|
|
1526
|
-
workflowId: 'resumable-workflow-789',
|
|
1527
|
-
enableNonLinearExecution: true,
|
|
1528
|
-
sharedBuffer: { completedPhases: [] },
|
|
1529
|
-
commonStatePersistence: {
|
|
1530
|
-
persistenceFunction: createCheckpoint,
|
|
1531
|
-
persistenceParams: { ttl: 7200 },
|
|
1532
|
-
loadBeforeHooks: true,
|
|
1533
|
-
storeAfterHooks: true
|
|
1534
|
-
},
|
|
1535
|
-
commonRequestData: { hostname: 'api.example.com' }
|
|
1536
1228
|
});
|
|
1229
|
+
|
|
1230
|
+
if (result.success) {
|
|
1231
|
+
console.log('Got data:', result.data);
|
|
1232
|
+
} else {
|
|
1233
|
+
console.log('Service offline, but we continue');
|
|
1234
|
+
}
|
|
1537
1235
|
```
|
|
1538
1236
|
|
|
1539
|
-
###
|
|
1237
|
+
### Handler Hooks
|
|
1540
1238
|
|
|
1541
|
-
|
|
1239
|
+
Custom logging and processing.
|
|
1240
|
+
|
|
1241
|
+
#### Success Handler
|
|
1542
1242
|
|
|
1543
|
-
**Request-Level Hooks**:
|
|
1544
1243
|
```typescript
|
|
1545
|
-
|
|
1546
|
-
|
|
1244
|
+
import { stableRequest } from '@emmvish/stable-request';
|
|
1245
|
+
|
|
1246
|
+
const result = await stableRequest({
|
|
1247
|
+
reqData: { path: '/data' },
|
|
1248
|
+
resReq: true,
|
|
1249
|
+
logAllSuccessfulAttempts: true,
|
|
1250
|
+
handleSuccessfulAttemptData: ({
|
|
1251
|
+
successfulAttemptData,
|
|
1252
|
+
reqData,
|
|
1253
|
+
maxSerializableChars,
|
|
1254
|
+
executionContext
|
|
1255
|
+
}) => {
|
|
1256
|
+
// Custom logging, metrics, state updates
|
|
1257
|
+
console.log(
|
|
1258
|
+
`Success in context ${executionContext.workflowId}`,
|
|
1259
|
+
`data:`,
|
|
1260
|
+
successfulAttemptData.data
|
|
1261
|
+
);
|
|
1262
|
+
}
|
|
1263
|
+
});
|
|
1264
|
+
```
|
|
1265
|
+
|
|
1266
|
+
#### Error Handler
|
|
1267
|
+
|
|
1268
|
+
```typescript
|
|
1269
|
+
const result = await stableRequest({
|
|
1270
|
+
reqData: { path: '/data' },
|
|
1547
1271
|
resReq: true,
|
|
1548
|
-
attempts: 3,
|
|
1549
|
-
|
|
1550
|
-
// Validate response content
|
|
1551
|
-
responseAnalyzer: async ({ data, reqData, params }) => {
|
|
1552
|
-
console.log('Analyzing response:', data);
|
|
1553
|
-
return data.status === 'success'; // false = retry
|
|
1554
|
-
},
|
|
1555
|
-
|
|
1556
|
-
// Custom error handling
|
|
1557
|
-
handleErrors: async ({ errorLog, reqData, commonBuffer }) => {
|
|
1558
|
-
console.error('Request failed:', errorLog);
|
|
1559
|
-
await sendToMonitoring(errorLog);
|
|
1560
|
-
},
|
|
1561
|
-
|
|
1562
|
-
// Log successful attempts
|
|
1563
|
-
handleSuccessfulAttemptData: async ({ successfulAttemptData, reqData }) => {
|
|
1564
|
-
console.log('Request succeeded:', successfulAttemptData);
|
|
1565
|
-
},
|
|
1566
|
-
|
|
1567
|
-
// Analyze final error after all retries
|
|
1568
|
-
finalErrorAnalyzer: async ({ error, reqData }) => {
|
|
1569
|
-
console.error('All retries exhausted:', error);
|
|
1570
|
-
return error.message.includes('404'); // true = return false instead of throw
|
|
1571
|
-
},
|
|
1572
|
-
|
|
1573
|
-
// Pass custom parameters to hooks
|
|
1574
|
-
hookParams: {
|
|
1575
|
-
responseAnalyzerParams: { expectedFormat: 'json' },
|
|
1576
|
-
handleErrorsParams: { alertChannel: 'slack' }
|
|
1577
|
-
},
|
|
1578
|
-
|
|
1579
1272
|
logAllErrors: true,
|
|
1580
|
-
|
|
1273
|
+
handleErrors: ({ errorLog, reqData, executionContext }) => {
|
|
1274
|
+
// Custom error logging, alerting, retry logic
|
|
1275
|
+
console.error(
|
|
1276
|
+
`Error in ${executionContext.workflowId}:`,
|
|
1277
|
+
errorLog.errorMessage,
|
|
1278
|
+
`Retryable: ${errorLog.isRetryable}`
|
|
1279
|
+
);
|
|
1280
|
+
}
|
|
1581
1281
|
});
|
|
1582
1282
|
```
|
|
1583
1283
|
|
|
1584
|
-
|
|
1284
|
+
#### Phase Handlers (Workflow)
|
|
1285
|
+
|
|
1585
1286
|
```typescript
|
|
1586
|
-
|
|
1587
|
-
|
|
1588
|
-
|
|
1589
|
-
|
|
1590
|
-
|
|
1591
|
-
|
|
1592
|
-
|
|
1593
|
-
|
|
1594
|
-
|
|
1595
|
-
|
|
1287
|
+
import { stableWorkflow } from '@emmvish/stable-request';
|
|
1288
|
+
import type { STABLE_WORKFLOW_PHASE } from '@emmvish/stable-request';
|
|
1289
|
+
|
|
1290
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
1291
|
+
{
|
|
1292
|
+
id: 'phase-1',
|
|
1293
|
+
requests: [{ id: 'r1', requestOptions: { reqData: { path: '/data' }, resReq: true } }]
|
|
1294
|
+
}
|
|
1295
|
+
];
|
|
1296
|
+
|
|
1297
|
+
const result = await stableWorkflow(phases, {
|
|
1298
|
+
workflowId: 'wf-handlers',
|
|
1299
|
+
handlePhaseCompletion: ({ phaseResult, workflowId }) => {
|
|
1300
|
+
console.log(`Phase ${phaseResult.phaseId} complete in ${workflowId}`);
|
|
1596
1301
|
},
|
|
1597
|
-
|
|
1598
|
-
// Called when a phase fails
|
|
1599
|
-
handlePhaseError: async ({ workflowId, error, phaseResult }) => {
|
|
1302
|
+
handlePhaseError: ({ phaseResult, error, workflowId }) => {
|
|
1600
1303
|
console.error(`Phase ${phaseResult.phaseId} failed:`, error);
|
|
1601
|
-
await alertOnCall(error);
|
|
1602
1304
|
},
|
|
1603
|
-
|
|
1604
|
-
// Monitor non-linear execution decisions
|
|
1605
|
-
handlePhaseDecision: async ({ decision, phaseResult }) => {
|
|
1305
|
+
handlePhaseDecision: ({ decision, phaseResult }) => {
|
|
1606
1306
|
console.log(`Phase decision: ${decision.action}`);
|
|
1607
|
-
|
|
1608
|
-
console.log(`Target: ${decision.targetPhaseId}`);
|
|
1609
|
-
}
|
|
1610
|
-
},
|
|
1611
|
-
|
|
1612
|
-
// Monitor branch completion
|
|
1613
|
-
handleBranchCompletion: async ({ workflowId, branchResult }) => {
|
|
1614
|
-
console.log(`Branch ${branchResult.branchId} completed`);
|
|
1615
|
-
},
|
|
1616
|
-
|
|
1617
|
-
// Monitor branch decisions
|
|
1618
|
-
handleBranchDecision: async ({ workflowId, branchId, branchResults, success }) => {
|
|
1619
|
-
console.log(`Branch ID: ${branchId}`);
|
|
1620
|
-
},
|
|
1621
|
-
|
|
1622
|
-
// Pass parameters to workflow hooks
|
|
1623
|
-
workflowHookParams: {
|
|
1624
|
-
handlePhaseCompletionParams: { environment: 'production' },
|
|
1625
|
-
handlePhaseErrorParams: { severity: 'high' }
|
|
1626
|
-
},
|
|
1627
|
-
|
|
1628
|
-
logPhaseResults: true,
|
|
1629
|
-
commonRequestData: { hostname: 'api.example.com' }
|
|
1307
|
+
}
|
|
1630
1308
|
});
|
|
1631
1309
|
```
|
|
1632
1310
|
|
|
1633
|
-
|
|
1311
|
+
### Decision Hooks
|
|
1312
|
+
|
|
1313
|
+
Dynamically determine workflow flow.
|
|
1314
|
+
|
|
1634
1315
|
```typescript
|
|
1316
|
+
import { stableWorkflow, PHASE_DECISION_ACTIONS } from '@emmvish/stable-request';
|
|
1317
|
+
import type { STABLE_WORKFLOW_PHASE } from '@emmvish/stable-request';
|
|
1318
|
+
|
|
1319
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
1320
|
+
{
|
|
1321
|
+
id: 'fetch-data',
|
|
1322
|
+
requests: [{ id: 'api', requestOptions: { reqData: { path: '/data' }, resReq: true } }],
|
|
1323
|
+
phaseDecisionHook: async ({ phaseResult, sharedBuffer, executionHistory }) => {
|
|
1324
|
+
if (!phaseResult.success) {
|
|
1325
|
+
return { action: PHASE_DECISION_ACTIONS.TERMINATE };
|
|
1326
|
+
}
|
|
1327
|
+
if (phaseResult.responses[0].data?.needsRetry) {
|
|
1328
|
+
return { action: PHASE_DECISION_ACTIONS.REPLAY };
|
|
1329
|
+
}
|
|
1330
|
+
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
1331
|
+
}
|
|
1332
|
+
}
|
|
1333
|
+
];
|
|
1334
|
+
|
|
1635
1335
|
const result = await stableWorkflow(phases, {
|
|
1636
|
-
enableNonLinearExecution: true
|
|
1637
|
-
workflowId: 'tracked-workflow',
|
|
1638
|
-
commonRequestData: { hostname: 'api.example.com' }
|
|
1336
|
+
enableNonLinearExecution: true
|
|
1639
1337
|
});
|
|
1338
|
+
```
|
|
1640
1339
|
|
|
1641
|
-
|
|
1642
|
-
|
|
1643
|
-
|
|
1644
|
-
phaseId: record.phaseId,
|
|
1645
|
-
executionNumber: record.executionNumber,
|
|
1646
|
-
decision: record.decision,
|
|
1647
|
-
timestamp: record.timestamp,
|
|
1648
|
-
metadata: record.metadata
|
|
1649
|
-
});
|
|
1650
|
-
});
|
|
1340
|
+
### Metrics & Logging
|
|
1341
|
+
|
|
1342
|
+
Automatic metrics collection across all execution modes.
|
|
1651
1343
|
|
|
1652
|
-
|
|
1653
|
-
|
|
1654
|
-
|
|
1655
|
-
|
|
1656
|
-
|
|
1657
|
-
|
|
1658
|
-
}
|
|
1344
|
+
#### Request Metrics
|
|
1345
|
+
|
|
1346
|
+
```typescript
|
|
1347
|
+
import { stableRequest } from '@emmvish/stable-request';
|
|
1348
|
+
|
|
1349
|
+
const result = await stableRequest({
|
|
1350
|
+
reqData: { path: '/data' },
|
|
1351
|
+
resReq: true,
|
|
1352
|
+
attempts: 3
|
|
1659
1353
|
});
|
|
1354
|
+
|
|
1355
|
+
console.log(result.metrics); // {
|
|
1356
|
+
// totalAttempts: 2,
|
|
1357
|
+
// successfulAttempts: 1,
|
|
1358
|
+
// failedAttempts: 1,
|
|
1359
|
+
// totalExecutionTime: 450,
|
|
1360
|
+
// averageAttemptTime: 225,
|
|
1361
|
+
// infrastructureMetrics: {
|
|
1362
|
+
// circuitBreaker: { /* state, stats, config */ },
|
|
1363
|
+
// cache: { /* hits, misses, size */ },
|
|
1364
|
+
// rateLimiter: { /* limit, current rate */ },
|
|
1365
|
+
// concurrencyLimiter: { /* limit, in-flight */ }
|
|
1366
|
+
// }
|
|
1367
|
+
// }
|
|
1660
1368
|
```
|
|
1661
1369
|
|
|
1662
|
-
|
|
1370
|
+
#### Workflow Metrics
|
|
1371
|
+
|
|
1372
|
+
```typescript
|
|
1373
|
+
import { stableWorkflow } from '@emmvish/stable-request';
|
|
1374
|
+
import type { STABLE_WORKFLOW_PHASE } from '@emmvish/stable-request';
|
|
1663
1375
|
|
|
1664
|
-
|
|
1376
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
1377
|
+
{ id: 'p1', requests: [{ id: 'r1', requestOptions: { reqData: { path: '/a' }, resReq: true } }] },
|
|
1378
|
+
{ id: 'p2', requests: [{ id: 'r2', requestOptions: { reqData: { path: '/b' }, resReq: true } }] }
|
|
1379
|
+
];
|
|
1380
|
+
|
|
1381
|
+
const result = await stableWorkflow(phases, {
|
|
1382
|
+
workflowId: 'wf-metrics'
|
|
1383
|
+
});
|
|
1384
|
+
|
|
1385
|
+
console.log(result); // {
|
|
1386
|
+
// workflowId: 'wf-metrics',
|
|
1387
|
+
// success: true,
|
|
1388
|
+
// totalPhases: 2,
|
|
1389
|
+
// completedPhases: 2,
|
|
1390
|
+
// totalRequests: 2,
|
|
1391
|
+
// successfulRequests: 2,
|
|
1392
|
+
// failedRequests: 0,
|
|
1393
|
+
// workflowExecutionTime: 1200,
|
|
1394
|
+
// phases: [
|
|
1395
|
+
// { phaseId: 'p1', success: true, responses: [...], ... },
|
|
1396
|
+
// { phaseId: 'p2', success: true, responses: [...], ... }
|
|
1397
|
+
// ]
|
|
1398
|
+
// }
|
|
1399
|
+
```
|
|
1400
|
+
|
|
1401
|
+
#### Structured Error Logs
|
|
1665
1402
|
|
|
1666
1403
|
```typescript
|
|
1667
|
-
await stableRequest({
|
|
1668
|
-
reqData: {
|
|
1404
|
+
const result = await stableRequest({
|
|
1405
|
+
reqData: { path: '/flaky' },
|
|
1669
1406
|
resReq: true,
|
|
1670
1407
|
attempts: 3,
|
|
1671
|
-
|
|
1672
|
-
|
|
1673
|
-
|
|
1674
|
-
|
|
1675
|
-
|
|
1408
|
+
logAllErrors: true,
|
|
1409
|
+
handleErrors: ({ errorLog }) => {
|
|
1410
|
+
console.log(errorLog); // {
|
|
1411
|
+
// attempt: '1/3',
|
|
1412
|
+
// type: 'NetworkError',
|
|
1413
|
+
// error: 'ECONNREFUSED',
|
|
1414
|
+
// isRetryable: true,
|
|
1415
|
+
// timestamp: 1234567890
|
|
1416
|
+
// }
|
|
1676
1417
|
}
|
|
1677
1418
|
});
|
|
1419
|
+
|
|
1420
|
+
if (result.errorLogs) {
|
|
1421
|
+
console.log(`${result.errorLogs.length} errors logged`);
|
|
1422
|
+
}
|
|
1678
1423
|
```
|
|
1679
1424
|
|
|
1680
|
-
|
|
1681
|
-
- Test retry logic without hitting APIs
|
|
1682
|
-
- Simulate failure scenarios
|
|
1683
|
-
- Load testing with controlled failure rates
|
|
1684
|
-
- Development without backend dependencies
|
|
1425
|
+
---
|
|
1685
1426
|
|
|
1686
|
-
##
|
|
1427
|
+
## Advanced Features
|
|
1687
1428
|
|
|
1688
|
-
###
|
|
1429
|
+
### Trial Mode
|
|
1430
|
+
|
|
1431
|
+
Dry-run workflows without side effects; simulate failures.
|
|
1689
1432
|
|
|
1690
1433
|
```typescript
|
|
1691
|
-
|
|
1692
|
-
|
|
1693
|
-
|
|
1694
|
-
|
|
1695
|
-
requests: [
|
|
1696
|
-
{ id: 'users', requestOptions: { reqData: { path: '/source/users' }, resReq: true } },
|
|
1697
|
-
{ id: 'orders', requestOptions: { reqData: { path: '/source/orders' }, resReq: true } }
|
|
1698
|
-
]
|
|
1699
|
-
},
|
|
1434
|
+
import { stableWorkflow } from '@emmvish/stable-request';
|
|
1435
|
+
import type { STABLE_WORKFLOW_PHASE } from '@emmvish/stable-request';
|
|
1436
|
+
|
|
1437
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
1700
1438
|
{
|
|
1701
|
-
id: '
|
|
1439
|
+
id: 'process',
|
|
1702
1440
|
requests: [
|
|
1703
|
-
{
|
|
1704
|
-
id: '
|
|
1705
|
-
requestOptions: {
|
|
1706
|
-
reqData: { path: '/
|
|
1707
|
-
resReq: true
|
|
1708
|
-
|
|
1441
|
+
{
|
|
1442
|
+
id: 'api-call',
|
|
1443
|
+
requestOptions: {
|
|
1444
|
+
reqData: { path: '/payment/charge' },
|
|
1445
|
+
resReq: true,
|
|
1446
|
+
trialMode: {
|
|
1447
|
+
enabled: true,
|
|
1448
|
+
requestFailureProbability: 0.3 // 30% simulated failure rate
|
|
1449
|
+
}
|
|
1450
|
+
}
|
|
1709
1451
|
}
|
|
1710
1452
|
]
|
|
1711
|
-
},
|
|
1712
|
-
{
|
|
1713
|
-
id: 'upload-to-destination',
|
|
1714
|
-
concurrentExecution: true,
|
|
1715
|
-
requests: [
|
|
1716
|
-
{ id: 'upload-users', requestOptions: { reqData: { path: '/dest/users', method: REQUEST_METHODS.POST }, resReq: false } },
|
|
1717
|
-
{ id: 'upload-orders', requestOptions: { reqData: { path: '/dest/orders', method: REQUEST_METHODS.POST }, resReq: false } }
|
|
1718
|
-
]
|
|
1719
1453
|
}
|
|
1720
1454
|
];
|
|
1721
1455
|
|
|
1722
|
-
await stableWorkflow(
|
|
1723
|
-
workflowId: '
|
|
1724
|
-
|
|
1725
|
-
|
|
1726
|
-
|
|
1727
|
-
|
|
1456
|
+
const result = await stableWorkflow(phases, {
|
|
1457
|
+
workflowId: 'payment-trial',
|
|
1458
|
+
trialMode: {
|
|
1459
|
+
enabled: true,
|
|
1460
|
+
functionFailureProbability: 0.2
|
|
1461
|
+
}
|
|
1728
1462
|
});
|
|
1463
|
+
|
|
1464
|
+
// Requests/functions execute but failures are simulated
|
|
1465
|
+
// Real API calls happen; real side effects occur only if enabled
|
|
1466
|
+
// Useful for testing retry logic, decision hooks, workflow topology
|
|
1729
1467
|
```
|
|
1730
1468
|
|
|
1731
|
-
###
|
|
1469
|
+
### State Persistence
|
|
1470
|
+
|
|
1471
|
+
Persist state across retry attempts for distributed tracing.
|
|
1732
1472
|
|
|
1733
1473
|
```typescript
|
|
1734
|
-
|
|
1735
|
-
|
|
1736
|
-
|
|
1737
|
-
|
|
1738
|
-
|
|
1739
|
-
|
|
1740
|
-
|
|
1741
|
-
|
|
1742
|
-
|
|
1743
|
-
|
|
1744
|
-
|
|
1745
|
-
|
|
1746
|
-
|
|
1747
|
-
|
|
1748
|
-
|
|
1749
|
-
|
|
1750
|
-
|
|
1751
|
-
|
|
1752
|
-
|
|
1474
|
+
import { stableRequest } from '@emmvish/stable-request';
|
|
1475
|
+
|
|
1476
|
+
const result = await stableRequest({
|
|
1477
|
+
reqData: { path: '/data' },
|
|
1478
|
+
resReq: true,
|
|
1479
|
+
attempts: 3,
|
|
1480
|
+
statePersistence: {
|
|
1481
|
+
save: async (state, executionContext) => {
|
|
1482
|
+
// Save state to database or distributed cache
|
|
1483
|
+
await saveToDatabase({
|
|
1484
|
+
key: `${executionContext.workflowId}:${executionContext.requestId}`,
|
|
1485
|
+
state
|
|
1486
|
+
});
|
|
1487
|
+
},
|
|
1488
|
+
load: async (executionContext) => {
|
|
1489
|
+
// Load state for recovery
|
|
1490
|
+
return await loadFromDatabase(
|
|
1491
|
+
`${executionContext.workflowId}:${executionContext.requestId}`
|
|
1492
|
+
);
|
|
1753
1493
|
}
|
|
1754
1494
|
}
|
|
1755
|
-
];
|
|
1756
|
-
|
|
1757
|
-
const results = await stableApiGateway(requests, {
|
|
1758
|
-
concurrentExecution: false, // Sequential: try fallback only if primary fails
|
|
1759
|
-
requestGroups: [
|
|
1760
|
-
{ groupId: 'critical', commonAttempts: 3 },
|
|
1761
|
-
{ groupId: 'fallback', commonAttempts: 1 }
|
|
1762
|
-
]
|
|
1763
1495
|
});
|
|
1764
1496
|
```
|
|
1765
1497
|
|
|
1766
|
-
###
|
|
1498
|
+
### Mixed Request & Function Phases
|
|
1499
|
+
|
|
1500
|
+
Combine API calls and computations in single phases.
|
|
1767
1501
|
|
|
1768
1502
|
```typescript
|
|
1769
|
-
|
|
1770
|
-
|
|
1771
|
-
|
|
1772
|
-
|
|
1773
|
-
|
|
1774
|
-
|
|
1775
|
-
|
|
1776
|
-
|
|
1777
|
-
|
|
1778
|
-
|
|
1779
|
-
|
|
1780
|
-
|
|
1503
|
+
import { stableWorkflow, RequestOrFunction } from '@emmvish/stable-request';
|
|
1504
|
+
import type { STABLE_WORKFLOW_PHASE, API_GATEWAY_ITEM } from '@emmvish/stable-request';
|
|
1505
|
+
|
|
1506
|
+
const phase: STABLE_WORKFLOW_PHASE = {
|
|
1507
|
+
id: 'mixed-phase',
|
|
1508
|
+
items: [
|
|
1509
|
+
{
|
|
1510
|
+
type: RequestOrFunction.REQUEST,
|
|
1511
|
+
request: {
|
|
1512
|
+
id: 'fetch-user',
|
|
1513
|
+
requestOptions: {
|
|
1514
|
+
reqData: { path: '/users/1' },
|
|
1515
|
+
resReq: true
|
|
1516
|
+
}
|
|
1781
1517
|
}
|
|
1782
|
-
|
|
1783
|
-
|
|
1784
|
-
|
|
1785
|
-
|
|
1786
|
-
|
|
1787
|
-
|
|
1788
|
-
|
|
1789
|
-
|
|
1790
|
-
|
|
1791
|
-
|
|
1792
|
-
|
|
1793
|
-
|
|
1794
|
-
|
|
1518
|
+
},
|
|
1519
|
+
{
|
|
1520
|
+
type: RequestOrFunction.FUNCTION,
|
|
1521
|
+
function: {
|
|
1522
|
+
id: 'transform',
|
|
1523
|
+
functionOptions: {
|
|
1524
|
+
fn: (user?: any) => ({ name: user?.name?.toUpperCase() }),
|
|
1525
|
+
args: [],
|
|
1526
|
+
returnResult: true
|
|
1527
|
+
}
|
|
1528
|
+
}
|
|
1529
|
+
},
|
|
1530
|
+
{
|
|
1531
|
+
type: RequestOrFunction.REQUEST,
|
|
1532
|
+
request: {
|
|
1533
|
+
id: 'store-transformed',
|
|
1534
|
+
requestOptions: {
|
|
1535
|
+
reqData: { path: '/cache/user-names' },
|
|
1536
|
+
resReq: false
|
|
1537
|
+
}
|
|
1795
1538
|
}
|
|
1796
1539
|
}
|
|
1797
|
-
|
|
1798
|
-
|
|
1799
|
-
|
|
1800
|
-
|
|
1801
|
-
|
|
1802
|
-
|
|
1540
|
+
]
|
|
1541
|
+
};
|
|
1542
|
+
|
|
1543
|
+
const result = await stableWorkflow([phase], {
|
|
1544
|
+
workflowId: 'mixed-execution'
|
|
1545
|
+
});
|
|
1546
|
+
```
|
|
1547
|
+
|
|
1548
|
+
---
|
|
1549
|
+
|
|
1550
|
+
## Best Practices
|
|
1551
|
+
|
|
1552
|
+
### 1. Start Conservative, Override When Needed
|
|
1553
|
+
|
|
1554
|
+
Define global defaults; override only where necessary.
|
|
1555
|
+
|
|
1556
|
+
```typescript
|
|
1557
|
+
await stableWorkflow(phases, {
|
|
1558
|
+
// Global defaults (conservative)
|
|
1559
|
+
commonAttempts: 3,
|
|
1560
|
+
commonWait: 500,
|
|
1561
|
+
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL,
|
|
1562
|
+
|
|
1563
|
+
// Override for specific phase
|
|
1564
|
+
phases: [
|
|
1565
|
+
{
|
|
1566
|
+
id: 'fast-phase',
|
|
1567
|
+
attempts: 1, // Override: no retries
|
|
1568
|
+
requests: [...]
|
|
1569
|
+
}
|
|
1570
|
+
]
|
|
1571
|
+
});
|
|
1572
|
+
```
|
|
1573
|
+
|
|
1574
|
+
### 2. Validate Responses
|
|
1575
|
+
|
|
1576
|
+
Use analyzers to ensure data shape and freshness.
|
|
1577
|
+
|
|
1578
|
+
```typescript
|
|
1579
|
+
type ApiResponse = { id: number; lastUpdated: string };
|
|
1580
|
+
|
|
1581
|
+
const result = await stableRequest<unknown, ApiResponse>({
|
|
1582
|
+
reqData: { path: '/data' },
|
|
1583
|
+
resReq: true,
|
|
1584
|
+
responseAnalyzer: ({ data }) => {
|
|
1585
|
+
if (!data || typeof data !== 'object') return false;
|
|
1586
|
+
if (!('id' in data && 'lastUpdated' in data)) return false;
|
|
1587
|
+
const age = Date.now() - new Date((data as any).lastUpdated).getTime();
|
|
1588
|
+
if (age > 60000) return false; // Data older than 1 minute
|
|
1589
|
+
return true;
|
|
1803
1590
|
}
|
|
1804
|
-
|
|
1591
|
+
});
|
|
1592
|
+
```
|
|
1805
1593
|
|
|
1806
|
-
|
|
1807
|
-
|
|
1808
|
-
|
|
1809
|
-
|
|
1594
|
+
### 3. Cache Idempotent Reads Aggressively
|
|
1595
|
+
|
|
1596
|
+
Reduce latency and load on dependencies.
|
|
1597
|
+
|
|
1598
|
+
```typescript
|
|
1599
|
+
const userCache = new CacheManager({
|
|
1600
|
+
enabled: true,
|
|
1601
|
+
ttl: 30000, // 30 seconds
|
|
1602
|
+
respectCacheControl: true
|
|
1603
|
+
});
|
|
1604
|
+
|
|
1605
|
+
await stableRequest({
|
|
1606
|
+
reqData: { path: '/users/1' },
|
|
1607
|
+
resReq: true,
|
|
1608
|
+
cache: userCache
|
|
1609
|
+
});
|
|
1610
|
+
|
|
1611
|
+
await stableRequest({
|
|
1612
|
+
reqData: { path: '/users/1' },
|
|
1613
|
+
resReq: true,
|
|
1614
|
+
cache: userCache // Cached within 30s
|
|
1810
1615
|
});
|
|
1811
1616
|
```
|
|
1812
1617
|
|
|
1813
|
-
###
|
|
1618
|
+
### 4. Use Circuit Breaker for Unstable Services
|
|
1619
|
+
|
|
1620
|
+
Protect against cascading failures.
|
|
1814
1621
|
|
|
1815
1622
|
```typescript
|
|
1816
|
-
|
|
1623
|
+
const unstabledServiceBreaker = new CircuitBreaker({
|
|
1624
|
+
failureThresholdPercentage: 40,
|
|
1625
|
+
minimumRequests: 5,
|
|
1626
|
+
recoveryTimeoutMs: 30000,
|
|
1627
|
+
successThresholdPercentage: 80
|
|
1628
|
+
});
|
|
1817
1629
|
|
|
1818
|
-
|
|
1819
|
-
|
|
1820
|
-
minimumRequests: 5, // Minimum 5 requests before evaluation
|
|
1821
|
-
recoveryTimeoutMs: 30000, // 30s timeout in open state
|
|
1822
|
-
successThresholdPercentage: 40 // 40% success rate closes circuit
|
|
1630
|
+
await stableApiGateway(requests, {
|
|
1631
|
+
circuitBreaker: unstabledServiceBreaker
|
|
1823
1632
|
});
|
|
1633
|
+
```
|
|
1824
1634
|
|
|
1825
|
-
|
|
1826
|
-
|
|
1827
|
-
|
|
1828
|
-
|
|
1829
|
-
|
|
1830
|
-
|
|
1831
|
-
|
|
1832
|
-
|
|
1833
|
-
|
|
1834
|
-
|
|
1835
|
-
|
|
1836
|
-
|
|
1837
|
-
|
|
1838
|
-
|
|
1839
|
-
|
|
1840
|
-
await queueForRetry(eventData);
|
|
1841
|
-
}
|
|
1842
|
-
});
|
|
1843
|
-
} catch (error) {
|
|
1844
|
-
console.error('Webhook permanently failed:', error);
|
|
1845
|
-
}
|
|
1846
|
-
}
|
|
1635
|
+
### 5. Apply Rate & Concurrency Limits
|
|
1636
|
+
|
|
1637
|
+
Respect external quotas and capacity.
|
|
1638
|
+
|
|
1639
|
+
```typescript
|
|
1640
|
+
// API allows 100 req/second, use 80% headroom
|
|
1641
|
+
const rateLimit = { maxRequests: 80, windowMs: 1000 };
|
|
1642
|
+
|
|
1643
|
+
// Database connection pool has 10 slots, use 5
|
|
1644
|
+
const maxConcurrent = 5;
|
|
1645
|
+
|
|
1646
|
+
await stableApiGateway(requests, {
|
|
1647
|
+
rateLimit,
|
|
1648
|
+
maxConcurrentRequests: maxConcurrent
|
|
1649
|
+
});
|
|
1847
1650
|
```
|
|
1848
1651
|
|
|
1849
|
-
###
|
|
1652
|
+
### 6. Use Shared Buffers for Cross-Phase Coordination
|
|
1653
|
+
|
|
1654
|
+
Avoid global state; pass computed data cleanly.
|
|
1850
1655
|
|
|
1851
1656
|
```typescript
|
|
1852
|
-
|
|
1853
|
-
import {
|
|
1854
|
-
stableWorkflow,
|
|
1855
|
-
PHASE_DECISION_ACTIONS,
|
|
1856
|
-
REQUEST_METHODS,
|
|
1857
|
-
VALID_REQUEST_PROTOCOLS
|
|
1858
|
-
} from '@emmvish/stable-request';
|
|
1657
|
+
const sharedBuffer = {};
|
|
1859
1658
|
|
|
1860
|
-
|
|
1659
|
+
await stableWorkflow(phases, {
|
|
1660
|
+
sharedBuffer,
|
|
1661
|
+
// Phase 1 writes userId to sharedBuffer
|
|
1662
|
+
// Phase 2 reads userId from sharedBuffer
|
|
1663
|
+
// Phase 3 uses both
|
|
1664
|
+
});
|
|
1665
|
+
```
|
|
1861
1666
|
|
|
1862
|
-
|
|
1863
|
-
|
|
1864
|
-
|
|
1865
|
-
|
|
1866
|
-
|
|
1867
|
-
|
|
1868
|
-
|
|
1869
|
-
|
|
1870
|
-
|
|
1871
|
-
|
|
1872
|
-
|
|
1873
|
-
completedPhases: [...new Set([
|
|
1874
|
-
...(existing.completedPhases || []),
|
|
1875
|
-
...(buffer.completedPhases || [])
|
|
1876
|
-
])],
|
|
1877
|
-
lastPhase: phaseId,
|
|
1878
|
-
updatedAt: new Date().toISOString()
|
|
1879
|
-
};
|
|
1880
|
-
await redis.setex(key, 86400, JSON.stringify(checkpoint));
|
|
1881
|
-
console.log(`💾 Checkpoint: ${checkpoint.recordsProcessed}/${checkpoint.totalRecords} records`);
|
|
1882
|
-
} else {
|
|
1883
|
-
// Load checkpoint
|
|
1884
|
-
const data = await redis.get(key);
|
|
1885
|
-
return data ? JSON.parse(data) : {
|
|
1886
|
-
completedPhases: [],
|
|
1887
|
-
recordsProcessed: 0,
|
|
1888
|
-
totalRecords: 0
|
|
1889
|
-
};
|
|
1667
|
+
### 7. Log Selectively with Max Serialization Cap
|
|
1668
|
+
|
|
1669
|
+
Prevent noisy logs from large payloads.
|
|
1670
|
+
|
|
1671
|
+
```typescript
|
|
1672
|
+
await stableRequest({
|
|
1673
|
+
reqData: { path: '/data' },
|
|
1674
|
+
resReq: true,
|
|
1675
|
+
maxSerializableChars: 500, // Truncate logs to 500 chars
|
|
1676
|
+
handleSuccessfulAttemptData: ({ successfulAttemptData, maxSerializableChars }) => {
|
|
1677
|
+
console.log(safelyStringify(successfulAttemptData, maxSerializableChars));
|
|
1890
1678
|
}
|
|
1891
|
-
|
|
1892
|
-
|
|
1679
|
+
});
|
|
1680
|
+
```
|
|
1893
1681
|
|
|
1894
|
-
|
|
1895
|
-
|
|
1896
|
-
|
|
1897
|
-
|
|
1898
|
-
|
|
1899
|
-
|
|
1900
|
-
reqData: {
|
|
1901
|
-
protocol: VALID_REQUEST_PROTOCOLS.HTTPS,
|
|
1902
|
-
hostname: 'source-api.example.com',
|
|
1903
|
-
path: '/data',
|
|
1904
|
-
method: REQUEST_METHODS.GET
|
|
1905
|
-
},
|
|
1906
|
-
resReq: true
|
|
1907
|
-
}
|
|
1908
|
-
}],
|
|
1909
|
-
phaseDecisionHook: async ({ phaseResult, sharedBuffer }) => {
|
|
1910
|
-
if (sharedBuffer.completedPhases?.includes('extract')) {
|
|
1911
|
-
console.log('✅ Extract already completed, skipping...');
|
|
1912
|
-
return {
|
|
1913
|
-
action: PHASE_DECISION_ACTIONS.SKIP,
|
|
1914
|
-
skipToPhaseId: 'transform'
|
|
1915
|
-
};
|
|
1916
|
-
}
|
|
1917
|
-
|
|
1918
|
-
if (phaseResult.success) {
|
|
1919
|
-
const records = phaseResult.responses[0]?.data?.records || [];
|
|
1920
|
-
sharedBuffer.extractedData = records;
|
|
1921
|
-
sharedBuffer.totalRecords = records.length;
|
|
1922
|
-
sharedBuffer.completedPhases = ['extract'];
|
|
1923
|
-
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
1924
|
-
}
|
|
1925
|
-
return { action: PHASE_DECISION_ACTIONS.TERMINATE };
|
|
1926
|
-
}
|
|
1927
|
-
},
|
|
1682
|
+
### 8. Use Non-Linear Workflows for Polling
|
|
1683
|
+
|
|
1684
|
+
REPLAY action simplifies polling logic.
|
|
1685
|
+
|
|
1686
|
+
```typescript
|
|
1687
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
1928
1688
|
{
|
|
1929
|
-
id: '
|
|
1689
|
+
id: 'wait-for-job',
|
|
1930
1690
|
allowReplay: true,
|
|
1931
|
-
maxReplayCount:
|
|
1932
|
-
requests: [
|
|
1933
|
-
|
|
1934
|
-
|
|
1935
|
-
|
|
1936
|
-
|
|
1937
|
-
|
|
1938
|
-
|
|
1939
|
-
method: REQUEST_METHODS.POST
|
|
1940
|
-
},
|
|
1941
|
-
resReq: true,
|
|
1942
|
-
preExecution: {
|
|
1943
|
-
preExecutionHook: ({ commonBuffer }) => {
|
|
1944
|
-
// Process in batches
|
|
1945
|
-
const batchSize = 100;
|
|
1946
|
-
const processed = commonBuffer.recordsProcessed || 0;
|
|
1947
|
-
const batch = commonBuffer.extractedData.slice(
|
|
1948
|
-
processed,
|
|
1949
|
-
processed + batchSize
|
|
1950
|
-
);
|
|
1951
|
-
return {
|
|
1952
|
-
reqData: { body: { records: batch } }
|
|
1953
|
-
};
|
|
1954
|
-
},
|
|
1955
|
-
applyPreExecutionConfigOverride: true
|
|
1956
|
-
}
|
|
1957
|
-
}
|
|
1958
|
-
}],
|
|
1959
|
-
phaseDecisionHook: async ({ phaseResult, sharedBuffer }) => {
|
|
1960
|
-
if (sharedBuffer.completedPhases?.includes('transform')) {
|
|
1961
|
-
return {
|
|
1962
|
-
action: PHASE_DECISION_ACTIONS.SKIP,
|
|
1963
|
-
skipToPhaseId: 'load'
|
|
1964
|
-
};
|
|
1965
|
-
}
|
|
1966
|
-
|
|
1967
|
-
if (phaseResult.success) {
|
|
1968
|
-
const transformed = phaseResult.responses[0]?.data?.transformed || [];
|
|
1969
|
-
sharedBuffer.recordsProcessed =
|
|
1970
|
-
(sharedBuffer.recordsProcessed || 0) + transformed.length;
|
|
1971
|
-
|
|
1972
|
-
// Continue transforming if more records remain
|
|
1973
|
-
if (sharedBuffer.recordsProcessed < sharedBuffer.totalRecords) {
|
|
1974
|
-
console.log(
|
|
1975
|
-
`🔄 Progress: ${sharedBuffer.recordsProcessed}/${sharedBuffer.totalRecords}`
|
|
1976
|
-
);
|
|
1977
|
-
return { action: PHASE_DECISION_ACTIONS.REPLAY };
|
|
1691
|
+
maxReplayCount: 10,
|
|
1692
|
+
requests: [
|
|
1693
|
+
{
|
|
1694
|
+
id: 'check-status',
|
|
1695
|
+
requestOptions: {
|
|
1696
|
+
reqData: { path: '/jobs/123' },
|
|
1697
|
+
resReq: true,
|
|
1698
|
+
attempts: 1
|
|
1978
1699
|
}
|
|
1979
|
-
|
|
1980
|
-
// All records transformed
|
|
1981
|
-
sharedBuffer.completedPhases = [
|
|
1982
|
-
...(sharedBuffer.completedPhases || []),
|
|
1983
|
-
'transform'
|
|
1984
|
-
];
|
|
1985
|
-
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
1986
1700
|
}
|
|
1987
|
-
|
|
1988
|
-
|
|
1989
|
-
|
|
1990
|
-
|
|
1991
|
-
|
|
1992
|
-
id: 'load',
|
|
1993
|
-
requests: [{
|
|
1994
|
-
id: 'upload-data',
|
|
1995
|
-
requestOptions: {
|
|
1996
|
-
reqData: {
|
|
1997
|
-
protocol: VALID_REQUEST_PROTOCOLS.HTTPS,
|
|
1998
|
-
hostname: 'dest-api.example.com',
|
|
1999
|
-
path: '/import',
|
|
2000
|
-
method: REQUEST_METHODS.POST
|
|
2001
|
-
},
|
|
2002
|
-
resReq: false
|
|
2003
|
-
}
|
|
2004
|
-
}],
|
|
2005
|
-
phaseDecisionHook: async ({ phaseResult, sharedBuffer }) => {
|
|
2006
|
-
if (phaseResult.success) {
|
|
2007
|
-
sharedBuffer.completedPhases = [
|
|
2008
|
-
...(sharedBuffer.completedPhases || []),
|
|
2009
|
-
'load'
|
|
2010
|
-
];
|
|
1701
|
+
],
|
|
1702
|
+
phaseDecisionHook: async ({ phaseResult }) => {
|
|
1703
|
+
const status = (phaseResult.responses[0].data as any)?.status;
|
|
1704
|
+
if (status === 'pending') {
|
|
1705
|
+
return { action: PHASE_DECISION_ACTIONS.REPLAY };
|
|
2011
1706
|
}
|
|
2012
1707
|
return { action: PHASE_DECISION_ACTIONS.CONTINUE };
|
|
2013
1708
|
}
|
|
2014
1709
|
}
|
|
2015
1710
|
];
|
|
2016
1711
|
|
|
2017
|
-
|
|
2018
|
-
|
|
2019
|
-
workflowId: 'data-migration-2024-01-08',
|
|
2020
|
-
enableNonLinearExecution: true,
|
|
2021
|
-
sharedBuffer: {
|
|
2022
|
-
completedPhases: [],
|
|
2023
|
-
recordsProcessed: 0,
|
|
2024
|
-
totalRecords: 0
|
|
2025
|
-
},
|
|
2026
|
-
commonStatePersistence: {
|
|
2027
|
-
persistenceFunction: createCheckpoint,
|
|
2028
|
-
loadBeforeHooks: true,
|
|
2029
|
-
storeAfterHooks: true
|
|
2030
|
-
},
|
|
2031
|
-
commonAttempts: 3,
|
|
2032
|
-
commonWait: 2000,
|
|
2033
|
-
stopOnFirstPhaseError: true,
|
|
2034
|
-
logPhaseResults: true
|
|
1712
|
+
await stableWorkflow(phases, {
|
|
1713
|
+
enableNonLinearExecution: true
|
|
2035
1714
|
});
|
|
1715
|
+
```
|
|
1716
|
+
|
|
1717
|
+
### 9. Use Graph Workflows for Complex Parallelism
|
|
1718
|
+
|
|
1719
|
+
DAGs make dependencies explicit and enable maximum parallelism.
|
|
1720
|
+
|
|
1721
|
+
```typescript
|
|
1722
|
+
// Clearer than 6 phases with conditional concurrency markers
|
|
1723
|
+
const graph = new WorkflowGraphBuilder()
|
|
1724
|
+
.addParallelGroup('fetch', ['fetch-users', 'fetch-posts', 'fetch-comments'])
|
|
1725
|
+
.addMergePoint('sync', ['fetch'])
|
|
1726
|
+
.addPhase('aggregate', {...})
|
|
1727
|
+
.connectSequence('fetch', 'sync', 'aggregate')
|
|
1728
|
+
.build();
|
|
1729
|
+
|
|
1730
|
+
await stableWorkflowGraph(graph);
|
|
1731
|
+
```
|
|
1732
|
+
|
|
1733
|
+
### 10. Prefer Dry-Run (Trial Mode) Before Production
|
|
1734
|
+
|
|
1735
|
+
Test workflows and retry logic safely.
|
|
2036
1736
|
|
|
2037
|
-
|
|
2038
|
-
|
|
1737
|
+
```typescript
|
|
1738
|
+
await stableWorkflow(phases, {
|
|
1739
|
+
workflowId: 'payment-pipeline',
|
|
1740
|
+
trialMode: { enabled: true }, // Dry-run before production
|
|
1741
|
+
handlePhaseCompletion: ({ phaseResult }) => {
|
|
1742
|
+
console.log(`Trial phase: ${phaseResult.phaseId}, success=${phaseResult.success}`);
|
|
1743
|
+
}
|
|
1744
|
+
});
|
|
2039
1745
|
|
|
2040
|
-
//
|
|
2041
|
-
// It will load the checkpoint and skip completed phases
|
|
1746
|
+
// If satisfied, deploy with trialMode: { enabled: false }
|
|
2042
1747
|
```
|
|
2043
1748
|
|
|
2044
1749
|
---
|
|
2045
1750
|
|
|
2046
|
-
##
|
|
1751
|
+
## Summary
|
|
1752
|
+
|
|
1753
|
+
@emmvish/stable-request provides a unified, type-safe framework for resilient execution:
|
|
1754
|
+
|
|
1755
|
+
- **Single calls** via `stableRequest` (APIs) or `stableFunction` (pure functions)
|
|
1756
|
+
- **Batch orchestration** via `stableApiGateway` (concurrent/sequential mixed items)
|
|
1757
|
+
- **Phased workflows** via `stableWorkflow` (array-based, non-linear, branched)
|
|
1758
|
+
- **Graph workflows** via `stableWorkflowGraph` (DAG, explicit parallelism)
|
|
1759
|
+
|
|
1760
|
+
All modes inherit robust resilience (retries, jitter, circuit breaking, caching, rate/concurrency limits), config cascading, shared state, hooks, and metrics. Use together or independently; compose freely.
|
|
2047
1761
|
|
|
2048
|
-
|
|
1762
|
+
Build resilient, observable, type-safe systems with confidence.
|