@emmvish/stable-request 1.5.1 → 1.5.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +721 -1748
- package/dist/core/stable-request.d.ts.map +1 -1
- package/dist/core/stable-request.js +48 -8
- package/dist/core/stable-request.js.map +1 -1
- package/dist/core/stable-workflow.d.ts.map +1 -1
- package/dist/core/stable-workflow.js +0 -2
- package/dist/core/stable-workflow.js.map +1 -1
- package/dist/types/index.d.ts +3 -0
- package/dist/types/index.d.ts.map +1 -1
- package/dist/utilities/circuit-breaker.d.ts +11 -0
- package/dist/utilities/circuit-breaker.d.ts.map +1 -1
- package/dist/utilities/circuit-breaker.js +52 -2
- package/dist/utilities/circuit-breaker.js.map +1 -1
- package/dist/utilities/execute-concurrently.d.ts.map +1 -1
- package/dist/utilities/execute-concurrently.js +8 -6
- package/dist/utilities/execute-concurrently.js.map +1 -1
- package/dist/utilities/execute-sequentially.d.ts.map +1 -1
- package/dist/utilities/execute-sequentially.js +8 -6
- package/dist/utilities/execute-sequentially.js.map +1 -1
- package/dist/utilities/req-fn.js +1 -1
- package/dist/utilities/req-fn.js.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -4,6 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
It ensures that **every request attempt**, whether it succeeds or fails, can be:
|
|
6
6
|
|
|
7
|
+
- Sent reliably
|
|
7
8
|
- Observed
|
|
8
9
|
- Analyzed
|
|
9
10
|
- Retried intelligently
|
|
@@ -17,1304 +18,416 @@ All without crashing your application or hiding context behind opaque errors.
|
|
|
17
18
|
> If you’ve ever logged `error.message` and thought
|
|
18
19
|
> **“This tells me absolutely nothing”** — this library is for you.
|
|
19
20
|
|
|
20
|
-
In addition, it enables **content-aware retries**, **hierarchical configuration**, **batch orchestration**, and **multi-phase workflows** with deep observability — all built on top of standard HTTP calls.
|
|
21
|
+
In addition, it enables **reliability** **content-aware retries**, **hierarchical configuration**, **batch orchestration**, and **multi-phase workflows** with deep observability — all built on top of standard HTTP calls.
|
|
21
22
|
|
|
22
23
|
All in all, it provides you with the **entire ecosystem** to build **API-integrations based workflows** with **complete flexibility**.
|
|
23
24
|
|
|
24
25
|
---
|
|
25
26
|
|
|
27
|
+
## Choose your entry point
|
|
28
|
+
|
|
29
|
+
| Need | Use |
|
|
30
|
+
|-----|-----|
|
|
31
|
+
| Reliable single API call | `stableRequest` |
|
|
32
|
+
| Batch or fan-out requests | `stableApiGateway` |
|
|
33
|
+
| Multi-step orchestration | `stableWorkflow` |
|
|
34
|
+
|
|
35
|
+
Start small and scale
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
26
39
|
## 📚 Table of Contents
|
|
27
40
|
<!-- TOC START -->
|
|
28
|
-
- [Why stable-request exists](#why-stable-request-exists)
|
|
29
|
-
- [What stable-request gives you](#what-stable-request-gives-you)
|
|
30
|
-
- [Core capabilities](#core-capabilities)
|
|
31
|
-
- [Scaling beyond single requests](#scaling-beyond-single-requests)
|
|
32
|
-
- [Full workflow orchestration](#full-workflow-orchestration)
|
|
33
|
-
- [How stable-request is different](#how-stable-request-is-different)
|
|
34
|
-
- [Choose your entry point](#choose-your-entry-point)
|
|
35
41
|
- [Installation](#installation)
|
|
42
|
+
- [Core Features](#core-features)
|
|
36
43
|
- [Quick Start](#quick-start)
|
|
37
|
-
|
|
38
|
-
- [
|
|
39
|
-
- [
|
|
40
|
-
- [
|
|
41
|
-
|
|
42
|
-
- [
|
|
43
|
-
- [
|
|
44
|
-
- [
|
|
45
|
-
- [
|
|
46
|
-
- [
|
|
47
|
-
- [
|
|
48
|
-
- [
|
|
49
|
-
- [
|
|
50
|
-
- [
|
|
51
|
-
- [
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
- [Shared Configuration (Common Options)](#shared-configuration-common-options)
|
|
55
|
-
- [Advanced: Request Grouping](#advanced-request-grouping)
|
|
56
|
-
- [Service Tiers](#example-service-tiers)
|
|
57
|
-
- [Multi-Region Configuration](#example-multi-region-configuration)
|
|
58
|
-
- [Shared Buffer Across Requests](#example-shared-buffer-is-common-across-the-entire-batch-of-requests)
|
|
59
|
-
- [Multi-Phase Workflows](#multi-phase-workflows)
|
|
60
|
-
- [Basic Workflow](#basic-workflow)
|
|
61
|
-
- [Phase Configuration](#phase-configuration)
|
|
62
|
-
- [Workflow with Request Groups](#workflow-with-request-groups)
|
|
63
|
-
- [Phase Observability Hooks](#phase-observability-hooks)
|
|
64
|
-
- [Workflow Buffer](#workflow-buffer)
|
|
65
|
-
- [Concurrent Execution of Phases](#concurrent-execution-of-phases)
|
|
66
|
-
- [Mixed Execution of Phases](#mixed-execution-of-phases)
|
|
67
|
-
- [Real-World Examples](#real-world-examples)
|
|
68
|
-
- [Polling for Job Completion](#1-polling-for-job-completion)
|
|
69
|
-
- [Database Replication Lag](#2-database-replication-lag)
|
|
70
|
-
- [Idempotent Payment Processing](#3-idempotent-payment-processing)
|
|
71
|
-
- [Batch User Creation](#4-batch-user-creation-with-error-handling)
|
|
72
|
-
- [Health Check Monitoring System](#5-health-check-monitoring-system)
|
|
73
|
-
- [Data Pipeline (ETL Workflow)](#6-data-pipeline-etl-workflow)
|
|
74
|
-
- [Complete API Reference](#complete-api-reference)
|
|
75
|
-
- [`stableRequest`](#stablerequestoptions)
|
|
76
|
-
- [`stableApiGateway`](#stableapigatewayrequests-options)
|
|
77
|
-
- [`stableWorkflow`](#stableworkflowphases-options)
|
|
78
|
-
- [Hooks Reference](#hooks-reference)
|
|
79
|
-
- [`preExecutionHook`](#preexecutionhook)
|
|
80
|
-
- [`responseAnalyzer`](#responseanalyzer)
|
|
81
|
-
- [`handleErrors`](#handleerrors)
|
|
82
|
-
- [`handleSuccessfulAttemptData`](#handlesuccessfulattemptdata)
|
|
83
|
-
- [`finalErrorAnalyzer`](#finalerroranalyzer)
|
|
84
|
-
- [`handlePhaseCompletion`](#handlephasecompletion)
|
|
85
|
-
- [`handlePhaseError`](#handlephaseerror)
|
|
86
|
-
- [Configuration Hierarchy](#configuration-hierarchy)
|
|
87
|
-
- [TypeScript Support](#typescript-support)
|
|
44
|
+
- [API Reference](#api-reference)
|
|
45
|
+
- [stableRequest](#stableRequest)
|
|
46
|
+
- [stableApiGateway](#stableApiGateway)
|
|
47
|
+
- [stableWorkflow](#stableWorkflow)
|
|
48
|
+
- [Advanced Features](#advanced-features)
|
|
49
|
+
- [Retry Strategies](#retry-strategies)
|
|
50
|
+
- [Circuit Breaker](#circuit-breaker)
|
|
51
|
+
- [Rate Limiting](#rate-limiting)
|
|
52
|
+
- [Caching](#caching)
|
|
53
|
+
- [Pre-Execution Hooks](#pre-execution-hooks)
|
|
54
|
+
- [Shared Buffer](#shared-buffer)
|
|
55
|
+
- [Request Grouping](#request-grouping)
|
|
56
|
+
- [Concurrency Control](#concurrency-control)
|
|
57
|
+
- [Response Analysis](#response-analysis)
|
|
58
|
+
- [Error Handling](#error-handling)
|
|
59
|
+
- [Advanced Use Cases](#advanced-use-cases)
|
|
60
|
+
- [Configuration Options](#configuration-options)
|
|
88
61
|
- [License](#license)
|
|
89
62
|
<!-- TOC END -->
|
|
90
63
|
|
|
91
64
|
---
|
|
92
65
|
|
|
93
|
-
##
|
|
94
|
-
|
|
95
|
-
Modern systems fail in subtle and dangerous ways:
|
|
96
|
-
|
|
97
|
-
- APIs return `200` but the resource isn’t ready
|
|
98
|
-
- Databases are eventually consistent
|
|
99
|
-
- Downstream services partially fail
|
|
100
|
-
- Some requests are critical, others are optional
|
|
101
|
-
- Blind retries amplify failures
|
|
102
|
-
- Workflows fail midway and leave systems inconsistent
|
|
103
|
-
|
|
104
|
-
Most HTTP clients answer only one question:
|
|
105
|
-
|
|
106
|
-
> “Did the request fail at the network or HTTP layer?”
|
|
107
|
-
|
|
108
|
-
**stable-request answers a different one:**
|
|
109
|
-
|
|
110
|
-
> “Is the system state actually correct yet?”
|
|
111
|
-
|
|
112
|
-
---
|
|
113
|
-
|
|
114
|
-
## What stable-request gives you
|
|
115
|
-
|
|
116
|
-
### Core capabilities
|
|
117
|
-
|
|
118
|
-
✅ **Content-aware retries**
|
|
119
|
-
|
|
120
|
-
Retry based on response validation, not just status codes
|
|
121
|
-
|
|
122
|
-
🔄 **Deterministic execution semantics**
|
|
123
|
-
|
|
124
|
-
Fixed, linear, or exponential retry strategies with hard limits
|
|
66
|
+
## Installation
|
|
125
67
|
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
68
|
+
```bash
|
|
69
|
+
npm install @emmvish/stable-request
|
|
70
|
+
```
|
|
129
71
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
72
|
+
## Core Features
|
|
73
|
+
|
|
74
|
+
- ✅ **Configurable Retry Strategies**: Fixed, Linear, and Exponential backoff
|
|
75
|
+
- ✅ **Circuit Breaker**: Prevent cascading failures with automatic circuit breaking
|
|
76
|
+
- ✅ **Rate Limiting**: Control request throughput across single or multiple requests
|
|
77
|
+
- ✅ **Response Caching**: Built-in TTL-based caching with global cache manager
|
|
78
|
+
- ✅ **Batch Processing**: Execute multiple requests concurrently or sequentially via API Gateway
|
|
79
|
+
- ✅ **Multi-Phase Workflows**: Orchestrate complex request workflows with phase dependencies
|
|
80
|
+
- ✅ **Pre-Execution Hooks**: Transform requests before execution with dynamic configuration
|
|
81
|
+
- ✅ **Shared Buffer**: Share state across requests in workflows and gateways
|
|
82
|
+
- ✅ **Request Grouping**: Apply different configurations to request groups
|
|
83
|
+
- ✅ **Observability Hooks**: Track errors, successful attempts, and phase completions
|
|
84
|
+
- ✅ **Response Analysis**: Validate responses and trigger retries based on content
|
|
85
|
+
- ✅ **Trial Mode**: Test configurations without making real API calls
|
|
86
|
+
- ✅ **TypeScript Support**: Full type safety with generics for request/response data
|
|
133
87
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
Inspect every failed and successful attempt
|
|
88
|
+
## Quick Start
|
|
137
89
|
|
|
138
|
-
|
|
90
|
+
### Basic Request with Retry
|
|
139
91
|
|
|
140
|
-
|
|
92
|
+
```typescript
|
|
93
|
+
import { stableRequest, RETRY_STRATEGIES } from '@emmvish/stable-request';
|
|
141
94
|
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
95
|
+
const data = await stableRequest({
|
|
96
|
+
reqData: {
|
|
97
|
+
hostname: 'api.example.com',
|
|
98
|
+
path: '/users/123',
|
|
99
|
+
method: 'GET'
|
|
100
|
+
},
|
|
101
|
+
resReq: true,
|
|
102
|
+
attempts: 3,
|
|
103
|
+
wait: 1000,
|
|
104
|
+
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL
|
|
105
|
+
});
|
|
145
106
|
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
Apply different reliability rules to critical, standard, and optional services
|
|
107
|
+
console.log(data);
|
|
108
|
+
```
|
|
149
109
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
Workflow → Phase → Group → Request (predictable overrides)
|
|
110
|
+
### Batch Requests via API Gateway
|
|
153
111
|
|
|
154
|
-
|
|
112
|
+
```typescript
|
|
113
|
+
import { stableApiGateway } from '@emmvish/stable-request';
|
|
155
114
|
|
|
156
|
-
|
|
115
|
+
const requests = [
|
|
116
|
+
{ id: 'user-1', requestOptions: { reqData: { path: '/users/1' }, resReq: true } },
|
|
117
|
+
{ id: 'user-2', requestOptions: { reqData: { path: '/users/2' }, resReq: true } },
|
|
118
|
+
{ id: 'user-3', requestOptions: { reqData: { path: '/users/3' }, resReq: true } }
|
|
119
|
+
];
|
|
157
120
|
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
121
|
+
const results = await stableApiGateway(requests, {
|
|
122
|
+
commonRequestData: { hostname: 'api.example.com' },
|
|
123
|
+
concurrentExecution: true,
|
|
124
|
+
maxConcurrentRequests: 10
|
|
125
|
+
});
|
|
161
126
|
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
127
|
+
results.forEach(result => {
|
|
128
|
+
if (result.success) {
|
|
129
|
+
console.log(`Request ${result.requestId}:`, result.data);
|
|
130
|
+
} else {
|
|
131
|
+
console.error(`Request ${result.requestId} failed:`, result.error);
|
|
132
|
+
}
|
|
133
|
+
});
|
|
134
|
+
```
|
|
165
135
|
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
Stop execution early or continue based on business criticality.
|
|
136
|
+
### Multi-Phase Workflow
|
|
169
137
|
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
Track execution time, success rates, and failure boundaries per phase.
|
|
138
|
+
```typescript
|
|
139
|
+
import { stableWorkflow, STABLE_WORKFLOW_PHASE } from '@emmvish/stable-request';
|
|
173
140
|
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
141
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
142
|
+
{
|
|
143
|
+
id: 'authentication',
|
|
144
|
+
requests: [
|
|
145
|
+
{ id: 'login', requestOptions: { reqData: { path: '/auth/login' }, resReq: true } }
|
|
146
|
+
]
|
|
147
|
+
},
|
|
148
|
+
{
|
|
149
|
+
id: 'data-fetching',
|
|
150
|
+
concurrentExecution: true,
|
|
151
|
+
requests: [
|
|
152
|
+
{ id: 'users', requestOptions: { reqData: { path: '/users' }, resReq: true } },
|
|
153
|
+
{ id: 'posts', requestOptions: { reqData: { path: '/posts' }, resReq: true } }
|
|
154
|
+
]
|
|
155
|
+
}
|
|
156
|
+
];
|
|
177
157
|
|
|
178
|
-
|
|
158
|
+
const result = await stableWorkflow(phases, {
|
|
159
|
+
workflowId: 'data-pipeline',
|
|
160
|
+
commonRequestData: { hostname: 'api.example.com' },
|
|
161
|
+
stopOnFirstPhaseError: true,
|
|
162
|
+
logPhaseResults: true
|
|
163
|
+
});
|
|
179
164
|
|
|
180
|
-
|
|
165
|
+
console.log(`Workflow completed: ${result.successfulRequests}/${result.totalRequests} successful`);
|
|
166
|
+
```
|
|
181
167
|
|
|
182
|
-
|
|
183
|
-
|--------------------------|---------------|
|
|
184
|
-
| Status-code based retries | Content-aware retries |
|
|
185
|
-
| Per-request thinking | System-level thinking |
|
|
186
|
-
| Fire-and-forget | Deterministic workflows |
|
|
187
|
-
| Best-effort retries | Business-aware execution |
|
|
188
|
-
| Little observability | Deep, structured hooks |
|
|
168
|
+
## API Reference
|
|
189
169
|
|
|
190
|
-
|
|
170
|
+
### stableRequest
|
|
191
171
|
|
|
192
|
-
|
|
172
|
+
Execute a single HTTP request with retry logic and observability.
|
|
193
173
|
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
174
|
+
**Signature:**
|
|
175
|
+
```typescript
|
|
176
|
+
async function stableRequest<RequestDataType, ResponseDataType>(
|
|
177
|
+
options: STABLE_REQUEST<RequestDataType, ResponseDataType>
|
|
178
|
+
): Promise<ResponseDataType | boolean>
|
|
179
|
+
```
|
|
199
180
|
|
|
200
|
-
|
|
181
|
+
**Key Options:**
|
|
182
|
+
- `reqData`: Request configuration (hostname, path, method, headers, body, etc.)
|
|
183
|
+
- `resReq`: If `true`, returns response data; if `false`, returns boolean success status
|
|
184
|
+
- `attempts`: Number of retry attempts (default: 1)
|
|
185
|
+
- `wait`: Base wait time between retries in milliseconds (default: 1000)
|
|
186
|
+
- `retryStrategy`: `FIXED`, `LINEAR`, or `EXPONENTIAL` (default: FIXED)
|
|
187
|
+
- `responseAnalyzer`: Custom function to validate response content
|
|
188
|
+
- `finalErrorAnalyzer`: Handle final errors gracefully (return `true` to suppress error)
|
|
189
|
+
- `cache`: Enable response caching with TTL
|
|
190
|
+
- `circuitBreaker`: Circuit breaker configuration
|
|
191
|
+
- `preExecution`: Pre-execution hooks for dynamic request transformation
|
|
192
|
+
- `commonBuffer`: Shared state object across hooks
|
|
201
193
|
|
|
202
|
-
|
|
194
|
+
### stableApiGateway
|
|
203
195
|
|
|
204
|
-
|
|
196
|
+
Execute multiple requests concurrently or sequentially with shared configuration.
|
|
205
197
|
|
|
206
|
-
|
|
207
|
-
|
|
198
|
+
**Signature:**
|
|
199
|
+
```typescript
|
|
200
|
+
async function stableApiGateway<RequestDataType, ResponseDataType>(
|
|
201
|
+
requests: API_GATEWAY_REQUEST<RequestDataType, ResponseDataType>[],
|
|
202
|
+
options: API_GATEWAY_OPTIONS<RequestDataType, ResponseDataType>
|
|
203
|
+
): Promise<API_GATEWAY_RESPONSE<ResponseDataType>[]>
|
|
208
204
|
```
|
|
209
205
|
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
206
|
+
**Key Options:**
|
|
207
|
+
- `concurrentExecution`: Execute requests concurrently (default: true)
|
|
208
|
+
- `stopOnFirstError`: Stop processing on first error (sequential mode only)
|
|
209
|
+
- `maxConcurrentRequests`: Limit concurrent execution
|
|
210
|
+
- `rateLimit`: Rate limiting configuration
|
|
211
|
+
- `circuitBreaker`: Shared circuit breaker across requests
|
|
212
|
+
- `requestGroups`: Apply different configurations to request groups
|
|
213
|
+
- `sharedBuffer`: Shared state across all requests
|
|
214
|
+
- `common*`: Common configuration applied to all requests (e.g., `commonAttempts`, `commonCache`)
|
|
213
215
|
|
|
214
|
-
|
|
215
|
-
import { stableRequest, REQUEST_METHODS } from '@emmvish/stable-request';
|
|
216
|
+
### stableWorkflow
|
|
216
217
|
|
|
217
|
-
|
|
218
|
-
id: number;
|
|
219
|
-
updates: {
|
|
220
|
-
name?: string;
|
|
221
|
-
age?: number;
|
|
222
|
-
}
|
|
223
|
-
}
|
|
218
|
+
Execute multi-phase workflows with sequential or concurrent phase execution.
|
|
224
219
|
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
220
|
+
**Signature:**
|
|
221
|
+
```typescript
|
|
222
|
+
async function stableWorkflow<RequestDataType, ResponseDataType>(
|
|
223
|
+
phases: STABLE_WORKFLOW_PHASE<RequestDataType, ResponseDataType>[],
|
|
224
|
+
options: STABLE_WORKFLOW_OPTIONS<RequestDataType, ResponseDataType>
|
|
225
|
+
): Promise<STABLE_WORKFLOW_RESULT<ResponseDataType>>
|
|
226
|
+
```
|
|
230
227
|
|
|
231
|
-
|
|
228
|
+
**Key Options:**
|
|
229
|
+
- `workflowId`: Unique workflow identifier
|
|
230
|
+
- `concurrentPhaseExecution`: Execute phases concurrently (default: false)
|
|
231
|
+
- `stopOnFirstPhaseError`: Stop workflow on first phase failure
|
|
232
|
+
- `enableMixedExecution`: Allow mixed concurrent/sequential phase execution
|
|
233
|
+
- `handlePhaseCompletion`: Hook called after each phase completes
|
|
234
|
+
- `handlePhaseError`: Hook called when phase fails
|
|
235
|
+
- `sharedBuffer`: Shared state across all phases and requests
|
|
232
236
|
|
|
233
|
-
|
|
237
|
+
**Phase Configuration:**
|
|
238
|
+
- `id`: Phase identifier
|
|
239
|
+
- `requests`: Array of requests in this phase
|
|
240
|
+
- `concurrentExecution`: Execute phase requests concurrently
|
|
241
|
+
- `stopOnFirstError`: Stop phase on first request error
|
|
242
|
+
- `markConcurrentPhase`: Mark phase for concurrent execution in mixed mode
|
|
243
|
+
- `commonConfig`: Phase-level configuration overrides
|
|
234
244
|
|
|
235
|
-
|
|
236
|
-
reqData: {
|
|
237
|
-
method: REQUEST_METHODS.PATCH,
|
|
238
|
-
hostname: 'api.example.com',
|
|
239
|
-
path: '/users',
|
|
240
|
-
headers: { Authorization: `Bearer ${token}` },
|
|
241
|
-
body: { id: 123, updates: { age: 27 } }
|
|
242
|
-
},
|
|
243
|
-
resReq: true // Return the response data
|
|
244
|
-
});
|
|
245
|
-
|
|
246
|
-
console.log(data); // { id: 123, name: 'MV', age: 27 }
|
|
247
|
-
}
|
|
245
|
+
## Advanced Features
|
|
248
246
|
|
|
249
|
-
|
|
250
|
-
```
|
|
247
|
+
### Retry Strategies
|
|
251
248
|
|
|
252
|
-
|
|
249
|
+
Control the delay between retry attempts:
|
|
253
250
|
|
|
254
251
|
```typescript
|
|
255
252
|
import { stableRequest, RETRY_STRATEGIES } from '@emmvish/stable-request';
|
|
256
253
|
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
attempts: 3, // Retry up to 3 times
|
|
265
|
-
wait: 1000, // Wait 1 second between retries
|
|
266
|
-
maxAllowedWait: 8000, // Maximum permissible wait time between retries
|
|
267
|
-
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL // 1s, 2s, 4s, 8s...
|
|
268
|
-
});
|
|
254
|
+
// Fixed delay: 1000ms between each retry
|
|
255
|
+
await stableRequest({
|
|
256
|
+
reqData: { hostname: 'api.example.com', path: '/data' },
|
|
257
|
+
attempts: 3,
|
|
258
|
+
wait: 1000,
|
|
259
|
+
retryStrategy: RETRY_STRATEGIES.FIXED
|
|
260
|
+
});
|
|
269
261
|
|
|
270
|
-
|
|
271
|
-
|
|
262
|
+
// Linear backoff: 1000ms, 2000ms, 3000ms
|
|
263
|
+
await stableRequest({
|
|
264
|
+
reqData: { hostname: 'api.example.com', path: '/data' },
|
|
265
|
+
attempts: 3,
|
|
266
|
+
wait: 1000,
|
|
267
|
+
retryStrategy: RETRY_STRATEGIES.LINEAR
|
|
268
|
+
});
|
|
272
269
|
|
|
273
|
-
|
|
270
|
+
// Exponential backoff: 1000ms, 2000ms, 4000ms
|
|
271
|
+
await stableRequest({
|
|
272
|
+
reqData: { hostname: 'api.example.com', path: '/data' },
|
|
273
|
+
attempts: 3,
|
|
274
|
+
wait: 1000,
|
|
275
|
+
maxAllowedWait: 10000,
|
|
276
|
+
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL
|
|
277
|
+
});
|
|
274
278
|
```
|
|
275
279
|
|
|
276
|
-
|
|
277
|
-
- `RETRY_STRATEGIES.FIXED` - Same delay every time (1s, 1s, 1s...)
|
|
278
|
-
- `RETRY_STRATEGIES.LINEAR` - Increasing delay (1s, 2s, 3s...)
|
|
279
|
-
- `RETRY_STRATEGIES.EXPONENTIAL` - Exponential backoff (1s, 2s, 4s, 8s...)
|
|
280
|
+
### Circuit Breaker
|
|
280
281
|
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
Sometimes an API returns HTTP 200 but the data isn't ready yet. Use `responseAnalyzer`:
|
|
282
|
+
Prevent cascading failures by automatically blocking requests when error thresholds are exceeded:
|
|
284
283
|
|
|
285
284
|
```typescript
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
// This hook validates the response content
|
|
296
|
-
responseAnalyzer: async ({ reqData, data, trialMode, params, commonBuffer }) => {
|
|
297
|
-
// Return true if response is valid, false to retry
|
|
298
|
-
if (data.status === 'completed') {
|
|
299
|
-
return true; // Success! Don't retry
|
|
300
|
-
}
|
|
301
|
-
|
|
302
|
-
console.log(`Job still processing... (${data.percentComplete}%)`);
|
|
303
|
-
return false; // Retry this request
|
|
285
|
+
import { stableApiGateway, CircuitBreakerState } from '@emmvish/stable-request';
|
|
286
|
+
|
|
287
|
+
const results = await stableApiGateway(requests, {
|
|
288
|
+
commonRequestData: { hostname: 'api.example.com' },
|
|
289
|
+
circuitBreaker: {
|
|
290
|
+
failureThresholdPercentage: 50, // Open circuit at 50% failure rate
|
|
291
|
+
minimumRequests: 5, // Need at least 5 requests to calculate
|
|
292
|
+
recoveryTimeoutMs: 30000, // Try recovery after 30 seconds
|
|
293
|
+
trackIndividualAttempts: false // Track per-request success/failure
|
|
304
294
|
}
|
|
305
295
|
});
|
|
306
296
|
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
responseAnalyzer?: (options: {
|
|
313
|
-
reqData: AxiosRequestConfig; // Request configuration
|
|
314
|
-
data: ResponseDataType; // Response data from API
|
|
315
|
-
trialMode?: TRIAL_MODE_OPTIONS; // Trial mode settings (if enabled)
|
|
316
|
-
params?: any; // Custom parameters (via hookParams)
|
|
317
|
-
commonBuffer: Record<string, any> // For communication between request hooks
|
|
318
|
-
}) => boolean | Promise<boolean>;
|
|
319
|
-
```
|
|
320
|
-
|
|
321
|
-
### 4. Monitor Errors (Observability)
|
|
322
|
-
|
|
323
|
-
Track every failed attempt with `handleErrors`:
|
|
324
|
-
|
|
325
|
-
```typescript
|
|
326
|
-
const data = await stableRequest({
|
|
327
|
-
reqData: {
|
|
328
|
-
hostname: 'api.example.com',
|
|
329
|
-
path: '/data'
|
|
330
|
-
},
|
|
331
|
-
resReq: true,
|
|
332
|
-
attempts: 5,
|
|
333
|
-
logAllErrors: true, // Enable error logging
|
|
334
|
-
|
|
335
|
-
// This hook is called on every failed attempt
|
|
336
|
-
handleErrors: async ({ reqData, errorLog, maxSerializableChars, commonBuffer }) => {
|
|
337
|
-
// Log to your monitoring service
|
|
338
|
-
await monitoring.logError({
|
|
339
|
-
url: reqData.url,
|
|
340
|
-
attempt: errorLog.attempt, // e.g., "3/5"
|
|
341
|
-
error: errorLog.error, // Error message
|
|
342
|
-
isRetryable: errorLog.isRetryable, // Can we retry?
|
|
343
|
-
type: errorLog.type, // 'HTTP_ERROR' or 'INVALID_CONTENT'
|
|
344
|
-
statusCode: errorLog.statusCode, // HTTP status code
|
|
345
|
-
timestamp: errorLog.timestamp, // ISO timestamp
|
|
346
|
-
executionTime: errorLog.executionTime // ms
|
|
347
|
-
});
|
|
348
|
-
}
|
|
297
|
+
// Circuit breaker can be shared across workflows
|
|
298
|
+
const breaker = new CircuitBreaker({
|
|
299
|
+
failureThresholdPercentage: 50,
|
|
300
|
+
minimumRequests: 10,
|
|
301
|
+
recoveryTimeoutMs: 60000
|
|
349
302
|
});
|
|
350
|
-
```
|
|
351
303
|
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
errorLog: ERROR_LOG; // Detailed error information
|
|
357
|
-
maxSerializableChars?: number; // Max chars for stringification
|
|
358
|
-
commonBuffer: Record<string, any> // For communication between request hooks
|
|
359
|
-
}) => any | Promise<any>;
|
|
360
|
-
```
|
|
304
|
+
const result = await stableWorkflow(phases, {
|
|
305
|
+
circuitBreaker: breaker,
|
|
306
|
+
// ... other options
|
|
307
|
+
});
|
|
361
308
|
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
timestamp: string; // ISO timestamp
|
|
366
|
-
executionTime: number; // Request duration in ms
|
|
367
|
-
statusCode: number; // HTTP status code (0 if network error)
|
|
368
|
-
attempt: string; // e.g., "3/5"
|
|
369
|
-
error: string; // Error message
|
|
370
|
-
type: 'HTTP_ERROR' | 'INVALID_CONTENT';
|
|
371
|
-
isRetryable: boolean; // Can this error be retried?
|
|
372
|
-
}
|
|
309
|
+
// Check circuit breaker state
|
|
310
|
+
const state = breaker.getState();
|
|
311
|
+
console.log(`Circuit breaker state: ${state.state}`); // CLOSED, OPEN, or HALF_OPEN
|
|
373
312
|
```
|
|
374
313
|
|
|
375
|
-
###
|
|
314
|
+
### Rate Limiting
|
|
376
315
|
|
|
377
|
-
|
|
316
|
+
Control request throughput to prevent overwhelming APIs:
|
|
378
317
|
|
|
379
318
|
```typescript
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
// This hook is called on every successful attempt
|
|
390
|
-
handleSuccessfulAttemptData: async ({ reqData, successfulAttemptData, maxSerializableChars }) => {
|
|
391
|
-
// Track metrics
|
|
392
|
-
await analytics.track('api_success', {
|
|
393
|
-
url: reqData.url,
|
|
394
|
-
attempt: successfulAttemptData.attempt, // e.g., "2/3"
|
|
395
|
-
duration: successfulAttemptData.executionTime, // ms
|
|
396
|
-
statusCode: successfulAttemptData.statusCode, // 200, 201, etc.
|
|
397
|
-
timestamp: successfulAttemptData.timestamp
|
|
398
|
-
});
|
|
319
|
+
import { stableApiGateway } from '@emmvish/stable-request';
|
|
320
|
+
|
|
321
|
+
const results = await stableApiGateway(requests, {
|
|
322
|
+
commonRequestData: { hostname: 'api.example.com' },
|
|
323
|
+
concurrentExecution: true,
|
|
324
|
+
rateLimit: {
|
|
325
|
+
maxRequests: 10, // Maximum 10 requests
|
|
326
|
+
windowMs: 1000 // Per 1 second window
|
|
399
327
|
}
|
|
400
328
|
});
|
|
401
|
-
```
|
|
402
329
|
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
maxSerializableChars?: number; // Max chars for stringification
|
|
409
|
-
commonBuffer: Record<string, any> // For communication between request hooks
|
|
410
|
-
}) => any | Promise<any>;
|
|
411
|
-
```
|
|
412
|
-
|
|
413
|
-
**SUCCESSFUL_ATTEMPT_DATA Structure:**
|
|
414
|
-
```typescript
|
|
415
|
-
interface SUCCESSFUL_ATTEMPT_DATA<ResponseDataType> {
|
|
416
|
-
attempt: string; // e.g., "2/3"
|
|
417
|
-
timestamp: string; // ISO timestamp
|
|
418
|
-
executionTime: number; // Request duration in ms
|
|
419
|
-
data: ResponseDataType; // Response data
|
|
420
|
-
statusCode: number; // HTTP status code
|
|
421
|
-
}
|
|
422
|
-
```
|
|
423
|
-
|
|
424
|
-
### 6. Handle Final Errors Gracefully
|
|
425
|
-
|
|
426
|
-
Decide what to do when all retries fail using `finalErrorAnalyzer`:
|
|
427
|
-
|
|
428
|
-
```typescript
|
|
429
|
-
const data = await stableRequest({
|
|
430
|
-
reqData: {
|
|
431
|
-
hostname: 'api.example.com',
|
|
432
|
-
path: '/optional-feature'
|
|
433
|
-
},
|
|
434
|
-
resReq: true,
|
|
435
|
-
attempts: 3,
|
|
436
|
-
|
|
437
|
-
// This hook is called when all retries are exhausted
|
|
438
|
-
finalErrorAnalyzer: async ({ reqData, error, trialMode, params }) => {
|
|
439
|
-
// Check if this is a non-critical error
|
|
440
|
-
if (error.message.includes('404')) {
|
|
441
|
-
console.log('Feature not available, continuing without it');
|
|
442
|
-
return true; // Suppress error, return false instead of throwing
|
|
443
|
-
}
|
|
444
|
-
|
|
445
|
-
// For critical errors
|
|
446
|
-
await alerting.sendAlert('Critical API failure', error);
|
|
447
|
-
return false; // Throw the error
|
|
330
|
+
// Rate limiting in workflows
|
|
331
|
+
const result = await stableWorkflow(phases, {
|
|
332
|
+
rateLimit: {
|
|
333
|
+
maxRequests: 5,
|
|
334
|
+
windowMs: 1000
|
|
448
335
|
}
|
|
449
336
|
});
|
|
450
|
-
|
|
451
|
-
if (data === false) {
|
|
452
|
-
console.log('Optional feature unavailable, using default');
|
|
453
|
-
}
|
|
454
|
-
```
|
|
455
|
-
|
|
456
|
-
**Hook Signature:**
|
|
457
|
-
```typescript
|
|
458
|
-
finalErrorAnalyzer?: (options: {
|
|
459
|
-
reqData: AxiosRequestConfig; // Request configuration
|
|
460
|
-
error: any; // The final error object
|
|
461
|
-
trialMode?: TRIAL_MODE_OPTIONS; // Trial mode settings (if enabled)
|
|
462
|
-
params?: any; // Custom parameters (via hookParams)
|
|
463
|
-
commonBuffer: Record<string, any> // For communication between request hooks
|
|
464
|
-
}) => boolean | Promise<boolean>;
|
|
465
337
|
```
|
|
466
338
|
|
|
467
|
-
|
|
468
|
-
- `true` - Suppress the error, function returns `false` instead of throwing
|
|
469
|
-
- `false` - Throw the error
|
|
470
|
-
|
|
471
|
-
### 7. Pass Custom Parameters to Hooks
|
|
339
|
+
### Caching
|
|
472
340
|
|
|
473
|
-
|
|
341
|
+
Cache responses with TTL to reduce redundant API calls:
|
|
474
342
|
|
|
475
343
|
```typescript
|
|
476
|
-
|
|
344
|
+
import { stableRequest, getGlobalCacheManager } from '@emmvish/stable-request';
|
|
477
345
|
|
|
346
|
+
// Enable caching for a request
|
|
478
347
|
const data = await stableRequest({
|
|
479
|
-
reqData: {
|
|
480
|
-
hostname: 'api.example.com',
|
|
481
|
-
path: '/data'
|
|
482
|
-
},
|
|
348
|
+
reqData: { hostname: 'api.example.com', path: '/users/123' },
|
|
483
349
|
resReq: true,
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
//
|
|
487
|
-
hookParams: {
|
|
488
|
-
responseAnalyzerParams: { expectedVersion, minItems: 10 },
|
|
489
|
-
finalErrorAnalyzerParams: { alertTeam: true }
|
|
490
|
-
},
|
|
491
|
-
|
|
492
|
-
responseAnalyzer: async ({ data, params }) => {
|
|
493
|
-
// Access custom parameters
|
|
494
|
-
return data.version >= params.expectedVersion &&
|
|
495
|
-
data.items.length >= params.minItems;
|
|
496
|
-
},
|
|
497
|
-
|
|
498
|
-
finalErrorAnalyzer: async ({ error, params }) => {
|
|
499
|
-
if (params.alertTeam) {
|
|
500
|
-
await pagerDuty.alert('API failure', error);
|
|
501
|
-
}
|
|
502
|
-
return false;
|
|
350
|
+
cache: {
|
|
351
|
+
enabled: true,
|
|
352
|
+
ttl: 60000 // Cache for 60 seconds
|
|
503
353
|
}
|
|
504
354
|
});
|
|
355
|
+
|
|
356
|
+
// Use global cache manager across requests
|
|
357
|
+
const results = await stableApiGateway(requests, {
|
|
358
|
+
commonRequestData: { hostname: 'api.example.com' },
|
|
359
|
+
commonCache: { enabled: true, ttl: 300000 } // 5 minutes
|
|
360
|
+
});
|
|
361
|
+
|
|
362
|
+
// Manage cache manually
|
|
363
|
+
const cacheManager = getGlobalCacheManager();
|
|
364
|
+
const stats = cacheManager.getStats();
|
|
365
|
+
console.log(`Cache size: ${stats.size}, Valid entries: ${stats.validEntries}`);
|
|
366
|
+
cacheManager.clear(); // Clear all cache
|
|
505
367
|
```
|
|
506
368
|
|
|
507
|
-
###
|
|
369
|
+
### Pre-Execution Hooks
|
|
508
370
|
|
|
509
|
-
|
|
371
|
+
Transform requests dynamically before execution:
|
|
510
372
|
|
|
511
373
|
```typescript
|
|
512
|
-
|
|
374
|
+
import { stableRequest } from '@emmvish/stable-request';
|
|
375
|
+
|
|
376
|
+
const commonBuffer: Record<string, any> = {};
|
|
513
377
|
|
|
514
378
|
const data = await stableRequest({
|
|
515
|
-
reqData: {
|
|
516
|
-
hostname: 'api.example.com',
|
|
517
|
-
path: '/protected-resource'
|
|
518
|
-
},
|
|
379
|
+
reqData: { hostname: 'api.example.com', path: '/data' },
|
|
519
380
|
resReq: true,
|
|
520
|
-
attempts: 3,
|
|
521
|
-
|
|
522
381
|
preExecution: {
|
|
523
|
-
// Hook executed before any request attempts
|
|
524
382
|
preExecutionHook: async ({ inputParams, commonBuffer }) => {
|
|
525
|
-
|
|
383
|
+
// Fetch authentication token
|
|
384
|
+
const token = await getAuthToken();
|
|
385
|
+
|
|
386
|
+
// Store in shared buffer
|
|
526
387
|
commonBuffer.token = token;
|
|
527
|
-
commonBuffer.
|
|
528
|
-
|
|
388
|
+
commonBuffer.timestamp = Date.now();
|
|
389
|
+
|
|
390
|
+
// Override request configuration
|
|
529
391
|
return {
|
|
530
392
|
reqData: {
|
|
531
393
|
hostname: 'api.example.com',
|
|
532
|
-
path: '/
|
|
533
|
-
headers: {
|
|
534
|
-
|
|
535
|
-
}
|
|
536
|
-
},
|
|
537
|
-
attempts: 5
|
|
394
|
+
path: '/authenticated-data',
|
|
395
|
+
headers: { Authorization: `Bearer ${token}` }
|
|
396
|
+
}
|
|
538
397
|
};
|
|
539
398
|
},
|
|
540
|
-
preExecutionHookParams: {
|
|
541
|
-
|
|
542
|
-
environment: 'production'
|
|
543
|
-
},
|
|
544
|
-
applyPreExecutionConfigOverride: true,
|
|
399
|
+
preExecutionHookParams: { userId: 'user123' },
|
|
400
|
+
applyPreExecutionConfigOverride: true, // Apply returned config
|
|
545
401
|
continueOnPreExecutionHookFailure: false
|
|
546
402
|
},
|
|
547
|
-
commonBuffer
|
|
548
|
-
});
|
|
549
|
-
|
|
550
|
-
console.log('Token used:',
|
|
551
|
-
console.log('Fetched at:', outputBuffer.fetchedAt);
|
|
552
|
-
```
|
|
553
|
-
|
|
554
|
-
**Pre-Execution Options:**
|
|
555
|
-
|
|
556
|
-
```typescript
|
|
557
|
-
interface PreExecutionOptions {
|
|
558
|
-
preExecutionHook: (options: {
|
|
559
|
-
inputParams: any; // Custom parameters you provide
|
|
560
|
-
}) => any | Promise<any>; // Returns config overrides
|
|
561
|
-
preExecutionHookParams?: any; // Custom input parameters
|
|
562
|
-
applyPreExecutionConfigOverride?: boolean; // Apply returned overrides (default: false)
|
|
563
|
-
continueOnPreExecutionHookFailure?: boolean; // Continue if hook fails (default: false)
|
|
564
|
-
}
|
|
565
|
-
```
|
|
566
|
-
|
|
567
|
-
## Intermediate Concepts
|
|
568
|
-
|
|
569
|
-
### Making POST/PUT/PATCH/DELETE Requests
|
|
570
|
-
|
|
571
|
-
```typescript
|
|
572
|
-
import { stableRequest, REQUEST_METHODS } from '@emmvish/stable-request';
|
|
573
|
-
|
|
574
|
-
const newUser = await stableRequest({
|
|
575
|
-
reqData: {
|
|
576
|
-
hostname: 'api.example.com',
|
|
577
|
-
path: '/users',
|
|
578
|
-
method: REQUEST_METHODS.POST,
|
|
579
|
-
headers: {
|
|
580
|
-
'Content-Type': 'application/json',
|
|
581
|
-
'Authorization': 'Bearer your-token'
|
|
582
|
-
},
|
|
583
|
-
body: {
|
|
584
|
-
name: 'John Doe',
|
|
585
|
-
email: 'john@example.com'
|
|
586
|
-
}
|
|
587
|
-
},
|
|
588
|
-
resReq: true,
|
|
589
|
-
attempts: 3
|
|
590
|
-
});
|
|
591
|
-
```
|
|
592
|
-
|
|
593
|
-
### Query Parameters
|
|
594
|
-
|
|
595
|
-
```typescript
|
|
596
|
-
const users = await stableRequest({
|
|
597
|
-
reqData: {
|
|
598
|
-
hostname: 'api.example.com',
|
|
599
|
-
path: '/users',
|
|
600
|
-
query: {
|
|
601
|
-
page: 1,
|
|
602
|
-
limit: 10,
|
|
603
|
-
sort: 'createdAt'
|
|
604
|
-
}
|
|
605
|
-
},
|
|
606
|
-
resReq: true
|
|
607
|
-
});
|
|
608
|
-
// Requests: https://api.example.com:443/users?page=1&limit=10&sort=createdAt
|
|
609
|
-
```
|
|
610
|
-
|
|
611
|
-
### Custom Timeout and Port
|
|
612
|
-
|
|
613
|
-
```typescript
|
|
614
|
-
const data = await stableRequest({
|
|
615
|
-
reqData: {
|
|
616
|
-
hostname: 'api.example.com',
|
|
617
|
-
path: '/slow-endpoint',
|
|
618
|
-
port: 8080,
|
|
619
|
-
protocol: 'http',
|
|
620
|
-
timeout: 30000 // 30 seconds
|
|
621
|
-
},
|
|
622
|
-
resReq: true,
|
|
623
|
-
attempts: 2
|
|
624
|
-
});
|
|
625
|
-
```
|
|
626
|
-
|
|
627
|
-
### Request Cancellation
|
|
628
|
-
|
|
629
|
-
```typescript
|
|
630
|
-
const controller = new AbortController();
|
|
631
|
-
|
|
632
|
-
// Cancel after 5 seconds
|
|
633
|
-
setTimeout(() => controller.abort(), 5000);
|
|
634
|
-
|
|
635
|
-
try {
|
|
636
|
-
await stableRequest({
|
|
637
|
-
reqData: {
|
|
638
|
-
hostname: 'api.example.com',
|
|
639
|
-
path: '/data',
|
|
640
|
-
signal: controller.signal
|
|
641
|
-
},
|
|
642
|
-
resReq: true
|
|
643
|
-
});
|
|
644
|
-
} catch (error) {
|
|
645
|
-
if (error.message.includes('cancelled')) {
|
|
646
|
-
console.log('Request was cancelled');
|
|
647
|
-
}
|
|
648
|
-
}
|
|
649
|
-
```
|
|
650
|
-
|
|
651
|
-
### Trial Mode (Testing Your Retry Logic)
|
|
652
|
-
|
|
653
|
-
Simulate failures without depending on actual API issues:
|
|
654
|
-
|
|
655
|
-
```typescript
|
|
656
|
-
await stableRequest({
|
|
657
|
-
reqData: {
|
|
658
|
-
hostname: 'api.example.com',
|
|
659
|
-
path: '/data'
|
|
660
|
-
},
|
|
661
|
-
resReq: true,
|
|
662
|
-
attempts: 5,
|
|
663
|
-
logAllErrors: true,
|
|
664
|
-
|
|
665
|
-
trialMode: {
|
|
666
|
-
enabled: true,
|
|
667
|
-
reqFailureProbability: 0.3, // 30% chance each request fails
|
|
668
|
-
retryFailureProbability: 0.2 // 20% chance error is non-retryable
|
|
669
|
-
}
|
|
670
|
-
});
|
|
671
|
-
```
|
|
672
|
-
|
|
673
|
-
**Use cases:**
|
|
674
|
-
- Test your error handling logic
|
|
675
|
-
- Verify monitoring alerts work
|
|
676
|
-
- Chaos engineering experiments
|
|
677
|
-
- Integration testing
|
|
678
|
-
|
|
679
|
-
## Batch Processing - Multiple Requests
|
|
680
|
-
|
|
681
|
-
### Basic Batch Request
|
|
682
|
-
|
|
683
|
-
```typescript
|
|
684
|
-
import { stableApiGateway } from '@emmvish/stable-request';
|
|
685
|
-
|
|
686
|
-
const requests = [
|
|
687
|
-
{
|
|
688
|
-
id: 'user-1',
|
|
689
|
-
requestOptions: {
|
|
690
|
-
reqData: { path: '/users/1' },
|
|
691
|
-
resReq: true
|
|
692
|
-
}
|
|
693
|
-
},
|
|
694
|
-
{
|
|
695
|
-
id: 'user-2',
|
|
696
|
-
requestOptions: {
|
|
697
|
-
reqData: { path: '/users/2' },
|
|
698
|
-
resReq: true
|
|
699
|
-
}
|
|
700
|
-
},
|
|
701
|
-
{
|
|
702
|
-
id: 'user-3',
|
|
703
|
-
requestOptions: {
|
|
704
|
-
reqData: { path: '/users/3' },
|
|
705
|
-
resReq: true
|
|
706
|
-
}
|
|
707
|
-
}
|
|
708
|
-
];
|
|
709
|
-
|
|
710
|
-
const results = await stableApiGateway(requests, {
|
|
711
|
-
// Common options applied to ALL requests
|
|
712
|
-
commonRequestData: {
|
|
713
|
-
hostname: 'api.example.com'
|
|
714
|
-
},
|
|
715
|
-
commonAttempts: 3,
|
|
716
|
-
commonWait: 1000,
|
|
717
|
-
concurrentExecution: true // Run all requests in parallel
|
|
718
|
-
});
|
|
719
|
-
|
|
720
|
-
// Process results
|
|
721
|
-
results.forEach(result => {
|
|
722
|
-
if (result.success) {
|
|
723
|
-
console.log(`${result.requestId} succeeded:`, result.data);
|
|
724
|
-
} else {
|
|
725
|
-
console.error(`${result.requestId} failed:`, result.error);
|
|
726
|
-
}
|
|
727
|
-
});
|
|
728
|
-
```
|
|
729
|
-
|
|
730
|
-
**Response Format:**
|
|
731
|
-
```typescript
|
|
732
|
-
interface API_GATEWAY_RESPONSE<ResponseDataType> {
|
|
733
|
-
requestId: string; // The ID you provided
|
|
734
|
-
groupId?: string; // Group ID (if request was grouped)
|
|
735
|
-
success: boolean; // Did the request succeed?
|
|
736
|
-
data?: ResponseDataType; // Response data (if success)
|
|
737
|
-
error?: string; // Error message (if failed)
|
|
738
|
-
}
|
|
739
|
-
```
|
|
740
|
-
|
|
741
|
-
### Sequential Execution (With Dependencies)
|
|
742
|
-
|
|
743
|
-
```typescript
|
|
744
|
-
const steps = [
|
|
745
|
-
{
|
|
746
|
-
id: 'step-1-create',
|
|
747
|
-
requestOptions: {
|
|
748
|
-
reqData: {
|
|
749
|
-
path: '/orders',
|
|
750
|
-
method: REQUEST_METHODS.POST,
|
|
751
|
-
body: { item: 'Widget' }
|
|
752
|
-
},
|
|
753
|
-
resReq: true
|
|
754
|
-
}
|
|
755
|
-
},
|
|
756
|
-
{
|
|
757
|
-
id: 'step-2-process',
|
|
758
|
-
requestOptions: {
|
|
759
|
-
reqData: {
|
|
760
|
-
path: '/orders/123/process',
|
|
761
|
-
method: REQUEST_METHODS.POST
|
|
762
|
-
},
|
|
763
|
-
resReq: true
|
|
764
|
-
}
|
|
765
|
-
},
|
|
766
|
-
{
|
|
767
|
-
id: 'step-3-ship',
|
|
768
|
-
requestOptions: {
|
|
769
|
-
reqData: { path: '/orders/123/ship' },
|
|
770
|
-
resReq: true
|
|
771
|
-
}
|
|
772
|
-
}
|
|
773
|
-
];
|
|
774
|
-
|
|
775
|
-
const results = await stableApiGateway(steps, {
|
|
776
|
-
concurrentExecution: false, // Run one at a time
|
|
777
|
-
stopOnFirstError: true, // Stop if any step fails
|
|
778
|
-
commonRequestData: {
|
|
779
|
-
hostname: 'api.example.com'
|
|
780
|
-
},
|
|
781
|
-
commonAttempts: 3
|
|
782
|
-
});
|
|
783
|
-
|
|
784
|
-
if (results.every(r => r.success)) {
|
|
785
|
-
console.log('Workflow completed successfully');
|
|
786
|
-
} else {
|
|
787
|
-
const failedStep = results.findIndex(r => !r.success);
|
|
788
|
-
console.error(`Workflow failed at step ${failedStep + 1}`);
|
|
789
|
-
}
|
|
790
|
-
```
|
|
791
|
-
|
|
792
|
-
### Shared Configuration (Common Options)
|
|
793
|
-
|
|
794
|
-
Instead of repeating configuration for each request:
|
|
795
|
-
|
|
796
|
-
```typescript
|
|
797
|
-
const results = await stableApiGateway(
|
|
798
|
-
[
|
|
799
|
-
{ id: 'req-1', requestOptions: { reqData: { path: '/users/1' } } },
|
|
800
|
-
{ id: 'req-2', requestOptions: { reqData: { path: '/users/2' } } },
|
|
801
|
-
{ id: 'req-3', requestOptions: { reqData: { path: '/users/3' } } }
|
|
802
|
-
],
|
|
803
|
-
{
|
|
804
|
-
// Applied to ALL requests
|
|
805
|
-
commonRequestData: {
|
|
806
|
-
hostname: 'api.example.com',
|
|
807
|
-
headers: { 'Authorization': `Bearer ${token}` }
|
|
808
|
-
},
|
|
809
|
-
commonResReq: true,
|
|
810
|
-
commonAttempts: 5,
|
|
811
|
-
commonWait: 2000,
|
|
812
|
-
commonRetryStrategy: RETRY_STRATEGIES.EXPONENTIAL,
|
|
813
|
-
commonLogAllErrors: true,
|
|
814
|
-
|
|
815
|
-
// Shared hooks
|
|
816
|
-
commonHandleErrors: async ({ reqData, errorLog }) => {
|
|
817
|
-
console.log(`Request to ${reqData.url} failed (${errorLog.attempt})`);
|
|
818
|
-
},
|
|
819
|
-
|
|
820
|
-
commonResponseAnalyzer: async ({ data }) => {
|
|
821
|
-
return data?.success === true;
|
|
822
|
-
}
|
|
823
|
-
}
|
|
824
|
-
);
|
|
825
|
-
```
|
|
826
|
-
|
|
827
|
-
## Advanced: Request Grouping
|
|
828
|
-
|
|
829
|
-
Group related requests with different configurations. Configuration priority:
|
|
830
|
-
|
|
831
|
-
**Individual Request** > **Group Config** > **Global Common Config**
|
|
832
|
-
|
|
833
|
-
### Example: Service Tiers
|
|
834
|
-
|
|
835
|
-
```typescript
|
|
836
|
-
const results = await stableApiGateway(
|
|
837
|
-
[
|
|
838
|
-
// Critical services - need high reliability
|
|
839
|
-
{
|
|
840
|
-
id: 'auth-check',
|
|
841
|
-
groupId: 'critical',
|
|
842
|
-
requestOptions: {
|
|
843
|
-
reqData: { path: '/auth/verify' },
|
|
844
|
-
resReq: true
|
|
845
|
-
}
|
|
846
|
-
},
|
|
847
|
-
{
|
|
848
|
-
id: 'payment-process',
|
|
849
|
-
groupId: 'critical',
|
|
850
|
-
requestOptions: {
|
|
851
|
-
reqData: { path: '/payments/charge' },
|
|
852
|
-
resReq: true,
|
|
853
|
-
// Individual override: even MORE attempts for payments
|
|
854
|
-
attempts: 15
|
|
855
|
-
}
|
|
856
|
-
},
|
|
857
|
-
|
|
858
|
-
// Analytics - failures are acceptable
|
|
859
|
-
{
|
|
860
|
-
id: 'track-event',
|
|
861
|
-
groupId: 'analytics',
|
|
862
|
-
requestOptions: {
|
|
863
|
-
reqData: { path: '/analytics/track' },
|
|
864
|
-
resReq: true
|
|
865
|
-
}
|
|
866
|
-
}
|
|
867
|
-
],
|
|
868
|
-
{
|
|
869
|
-
// Global defaults (lowest priority)
|
|
870
|
-
commonRequestData: {
|
|
871
|
-
hostname: 'api.example.com'
|
|
872
|
-
},
|
|
873
|
-
commonAttempts: 2,
|
|
874
|
-
commonWait: 500,
|
|
875
|
-
|
|
876
|
-
// Define groups with their own configs
|
|
877
|
-
requestGroups: [
|
|
878
|
-
{
|
|
879
|
-
id: 'critical',
|
|
880
|
-
commonConfig: {
|
|
881
|
-
// Critical services: aggressive retries
|
|
882
|
-
commonAttempts: 10,
|
|
883
|
-
commonWait: 2000,
|
|
884
|
-
commonRetryStrategy: RETRY_STRATEGIES.EXPONENTIAL,
|
|
885
|
-
|
|
886
|
-
commonHandleErrors: async ({ errorLog }) => {
|
|
887
|
-
// Alert on critical failures
|
|
888
|
-
await pagerDuty.alert('Critical service failure', errorLog);
|
|
889
|
-
},
|
|
890
|
-
|
|
891
|
-
commonResponseAnalyzer: async ({ data }) => {
|
|
892
|
-
// Strict validation
|
|
893
|
-
return data?.status === 'success' && !data?.errors;
|
|
894
|
-
}
|
|
895
|
-
}
|
|
896
|
-
},
|
|
897
|
-
{
|
|
898
|
-
id: 'analytics',
|
|
899
|
-
commonConfig: {
|
|
900
|
-
// Analytics: minimal retries, don't throw on failure
|
|
901
|
-
commonAttempts: 1,
|
|
902
|
-
|
|
903
|
-
commonFinalErrorAnalyzer: async () => {
|
|
904
|
-
return true; // Suppress errors
|
|
905
|
-
}
|
|
906
|
-
}
|
|
907
|
-
}
|
|
908
|
-
]
|
|
909
|
-
}
|
|
910
|
-
);
|
|
911
|
-
|
|
912
|
-
// Analyze by group
|
|
913
|
-
const criticalOk = results
|
|
914
|
-
.filter(r => r.groupId === 'critical')
|
|
915
|
-
.every(r => r.success);
|
|
916
|
-
|
|
917
|
-
const analyticsCount = results
|
|
918
|
-
.filter(r => r.groupId === 'analytics' && r.success)
|
|
919
|
-
.length;
|
|
920
|
-
|
|
921
|
-
console.log('Critical services:', criticalOk ? 'HEALTHY' : 'DEGRADED');
|
|
922
|
-
console.log('Analytics events tracked:', analyticsCount);
|
|
923
|
-
```
|
|
924
|
-
|
|
925
|
-
### Example: Multi-Region Configuration
|
|
926
|
-
|
|
927
|
-
```typescript
|
|
928
|
-
const results = await stableApiGateway(
|
|
929
|
-
[
|
|
930
|
-
{ id: 'us-data', groupId: 'us-east', requestOptions: { reqData: { path: '/data' }, resReq: true } },
|
|
931
|
-
{ id: 'eu-data', groupId: 'eu-west', requestOptions: { reqData: { path: '/data' }, resReq: true } },
|
|
932
|
-
{ id: 'ap-data', groupId: 'ap-southeast', requestOptions: { reqData: { path: '/data' }, resReq: true } }
|
|
933
|
-
],
|
|
934
|
-
{
|
|
935
|
-
commonAttempts: 3,
|
|
936
|
-
|
|
937
|
-
requestGroups: [
|
|
938
|
-
{
|
|
939
|
-
id: 'us-east',
|
|
940
|
-
commonConfig: {
|
|
941
|
-
commonRequestData: {
|
|
942
|
-
hostname: 'api-us.example.com',
|
|
943
|
-
timeout: 5000, // Low latency expected
|
|
944
|
-
headers: { 'X-Region': 'us-east-1' }
|
|
945
|
-
},
|
|
946
|
-
commonAttempts: 3
|
|
947
|
-
}
|
|
948
|
-
},
|
|
949
|
-
{
|
|
950
|
-
id: 'eu-west',
|
|
951
|
-
commonConfig: {
|
|
952
|
-
commonRequestData: {
|
|
953
|
-
hostname: 'api-eu.example.com',
|
|
954
|
-
timeout: 8000, // Medium latency
|
|
955
|
-
headers: { 'X-Region': 'eu-west-1' }
|
|
956
|
-
},
|
|
957
|
-
commonAttempts: 5
|
|
958
|
-
}
|
|
959
|
-
},
|
|
960
|
-
{
|
|
961
|
-
id: 'ap-southeast',
|
|
962
|
-
commonConfig: {
|
|
963
|
-
commonRequestData: {
|
|
964
|
-
hostname: 'api-ap.example.com',
|
|
965
|
-
timeout: 12000, // Higher latency expected
|
|
966
|
-
headers: { 'X-Region': 'ap-southeast-1' }
|
|
967
|
-
},
|
|
968
|
-
commonAttempts: 7,
|
|
969
|
-
commonRetryStrategy: RETRY_STRATEGIES.EXPONENTIAL
|
|
970
|
-
}
|
|
971
|
-
}
|
|
972
|
-
]
|
|
973
|
-
}
|
|
974
|
-
);
|
|
975
|
-
```
|
|
976
|
-
|
|
977
|
-
### Example: Shared Buffer is common across the entire batch of requests
|
|
978
|
-
|
|
979
|
-
```typescript
|
|
980
|
-
const sharedBuffer: Record<string, any> = {};
|
|
981
|
-
|
|
982
|
-
const requests = [
|
|
983
|
-
{
|
|
984
|
-
id: 'a',
|
|
985
|
-
requestOptions: {
|
|
986
|
-
reqData: { path: '/a' },
|
|
987
|
-
resReq: true,
|
|
988
|
-
preExecution: {
|
|
989
|
-
preExecutionHook: ({ commonBuffer }: any) => {
|
|
990
|
-
commonBuffer.fromA = true;
|
|
991
|
-
return {};
|
|
992
|
-
},
|
|
993
|
-
preExecutionHookParams: {},
|
|
994
|
-
applyPreExecutionConfigOverride: false,
|
|
995
|
-
continueOnPreExecutionHookFailure: false
|
|
996
|
-
}
|
|
997
|
-
}
|
|
998
|
-
},
|
|
999
|
-
{
|
|
1000
|
-
id: 'b',
|
|
1001
|
-
requestOptions: {
|
|
1002
|
-
reqData: { path: '/b' },
|
|
1003
|
-
resReq: true,
|
|
1004
|
-
preExecution: {
|
|
1005
|
-
preExecutionHook: ({ commonBuffer }: any) => {
|
|
1006
|
-
commonBuffer.fromB = true;
|
|
1007
|
-
return {};
|
|
1008
|
-
},
|
|
1009
|
-
preExecutionHookParams: {},
|
|
1010
|
-
applyPreExecutionConfigOverride: false,
|
|
1011
|
-
continueOnPreExecutionHookFailure: false
|
|
1012
|
-
}
|
|
1013
|
-
}
|
|
1014
|
-
}
|
|
1015
|
-
] satisfies API_GATEWAY_REQUEST[];
|
|
1016
|
-
|
|
1017
|
-
const results = await stableApiGateway(requests, {
|
|
1018
|
-
concurrentExecution: true,
|
|
1019
|
-
commonRequestData: { hostname: 'api.example.com' },
|
|
1020
|
-
sharedBuffer
|
|
1021
|
-
});
|
|
1022
|
-
|
|
1023
|
-
console.log(sharedBuffer); // { fromA: true, fromB: true }
|
|
1024
|
-
```
|
|
1025
|
-
|
|
1026
|
-
|
|
1027
|
-
## Multi-Phase Workflows
|
|
1028
|
-
|
|
1029
|
-
For complex operations that require multiple stages of execution, use `stableWorkflow` to orchestrate phase-based workflows with full control over execution order and error handling.
|
|
1030
|
-
|
|
1031
|
-
### Basic Workflow
|
|
1032
|
-
|
|
1033
|
-
```typescript
|
|
1034
|
-
import { stableWorkflow } from '@emmvish/stable-request';
|
|
1035
|
-
|
|
1036
|
-
const workflow = await stableWorkflow(
|
|
1037
|
-
[
|
|
1038
|
-
{
|
|
1039
|
-
id: 'validation',
|
|
1040
|
-
concurrentExecution: true,
|
|
1041
|
-
requests: [
|
|
1042
|
-
{
|
|
1043
|
-
id: 'check-inventory',
|
|
1044
|
-
requestOptions: {
|
|
1045
|
-
reqData: { path: '/inventory/check' },
|
|
1046
|
-
resReq: true
|
|
1047
|
-
}
|
|
1048
|
-
},
|
|
1049
|
-
{
|
|
1050
|
-
id: 'validate-payment',
|
|
1051
|
-
requestOptions: {
|
|
1052
|
-
reqData: { path: '/payment/validate' },
|
|
1053
|
-
resReq: true
|
|
1054
|
-
}
|
|
1055
|
-
}
|
|
1056
|
-
]
|
|
1057
|
-
},
|
|
1058
|
-
{
|
|
1059
|
-
id: 'processing',
|
|
1060
|
-
concurrentExecution: false,
|
|
1061
|
-
stopOnFirstError: true,
|
|
1062
|
-
requests: [
|
|
1063
|
-
{
|
|
1064
|
-
id: 'charge-payment',
|
|
1065
|
-
requestOptions: {
|
|
1066
|
-
reqData: { path: '/payment/charge', method: REQUEST_METHODS.POST },
|
|
1067
|
-
resReq: true
|
|
1068
|
-
}
|
|
1069
|
-
},
|
|
1070
|
-
{
|
|
1071
|
-
id: 'reserve-inventory',
|
|
1072
|
-
requestOptions: {
|
|
1073
|
-
reqData: { path: '/inventory/reserve', method: REQUEST_METHODS.POST },
|
|
1074
|
-
resReq: true
|
|
1075
|
-
}
|
|
1076
|
-
}
|
|
1077
|
-
]
|
|
1078
|
-
}
|
|
1079
|
-
],
|
|
1080
|
-
{
|
|
1081
|
-
workflowId: 'order-processing-123',
|
|
1082
|
-
stopOnFirstPhaseError: true,
|
|
1083
|
-
logPhaseResults: true,
|
|
1084
|
-
commonRequestData: {
|
|
1085
|
-
hostname: 'api.example.com',
|
|
1086
|
-
headers: { 'X-Transaction-Id': 'txn-123' }
|
|
1087
|
-
},
|
|
1088
|
-
commonAttempts: 3,
|
|
1089
|
-
commonWait: 1000
|
|
1090
|
-
}
|
|
1091
|
-
);
|
|
1092
|
-
|
|
1093
|
-
console.log('Workflow completed:', workflow.success);
|
|
1094
|
-
console.log(`${workflow.successfulRequests}/${workflow.totalRequests} requests succeeded`);
|
|
1095
|
-
console.log(`Completed in ${workflow.executionTime}ms`);
|
|
1096
|
-
```
|
|
1097
|
-
|
|
1098
|
-
**Workflow Result:**
|
|
1099
|
-
```typescript
|
|
1100
|
-
interface STABLE_WORKFLOW_RESULT {
|
|
1101
|
-
workflowId: string; // Workflow identifier
|
|
1102
|
-
success: boolean; // Did all phases succeed?
|
|
1103
|
-
executionTime: number; // Total workflow duration (ms)
|
|
1104
|
-
timestamp: string; // ISO timestamp
|
|
1105
|
-
totalPhases: number; // Number of phases defined
|
|
1106
|
-
completedPhases: number; // Number of phases executed
|
|
1107
|
-
totalRequests: number; // Total requests across all phases
|
|
1108
|
-
successfulRequests: number; // Successful requests
|
|
1109
|
-
failedRequests: number; // Failed requests
|
|
1110
|
-
phases: PHASE_RESULT[]; // Detailed results per phase
|
|
1111
|
-
error?: string; // Workflow-level error (if any)
|
|
1112
|
-
}
|
|
1113
|
-
```
|
|
1114
|
-
|
|
1115
|
-
### Phase Configuration
|
|
1116
|
-
|
|
1117
|
-
Each phase can have its own execution mode and error handling, and can also work upon the shared buffer:
|
|
1118
|
-
|
|
1119
|
-
```typescript
|
|
1120
|
-
{
|
|
1121
|
-
id: 'phase-name', // Optional: phase identifier
|
|
1122
|
-
concurrentExecution?: boolean, // true = parallel, false = sequential
|
|
1123
|
-
stopOnFirstError?: boolean, // Stop phase on first request failure
|
|
1124
|
-
commonConfig?: { /* phase-level common config */ },
|
|
1125
|
-
requests: [/* array of requests */],
|
|
1126
|
-
sharedBuffer?: Record<string, any> // State to be shared among all requests in the phase
|
|
1127
|
-
}
|
|
1128
|
-
```
|
|
1129
|
-
|
|
1130
|
-
**Configuration Priority:**
|
|
1131
|
-
Individual Request > Phase Common Config > Workflow Common Config
|
|
1132
|
-
|
|
1133
|
-
### Workflow with Request Groups
|
|
1134
|
-
|
|
1135
|
-
Combine workflows with request groups for fine-grained control:
|
|
1136
|
-
|
|
1137
|
-
```typescript
|
|
1138
|
-
const workflow = await stableWorkflow(
|
|
1139
|
-
[
|
|
1140
|
-
{
|
|
1141
|
-
id: 'critical-validation',
|
|
1142
|
-
concurrentExecution: true,
|
|
1143
|
-
requests: [
|
|
1144
|
-
{
|
|
1145
|
-
id: 'auth-check',
|
|
1146
|
-
groupId: 'critical',
|
|
1147
|
-
requestOptions: {
|
|
1148
|
-
reqData: { path: '/auth/verify' },
|
|
1149
|
-
resReq: true
|
|
1150
|
-
}
|
|
1151
|
-
},
|
|
1152
|
-
{
|
|
1153
|
-
id: 'rate-limit-check',
|
|
1154
|
-
groupId: 'critical',
|
|
1155
|
-
requestOptions: {
|
|
1156
|
-
reqData: { path: '/ratelimit/check' },
|
|
1157
|
-
resReq: true
|
|
1158
|
-
}
|
|
1159
|
-
}
|
|
1160
|
-
]
|
|
1161
|
-
},
|
|
1162
|
-
{
|
|
1163
|
-
id: 'data-processing',
|
|
1164
|
-
concurrentExecution: false,
|
|
1165
|
-
commonConfig: {
|
|
1166
|
-
// Phase-specific overrides
|
|
1167
|
-
commonAttempts: 5,
|
|
1168
|
-
commonRetryStrategy: RETRY_STRATEGIES.EXPONENTIAL
|
|
1169
|
-
},
|
|
1170
|
-
requests: [
|
|
1171
|
-
{
|
|
1172
|
-
id: 'process-data',
|
|
1173
|
-
groupId: 'standard',
|
|
1174
|
-
requestOptions: {
|
|
1175
|
-
reqData: { path: '/data/process', method: REQUEST_METHODS.POST },
|
|
1176
|
-
resReq: true
|
|
1177
|
-
}
|
|
1178
|
-
},
|
|
1179
|
-
{
|
|
1180
|
-
id: 'store-result',
|
|
1181
|
-
groupId: 'standard',
|
|
1182
|
-
requestOptions: {
|
|
1183
|
-
reqData: { path: '/data/store', method: REQUEST_METHODS.POST },
|
|
1184
|
-
resReq: true
|
|
1185
|
-
}
|
|
1186
|
-
}
|
|
1187
|
-
]
|
|
1188
|
-
},
|
|
1189
|
-
{
|
|
1190
|
-
id: 'notifications',
|
|
1191
|
-
concurrentExecution: true,
|
|
1192
|
-
requests: [
|
|
1193
|
-
{
|
|
1194
|
-
id: 'email-notification',
|
|
1195
|
-
groupId: 'optional',
|
|
1196
|
-
requestOptions: {
|
|
1197
|
-
reqData: { path: '/notify/email' },
|
|
1198
|
-
resReq: true
|
|
1199
|
-
}
|
|
1200
|
-
},
|
|
1201
|
-
{
|
|
1202
|
-
id: 'webhook-notification',
|
|
1203
|
-
groupId: 'optional',
|
|
1204
|
-
requestOptions: {
|
|
1205
|
-
reqData: { path: '/notify/webhook' },
|
|
1206
|
-
resReq: true
|
|
1207
|
-
}
|
|
1208
|
-
}
|
|
1209
|
-
]
|
|
1210
|
-
}
|
|
1211
|
-
],
|
|
1212
|
-
{
|
|
1213
|
-
workflowId: 'data-pipeline-workflow',
|
|
1214
|
-
stopOnFirstPhaseError: true,
|
|
1215
|
-
logPhaseResults: true,
|
|
1216
|
-
|
|
1217
|
-
commonRequestData: {
|
|
1218
|
-
hostname: 'api.example.com'
|
|
1219
|
-
},
|
|
1220
|
-
commonAttempts: 3,
|
|
1221
|
-
commonWait: 1000,
|
|
1222
|
-
|
|
1223
|
-
// Request groups with different reliability requirements
|
|
1224
|
-
requestGroups: [
|
|
1225
|
-
{
|
|
1226
|
-
id: 'critical',
|
|
1227
|
-
commonConfig: {
|
|
1228
|
-
commonAttempts: 10,
|
|
1229
|
-
commonRetryStrategy: RETRY_STRATEGIES.EXPONENTIAL,
|
|
1230
|
-
commonWait: 2000,
|
|
1231
|
-
commonHandleErrors: async ({ errorLog }) => {
|
|
1232
|
-
await alerting.critical('Critical service failure', errorLog);
|
|
1233
|
-
}
|
|
1234
|
-
}
|
|
1235
|
-
},
|
|
1236
|
-
{
|
|
1237
|
-
id: 'standard',
|
|
1238
|
-
commonConfig: {
|
|
1239
|
-
commonAttempts: 5,
|
|
1240
|
-
commonRetryStrategy: RETRY_STRATEGIES.LINEAR,
|
|
1241
|
-
commonWait: 1000
|
|
1242
|
-
}
|
|
1243
|
-
},
|
|
1244
|
-
{
|
|
1245
|
-
id: 'optional',
|
|
1246
|
-
commonConfig: {
|
|
1247
|
-
commonAttempts: 2,
|
|
1248
|
-
commonFinalErrorAnalyzer: async () => true // Suppress errors
|
|
1249
|
-
}
|
|
1250
|
-
}
|
|
1251
|
-
]
|
|
1252
|
-
}
|
|
1253
|
-
);
|
|
1254
|
-
```
|
|
1255
|
-
|
|
1256
|
-
### Phase Observability Hooks
|
|
1257
|
-
|
|
1258
|
-
Monitor workflow execution with phase-level hooks:
|
|
1259
|
-
|
|
1260
|
-
```typescript
|
|
1261
|
-
const workflow = await stableWorkflow(
|
|
1262
|
-
[
|
|
1263
|
-
// ...phases...
|
|
1264
|
-
],
|
|
1265
|
-
{
|
|
1266
|
-
workflowId: 'monitored-workflow',
|
|
1267
|
-
|
|
1268
|
-
// Called after each phase completes successfully
|
|
1269
|
-
handlePhaseCompletion: async ({ workflowId, phaseResult, sharedBuffer }) => {
|
|
1270
|
-
console.log(`Phase ${phaseResult.phaseId} completed`);
|
|
1271
|
-
|
|
1272
|
-
await analytics.track('workflow_phase_complete', {
|
|
1273
|
-
workflowId,
|
|
1274
|
-
phaseId: phaseResult.phaseId,
|
|
1275
|
-
duration: phaseResult.executionTime,
|
|
1276
|
-
successRate: phaseResult.successfulRequests / phaseResult.totalRequests
|
|
1277
|
-
});
|
|
1278
|
-
},
|
|
1279
|
-
|
|
1280
|
-
// Called when a phase fails
|
|
1281
|
-
handlePhaseError: async ({ workflowId, phaseResult, error, sharedBuffer }) => {
|
|
1282
|
-
console.error(`Phase ${phaseResult.phaseId} failed`);
|
|
1283
|
-
|
|
1284
|
-
await alerting.notify('workflow_phase_failed', {
|
|
1285
|
-
workflowId,
|
|
1286
|
-
phaseId: phaseResult.phaseId,
|
|
1287
|
-
error: error.message,
|
|
1288
|
-
failedRequests: phaseResult.failedRequests
|
|
1289
|
-
});
|
|
1290
|
-
},
|
|
1291
|
-
|
|
1292
|
-
logPhaseResults: true // Enable console logging
|
|
1293
|
-
}
|
|
1294
|
-
);
|
|
403
|
+
commonBuffer
|
|
404
|
+
});
|
|
405
|
+
|
|
406
|
+
console.log('Token used:', commonBuffer.token);
|
|
1295
407
|
```
|
|
1296
408
|
|
|
1297
|
-
###
|
|
409
|
+
### Shared Buffer
|
|
1298
410
|
|
|
1299
|
-
|
|
411
|
+
Share state across requests in gateways and workflows:
|
|
1300
412
|
|
|
1301
413
|
```typescript
|
|
1302
|
-
|
|
414
|
+
import { stableWorkflow } from '@emmvish/stable-request';
|
|
415
|
+
|
|
416
|
+
const sharedBuffer: Record<string, any> = { requestCount: 0 };
|
|
1303
417
|
|
|
1304
|
-
const phases = [
|
|
418
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
1305
419
|
{
|
|
1306
|
-
id: '
|
|
1307
|
-
concurrentExecution: false,
|
|
420
|
+
id: 'phase-1',
|
|
1308
421
|
requests: [
|
|
1309
422
|
{
|
|
1310
|
-
id: '
|
|
423
|
+
id: 'req-1',
|
|
1311
424
|
requestOptions: {
|
|
1312
|
-
reqData: { path: '/
|
|
425
|
+
reqData: { path: '/step1' },
|
|
1313
426
|
resReq: true,
|
|
1314
427
|
preExecution: {
|
|
1315
|
-
preExecutionHook: ({ commonBuffer }
|
|
1316
|
-
commonBuffer.
|
|
1317
|
-
commonBuffer.
|
|
428
|
+
preExecutionHook: ({ commonBuffer }) => {
|
|
429
|
+
commonBuffer.requestCount++;
|
|
430
|
+
commonBuffer.phase1Data = 'initialized';
|
|
1318
431
|
return {};
|
|
1319
432
|
},
|
|
1320
433
|
preExecutionHookParams: {},
|
|
@@ -1326,17 +439,18 @@ const phases = [
|
|
|
1326
439
|
]
|
|
1327
440
|
},
|
|
1328
441
|
{
|
|
1329
|
-
id: '
|
|
1330
|
-
concurrentExecution: false,
|
|
442
|
+
id: 'phase-2',
|
|
1331
443
|
requests: [
|
|
1332
444
|
{
|
|
1333
|
-
id: '
|
|
445
|
+
id: 'req-2',
|
|
1334
446
|
requestOptions: {
|
|
1335
|
-
reqData: { path: '/
|
|
447
|
+
reqData: { path: '/step2' },
|
|
1336
448
|
resReq: true,
|
|
1337
449
|
preExecution: {
|
|
1338
|
-
preExecutionHook: ({ commonBuffer }
|
|
1339
|
-
commonBuffer.
|
|
450
|
+
preExecutionHook: ({ commonBuffer }) => {
|
|
451
|
+
commonBuffer.requestCount++;
|
|
452
|
+
// Access data from phase-1
|
|
453
|
+
console.log('Phase 1 data:', commonBuffer.phase1Data);
|
|
1340
454
|
return {};
|
|
1341
455
|
},
|
|
1342
456
|
preExecutionHookParams: {},
|
|
@@ -1347,713 +461,572 @@ const phases = [
|
|
|
1347
461
|
}
|
|
1348
462
|
]
|
|
1349
463
|
}
|
|
1350
|
-
]
|
|
464
|
+
];
|
|
1351
465
|
|
|
1352
466
|
const result = await stableWorkflow(phases, {
|
|
1353
|
-
workflowId: '
|
|
467
|
+
workflowId: 'stateful-workflow',
|
|
1354
468
|
commonRequestData: { hostname: 'api.example.com' },
|
|
1355
|
-
sharedBuffer
|
|
469
|
+
sharedBuffer
|
|
1356
470
|
});
|
|
1357
471
|
|
|
1358
|
-
console.log(
|
|
472
|
+
console.log('Total requests processed:', sharedBuffer.requestCount);
|
|
1359
473
|
```
|
|
1360
|
-
|
|
474
|
+
|
|
475
|
+
### Request Grouping
|
|
476
|
+
|
|
477
|
+
Apply different configurations to request groups:
|
|
1361
478
|
|
|
1362
479
|
```typescript
|
|
1363
|
-
|
|
480
|
+
import { stableApiGateway } from '@emmvish/stable-request';
|
|
481
|
+
|
|
482
|
+
const requests = [
|
|
1364
483
|
{
|
|
1365
|
-
id: '
|
|
1366
|
-
|
|
1367
|
-
|
|
1368
|
-
]
|
|
484
|
+
id: 'critical-1',
|
|
485
|
+
groupId: 'critical',
|
|
486
|
+
requestOptions: { reqData: { path: '/critical/1' }, resReq: true }
|
|
1369
487
|
},
|
|
1370
488
|
{
|
|
1371
|
-
id: '
|
|
1372
|
-
|
|
1373
|
-
|
|
1374
|
-
]
|
|
489
|
+
id: 'critical-2',
|
|
490
|
+
groupId: 'critical',
|
|
491
|
+
requestOptions: { reqData: { path: '/critical/2' }, resReq: true }
|
|
1375
492
|
},
|
|
1376
493
|
{
|
|
1377
|
-
id: '
|
|
1378
|
-
|
|
1379
|
-
|
|
1380
|
-
]
|
|
494
|
+
id: 'optional-1',
|
|
495
|
+
groupId: 'optional',
|
|
496
|
+
requestOptions: { reqData: { path: '/optional/1' }, resReq: true }
|
|
1381
497
|
}
|
|
1382
|
-
]
|
|
498
|
+
];
|
|
1383
499
|
|
|
1384
|
-
const
|
|
1385
|
-
workflowId: 'wf-concurrent-phases',
|
|
500
|
+
const results = await stableApiGateway(requests, {
|
|
1386
501
|
commonRequestData: { hostname: 'api.example.com' },
|
|
1387
502
|
commonAttempts: 1,
|
|
1388
|
-
commonWait:
|
|
1389
|
-
|
|
1390
|
-
});
|
|
1391
|
-
```
|
|
1392
|
-
### Mixed Execution of Phases
|
|
1393
|
-
|
|
1394
|
-
```typescript
|
|
1395
|
-
const workflow = await stableWorkflow(
|
|
1396
|
-
[
|
|
1397
|
-
{
|
|
1398
|
-
id: 'phase-1-sequential',
|
|
1399
|
-
requests: [/* ... */]
|
|
1400
|
-
},
|
|
1401
|
-
{
|
|
1402
|
-
id: 'phase-2-concurrent-start',
|
|
1403
|
-
markConcurrentPhase: true, // Will run concurrently with phase-3
|
|
1404
|
-
requests: [/* ... */]
|
|
1405
|
-
},
|
|
503
|
+
commonWait: 100,
|
|
504
|
+
requestGroups: [
|
|
1406
505
|
{
|
|
1407
|
-
id: '
|
|
1408
|
-
|
|
1409
|
-
|
|
506
|
+
id: 'critical',
|
|
507
|
+
commonConfig: {
|
|
508
|
+
commonAttempts: 5, // More retries for critical requests
|
|
509
|
+
commonWait: 2000,
|
|
510
|
+
commonRetryStrategy: RETRY_STRATEGIES.EXPONENTIAL
|
|
511
|
+
}
|
|
1410
512
|
},
|
|
1411
513
|
{
|
|
1412
|
-
id: '
|
|
1413
|
-
|
|
514
|
+
id: 'optional',
|
|
515
|
+
commonConfig: {
|
|
516
|
+
commonAttempts: 1, // No retries for optional requests
|
|
517
|
+
commonFinalErrorAnalyzer: async () => true // Suppress errors
|
|
518
|
+
}
|
|
1414
519
|
}
|
|
1415
|
-
]
|
|
1416
|
-
|
|
1417
|
-
workflowId: 'mixed-execution-workflow',
|
|
1418
|
-
allowExecutionMixing: true // Enable mixed execution mode
|
|
1419
|
-
}
|
|
1420
|
-
);
|
|
520
|
+
]
|
|
521
|
+
});
|
|
1421
522
|
```
|
|
1422
523
|
|
|
1423
|
-
|
|
524
|
+
### Concurrency Control
|
|
1424
525
|
|
|
1425
|
-
|
|
526
|
+
Limit concurrent request execution:
|
|
1426
527
|
|
|
1427
528
|
```typescript
|
|
1428
|
-
|
|
1429
|
-
|
|
1430
|
-
|
|
1431
|
-
|
|
1432
|
-
},
|
|
1433
|
-
|
|
1434
|
-
|
|
1435
|
-
wait: 3000, // Wait 3 seconds between polls
|
|
1436
|
-
retryStrategy: RETRY_STRATEGIES.FIXED,
|
|
1437
|
-
|
|
1438
|
-
responseAnalyzer: async ({ data }) => {
|
|
1439
|
-
if (data.status === 'completed') {
|
|
1440
|
-
console.log('Job completed!');
|
|
1441
|
-
return true; // Success
|
|
1442
|
-
}
|
|
1443
|
-
|
|
1444
|
-
if (data.status === 'failed') {
|
|
1445
|
-
throw new Error(`Job failed: ${data.error}`);
|
|
1446
|
-
}
|
|
1447
|
-
|
|
1448
|
-
console.log(`Job ${data.status}... ${data.progress}%`);
|
|
1449
|
-
return false; // Keep polling
|
|
1450
|
-
},
|
|
1451
|
-
|
|
1452
|
-
handleErrors: async ({ errorLog }) => {
|
|
1453
|
-
console.log(`Poll attempt ${errorLog.attempt}`);
|
|
1454
|
-
}
|
|
529
|
+
import { stableApiGateway } from '@emmvish/stable-request';
|
|
530
|
+
|
|
531
|
+
// Limit to 5 concurrent requests
|
|
532
|
+
const results = await stableApiGateway(requests, {
|
|
533
|
+
commonRequestData: { hostname: 'api.example.com' },
|
|
534
|
+
concurrentExecution: true,
|
|
535
|
+
maxConcurrentRequests: 5
|
|
1455
536
|
});
|
|
1456
537
|
|
|
1457
|
-
|
|
538
|
+
// Phase-level concurrency control
|
|
539
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
540
|
+
{
|
|
541
|
+
id: 'limited-phase',
|
|
542
|
+
concurrentExecution: true,
|
|
543
|
+
maxConcurrentRequests: 3,
|
|
544
|
+
requests: [/* ... */]
|
|
545
|
+
}
|
|
546
|
+
];
|
|
1458
547
|
```
|
|
1459
548
|
|
|
1460
|
-
###
|
|
549
|
+
### Response Analysis
|
|
550
|
+
|
|
551
|
+
Validate response content and trigger retries:
|
|
1461
552
|
|
|
1462
553
|
```typescript
|
|
1463
|
-
|
|
554
|
+
import { stableRequest } from '@emmvish/stable-request';
|
|
1464
555
|
|
|
1465
556
|
const data = await stableRequest({
|
|
1466
|
-
reqData: {
|
|
1467
|
-
hostname: 'replica.db.example.com',
|
|
1468
|
-
path: '/records/123'
|
|
1469
|
-
},
|
|
557
|
+
reqData: { hostname: 'api.example.com', path: '/job/status' },
|
|
1470
558
|
resReq: true,
|
|
1471
559
|
attempts: 10,
|
|
1472
|
-
wait:
|
|
1473
|
-
|
|
1474
|
-
|
|
1475
|
-
|
|
1476
|
-
|
|
1477
|
-
|
|
1478
|
-
|
|
1479
|
-
responseAnalyzer: async ({ data, params }) => {
|
|
1480
|
-
// Wait until replica catches up
|
|
1481
|
-
if (data.version >= params.expectedVersion) {
|
|
1482
|
-
return true;
|
|
560
|
+
wait: 2000,
|
|
561
|
+
responseAnalyzer: async ({ data, reqData, params }) => {
|
|
562
|
+
// Retry until job is completed
|
|
563
|
+
if (data.status === 'processing') {
|
|
564
|
+
console.log('Job still processing, will retry...');
|
|
565
|
+
return false; // Trigger retry
|
|
1483
566
|
}
|
|
1484
|
-
|
|
1485
|
-
console.log(`Replica at version ${data.version}, waiting for ${params.expectedVersion}`);
|
|
1486
|
-
return false;
|
|
567
|
+
return data.status === 'completed';
|
|
1487
568
|
}
|
|
1488
569
|
});
|
|
570
|
+
|
|
571
|
+
console.log('Job completed:', data);
|
|
1489
572
|
```
|
|
1490
573
|
|
|
1491
|
-
###
|
|
574
|
+
### Error Handling
|
|
575
|
+
|
|
576
|
+
Comprehensive error handling with observability hooks:
|
|
1492
577
|
|
|
1493
578
|
```typescript
|
|
1494
|
-
|
|
1495
|
-
|
|
1496
|
-
|
|
1497
|
-
|
|
1498
|
-
method: REQUEST_METHODS.POST,
|
|
1499
|
-
headers: {
|
|
1500
|
-
'Authorization': 'Bearer sk_...',
|
|
1501
|
-
'Idempotency-Key': crypto.randomUUID() // Ensure idempotency
|
|
1502
|
-
},
|
|
1503
|
-
body: {
|
|
1504
|
-
amount: 1000,
|
|
1505
|
-
currency: 'usd',
|
|
1506
|
-
source: 'tok_visa'
|
|
1507
|
-
}
|
|
1508
|
-
},
|
|
579
|
+
import { stableRequest } from '@emmvish/stable-request';
|
|
580
|
+
|
|
581
|
+
const data = await stableRequest({
|
|
582
|
+
reqData: { hostname: 'api.example.com', path: '/data' },
|
|
1509
583
|
resReq: true,
|
|
1510
|
-
attempts:
|
|
1511
|
-
wait:
|
|
1512
|
-
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL,
|
|
1513
|
-
|
|
584
|
+
attempts: 3,
|
|
585
|
+
wait: 1000,
|
|
1514
586
|
logAllErrors: true,
|
|
1515
|
-
|
|
1516
|
-
|
|
1517
|
-
|
|
1518
|
-
|
|
587
|
+
handleErrors: ({ reqData, errorLog, params }) => {
|
|
588
|
+
// Custom error logging
|
|
589
|
+
console.error('Request failed:', {
|
|
590
|
+
url: reqData.url,
|
|
1519
591
|
attempt: errorLog.attempt,
|
|
592
|
+
statusCode: errorLog.statusCode,
|
|
1520
593
|
error: errorLog.error,
|
|
1521
594
|
isRetryable: errorLog.isRetryable
|
|
1522
595
|
});
|
|
596
|
+
|
|
597
|
+
// Send to monitoring service
|
|
598
|
+
monitoringService.trackError(errorLog);
|
|
1523
599
|
},
|
|
1524
|
-
|
|
1525
|
-
|
|
1526
|
-
|
|
1527
|
-
return data.status === 'succeeded' && data.paid === true;
|
|
600
|
+
logAllSuccessfulAttempts: true,
|
|
601
|
+
handleSuccessfulAttemptData: ({ successfulAttemptData }) => {
|
|
602
|
+
console.log('Request succeeded on attempt:', successfulAttemptData.attempt);
|
|
1528
603
|
},
|
|
1529
|
-
|
|
1530
|
-
|
|
1531
|
-
|
|
1532
|
-
|
|
604
|
+
finalErrorAnalyzer: async ({ error, reqData }) => {
|
|
605
|
+
// Gracefully handle specific errors
|
|
606
|
+
if (error.response?.status === 404) {
|
|
607
|
+
console.warn('Resource not found, continuing...');
|
|
608
|
+
return true; // Return false to suppress error
|
|
609
|
+
}
|
|
1533
610
|
return false; // Throw error
|
|
1534
611
|
}
|
|
1535
612
|
});
|
|
1536
613
|
```
|
|
1537
614
|
|
|
1538
|
-
|
|
1539
|
-
|
|
1540
|
-
```typescript
|
|
1541
|
-
const users = [
|
|
1542
|
-
{ name: 'Alice', email: 'alice@example.com' },
|
|
1543
|
-
{ name: 'Bob', email: 'bob@example.com' },
|
|
1544
|
-
{ name: 'Charlie', email: 'charlie@example.com' }
|
|
1545
|
-
];
|
|
1546
|
-
|
|
1547
|
-
const requests = users.map((user, index) => ({
|
|
1548
|
-
id: `user-${index}`,
|
|
1549
|
-
requestOptions: {
|
|
1550
|
-
reqData: {
|
|
1551
|
-
body: user
|
|
1552
|
-
},
|
|
1553
|
-
resReq: true
|
|
1554
|
-
}
|
|
1555
|
-
}));
|
|
1556
|
-
|
|
1557
|
-
const results = await stableApiGateway(requests, {
|
|
1558
|
-
concurrentExecution: true,
|
|
1559
|
-
|
|
1560
|
-
commonRequestData: {
|
|
1561
|
-
hostname: 'api.example.com',
|
|
1562
|
-
path: '/users',
|
|
1563
|
-
method: REQUEST_METHODS.POST,
|
|
1564
|
-
headers: {
|
|
1565
|
-
'Content-Type': 'application/json'
|
|
1566
|
-
}
|
|
1567
|
-
},
|
|
1568
|
-
|
|
1569
|
-
commonAttempts: 3,
|
|
1570
|
-
commonWait: 1000,
|
|
1571
|
-
commonRetryStrategy: RETRY_STRATEGIES.EXPONENTIAL,
|
|
1572
|
-
commonResReq: true,
|
|
1573
|
-
commonLogAllErrors: true,
|
|
1574
|
-
|
|
1575
|
-
commonHandleErrors: async ({ reqData, errorLog }) => {
|
|
1576
|
-
const user = reqData.data;
|
|
1577
|
-
console.error(`Failed to create user ${user.name}: ${errorLog.error}`);
|
|
1578
|
-
},
|
|
1579
|
-
|
|
1580
|
-
commonResponseAnalyzer: async ({ data }) => {
|
|
1581
|
-
// Ensure user was created with an ID
|
|
1582
|
-
return data?.id && data?.email;
|
|
1583
|
-
}
|
|
1584
|
-
});
|
|
1585
|
-
|
|
1586
|
-
const successful = results.filter(r => r.success);
|
|
1587
|
-
const failed = results.filter(r => !r.success);
|
|
1588
|
-
|
|
1589
|
-
console.log(`✓ Created ${successful.length} users`);
|
|
1590
|
-
console.log(`✗ Failed to create ${failed.length} users`);
|
|
1591
|
-
|
|
1592
|
-
failed.forEach(r => {
|
|
1593
|
-
console.error(` - ${r.requestId}: ${r.error}`);
|
|
1594
|
-
});
|
|
1595
|
-
```
|
|
615
|
+
## Advanced Use Cases
|
|
1596
616
|
|
|
1597
|
-
###
|
|
617
|
+
### Use Case 1: Multi-Tenant API with Dynamic Authentication
|
|
1598
618
|
|
|
1599
619
|
```typescript
|
|
1600
|
-
|
|
1601
|
-
[
|
|
1602
|
-
// Core services - must be healthy
|
|
1603
|
-
{ id: 'auth', groupId: 'core', requestOptions: { reqData: { hostname: 'auth.internal', path: '/health' } } },
|
|
1604
|
-
{ id: 'database', groupId: 'core', requestOptions: { reqData: { hostname: 'db.internal', path: '/health' } } },
|
|
1605
|
-
{ id: 'api', groupId: 'core', requestOptions: { reqData: { hostname: 'api.internal', path: '/health' } } },
|
|
1606
|
-
|
|
1607
|
-
// Optional services
|
|
1608
|
-
{ id: 'cache', groupId: 'optional', requestOptions: { reqData: { hostname: 'cache.internal', path: '/health' } } },
|
|
1609
|
-
{ id: 'search', groupId: 'optional', requestOptions: { reqData: { hostname: 'search.internal', path: '/health' } } }
|
|
1610
|
-
],
|
|
1611
|
-
{
|
|
1612
|
-
commonResReq: true,
|
|
1613
|
-
concurrentExecution: true,
|
|
1614
|
-
|
|
1615
|
-
requestGroups: [
|
|
1616
|
-
{
|
|
1617
|
-
id: 'core',
|
|
1618
|
-
commonConfig: {
|
|
1619
|
-
commonAttempts: 5,
|
|
1620
|
-
commonWait: 2000,
|
|
1621
|
-
commonRetryStrategy: RETRY_STRATEGIES.EXPONENTIAL,
|
|
1622
|
-
|
|
1623
|
-
commonResponseAnalyzer: async ({ data }) => {
|
|
1624
|
-
// Core services need strict validation
|
|
1625
|
-
return data?.status === 'healthy' &&
|
|
1626
|
-
data?.uptime > 0 &&
|
|
1627
|
-
data?.dependencies?.every(d => d.healthy);
|
|
1628
|
-
},
|
|
1629
|
-
|
|
1630
|
-
commonHandleErrors: async ({ reqData, errorLog }) => {
|
|
1631
|
-
// Alert on core service issues
|
|
1632
|
-
await pagerDuty.trigger({
|
|
1633
|
-
severity: 'critical',
|
|
1634
|
-
service: reqData.baseURL,
|
|
1635
|
-
message: errorLog.error
|
|
1636
|
-
});
|
|
1637
|
-
}
|
|
1638
|
-
}
|
|
1639
|
-
},
|
|
1640
|
-
{
|
|
1641
|
-
id: 'optional',
|
|
1642
|
-
commonConfig: {
|
|
1643
|
-
commonAttempts: 2,
|
|
1644
|
-
|
|
1645
|
-
commonResponseAnalyzer: async ({ data }) => {
|
|
1646
|
-
// Optional services: basic check
|
|
1647
|
-
return data?.status === 'ok';
|
|
1648
|
-
},
|
|
1649
|
-
|
|
1650
|
-
commonFinalErrorAnalyzer: async ({ reqData, error }) => {
|
|
1651
|
-
// Log but don't alert
|
|
1652
|
-
console.warn(`Optional service ${reqData.baseURL} unhealthy`);
|
|
1653
|
-
return true; // Don't throw
|
|
1654
|
-
}
|
|
1655
|
-
}
|
|
1656
|
-
}
|
|
1657
|
-
]
|
|
1658
|
-
}
|
|
1659
|
-
);
|
|
620
|
+
import { stableWorkflow, RETRY_STRATEGIES } from '@emmvish/stable-request';
|
|
1660
621
|
|
|
1661
|
-
|
|
1662
|
-
|
|
1663
|
-
|
|
1664
|
-
|
|
1665
|
-
|
|
1666
|
-
};
|
|
1667
|
-
|
|
1668
|
-
console.log('System Health:', report);
|
|
1669
|
-
```
|
|
622
|
+
interface TenantConfig {
|
|
623
|
+
tenantId: string;
|
|
624
|
+
apiKey: string;
|
|
625
|
+
baseUrl: string;
|
|
626
|
+
}
|
|
1670
627
|
|
|
1671
|
-
|
|
628
|
+
async function executeTenantWorkflow(tenantConfig: TenantConfig) {
|
|
629
|
+
const sharedBuffer: Record<string, any> = {
|
|
630
|
+
tenantId: tenantConfig.tenantId,
|
|
631
|
+
authToken: null,
|
|
632
|
+
processedItems: []
|
|
633
|
+
};
|
|
1672
634
|
|
|
1673
|
-
|
|
1674
|
-
const etlWorkflow = await stableWorkflow(
|
|
1675
|
-
[
|
|
635
|
+
const phases: STABLE_WORKFLOW_PHASE[] = [
|
|
1676
636
|
{
|
|
1677
|
-
id: '
|
|
1678
|
-
concurrentExecution: true,
|
|
1679
|
-
commonConfig: {
|
|
1680
|
-
commonAttempts: 5,
|
|
1681
|
-
commonRetryStrategy: RETRY_STRATEGIES.EXPONENTIAL
|
|
1682
|
-
},
|
|
637
|
+
id: 'authentication',
|
|
1683
638
|
requests: [
|
|
1684
|
-
{
|
|
1685
|
-
|
|
1686
|
-
|
|
639
|
+
{
|
|
640
|
+
id: 'get-token',
|
|
641
|
+
requestOptions: {
|
|
642
|
+
reqData: {
|
|
643
|
+
path: '/auth/token',
|
|
644
|
+
method: 'POST',
|
|
645
|
+
headers: { 'X-API-Key': tenantConfig.apiKey }
|
|
646
|
+
},
|
|
647
|
+
resReq: true,
|
|
648
|
+
attempts: 3,
|
|
649
|
+
wait: 2000,
|
|
650
|
+
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL,
|
|
651
|
+
responseAnalyzer: async ({ data, commonBuffer }) => {
|
|
652
|
+
if (data?.token) {
|
|
653
|
+
commonBuffer.authToken = data.token;
|
|
654
|
+
commonBuffer.tokenExpiry = Date.now() + (data.expiresIn * 1000);
|
|
655
|
+
return true;
|
|
656
|
+
}
|
|
657
|
+
return false;
|
|
658
|
+
}
|
|
659
|
+
}
|
|
660
|
+
}
|
|
1687
661
|
]
|
|
1688
662
|
},
|
|
1689
663
|
{
|
|
1690
|
-
id: '
|
|
1691
|
-
concurrentExecution:
|
|
1692
|
-
|
|
664
|
+
id: 'data-fetching',
|
|
665
|
+
concurrentExecution: true,
|
|
666
|
+
maxConcurrentRequests: 5,
|
|
1693
667
|
requests: [
|
|
1694
668
|
{
|
|
1695
|
-
id: '
|
|
1696
|
-
requestOptions: {
|
|
1697
|
-
reqData: { path: '/transform/clean', method: REQUEST_METHODS.POST },
|
|
1698
|
-
resReq: true
|
|
1699
|
-
}
|
|
1700
|
-
},
|
|
1701
|
-
{
|
|
1702
|
-
id: 'enrich-data',
|
|
669
|
+
id: 'fetch-users',
|
|
1703
670
|
requestOptions: {
|
|
1704
|
-
reqData: { path: '/
|
|
1705
|
-
resReq: true
|
|
671
|
+
reqData: { path: '/users' },
|
|
672
|
+
resReq: true,
|
|
673
|
+
preExecution: {
|
|
674
|
+
preExecutionHook: ({ commonBuffer }) => ({
|
|
675
|
+
reqData: {
|
|
676
|
+
path: '/users',
|
|
677
|
+
headers: { Authorization: `Bearer ${commonBuffer.authToken}` }
|
|
678
|
+
}
|
|
679
|
+
}),
|
|
680
|
+
applyPreExecutionConfigOverride: true
|
|
681
|
+
}
|
|
1706
682
|
}
|
|
1707
683
|
},
|
|
1708
684
|
{
|
|
1709
|
-
id: '
|
|
685
|
+
id: 'fetch-settings',
|
|
1710
686
|
requestOptions: {
|
|
1711
|
-
reqData: { path: '/
|
|
687
|
+
reqData: { path: '/settings' },
|
|
1712
688
|
resReq: true,
|
|
1713
|
-
|
|
1714
|
-
|
|
689
|
+
preExecution: {
|
|
690
|
+
preExecutionHook: ({ commonBuffer }) => ({
|
|
691
|
+
reqData: {
|
|
692
|
+
path: '/settings',
|
|
693
|
+
headers: { Authorization: `Bearer ${commonBuffer.authToken}` }
|
|
694
|
+
}
|
|
695
|
+
}),
|
|
696
|
+
applyPreExecutionConfigOverride: true
|
|
1715
697
|
}
|
|
1716
698
|
}
|
|
1717
699
|
}
|
|
1718
700
|
]
|
|
1719
701
|
},
|
|
1720
702
|
{
|
|
1721
|
-
id: '
|
|
703
|
+
id: 'data-processing',
|
|
1722
704
|
concurrentExecution: true,
|
|
1723
705
|
requests: [
|
|
1724
706
|
{
|
|
1725
|
-
id: '
|
|
1726
|
-
requestOptions: {
|
|
1727
|
-
reqData: { path: '/load/warehouse', method: REQUEST_METHODS.POST },
|
|
1728
|
-
resReq: true
|
|
1729
|
-
}
|
|
1730
|
-
},
|
|
1731
|
-
{
|
|
1732
|
-
id: 'update-indexes',
|
|
707
|
+
id: 'process-users',
|
|
1733
708
|
requestOptions: {
|
|
1734
|
-
reqData: { path: '/
|
|
1735
|
-
resReq: true
|
|
1736
|
-
|
|
1737
|
-
|
|
1738
|
-
|
|
1739
|
-
|
|
1740
|
-
|
|
1741
|
-
|
|
1742
|
-
|
|
709
|
+
reqData: { path: '/process/users', method: 'POST' },
|
|
710
|
+
resReq: true,
|
|
711
|
+
preExecution: {
|
|
712
|
+
preExecutionHook: ({ commonBuffer }) => {
|
|
713
|
+
const usersPhase = commonBuffer.phases?.find(p => p.phaseId === 'data-fetching');
|
|
714
|
+
const usersData = usersPhase?.responses?.find(r => r.requestId === 'fetch-users')?.data;
|
|
715
|
+
|
|
716
|
+
return {
|
|
717
|
+
reqData: {
|
|
718
|
+
path: '/process/users',
|
|
719
|
+
method: 'POST',
|
|
720
|
+
headers: { Authorization: `Bearer ${commonBuffer.authToken}` },
|
|
721
|
+
body: { users: usersData }
|
|
722
|
+
}
|
|
723
|
+
};
|
|
724
|
+
},
|
|
725
|
+
applyPreExecutionConfigOverride: true
|
|
726
|
+
},
|
|
727
|
+
responseAnalyzer: async ({ data, commonBuffer }) => {
|
|
728
|
+
if (data?.processed) {
|
|
729
|
+
commonBuffer.processedItems.push(...data.processed);
|
|
730
|
+
return true;
|
|
731
|
+
}
|
|
732
|
+
return false;
|
|
733
|
+
}
|
|
1743
734
|
}
|
|
1744
735
|
}
|
|
1745
736
|
]
|
|
1746
737
|
}
|
|
1747
|
-
]
|
|
1748
|
-
|
|
1749
|
-
|
|
738
|
+
];
|
|
739
|
+
|
|
740
|
+
const result = await stableWorkflow(phases, {
|
|
741
|
+
workflowId: `tenant-${tenantConfig.tenantId}-workflow`,
|
|
742
|
+
commonRequestData: {
|
|
743
|
+
hostname: tenantConfig.baseUrl,
|
|
744
|
+
headers: { 'X-Tenant-ID': tenantConfig.tenantId }
|
|
745
|
+
},
|
|
1750
746
|
stopOnFirstPhaseError: true,
|
|
1751
747
|
logPhaseResults: true,
|
|
1752
|
-
|
|
1753
|
-
|
|
1754
|
-
|
|
748
|
+
sharedBuffer,
|
|
749
|
+
circuitBreaker: {
|
|
750
|
+
failureThresholdPercentage: 40,
|
|
751
|
+
minimumRequests: 5,
|
|
752
|
+
recoveryTimeoutMs: 30000
|
|
1755
753
|
},
|
|
1756
|
-
|
|
1757
|
-
|
|
1758
|
-
|
|
1759
|
-
handlePhaseCompletion: async ({ phaseResult }) => {
|
|
1760
|
-
const recordsProcessed = phaseResult.responses
|
|
1761
|
-
.filter(r => r.success)
|
|
1762
|
-
.reduce((sum, r) => sum + (r.data?.recordCount || 0), 0);
|
|
1763
|
-
|
|
1764
|
-
await metrics.gauge('etl.phase.records', recordsProcessed, {
|
|
1765
|
-
phase: phaseResult.phaseId
|
|
1766
|
-
});
|
|
754
|
+
rateLimit: {
|
|
755
|
+
maxRequests: 20,
|
|
756
|
+
windowMs: 1000
|
|
1767
757
|
},
|
|
1768
|
-
|
|
1769
|
-
|
|
1770
|
-
|
|
1771
|
-
|
|
1772
|
-
|
|
1773
|
-
|
|
758
|
+
commonCache: {
|
|
759
|
+
enabled: true,
|
|
760
|
+
ttl: 300000 // Cache for 5 minutes
|
|
761
|
+
},
|
|
762
|
+
handlePhaseCompletion: ({ workflowId, phaseResult }) => {
|
|
763
|
+
console.log(`[${workflowId}] Phase ${phaseResult.phaseId} completed:`, {
|
|
764
|
+
success: phaseResult.success,
|
|
765
|
+
successfulRequests: phaseResult.successfulRequests,
|
|
766
|
+
executionTime: `${phaseResult.executionTime}ms`
|
|
1774
767
|
});
|
|
768
|
+
},
|
|
769
|
+
handlePhaseError: ({ workflowId, error, phaseResult }) => {
|
|
770
|
+
console.error(`[${workflowId}] Phase ${phaseResult.phaseId} failed:`, error);
|
|
771
|
+
// Send to monitoring
|
|
772
|
+
monitoringService.trackPhaseError(workflowId, phaseResult.phaseId, error);
|
|
1775
773
|
}
|
|
1776
|
-
}
|
|
1777
|
-
);
|
|
774
|
+
});
|
|
1778
775
|
|
|
1779
|
-
|
|
1780
|
-
|
|
1781
|
-
|
|
1782
|
-
|
|
776
|
+
return {
|
|
777
|
+
success: result.success,
|
|
778
|
+
tenantId: tenantConfig.tenantId,
|
|
779
|
+
processedItems: sharedBuffer.processedItems,
|
|
780
|
+
executionTime: result.executionTime,
|
|
781
|
+
phases: result.phases.map(p => ({
|
|
782
|
+
id: p.phaseId,
|
|
783
|
+
success: p.success,
|
|
784
|
+
requestCount: p.totalRequests
|
|
785
|
+
}))
|
|
786
|
+
};
|
|
787
|
+
}
|
|
1783
788
|
|
|
1784
|
-
|
|
1785
|
-
|
|
1786
|
-
|
|
1787
|
-
|
|
1788
|
-
|
|
1789
|
-
|--------|------|---------|-------------|
|
|
1790
|
-
| `reqData` | `REQUEST_DATA` | **required** | Request configuration |
|
|
1791
|
-
| `resReq` | `boolean` | `false` | Return response data vs. just boolean |
|
|
1792
|
-
| `attempts` | `number` | `1` | Max retry attempts |
|
|
1793
|
-
| `wait` | `number` | `1000` | Base delay between retries (ms) |
|
|
1794
|
-
| `maxAllowedWait` | `number` | `60000` | Maximum permitted wait duration between retries (ms) |
|
|
1795
|
-
| `retryStrategy` | `RETRY_STRATEGY_TYPES` | `'fixed'` | Retry backoff strategy |
|
|
1796
|
-
| `performAllAttempts` | `boolean` | `false` | Execute all attempts regardless |
|
|
1797
|
-
| `logAllErrors` | `boolean` | `false` | Enable error logging |
|
|
1798
|
-
| `logAllSuccessfulAttempts` | `boolean` | `false` | Enable success logging |
|
|
1799
|
-
| `maxSerializableChars` | `number` | `1000` | Max chars for logs |
|
|
1800
|
-
| `trialMode` | `TRIAL_MODE_OPTIONS` | `{ enabled: false }` | Failure simulation |
|
|
1801
|
-
| `hookParams` | `HookParams` | `{}` | Custom parameters for hooks |
|
|
1802
|
-
| `preExecution` | `RequestPreExecutionOptions` | `{}` | Executes before actually sending request, can modify request config |
|
|
1803
|
-
| `commonBuffer` | `Record<string, any>` | `{}` | For communication between various request hooks |
|
|
1804
|
-
| `responseAnalyzer` | `function` | `() => true` | Validate response content |
|
|
1805
|
-
| `handleErrors` | `function` | `console.log` | Error handler |
|
|
1806
|
-
| `handleSuccessfulAttemptData` | `function` | `console.log` | Success handler |
|
|
1807
|
-
| `finalErrorAnalyzer` | `function` | `() => false` | Final error handler |
|
|
1808
|
-
|
|
1809
|
-
### REQUEST_DATA
|
|
789
|
+
// Execute workflows for multiple tenants
|
|
790
|
+
const tenants: TenantConfig[] = [
|
|
791
|
+
{ tenantId: 'tenant-1', apiKey: 'key1', baseUrl: 'api.tenant1.com' },
|
|
792
|
+
{ tenantId: 'tenant-2', apiKey: 'key2', baseUrl: 'api.tenant2.com' }
|
|
793
|
+
];
|
|
1810
794
|
|
|
1811
|
-
|
|
1812
|
-
|
|
1813
|
-
|
|
1814
|
-
|
|
1815
|
-
method?: 'GET' | 'POST' | 'PUT' | 'PATCH' | 'DELETE'; // Default: 'GET'
|
|
1816
|
-
path?: `/${string}`; // Default: ''
|
|
1817
|
-
port?: number; // Default: 443
|
|
1818
|
-
headers?: Record<string, any>; // Default: {}
|
|
1819
|
-
body?: RequestDataType; // Request body
|
|
1820
|
-
query?: Record<string, any>; // Query parameters
|
|
1821
|
-
timeout?: number; // Default: 15000ms
|
|
1822
|
-
signal?: AbortSignal; // For cancellation
|
|
1823
|
-
}
|
|
795
|
+
const results = await Promise.all(tenants.map(executeTenantWorkflow));
|
|
796
|
+
results.forEach(result => {
|
|
797
|
+
console.log(`Tenant ${result.tenantId}:`, result.success ? 'Success' : 'Failed');
|
|
798
|
+
});
|
|
1824
799
|
```
|
|
1825
800
|
|
|
1826
|
-
###
|
|
1827
|
-
|
|
1828
|
-
| Option | Type | Default | Description |
|
|
1829
|
-
|--------|------|---------|-------------|
|
|
1830
|
-
| `concurrentExecution` | `boolean` | `true` | Execute requests concurrently or sequentially |
|
|
1831
|
-
| `stopOnFirstError` | `boolean` | `false` | Stop execution on first error (sequential only) |
|
|
1832
|
-
| `requestGroups` | `RequestGroup[]` | `[]` | Define groups with their own common configurations |
|
|
1833
|
-
| `commonAttempts` | `number` | `1` | Default attempts for all requests |
|
|
1834
|
-
| `commonPerformAllAttempts` | `boolean` | `false` | Default performAllAttempts for all requests |
|
|
1835
|
-
| `commonWait` | `number` | `1000` | Default wait time for all requests |
|
|
1836
|
-
| `commonMaxAllowedWait` | `number` | `60000` | Default maximum permitted wait time for all requests |
|
|
1837
|
-
| `commonRetryStrategy` | `RETRY_STRATEGY_TYPES` | `'fixed'` | Default retry strategy for all requests |
|
|
1838
|
-
| `commonLogAllErrors` | `boolean` | `false` | Default error logging for all requests |
|
|
1839
|
-
| `commonLogAllSuccessfulAttempts` | `boolean` | `false` | Default success logging for all requests |
|
|
1840
|
-
| `commonMaxSerializableChars` | `number` | `1000` | Default max chars for serialization |
|
|
1841
|
-
| `commonTrialMode` | `TRIAL_MODE_OPTIONS` | `{ enabled: false }` | Default trial mode for all requests |
|
|
1842
|
-
| `commonResponseAnalyzer` | `function` | `() => true` | Default response analyzer for all requests |
|
|
1843
|
-
| `commonResReq` | `boolean` | `false` | Default resReq for all requests |
|
|
1844
|
-
| `commonFinalErrorAnalyzer` | `function` | `() => false` | Default final error analyzer for all requests |
|
|
1845
|
-
| `commonHandleErrors` | `function` | console.log | Default error handler for all requests |
|
|
1846
|
-
| `commonHandleSuccessfulAttemptData` | `function` | console.log | Default success handler for all requests |
|
|
1847
|
-
| `commonRequestData` | `Partial<REQUEST_DATA>` | `{ hostname: '' }` | Common set of request options for each request |
|
|
1848
|
-
| `commonHookParams` | `HookParams` | `{ }` | Common options for each request hook |
|
|
1849
|
-
| `sharedBuffer` | `Record<string, any>` | `undefined` | For communication between various requests |
|
|
1850
|
-
|
|
1851
|
-
### `stableWorkflow(phases, options)`
|
|
1852
|
-
|
|
1853
|
-
Execute a multi-phase workflow with full control over execution order and error handling.
|
|
1854
|
-
|
|
1855
|
-
**Phases Array:**
|
|
1856
|
-
```typescript
|
|
1857
|
-
interface STABLE_WORKFLOW_PHASE {
|
|
1858
|
-
id?: string; // Phase identifier (auto-generated if omitted)
|
|
1859
|
-
concurrentExecution?: boolean; // true = parallel, false = sequential (default: true)
|
|
1860
|
-
stopOnFirstError?: boolean; // Stop phase on first request failure (default: false)
|
|
1861
|
-
commonConfig?: Omit<API_GATEWAY_OPTIONS; 'concurrentExecution' | 'stopOnFirstError' | 'requestGroups'>;
|
|
1862
|
-
requests: API_GATEWAY_REQUEST[]; // Array of requests for this phase
|
|
1863
|
-
markConcurrentPhase?: boolean; // Allows this phase to be executed concurrently with immediately next phase marked as concurrent
|
|
1864
|
-
}
|
|
1865
|
-
```
|
|
801
|
+
### Use Case 2: Resilient Data Pipeline with Fallback Strategies
|
|
1866
802
|
|
|
1867
|
-
**Workflow Options:**
|
|
1868
|
-
|
|
1869
|
-
| Option | Type | Default | Description |
|
|
1870
|
-
|--------|------|---------|-------------|
|
|
1871
|
-
| `workflowId` | `string` | `workflow-{timestamp}` | Workflow identifier |
|
|
1872
|
-
| `stopOnFirstPhaseError` | `boolean` | `false` | Stop workflow if any phase fails |
|
|
1873
|
-
| `logPhaseResults` | `boolean` | `false` | Log phase execution to console |
|
|
1874
|
-
| `concurrentPhaseExecution` | `boolean` | `false` | Execute all phases in parallel. Overrides `enableMixedExecution` |
|
|
1875
|
-
| `handlePhaseCompletion` | `function` | `undefined` | Hook called after each successful phase |
|
|
1876
|
-
| `handlePhaseError` | `function` | `undefined` | Hook called when a phase fails |
|
|
1877
|
-
| `maxSerializableChars` | `number` | `1000` | Max chars for serialization in hooks |
|
|
1878
|
-
| `workflowHookParams` | `WorkflowHookParams` | {} | Custom set of params passed to hooks |
|
|
1879
|
-
| `sharedBuffer` | `Record<string, any>` | `undefined` | Buffer shared by all phases and all requests within them |
|
|
1880
|
-
| `enableMixedExecution` | `boolean` | `false` | Enables mixing of sequential and parallel sub-workflows |
|
|
1881
|
-
| All `stableApiGateway` options | - | - | Applied as workflow-level defaults |
|
|
1882
|
-
|
|
1883
|
-
**STABLE_WORKFLOW_RESULT response:**
|
|
1884
803
|
```typescript
|
|
1885
|
-
|
|
1886
|
-
workflowId: string;
|
|
1887
|
-
success: boolean; // All phases successful?
|
|
1888
|
-
executionTime: number; // Total workflow duration (ms)
|
|
1889
|
-
timestamp: string; // ISO timestamp
|
|
1890
|
-
totalPhases: number;
|
|
1891
|
-
completedPhases: number;
|
|
1892
|
-
totalRequests: number;
|
|
1893
|
-
successfulRequests: number;
|
|
1894
|
-
failedRequests: number;
|
|
1895
|
-
phases: PHASE_RESULT[]; // Detailed results per phase
|
|
1896
|
-
error?: string; // Workflow-level error
|
|
1897
|
-
}
|
|
1898
|
-
```
|
|
804
|
+
import { stableApiGateway, RETRY_STRATEGIES, CircuitBreaker } from '@emmvish/stable-request';
|
|
1899
805
|
|
|
1900
|
-
|
|
806
|
+
interface DataSource {
|
|
807
|
+
id: string;
|
|
808
|
+
priority: number;
|
|
809
|
+
endpoint: string;
|
|
810
|
+
hostname: string;
|
|
811
|
+
}
|
|
1901
812
|
|
|
1902
|
-
|
|
813
|
+
async function fetchDataWithFallback(dataSources: DataSource[]) {
|
|
814
|
+
// Sort by priority
|
|
815
|
+
const sortedSources = [...dataSources].sort((a, b) => a.priority - b.priority);
|
|
816
|
+
|
|
817
|
+
// Create circuit breakers for each source
|
|
818
|
+
const circuitBreakers = new Map(
|
|
819
|
+
sortedSources.map(source => [
|
|
820
|
+
source.id,
|
|
821
|
+
new CircuitBreaker({
|
|
822
|
+
failureThresholdPercentage: 50,
|
|
823
|
+
minimumRequests: 3,
|
|
824
|
+
recoveryTimeoutMs: 60000
|
|
825
|
+
})
|
|
826
|
+
])
|
|
827
|
+
);
|
|
828
|
+
|
|
829
|
+
// Try each data source in priority order
|
|
830
|
+
for (const source of sortedSources) {
|
|
831
|
+
const breaker = circuitBreakers.get(source.id)!;
|
|
832
|
+
const breakerState = breaker.getState();
|
|
833
|
+
|
|
834
|
+
// Skip if circuit is open
|
|
835
|
+
if (breakerState.state === 'OPEN') {
|
|
836
|
+
console.warn(`Circuit breaker open for ${source.id}, skipping...`);
|
|
837
|
+
continue;
|
|
838
|
+
}
|
|
1903
839
|
|
|
1904
|
-
|
|
840
|
+
console.log(`Attempting to fetch from ${source.id}...`);
|
|
1905
841
|
|
|
1906
|
-
|
|
1907
|
-
|
|
1908
|
-
|
|
1909
|
-
|
|
1910
|
-
|
|
1911
|
-
|
|
1912
|
-
|
|
1913
|
-
|
|
1914
|
-
|
|
1915
|
-
|
|
1916
|
-
|
|
1917
|
-
|
|
1918
|
-
|
|
1919
|
-
|
|
1920
|
-
|
|
1921
|
-
|
|
1922
|
-
|
|
1923
|
-
|
|
1924
|
-
|
|
1925
|
-
|
|
1926
|
-
|
|
1927
|
-
}
|
|
1928
|
-
|
|
842
|
+
try {
|
|
843
|
+
const requests = [
|
|
844
|
+
{
|
|
845
|
+
id: 'users',
|
|
846
|
+
requestOptions: {
|
|
847
|
+
reqData: { path: `${source.endpoint}/users` },
|
|
848
|
+
resReq: true,
|
|
849
|
+
attempts: 3,
|
|
850
|
+
wait: 1000,
|
|
851
|
+
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL
|
|
852
|
+
}
|
|
853
|
+
},
|
|
854
|
+
{
|
|
855
|
+
id: 'products',
|
|
856
|
+
requestOptions: {
|
|
857
|
+
reqData: { path: `${source.endpoint}/products` },
|
|
858
|
+
resReq: true,
|
|
859
|
+
attempts: 3,
|
|
860
|
+
wait: 1000,
|
|
861
|
+
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL
|
|
862
|
+
}
|
|
863
|
+
},
|
|
864
|
+
{
|
|
865
|
+
id: 'orders',
|
|
866
|
+
requestOptions: {
|
|
867
|
+
reqData: { path: `${source.endpoint}/orders` },
|
|
868
|
+
resReq: true,
|
|
869
|
+
attempts: 3,
|
|
870
|
+
wait: 1000,
|
|
871
|
+
retryStrategy: RETRY_STRATEGIES.EXPONENTIAL
|
|
872
|
+
}
|
|
873
|
+
}
|
|
874
|
+
];
|
|
1929
875
|
|
|
1930
|
-
|
|
876
|
+
const results = await stableApiGateway(requests, {
|
|
877
|
+
commonRequestData: {
|
|
878
|
+
hostname: source.hostname,
|
|
879
|
+
headers: { 'X-Source-ID': source.id }
|
|
880
|
+
},
|
|
881
|
+
concurrentExecution: true,
|
|
882
|
+
maxConcurrentRequests: 10,
|
|
883
|
+
circuitBreaker: breaker,
|
|
884
|
+
rateLimit: {
|
|
885
|
+
maxRequests: 50,
|
|
886
|
+
windowMs: 1000
|
|
887
|
+
},
|
|
888
|
+
commonCache: {
|
|
889
|
+
enabled: true,
|
|
890
|
+
ttl: 60000
|
|
891
|
+
},
|
|
892
|
+
commonResponseAnalyzer: async ({ data }) => {
|
|
893
|
+
// Validate data structure
|
|
894
|
+
return data && typeof data === 'object' && !data.error;
|
|
895
|
+
},
|
|
896
|
+
commonHandleErrors: ({ errorLog }) => {
|
|
897
|
+
console.error(`Error from ${source.id}:`, errorLog);
|
|
898
|
+
}
|
|
899
|
+
});
|
|
1931
900
|
|
|
1932
|
-
|
|
901
|
+
// Check if all requests succeeded
|
|
902
|
+
const allSuccessful = results.every(r => r.success);
|
|
903
|
+
|
|
904
|
+
if (allSuccessful) {
|
|
905
|
+
console.log(`Successfully fetched data from ${source.id}`);
|
|
906
|
+
return {
|
|
907
|
+
source: source.id,
|
|
908
|
+
data: {
|
|
909
|
+
users: results.find(r => r.requestId === 'users')?.data,
|
|
910
|
+
products: results.find(r => r.requestId === 'products')?.data,
|
|
911
|
+
orders: results.find(r => r.requestId === 'orders')?.data
|
|
912
|
+
}
|
|
913
|
+
};
|
|
914
|
+
} else {
|
|
915
|
+
console.warn(`Partial failure from ${source.id}, trying next source...`);
|
|
916
|
+
}
|
|
917
|
+
} catch (error) {
|
|
918
|
+
console.error(`Failed to fetch from ${source.id}:`, error);
|
|
919
|
+
// Continue to next source
|
|
920
|
+
}
|
|
921
|
+
}
|
|
1933
922
|
|
|
1934
|
-
|
|
1935
|
-
responseAnalyzer: async ({ reqData, data, trialMode, params, commonBuffer }) => {
|
|
1936
|
-
// Return true if valid, false to retry
|
|
1937
|
-
return data.status === 'ready';
|
|
923
|
+
throw new Error('All data sources failed');
|
|
1938
924
|
}
|
|
925
|
+
|
|
926
|
+
// Usage
|
|
927
|
+
const dataSources: DataSource[] = [
|
|
928
|
+
{
|
|
929
|
+
id: 'primary-db',
|
|
930
|
+
priority: 1,
|
|
931
|
+
endpoint: '/api/v1',
|
|
932
|
+
hostname: 'primary.example.com'
|
|
933
|
+
},
|
|
934
|
+
{
|
|
935
|
+
id: 'replica-db',
|
|
936
|
+
priority: 2,
|
|
937
|
+
endpoint: '/api/v1',
|
|
938
|
+
hostname: 'replica.example.com'
|
|
939
|
+
},
|
|
940
|
+
{
|
|
941
|
+
id: 'backup-cache',
|
|
942
|
+
priority: 3,
|
|
943
|
+
endpoint: '/cached',
|
|
944
|
+
hostname: 'cache.example.com'
|
|
945
|
+
}
|
|
946
|
+
];
|
|
947
|
+
|
|
948
|
+
const result = await fetchDataWithFallback(dataSources);
|
|
949
|
+
console.log('Data fetched from:', result.source);
|
|
950
|
+
console.log('Users:', result.data.users?.length);
|
|
951
|
+
console.log('Products:', result.data.products?.length);
|
|
952
|
+
console.log('Orders:', result.data.orders?.length);
|
|
1939
953
|
```
|
|
1940
954
|
|
|
1941
|
-
|
|
955
|
+
## Configuration Options
|
|
1942
956
|
|
|
1943
|
-
|
|
957
|
+
### Request Data Configuration
|
|
1944
958
|
|
|
1945
959
|
```typescript
|
|
1946
|
-
|
|
1947
|
-
|
|
1948
|
-
|
|
1949
|
-
|
|
1950
|
-
|
|
1951
|
-
|
|
960
|
+
interface REQUEST_DATA<RequestDataType> {
|
|
961
|
+
hostname: string;
|
|
962
|
+
protocol?: 'http' | 'https'; // default: 'https'
|
|
963
|
+
method?: 'GET' | 'POST' | 'PUT' | 'PATCH' | 'DELETE'; // default: 'GET'
|
|
964
|
+
path?: `/${string}`;
|
|
965
|
+
port?: number; // default: 443
|
|
966
|
+
headers?: Record<string, any>;
|
|
967
|
+
body?: RequestDataType;
|
|
968
|
+
query?: Record<string, any>;
|
|
969
|
+
timeout?: number; // default: 15000ms
|
|
970
|
+
signal?: AbortSignal;
|
|
1952
971
|
}
|
|
1953
972
|
```
|
|
1954
973
|
|
|
1955
|
-
|
|
1956
|
-
|
|
1957
|
-
**Purpose:** Monitor and log successful attempts
|
|
974
|
+
### Retry Configuration
|
|
1958
975
|
|
|
1959
976
|
```typescript
|
|
1960
|
-
|
|
1961
|
-
|
|
1962
|
-
|
|
1963
|
-
|
|
1964
|
-
|
|
977
|
+
interface RetryConfig {
|
|
978
|
+
attempts?: number; // default: 1
|
|
979
|
+
wait?: number; // default: 1000ms
|
|
980
|
+
maxAllowedWait?: number; // default: 60000ms
|
|
981
|
+
retryStrategy?: 'fixed' | 'linear' | 'exponential'; // default: 'fixed'
|
|
982
|
+
performAllAttempts?: boolean; // default: false
|
|
1965
983
|
}
|
|
1966
984
|
```
|
|
1967
985
|
|
|
1968
|
-
|
|
1969
|
-
|
|
1970
|
-
**Purpose:** Handle final error after all retries exhausted
|
|
986
|
+
### Circuit Breaker Configuration
|
|
1971
987
|
|
|
1972
988
|
```typescript
|
|
1973
|
-
|
|
1974
|
-
//
|
|
1975
|
-
|
|
1976
|
-
|
|
1977
|
-
|
|
1978
|
-
}
|
|
1979
|
-
return false; // Throw
|
|
989
|
+
interface CircuitBreakerConfig {
|
|
990
|
+
failureThresholdPercentage: number; // 0-100
|
|
991
|
+
minimumRequests: number;
|
|
992
|
+
recoveryTimeoutMs: number;
|
|
993
|
+
trackIndividualAttempts?: boolean; // default: false
|
|
1980
994
|
}
|
|
1981
995
|
```
|
|
1982
996
|
|
|
1983
|
-
|
|
1984
|
-
|
|
1985
|
-
**Purpose:** Execute phase-bridging code upon successful completion of a phase
|
|
997
|
+
### Rate Limit Configuration
|
|
1986
998
|
|
|
1987
999
|
```typescript
|
|
1988
|
-
|
|
1989
|
-
|
|
1000
|
+
interface RateLimitConfig {
|
|
1001
|
+
maxRequests: number;
|
|
1002
|
+
windowMs: number;
|
|
1990
1003
|
}
|
|
1991
1004
|
```
|
|
1992
1005
|
|
|
1993
|
-
|
|
1994
|
-
|
|
1995
|
-
**Purpose:** Execute error handling code if a phase runs into an error
|
|
1006
|
+
### Cache Configuration
|
|
1996
1007
|
|
|
1997
1008
|
```typescript
|
|
1998
|
-
|
|
1999
|
-
|
|
1009
|
+
interface CacheConfig {
|
|
1010
|
+
enabled: boolean;
|
|
1011
|
+
ttl?: number; // milliseconds, default: 300000 (5 minutes)
|
|
2000
1012
|
}
|
|
2001
1013
|
```
|
|
2002
1014
|
|
|
2003
|
-
|
|
2004
|
-
|
|
2005
|
-
Configuration precedence across orchestration:
|
|
2006
|
-
|
|
2007
|
-
- Workflow-level (options.sharedBuffer)
|
|
2008
|
-
- Phase-level (commonConfig)
|
|
2009
|
-
- Request group (requestGroups[].commonConfig)
|
|
2010
|
-
- Individual request options (highest priority)
|
|
2011
|
-
|
|
2012
|
-
Buffers are state (not config):
|
|
2013
|
-
|
|
2014
|
-
- Request scope: commonBuffer
|
|
2015
|
-
- Gateway scope: Gateway's / Phase's sharedBuffer
|
|
2016
|
-
- Workflow scope: Workflow's sharedBuffer
|
|
2017
|
-
|
|
2018
|
-
## TypeScript Support
|
|
2019
|
-
|
|
2020
|
-
Fully typed with generics:
|
|
1015
|
+
### Pre-Execution Configuration
|
|
2021
1016
|
|
|
2022
1017
|
```typescript
|
|
2023
|
-
interface
|
|
2024
|
-
|
|
2025
|
-
|
|
2026
|
-
|
|
2027
|
-
|
|
2028
|
-
interface UserResponse {
|
|
2029
|
-
id: string;
|
|
2030
|
-
name: string;
|
|
2031
|
-
email: string;
|
|
2032
|
-
createdAt: string;
|
|
1018
|
+
interface RequestPreExecutionOptions {
|
|
1019
|
+
preExecutionHook: (options: PreExecutionHookOptions) => any | Promise<any>;
|
|
1020
|
+
preExecutionHookParams?: any;
|
|
1021
|
+
applyPreExecutionConfigOverride?: boolean; // default: false
|
|
1022
|
+
continueOnPreExecutionHookFailure?: boolean; // default: false
|
|
2033
1023
|
}
|
|
2034
|
-
|
|
2035
|
-
const user = await stableRequest<CreateUserRequest, UserResponse>({
|
|
2036
|
-
reqData: {
|
|
2037
|
-
hostname: 'api.example.com',
|
|
2038
|
-
path: '/users',
|
|
2039
|
-
method: REQUEST_METHODS.POST,
|
|
2040
|
-
body: {
|
|
2041
|
-
name: 'John Doe',
|
|
2042
|
-
email: 'john@example.com'
|
|
2043
|
-
}
|
|
2044
|
-
},
|
|
2045
|
-
resReq: true
|
|
2046
|
-
});
|
|
2047
|
-
|
|
2048
|
-
// user is typed as UserResponse
|
|
2049
|
-
console.log(user.id); // TypeScript knows this exists
|
|
2050
1024
|
```
|
|
2051
1025
|
|
|
2052
1026
|
## License
|
|
2053
1027
|
|
|
2054
1028
|
MIT © Manish Varma
|
|
2055
1029
|
|
|
2056
|
-
|
|
2057
1030
|
[](https://opensource.org/licenses/MIT)
|
|
2058
1031
|
|
|
2059
1032
|
---
|