@backendkit-labs/auto-learning 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +741 -0
- package/dist/{index-DnQ9xssn.d.cts → index-CtdA-dkB.d.cts} +15 -5
- package/dist/{index-DnQ9xssn.d.ts → index-CtdA-dkB.d.ts} +15 -5
- package/dist/index.cjs +147 -44
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +2 -2
- package/dist/index.d.ts +2 -2
- package/dist/index.js +133 -40
- package/dist/index.js.map +1 -1
- package/dist/nestjs/index.cjs +147 -44
- package/dist/nestjs/index.cjs.map +1 -1
- package/dist/nestjs/index.d.cts +1 -1
- package/dist/nestjs/index.d.ts +1 -1
- package/dist/nestjs/index.js +133 -40
- package/dist/nestjs/index.js.map +1 -1
- package/package.json +9 -1
package/README.md
ADDED
|
@@ -0,0 +1,741 @@
|
|
|
1
|
+
# @backendkit-labs/auto-learning
|
|
2
|
+
|
|
3
|
+
[](https://www.npmjs.com/package/@backendkit-labs/auto-learning)
|
|
4
|
+
[](https://github.com/BackendKit-labs/backendkit-monorepo/actions/workflows/ci.yml)
|
|
5
|
+
[](LICENSE)
|
|
6
|
+
[](package.json)
|
|
7
|
+
[](https://backendkitlabs.dev/docs/auto-learning/)
|
|
8
|
+
|
|
9
|
+
> Adaptive resilience configuration for Node.js — automatically tunes circuit breakers, bulkheads, and HTTP clients based on real traffic patterns.
|
|
10
|
+
|
|
11
|
+
Static resilience configuration is a guess. `@backendkit-labs/auto-learning` observes your actual traffic, detects anomalies, and adjusts thresholds continuously — so your circuit breaker opens at the right rate, your bulkhead concurrency matches real load, and your HTTP timeouts reflect actual p95 latency rather than a number someone typed four years ago.
|
|
12
|
+
|
|
13
|
+
Optional NestJS integration included — global interceptor that records patterns automatically, and adapters that push config changes directly to `CircuitBreakerRegistry` and `BulkheadRegistry`.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Table of Contents
|
|
18
|
+
|
|
19
|
+
- [Installation](#installation)
|
|
20
|
+
- [Quick Start](#quick-start)
|
|
21
|
+
- [Core Concepts](#core-concepts)
|
|
22
|
+
- [Pattern Recording](#pattern-recording)
|
|
23
|
+
- [Feedback Loop](#feedback-loop)
|
|
24
|
+
- [Anomaly Detection](#anomaly-detection)
|
|
25
|
+
- [Config Tuning](#config-tuning)
|
|
26
|
+
- [TunableConfig](#tunableconfig)
|
|
27
|
+
- [AutoLearningCore API](#autolearningcore-api)
|
|
28
|
+
- [create()](#create)
|
|
29
|
+
- [recordPattern()](#recordpattern)
|
|
30
|
+
- [runOnce()](#runonce)
|
|
31
|
+
- [startFeedbackLoop() / stopFeedbackLoop()](#startfeedbackloop--stopfeedbackloop)
|
|
32
|
+
- [onConfigChange()](#onconfigchange)
|
|
33
|
+
- [onCycle()](#oncycle)
|
|
34
|
+
- [getCurrentConfig()](#getcurrentconfig)
|
|
35
|
+
- [Configuration Reference](#configuration-reference)
|
|
36
|
+
- [AnomalyDetectorConfig](#anomalydetectorconfig)
|
|
37
|
+
- [ConfigTunerConfig](#configtunerconfig)
|
|
38
|
+
- [FeedbackLoopConfig](#feedbackloopconfig)
|
|
39
|
+
- [Storage Adapters](#storage-adapters)
|
|
40
|
+
- [InMemoryStorage](#inmemorystorage)
|
|
41
|
+
- [FileStorageAdapter](#filestorageadapter)
|
|
42
|
+
- [NestJS Integration](#nestjs-integration)
|
|
43
|
+
- [Module Setup](#module-setup)
|
|
44
|
+
- [@AutoLearn — per-route recording](#autolearn--per-route-recording)
|
|
45
|
+
- [Adapters — automatic config propagation](#adapters--automatic-config-propagation)
|
|
46
|
+
- [Integration with Circuit Breaker and Bulkhead](#integration-with-circuit-breaker-and-bulkhead)
|
|
47
|
+
- [Automatic (NestJS)](#automatic-nestjs)
|
|
48
|
+
- [Manual (framework-agnostic)](#manual-framework-agnostic)
|
|
49
|
+
- [Architecture](#architecture)
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
## Installation
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
npm install @backendkit-labs/auto-learning
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
NestJS peer dependencies (only needed for the `/nestjs` subpath):
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
npm install @nestjs/common @nestjs/core rxjs
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
To connect to `CircuitBreakerRegistry` or `BulkheadRegistry` via adapters:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
npm install @backendkit-labs/circuit-breaker @backendkit-labs/bulkhead
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## TypeScript Configuration
|
|
74
|
+
|
|
75
|
+
### Subpath exports (`/nestjs`)
|
|
76
|
+
|
|
77
|
+
This package uses the `exports` field in `package.json` to expose the `/nestjs` subpath. TypeScript's ability to resolve it depends on the `moduleResolution` setting in your `tsconfig.json`.
|
|
78
|
+
|
|
79
|
+
**Modern resolution (recommended) — no extra config needed:**
|
|
80
|
+
|
|
81
|
+
```json
|
|
82
|
+
{
|
|
83
|
+
"compilerOptions": {
|
|
84
|
+
"moduleResolution": "bundler"
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
`"bundler"`, `"node16"`, and `"nodenext"` all understand the `exports` field natively.
|
|
90
|
+
|
|
91
|
+
**Legacy resolution (`"node"`) — add a `paths` alias:**
|
|
92
|
+
|
|
93
|
+
```json
|
|
94
|
+
{
|
|
95
|
+
"compilerOptions": {
|
|
96
|
+
"moduleResolution": "node",
|
|
97
|
+
"paths": {
|
|
98
|
+
"@backendkit-labs/auto-learning/nestjs": [
|
|
99
|
+
"./node_modules/@backendkit-labs/auto-learning/dist/nestjs/index"
|
|
100
|
+
]
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
}
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
### NestJS decorator support
|
|
107
|
+
|
|
108
|
+
```json
|
|
109
|
+
{
|
|
110
|
+
"compilerOptions": {
|
|
111
|
+
"experimentalDecorators": true,
|
|
112
|
+
"emitDecoratorMetadata": true
|
|
113
|
+
}
|
|
114
|
+
}
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
---
|
|
118
|
+
|
|
119
|
+
## Quick Start
|
|
120
|
+
|
|
121
|
+
### Framework-agnostic
|
|
122
|
+
|
|
123
|
+
```typescript
|
|
124
|
+
import { AutoLearningCore } from '@backendkit-labs/auto-learning';
|
|
125
|
+
|
|
126
|
+
const core = AutoLearningCore.create();
|
|
127
|
+
|
|
128
|
+
// Record a pattern after each request
|
|
129
|
+
core.recordPattern({
|
|
130
|
+
method: 'GET',
|
|
131
|
+
path: '/api/orders',
|
|
132
|
+
statusCode: 200,
|
|
133
|
+
durationMs: 142,
|
|
134
|
+
timestamp: new Date(),
|
|
135
|
+
});
|
|
136
|
+
|
|
137
|
+
// React to config changes
|
|
138
|
+
core.onConfigChange((config) => {
|
|
139
|
+
console.log('New timeout:', config.httpClient.timeoutMs);
|
|
140
|
+
console.log('New CB threshold:', config.circuitBreaker.failureThreshold);
|
|
141
|
+
});
|
|
142
|
+
|
|
143
|
+
// Start the feedback loop — runs a cycle every 60s by default
|
|
144
|
+
core.startFeedbackLoop();
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
### NestJS — zero-config
|
|
148
|
+
|
|
149
|
+
```typescript
|
|
150
|
+
import { AutoLearningModule } from '@backendkit-labs/auto-learning/nestjs';
|
|
151
|
+
|
|
152
|
+
@Module({
|
|
153
|
+
imports: [
|
|
154
|
+
AutoLearningModule.forRoot({ intervalMs: 60_000 }),
|
|
155
|
+
],
|
|
156
|
+
})
|
|
157
|
+
export class AppModule {}
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
Then decorate the routes you want to observe:
|
|
161
|
+
|
|
162
|
+
```typescript
|
|
163
|
+
import { AutoLearn } from '@backendkit-labs/auto-learning/nestjs';
|
|
164
|
+
|
|
165
|
+
@Controller('orders')
|
|
166
|
+
export class OrdersController {
|
|
167
|
+
@Get()
|
|
168
|
+
@AutoLearn()
|
|
169
|
+
findAll() { ... }
|
|
170
|
+
}
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
That's it. Every request to `GET /orders` is recorded automatically. The feedback loop runs in the background and adjusts `TunableConfig` as it learns.
|
|
174
|
+
|
|
175
|
+
---
|
|
176
|
+
|
|
177
|
+
## Core Concepts
|
|
178
|
+
|
|
179
|
+
### Pattern Recording
|
|
180
|
+
|
|
181
|
+
A **pattern** is a single observation of one HTTP request: method, path, status code, duration, and timestamp. Patterns are the raw data from which everything else is derived.
|
|
182
|
+
|
|
183
|
+
```typescript
|
|
184
|
+
core.recordPattern({
|
|
185
|
+
method: 'POST',
|
|
186
|
+
path: '/api/payments',
|
|
187
|
+
statusCode: 500,
|
|
188
|
+
durationMs: 3200,
|
|
189
|
+
timestamp: new Date(),
|
|
190
|
+
correlationId: 'req-abc123', // optional — for tracing
|
|
191
|
+
metadata: { region: 'us-east' }, // optional — custom dimensions
|
|
192
|
+
});
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Patterns are stored in a time-windowed buffer (default: last 5 minutes). Older patterns are pruned automatically.
|
|
196
|
+
|
|
197
|
+
### Feedback Loop
|
|
198
|
+
|
|
199
|
+
The feedback loop is the heart of the system. On each cycle it:
|
|
200
|
+
|
|
201
|
+
1. **Collects** all patterns recorded in the current time window
|
|
202
|
+
2. **Aggregates** them by `method:path` (avg latency, p50/p95/p99, error rate)
|
|
203
|
+
3. **Detects** anomalies against the learned baseline
|
|
204
|
+
4. **Tunes** config based on what the aggregates and anomalies reveal
|
|
205
|
+
5. **Fires** `onConfigChange` listeners if anything changed
|
|
206
|
+
6. **Persists** the new config and a cycle event to storage
|
|
207
|
+
|
|
208
|
+
The loop requires a minimum number of samples before it tunes (default: 10). Below that threshold, it skips tuning and returns a cycle event with empty `configChanges`.
|
|
209
|
+
|
|
210
|
+
### Anomaly Detection
|
|
211
|
+
|
|
212
|
+
The anomaly detector compares the current window against the historical aggregate baseline:
|
|
213
|
+
|
|
214
|
+
| Metric | Anomaly condition | Severity |
|
|
215
|
+
|--------|------------------|----------|
|
|
216
|
+
| Latency | actual > baseline × `latencyStdDevThreshold` | `high` / `critical` |
|
|
217
|
+
| Error rate | actual > `errorRateThreshold` AND actual > baseline × 2 | `high` |
|
|
218
|
+
| Frequency | request count deviates > `frequencyDeviationThreshold` σ | `medium` |
|
|
219
|
+
| Unknown endpoint | path not seen before | `low` |
|
|
220
|
+
|
|
221
|
+
Severity influences how aggressively config is tightened.
|
|
222
|
+
|
|
223
|
+
### Config Tuning
|
|
224
|
+
|
|
225
|
+
The tuner adjusts three sections of `TunableConfig` based on what it observes:
|
|
226
|
+
|
|
227
|
+
**`httpClient.timeoutMs`** — smoothed toward `p95 × 2`, clamped between `minTimeoutMs` and `maxTimeoutMs`:
|
|
228
|
+
```
|
|
229
|
+
newTimeout = current + (target − current) × smoothingFactor
|
|
230
|
+
```
|
|
231
|
+
A smoothing factor of 0.3 means changes are gradual — a single spike doesn't immediately inflate the timeout.
|
|
232
|
+
|
|
233
|
+
**`httpClient.maxRetries`** — increases by 1 when error rate > 10%, decreases by 1 when error rate < 1%. Never drops below 0.
|
|
234
|
+
|
|
235
|
+
**`circuitBreaker.failureThreshold`** — decreases by `10 × criticalAnomalyCount` when anomalies are detected (min 10), increases by 5 per clean cycle (max 80). A circuit breaker that sees 3 critical anomalies in one cycle will tighten from 50 → 20.
|
|
236
|
+
|
|
237
|
+
**`bulkhead.maxConcurrentCalls`** — currently preserved at its configured value; future versions will tune it based on concurrency patterns.
|
|
238
|
+
|
|
239
|
+
### TunableConfig
|
|
240
|
+
|
|
241
|
+
The config emitted on every change:
|
|
242
|
+
|
|
243
|
+
```typescript
|
|
244
|
+
type TunableConfig = {
|
|
245
|
+
circuitBreaker: {
|
|
246
|
+
failureThreshold: number; // 0–100 (% of calls that must fail to open the circuit)
|
|
247
|
+
openTimeoutMs: number; // ms to wait in OPEN before probing
|
|
248
|
+
};
|
|
249
|
+
bulkhead: {
|
|
250
|
+
maxConcurrentCalls: number;
|
|
251
|
+
};
|
|
252
|
+
httpClient: {
|
|
253
|
+
timeoutMs: number;
|
|
254
|
+
maxRetries: number;
|
|
255
|
+
};
|
|
256
|
+
};
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
Defaults:
|
|
260
|
+
|
|
261
|
+
```typescript
|
|
262
|
+
{
|
|
263
|
+
circuitBreaker: { failureThreshold: 50, openTimeoutMs: 30_000 },
|
|
264
|
+
bulkhead: { maxConcurrentCalls: 10 },
|
|
265
|
+
httpClient: { timeoutMs: 10_000, maxRetries: 3 },
|
|
266
|
+
}
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
---
|
|
270
|
+
|
|
271
|
+
## AutoLearningCore API
|
|
272
|
+
|
|
273
|
+
### `create()`
|
|
274
|
+
|
|
275
|
+
Factory method. All internal components are wired automatically.
|
|
276
|
+
|
|
277
|
+
```typescript
|
|
278
|
+
const core = AutoLearningCore.create();
|
|
279
|
+
|
|
280
|
+
// With options
|
|
281
|
+
const core = AutoLearningCore.create({
|
|
282
|
+
storage: new FileStorageAdapter('./config/auto-learning.json'),
|
|
283
|
+
observability: myLogger,
|
|
284
|
+
anomalyConfig: { errorRateThreshold: 0.1 },
|
|
285
|
+
tunerConfig: { smoothingFactor: 0.2 },
|
|
286
|
+
loopConfig: { minSamplesBeforeTuning: 20 },
|
|
287
|
+
});
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
### `recordPattern()`
|
|
291
|
+
|
|
292
|
+
Records a single request observation. Call this after every request you want to track.
|
|
293
|
+
|
|
294
|
+
```typescript
|
|
295
|
+
const result = core.recordPattern({
|
|
296
|
+
method: 'GET',
|
|
297
|
+
path: '/api/users',
|
|
298
|
+
statusCode: 200,
|
|
299
|
+
durationMs: 85,
|
|
300
|
+
timestamp: new Date(),
|
|
301
|
+
});
|
|
302
|
+
|
|
303
|
+
if (!result.ok) {
|
|
304
|
+
console.error('Failed to record pattern:', result.error);
|
|
305
|
+
}
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
### `runOnce()`
|
|
309
|
+
|
|
310
|
+
Executes a single feedback cycle immediately — useful for testing or manual triggering.
|
|
311
|
+
|
|
312
|
+
```typescript
|
|
313
|
+
const result = await core.runOnce();
|
|
314
|
+
|
|
315
|
+
if (result.ok) {
|
|
316
|
+
const { cycleId, patternsProcessed, anomaliesFound, configChanges, durationMs } = result.value;
|
|
317
|
+
console.log(`Cycle ${cycleId}: ${patternsProcessed} patterns, ${anomaliesFound} anomalies`);
|
|
318
|
+
console.log('Config sections changed:', Object.keys(configChanges));
|
|
319
|
+
}
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
### `startFeedbackLoop()` / `stopFeedbackLoop()`
|
|
323
|
+
|
|
324
|
+
Starts or stops the background `setInterval` loop.
|
|
325
|
+
|
|
326
|
+
```typescript
|
|
327
|
+
// Start with default interval (60s)
|
|
328
|
+
core.startFeedbackLoop();
|
|
329
|
+
|
|
330
|
+
// Start with custom interval
|
|
331
|
+
core.startFeedbackLoop(30_000); // every 30s
|
|
332
|
+
|
|
333
|
+
// Stop
|
|
334
|
+
core.stopFeedbackLoop();
|
|
335
|
+
|
|
336
|
+
// Check status
|
|
337
|
+
core.isFeedbackLoopRunning(); // boolean
|
|
338
|
+
```
|
|
339
|
+
|
|
340
|
+
### `onConfigChange()`
|
|
341
|
+
|
|
342
|
+
Registers a callback that fires every time the tuner produces a new config. Multiple listeners are supported.
|
|
343
|
+
|
|
344
|
+
```typescript
|
|
345
|
+
core.onConfigChange((config: TunableConfig) => {
|
|
346
|
+
// Update your HTTP client
|
|
347
|
+
httpClient.setDefaults({ timeout: config.httpClient.timeoutMs });
|
|
348
|
+
|
|
349
|
+
// Update circuit breaker manually
|
|
350
|
+
myCircuitBreaker.updateConfig({
|
|
351
|
+
failureThreshold: config.circuitBreaker.failureThreshold,
|
|
352
|
+
openTimeoutMs: config.circuitBreaker.openTimeoutMs,
|
|
353
|
+
});
|
|
354
|
+
});
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
The callback fires only when at least one section of `TunableConfig` actually changed — identical configs are suppressed.
|
|
358
|
+
|
|
359
|
+
### `onCycle()`
|
|
360
|
+
|
|
361
|
+
Fires after every completed feedback cycle, regardless of whether config changed.
|
|
362
|
+
|
|
363
|
+
```typescript
|
|
364
|
+
core.onCycle((event) => {
|
|
365
|
+
metrics.record('auto_learning.patterns_processed', event.patternsProcessed);
|
|
366
|
+
metrics.record('auto_learning.anomalies_found', event.anomaliesFound);
|
|
367
|
+
metrics.record('auto_learning.cycle_duration_ms', event.durationMs);
|
|
368
|
+
});
|
|
369
|
+
```
|
|
370
|
+
|
|
371
|
+
`LearningCycleEvent` shape:
|
|
372
|
+
|
|
373
|
+
```typescript
|
|
374
|
+
{
|
|
375
|
+
cycleId: string; // UUID for this cycle
|
|
376
|
+
timestamp: Date;
|
|
377
|
+
patternsProcessed: number; // patterns in the time window
|
|
378
|
+
anomaliesFound: number;
|
|
379
|
+
configChanges: Partial<TunableConfig>; // only changed sections
|
|
380
|
+
durationMs: number; // total cycle execution time
|
|
381
|
+
}
|
|
382
|
+
```
|
|
383
|
+
|
|
384
|
+
### `getCurrentConfig()`
|
|
385
|
+
|
|
386
|
+
Returns a deep copy of the current `TunableConfig` without triggering a cycle.
|
|
387
|
+
|
|
388
|
+
```typescript
|
|
389
|
+
const config = core.getCurrentConfig();
|
|
390
|
+
console.log(config.httpClient.timeoutMs); // 10000 (default until first cycle)
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
---
|
|
394
|
+
|
|
395
|
+
## Configuration Reference
|
|
396
|
+
|
|
397
|
+
### `AnomalyDetectorConfig`
|
|
398
|
+
|
|
399
|
+
```typescript
|
|
400
|
+
const core = AutoLearningCore.create({
|
|
401
|
+
anomalyConfig: {
|
|
402
|
+
// Latency deviation multiplier — actual > baseline × this triggers anomaly
|
|
403
|
+
// Default: 2.5
|
|
404
|
+
latencyStdDevThreshold: 2.5,
|
|
405
|
+
|
|
406
|
+
// Error rate above which an anomaly is flagged (0–1)
|
|
407
|
+
// Default: 0.05 (5%)
|
|
408
|
+
errorRateThreshold: 0.05,
|
|
409
|
+
|
|
410
|
+
// Frequency deviation in standard deviations before flagging unusual volume
|
|
411
|
+
// Default: 3.0
|
|
412
|
+
frequencyDeviationThreshold: 3.0,
|
|
413
|
+
|
|
414
|
+
// Flag endpoints that have never been seen before
|
|
415
|
+
// Default: true
|
|
416
|
+
enableUnknownEndpointDetection: true,
|
|
417
|
+
},
|
|
418
|
+
});
|
|
419
|
+
```
|
|
420
|
+
|
|
421
|
+
### `ConfigTunerConfig`
|
|
422
|
+
|
|
423
|
+
```typescript
|
|
424
|
+
const core = AutoLearningCore.create({
|
|
425
|
+
tunerConfig: {
|
|
426
|
+
// Lower bound for httpClient.timeoutMs
|
|
427
|
+
// Default: 1000
|
|
428
|
+
minTimeoutMs: 1000,
|
|
429
|
+
|
|
430
|
+
// Upper bound for httpClient.timeoutMs
|
|
431
|
+
// Default: 30000
|
|
432
|
+
maxTimeoutMs: 30_000,
|
|
433
|
+
|
|
434
|
+
// Controls how fast timeoutMs moves toward the target (0–1)
|
|
435
|
+
// Lower = smoother but slower. Higher = reactive but noisy.
|
|
436
|
+
// Default: 0.3
|
|
437
|
+
smoothingFactor: 0.3,
|
|
438
|
+
|
|
439
|
+
// Step size in ms for timeout adjustments
|
|
440
|
+
// Default: 500
|
|
441
|
+
adjustmentStepMs: 500,
|
|
442
|
+
},
|
|
443
|
+
});
|
|
444
|
+
```
|
|
445
|
+
|
|
446
|
+
### `FeedbackLoopConfig`
|
|
447
|
+
|
|
448
|
+
```typescript
|
|
449
|
+
const core = AutoLearningCore.create({
|
|
450
|
+
loopConfig: {
|
|
451
|
+
// Interval between automatic cycles when started with startFeedbackLoop()
|
|
452
|
+
// Default: 60_000 (1 minute)
|
|
453
|
+
defaultIntervalMs: 60_000,
|
|
454
|
+
|
|
455
|
+
// How far back patterns are collected for each cycle
|
|
456
|
+
// Default: 5 (minutes)
|
|
457
|
+
windowSizeMinutes: 5,
|
|
458
|
+
|
|
459
|
+
// Minimum patterns required in the window before tuning runs
|
|
460
|
+
// Below this count the cycle completes but skips the tuning step
|
|
461
|
+
// Default: 10
|
|
462
|
+
minSamplesBeforeTuning: 10,
|
|
463
|
+
|
|
464
|
+
// Minimum time between two consecutive config changes (ms)
|
|
465
|
+
// Prevents thrashing when anomalies appear in consecutive cycles
|
|
466
|
+
// Default: 120_000 (2 minutes)
|
|
467
|
+
cooldownBetweenChangesMs: 120_000,
|
|
468
|
+
},
|
|
469
|
+
});
|
|
470
|
+
```
|
|
471
|
+
|
|
472
|
+
---
|
|
473
|
+
|
|
474
|
+
## Storage Adapters
|
|
475
|
+
|
|
476
|
+
### `InMemoryStorage`
|
|
477
|
+
|
|
478
|
+
The default. Patterns, anomalies, and cycle events live in process memory. Config is also in-memory and resets to defaults on restart.
|
|
479
|
+
|
|
480
|
+
```typescript
|
|
481
|
+
import { InMemoryStorage } from '@backendkit-labs/auto-learning';
|
|
482
|
+
|
|
483
|
+
const core = AutoLearningCore.create({
|
|
484
|
+
storage: new InMemoryStorage(), // this is the default
|
|
485
|
+
});
|
|
486
|
+
```
|
|
487
|
+
|
|
488
|
+
Use this in development, tests, or when you don't need config to survive restarts.
|
|
489
|
+
|
|
490
|
+
### `FileStorageAdapter`
|
|
491
|
+
|
|
492
|
+
Extends `InMemoryStorage` with config persistence to a JSON file. Patterns, anomalies, and cycle events remain in-memory (re-learned on restart). Only the tuned `TunableConfig` survives across restarts.
|
|
493
|
+
|
|
494
|
+
```typescript
|
|
495
|
+
import { FileStorageAdapter } from '@backendkit-labs/auto-learning';
|
|
496
|
+
|
|
497
|
+
const core = AutoLearningCore.create({
|
|
498
|
+
storage: new FileStorageAdapter('./config/auto-learning.json'),
|
|
499
|
+
});
|
|
500
|
+
```
|
|
501
|
+
|
|
502
|
+
The directory is created automatically if it doesn't exist. The file is written synchronously on every config change to prevent partial writes.
|
|
503
|
+
|
|
504
|
+
Use this in production when you want to preserve learned thresholds across deploys or restarts without an external database.
|
|
505
|
+
|
|
506
|
+
**Custom `StorageAdapter`:** implement the `StorageAdapter` interface to plug in Redis, PostgreSQL, or any other backend.
|
|
507
|
+
|
|
508
|
+
---
|
|
509
|
+
|
|
510
|
+
## NestJS Integration
|
|
511
|
+
|
|
512
|
+
Import from the `/nestjs` subpath — framework code is tree-shaken from the core bundle.
|
|
513
|
+
|
|
514
|
+
### Module Setup
|
|
515
|
+
|
|
516
|
+
```typescript
|
|
517
|
+
import { AutoLearningModule } from '@backendkit-labs/auto-learning/nestjs';
|
|
518
|
+
|
|
519
|
+
@Module({
|
|
520
|
+
imports: [
|
|
521
|
+
AutoLearningModule.forRoot({
|
|
522
|
+
// Feedback loop interval
|
|
523
|
+
// Default: 60_000
|
|
524
|
+
intervalMs: 60_000,
|
|
525
|
+
|
|
526
|
+
// Observability — pass NestJS Logger or any LoggerService
|
|
527
|
+
observability: {
|
|
528
|
+
logger: new Logger('AutoLearning'),
|
|
529
|
+
metrics: {
|
|
530
|
+
increment: (name, val, tags) => statsd.increment(name, val, tags),
|
|
531
|
+
gauge: (name, val, tags) => statsd.gauge(name, val, tags),
|
|
532
|
+
histogram: (name, val, tags) => statsd.histogram(name, val, tags),
|
|
533
|
+
},
|
|
534
|
+
},
|
|
535
|
+
|
|
536
|
+
// Fine-tune the core components
|
|
537
|
+
coreOptions: {
|
|
538
|
+
storage: new FileStorageAdapter('./config/auto-learning.json'),
|
|
539
|
+
anomalyConfig: { errorRateThreshold: 0.1 },
|
|
540
|
+
tunerConfig: { smoothingFactor: 0.2 },
|
|
541
|
+
loopConfig: { minSamplesBeforeTuning: 20 },
|
|
542
|
+
},
|
|
543
|
+
}),
|
|
544
|
+
],
|
|
545
|
+
})
|
|
546
|
+
export class AppModule {}
|
|
547
|
+
```
|
|
548
|
+
|
|
549
|
+
`AutoLearningModule.forRoot()` is **global** — no need to re-import it in feature modules.
|
|
550
|
+
|
|
551
|
+
It registers:
|
|
552
|
+
- `AUTO_LEARNING_INSTANCE` — the `AutoLearningCore` instance (injectable by token)
|
|
553
|
+
- `AutoLearningInterceptor` — global APP_INTERCEPTOR that records patterns automatically
|
|
554
|
+
- `AutoLearningAdaptersService` — wires CB/BH registries when `adapters` is configured
|
|
555
|
+
|
|
556
|
+
### `@AutoLearn` — per-route recording
|
|
557
|
+
|
|
558
|
+
Add `@AutoLearn()` to any controller method to start recording its traffic. The global interceptor handles the rest — no manual `recordPattern()` calls needed.
|
|
559
|
+
|
|
560
|
+
```typescript
|
|
561
|
+
import { AutoLearn } from '@backendkit-labs/auto-learning/nestjs';
|
|
562
|
+
|
|
563
|
+
@Controller('payments')
|
|
564
|
+
export class PaymentsController {
|
|
565
|
+
// Basic — records method, path, status code, and duration
|
|
566
|
+
@Post()
|
|
567
|
+
@AutoLearn()
|
|
568
|
+
charge(@Body() dto: ChargeDto) { ... }
|
|
569
|
+
|
|
570
|
+
// With custom metadata attached to each pattern
|
|
571
|
+
@Get(':id')
|
|
572
|
+
@AutoLearn({
|
|
573
|
+
customMetadata: (req) => ({
|
|
574
|
+
region: req.headers['x-region'],
|
|
575
|
+
clientId: req.headers['x-client-id'],
|
|
576
|
+
}),
|
|
577
|
+
})
|
|
578
|
+
getCharge(@Param('id') id: string) { ... }
|
|
579
|
+
}
|
|
580
|
+
```
|
|
581
|
+
|
|
582
|
+
`@AutoLearn` options:
|
|
583
|
+
|
|
584
|
+
| Option | Type | Default | Description |
|
|
585
|
+
|--------|------|---------|-------------|
|
|
586
|
+
| `customMetadata` | `(req) => Record<string, unknown>` | `undefined` | Attach arbitrary data to each recorded pattern |
|
|
587
|
+
|
|
588
|
+
Routes without `@AutoLearn()` are silently ignored — the interceptor is a no-op for undecorated handlers.
|
|
589
|
+
|
|
590
|
+
### Inject `AutoLearningCore` directly
|
|
591
|
+
|
|
592
|
+
```typescript
|
|
593
|
+
import { Inject } from '@nestjs/common';
|
|
594
|
+
import { AUTO_LEARNING_INSTANCE } from '@backendkit-labs/auto-learning/nestjs';
|
|
595
|
+
import type { AutoLearningCore } from '@backendkit-labs/auto-learning';
|
|
596
|
+
|
|
597
|
+
@Injectable()
|
|
598
|
+
export class StatsService {
|
|
599
|
+
constructor(
|
|
600
|
+
@Inject(AUTO_LEARNING_INSTANCE)
|
|
601
|
+
private readonly learning: AutoLearningCore,
|
|
602
|
+
) {}
|
|
603
|
+
|
|
604
|
+
getConfig() {
|
|
605
|
+
return this.learning.getCurrentConfig();
|
|
606
|
+
}
|
|
607
|
+
|
|
608
|
+
async triggerCycle() {
|
|
609
|
+
return this.learning.runOnce();
|
|
610
|
+
}
|
|
611
|
+
}
|
|
612
|
+
```
|
|
613
|
+
|
|
614
|
+
### Adapters — automatic config propagation
|
|
615
|
+
|
|
616
|
+
The `adapters` option connects auto-learning directly to `CircuitBreakerRegistry` and `BulkheadRegistry`. When the tuner produces a new config, every registered instance is updated automatically — no `onConfigChange` wiring needed.
|
|
617
|
+
|
|
618
|
+
```typescript
|
|
619
|
+
import { AutoLearningModule } from '@backendkit-labs/auto-learning/nestjs';
|
|
620
|
+
import { CircuitBreakerModule } from '@backendkit-labs/circuit-breaker/nestjs';
|
|
621
|
+
import { BulkheadModule } from '@backendkit-labs/bulkhead/nestjs';
|
|
622
|
+
|
|
623
|
+
@Module({
|
|
624
|
+
imports: [
|
|
625
|
+
CircuitBreakerModule,
|
|
626
|
+
BulkheadModule,
|
|
627
|
+
AutoLearningModule.forRoot({
|
|
628
|
+
adapters: {
|
|
629
|
+
circuitBreaker: true, // auto-updates all CircuitBreaker instances
|
|
630
|
+
bulkhead: true, // auto-updates all Bulkhead instances
|
|
631
|
+
},
|
|
632
|
+
}),
|
|
633
|
+
],
|
|
634
|
+
})
|
|
635
|
+
export class AppModule {}
|
|
636
|
+
```
|
|
637
|
+
|
|
638
|
+
**How it works:** on module init, `AutoLearningAdaptersService` resolves `CircuitBreakerRegistry` and `BulkheadRegistry` from the NestJS DI container. On every `onConfigChange` event, it calls `updateConfig()` on all registered instances.
|
|
639
|
+
|
|
640
|
+
If `CircuitBreakerModule` or `BulkheadModule` is not imported, the adapter logs a warning and skips gracefully — it does not throw.
|
|
641
|
+
|
|
642
|
+
---
|
|
643
|
+
|
|
644
|
+
## Integration with Circuit Breaker and Bulkhead
|
|
645
|
+
|
|
646
|
+
### Automatic (NestJS)
|
|
647
|
+
|
|
648
|
+
See [Adapters](#adapters--automatic-config-propagation) above — one flag, no wiring.
|
|
649
|
+
|
|
650
|
+
### Manual (framework-agnostic)
|
|
651
|
+
|
|
652
|
+
Wire `onConfigChange` to call `updateConfig()` on your instances directly:
|
|
653
|
+
|
|
654
|
+
```typescript
|
|
655
|
+
import { AutoLearningCore } from '@backendkit-labs/auto-learning';
|
|
656
|
+
import { CircuitBreakerRegistry } from '@backendkit-labs/circuit-breaker';
|
|
657
|
+
import { BulkheadRegistry } from '@backendkit-labs/bulkhead';
|
|
658
|
+
|
|
659
|
+
const core = AutoLearningCore.create();
|
|
660
|
+
const cbRegistry = new CircuitBreakerRegistry();
|
|
661
|
+
const bhRegistry = new BulkheadRegistry();
|
|
662
|
+
|
|
663
|
+
// Create your instances
|
|
664
|
+
const paymentsCB = cbRegistry.getOrCreate({ name: 'payments' });
|
|
665
|
+
const paymentsBH = bhRegistry.getOrCreate({ name: 'payments' });
|
|
666
|
+
|
|
667
|
+
// Wire config propagation
|
|
668
|
+
core.onConfigChange((config) => {
|
|
669
|
+
// Update every registered circuit breaker
|
|
670
|
+
for (const name of Object.keys(cbRegistry.getAllMetrics())) {
|
|
671
|
+
cbRegistry.getOrCreate({ name }).updateConfig({
|
|
672
|
+
failureThreshold: config.circuitBreaker.failureThreshold,
|
|
673
|
+
openTimeoutMs: config.circuitBreaker.openTimeoutMs,
|
|
674
|
+
});
|
|
675
|
+
}
|
|
676
|
+
|
|
677
|
+
// Update every registered bulkhead
|
|
678
|
+
for (const name of Object.keys(bhRegistry.getAllMetrics())) {
|
|
679
|
+
bhRegistry.getOrCreate({ name }).updateConfig({
|
|
680
|
+
maxConcurrentCalls: config.bulkhead.maxConcurrentCalls,
|
|
681
|
+
});
|
|
682
|
+
}
|
|
683
|
+
});
|
|
684
|
+
|
|
685
|
+
// Start learning
|
|
686
|
+
core.startFeedbackLoop();
|
|
687
|
+
|
|
688
|
+
// Record traffic
|
|
689
|
+
core.recordPattern({
|
|
690
|
+
method: 'POST', path: '/payments', statusCode: 200, durationMs: 120, timestamp: new Date(),
|
|
691
|
+
});
|
|
692
|
+
```
|
|
693
|
+
|
|
694
|
+
**What happens when an anomaly is detected:**
|
|
695
|
+
|
|
696
|
+
```
|
|
697
|
+
12 ok + 3 errors (20% error rate) in one window
|
|
698
|
+
→ AnomalyDetector: 3 HIGH anomalies
|
|
699
|
+
→ ConfigTuner: failureThreshold = max(50 − 10×3, 10) = 20
|
|
700
|
+
→ onConfigChange fires
|
|
701
|
+
→ CircuitBreaker.updateConfig({ failureThreshold: 20 }) ← tighter, reacts sooner
|
|
702
|
+
→ 2 clean cycles later: failureThreshold recovers toward 30, 35, ...
|
|
703
|
+
```
|
|
704
|
+
|
|
705
|
+
---
|
|
706
|
+
|
|
707
|
+
## Architecture
|
|
708
|
+
|
|
709
|
+
```
|
|
710
|
+
@backendkit-labs/auto-learning (core — zero framework dependencies)
|
|
711
|
+
AutoLearningCore facade — wires all components together
|
|
712
|
+
PatternRegistry time-windowed pattern buffer + aggregation
|
|
713
|
+
AnomalyDetector statistical analysis against baselines
|
|
714
|
+
ConfigTuner smoothed config adjustment + persistence
|
|
715
|
+
FeedbackLoop setInterval orchestrator
|
|
716
|
+
InMemoryStorage default in-process storage
|
|
717
|
+
FileStorageAdapter config persistence across restarts
|
|
718
|
+
|
|
719
|
+
@backendkit-labs/auto-learning/nestjs (optional NestJS layer)
|
|
720
|
+
AutoLearningModule DynamicModule — registers all providers
|
|
721
|
+
AutoLearningInterceptor global APP_INTERCEPTOR — auto-records @AutoLearn routes
|
|
722
|
+
AutoLearningAdaptersService wires CB/BH registries on config change
|
|
723
|
+
@AutoLearn route decorator — opts a handler into recording
|
|
724
|
+
```
|
|
725
|
+
|
|
726
|
+
**Dependency direction:**
|
|
727
|
+
|
|
728
|
+
```
|
|
729
|
+
auto-learning ──→ circuit-breaker (optional peer — adapters only)
|
|
730
|
+
auto-learning ──→ bulkhead (optional peer — adapters only)
|
|
731
|
+
auto-learning ──→ observability (optional peer — NestJS adapter)
|
|
732
|
+
auto-learning ──→ result (core utility)
|
|
733
|
+
```
|
|
734
|
+
|
|
735
|
+
`circuit-breaker` and `bulkhead` do **not** depend on `auto-learning` — the integration is one-directional. This avoids circular dependencies and keeps resilience primitives standalone.
|
|
736
|
+
|
|
737
|
+
---
|
|
738
|
+
|
|
739
|
+
## License
|
|
740
|
+
|
|
741
|
+
Apache-2.0 — [BackendKit Labs](https://github.com/BackendKit-labs)
|