immortal-js 1.0.0

package/docs/api/monitoring.md

@@ -0,0 +1,313 @@
+# API Reference — Monitoring (Layer 5: Observability)
+
+The **Monitoring** layer collects system metrics, detects anomalies, monitors memory growth, and exposes health status. It feeds data to the Supervision layer so restarts can be triggered proactively.
+
+---
+
+## MetricsCollector
+
+Collects event loop lag, CPU, memory, GC frequency, and per-service circuit/bulkhead statistics.
+
+### `new MetricsCollector()`
+
+```typescript
+import { MetricsCollector } from '@immortal/core';
+
+const collector = new MetricsCollector();
+```
+
+The collector starts tracking automatically on construction (GC observers, CPU sampling).
+
+### `collector.getSnapshot()`
+
+Returns the latest metrics snapshot:
+
+```typescript
+const snapshot = collector.getSnapshot();
+```
+
+**Returns:** `SystemMetrics`
+
+```typescript
+interface SystemMetrics {
+  timestamp: number;
+  eventLoopLag: {
+    p50: number;   // ms
+    p95: number;
+    p99: number;
+    max: number;
+  };
+  memory: {
+    heapUsedMb: number;
+    heapTotalMb: number;
+    externalMb: number;
+    rssMb: number;
+    heapGrowthRate: number;  // MB/sample
+    gcFrequency: number;     // GC events/sec
+  };
+  cpu: {
+    percentUser: number;
+    percentSystem: number;
+  };
+  handles: {
+    active: number;
+  };
+  requests: {
+    total: number;
+    active: number;
+    perSecond: number;
+    errorRate: number;
+  };
+  circuits: Record<string, CircuitBreakerStatus>;
+  bulkheads: Record<string, BulkheadStatus>;
+  workers: WorkerStatus[];
+}
+```
+
+### `collector.recordRequestStart()` / `collector.recordRequestEnd(success)`
+
+Track in-flight requests for RPS and error rate calculation.
+
+```typescript
+collector.recordRequestStart();
+try {
+  await handler(req, res);
+  collector.recordRequestEnd(true);
+} catch (err) {
+  collector.recordRequestEnd(false);
+}
+```
+
+### `collector.getHeapGrowthRate()`
+
+Returns heap growth rate in MB/sample.
+
+### `collector.getGcFrequency()`
+
+Returns GC events per second.
+
+---
+
+## HealthMonitor
+
+Periodically evaluates system health and emits warning/critical events on the event bus.
+
+### `new HealthMonitor(config, bus, metricsCollector)`
+
+```typescript
+import { HealthMonitor } from '@immortal/core';
+
+const monitor = new HealthMonitor(
+  {
+    checkIntervalMs: 30_000,
+    eventLoopLagWarningMs: 50,
+    eventLoopLagCriticalMs: 100,
+    heapUsageWarningPercent: 80,
+    heapUsageCriticalPercent: 90,
+    cpuWarningPercent: 80,
+    cpuCriticalPercent: 95,
+  },
+  getEventBus(),
+  metricsCollector
+);
+```
+
+**Config:**
+
+```typescript
+interface HealthMonitorConfig {
+  enabled?: boolean;
+  checkIntervalMs?: number;         // default: 30_000
+  eventLoopLagWarningMs?: number;   // default: 50
+  eventLoopLagCriticalMs?: number;  // default: 100
+  heapUsageWarningPercent?: number; // default: 80
+  heapUsageCriticalPercent?: number;// default: 90
+  cpuWarningPercent?: number;       // default: 80
+  cpuCriticalPercent?: number;      // default: 95
+}
+```
+
+### `monitor.start()` / `monitor.stop()`
+
+```typescript
+monitor.start();
+// ...
+monitor.stop();
+```
+
+**Events emitted:**
+
+| Event | Trigger |
+|-------|---------|
+| `eventloop:lag-warning` | Event loop lag > `eventLoopLagWarningMs` |
+| `eventloop:lag-critical` | Event loop lag > `eventLoopLagCriticalMs` |
+| `cpu:warning` | CPU usage > `cpuWarningPercent` |
+| `cpu:critical` | CPU usage > `cpuCriticalPercent` |
+| `memory:warning` | Heap usage > `heapUsageWarningPercent` |
+| `memory:critical` | Heap usage > `heapUsageCriticalPercent` |
+
+---
+
+## MemoryLeakGuard
+
+Detects memory leaks by tracking heap growth over a sliding window. Can trigger proactive restarts before OOM.
+
+### `new MemoryLeakGuard(config, bus)`
+
+```typescript
+import { MemoryLeakGuard } from '@immortal/core';
+
+const guard = new MemoryLeakGuard(
+  {
+    checkIntervalMs: 30_000,
+    warningThresholdMb: 400,
+    restartThresholdMb: 600,
+    growthWindowCount: 5,
+    adaptive: true,
+  },
+  getEventBus()
+);
+```
+
+**Config:**
+
+```typescript
+interface MemoryGuardConfig {
+  enabled?: boolean;
+  checkIntervalMs?: number;       // default: 60_000
+  warningThresholdMb?: number;    // default: 400
+  restartThresholdMb?: number;    // default: 600
+  growthWindowCount?: number;     // default: 5 — samples for growth detection
+  adaptive?: boolean;             // default: true — adjust thresholds based on baseline
+}
+```
+
+### `guard.start()` / `guard.stop()`
+
+```typescript
+guard.start();
+guard.stop();
+```
+
+### `guard.forceSample()`
+
+Force an immediate memory check (used in tests and admin endpoints).
+
+```typescript
+guard.forceSample();
+```
+
+**Events emitted:**
+
+| Event | Trigger |
+|-------|---------|
+| `memory:warning` | Heap above warning threshold or growth detected |
+| `memory:critical` | Heap above restart threshold |
+| `memory:leak-detected` | Sustained heap growth over `growthWindowCount` samples |
+| `memory:proactive-restart` | Guard is requesting a process restart |
+
+### Adaptive mode
+
+With `adaptive: true`, the guard establishes a baseline during the first few samples and adjusts thresholds relative to the baseline. This prevents false positives in processes with legitimately large heaps (e.g. image processing).
+
+---
+
+## DiagnosticsChannel
+
+OpenTelemetry-compatible tracing and metrics export via OTLP.
+
+### `new DiagnosticsChannel(options?)`
+
+```typescript
+import { DiagnosticsChannel } from '@immortal/core';
+
+const dc = new DiagnosticsChannel({
+  otlpEndpoint: 'http://otel-collector:4318',
+  metricsIntervalMs: 60_000,
+});
+```
+
+**Options:**
+
+```typescript
+interface DiagnosticsChannelOptions {
+  otlpEndpoint?: string;
+  metricsIntervalMs?: number;  // default: 60_000
+}
+```
+
+### `dc.recordRequest(info)`
+
+Record an HTTP request for metrics aggregation.
+
+```typescript
+dc.recordRequest({
+  route: '/api/orders',
+  method: 'POST',
+  statusCode: 201,
+  durationMs: 145,
+});
+```
+
+### `dc.startSpan(name, attributes?)` / `dc.endSpan(spanId)`
+
+Create and close trace spans.
+
+```typescript
+const spanId = dc.startSpan('db.query', { 'db.statement': sql });
+try {
+  await db.query(sql);
+} finally {
+  dc.endSpan(spanId);
+}
+```
+
+### `dc.destroy()`
+
+Stop the metrics export interval.
+
+---
+
+## Event Bus
+
+The monitoring layer communicates through the Immortal event bus. Subscribe to any event from any layer.
+
+```typescript
+import { getEventBus } from '@immortal/core';
+
+const bus = getEventBus();
+
+bus.on('memory:critical', (event) => {
+  console.error('Memory critical!', event.data);
+});
+
+bus.on('circuit:opened', (event) => {
+  console.warn(`Circuit ${event.data?.name} opened`);
+});
+
+// Get history
+const recentEvents = bus.getHistory({
+  type: 'worker:crashed',
+  since: Date.now() - 60_000,
+  limit: 10,
+});
+
+// Event counts
+const counts = bus.getEventCounts(300_000); // last 5 minutes
+```
+
+**All event types:**
+
+```
+error:captured     error:escalated
+retry:attempt      retry:exhausted      retry:success
+circuit:opened     circuit:closed       circuit:half-open   circuit:rejected
+bulkhead:queued    bulkhead:rejected    bulkhead:timeout
+worker:started     worker:restarted     worker:escalated    worker:crashed
+memory:warning     memory:critical      memory:leak-detected memory:proactive-restart
+eventloop:lag-warning  eventloop:lag-critical
+cpu:warning        cpu:critical
+shutdown:initiated shutdown:complete    shutdown:forced
+fallback:cache-hit fallback:default-used
+chaos:latency-injected chaos:error-injected chaos:worker-killed
+```

package/docs/api/plugins.md

@@ -0,0 +1,217 @@
+# API Reference — Plugin System
+
+Immortal.js has a first-class plugin system that lets you extend the runtime with cross-cutting concerns: logging pipelines, distributed tracing, alerting, anomaly detection, and more.
+
+---
+
+## Plugin Interface
+
+```typescript
+interface ImmortalPlugin {
+  /** Unique plugin name (for logging and deduplication) */
+  name: string;
+
+  /**
+   * Called during ImmortalRuntime.initialize().
+   * Use to connect to external services, register listeners, etc.
+   */
+  onInit?(ctx: PluginContext): Promise<void> | void;
+
+  /**
+   * Called during ImmortalRuntime.shutdown().
+   * Flush buffers, close connections, etc.
+   */
+  onShutdown?(signal: string): Promise<void> | void;
+}
+
+interface PluginContext {
+  config: Required<ImmortalConfig>;
+  emit: (event: Partial<ImmortalEvent>) => void;
+  getMetrics: () => SystemMetrics;
+  logger: ImmortalLogger;
+}
+```
+
+---
+
+## Built-in Plugins
+
+### `ConsoleLogPlugin`
+
+Structured console logging. Prints JSON lines to stdout.
+
+```typescript
+import { ConsoleLogPlugin } from '@immortal/core';
+
+const immortal = await createImmortal({
+  plugins: [ConsoleLogPlugin],
+});
+```
+
+Subscribes to all Immortal events and logs them with contextual metadata.
+
+---
+
+### `RequestTracingPlugin`
+
+Propagates request IDs through async context using `AsyncLocalStorage`.
+
+```typescript
+import { RequestTracingPlugin } from '@immortal/core';
+```
+
+After initialization, every async operation can retrieve the current request ID:
+
+```typescript
+import { AsyncBoundary } from '@immortal/core';
+
+const requestId = AsyncBoundary.getRequestId();
+// → 'req-abc-123' | undefined
+```
+
+Pairs well with `asyncBoundary()`:
+
+```typescript
+const handler = asyncBoundary(async (req) => {
+  const reqId = AsyncBoundary.getRequestId();
+  logger.info('Handling request', { reqId });
+  return processRequest(req);
+});
+```
+
+---
+
+### `createAnomalyDetectionPlugin(options)`
+
+Monitors metrics for statistical anomalies and emits `error:escalated` events.
+
+```typescript
+import { createAnomalyDetectionPlugin } from '@immortal/core';
+
+const plugin = createAnomalyDetectionPlugin({
+  threshold: 0.1,           // anomaly score threshold (0–1)
+  windowMs: 300_000,        // 5-minute rolling window
+  metrics: ['errorRate', 'eventLoopLag', 'heapGrowthRate'],
+});
+```
+
+**Options:**
+
+```typescript
+interface AnomalyDetectionOptions {
+  threshold?: number;       // default: 0.2
+  windowMs?: number;        // default: 300_000
+  metrics?: string[];       // which metrics to monitor
+  onAnomaly?: (info: AnomalyInfo) => void;
+}
+```
+
+---
+
+### `createSlackAlertPlugin(options)`
+
+Sends Slack webhook notifications for critical Immortal events.
+
+```typescript
+import { createSlackAlertPlugin } from '@immortal/core';
+
+const plugin = createSlackAlertPlugin({
+  webhookUrl: process.env.SLACK_WEBHOOK!,
+  events: ['worker:escalated', 'circuit:opened', 'memory:critical'],
+  throttleMs: 60_000,   // max 1 alert per event type per minute
+});
+```
+
+**Options:**
+
+```typescript
+interface SlackAlertOptions {
+  webhookUrl: string;
+  events?: ImmortalEventType[];   // default: all critical events
+  throttleMs?: number;            // default: 60_000
+  channel?: string;               // override default channel
+  username?: string;              // default: 'Immortal.js'
+}
+```
+
+---
+
+### `BUILTIN_PLUGINS`
+
+All built-in plugins as an array, for convenience:
+
+```typescript
+import { BUILTIN_PLUGINS } from '@immortal/core';
+
+const immortal = await createImmortal({
+  plugins: [...BUILTIN_PLUGINS],
+});
+```
+
+---
+
+## Writing a Custom Plugin
+
+```typescript
+import type { ImmortalPlugin } from '@immortal/core';
+import { getEventBus } from '@immortal/core';
+
+export const DatadogPlugin: ImmortalPlugin = {
+  name: 'datadog',
+
+  async onInit(ctx) {
+    const bus = getEventBus();
+
+    // Subscribe to all events
+    bus.onAny((event) => {
+      dogstatsd.increment(`immortal.event.${event.type}`, 1, {
+        env: ctx.config.logger.prefix ?? 'default',
+      });
+    });
+
+    // Push metrics every 30s
+    const interval = setInterval(() => {
+      const metrics = ctx.getMetrics();
+      dogstatsd.gauge('immortal.heap_mb', metrics.memory.heapUsedMb);
+      dogstatsd.gauge('immortal.eventloop_lag_p99', metrics.eventLoopLag.p99);
+    }, 30_000).unref();
+
+    ctx.logger.info('Datadog plugin initialized');
+  },
+
+  async onShutdown() {
+    await dogstatsd.close();
+  },
+};
+```
+
+---
+
+## Plugin registration
+
+Plugins are registered in `createImmortal()`:
+
+```typescript
+const immortal = await createImmortal({
+  plugins: [
+    ConsoleLogPlugin,
+    RequestTracingPlugin,
+    createAnomalyDetectionPlugin({ threshold: 0.15 }),
+    createSlackAlertPlugin({ webhookUrl: process.env.SLACK_WEBHOOK! }),
+    DatadogPlugin,
+  ],
+});
+```
+
+Or added dynamically after initialization:
+
+```typescript
+immortal.addPlugin(DatadogPlugin);
+// → onInit is called immediately
+```
+
+---
+
+## Plugin execution order
+
+Plugins are initialized in **array order** during `initialize()`, and shut down in **reverse order** during `shutdown()`. Place lower-level plugins (logging, tracing) before higher-level ones (alerting, metrics export).