@ai-orchestration/core 0.1.0 → 0.3.0

package/README.md

@@ -21,6 +21,22 @@ A modular and extensible framework for orchestrating multiple AI/LLM providers c
 npm install @ai-orchestration/core
 ```
 
+### Module System Compatibility
+
+This package supports both **ESM** (ECMAScript Modules) and **CommonJS**, so you can use it in any Node.js project:
+
+**ESM Projects** (recommended):
+```typescript
+import { createOrchestrator } from '@ai-orchestration/core';
+```
+
+**CommonJS Projects**:
+```javascript
+const { createOrchestrator } = require('@ai-orchestration/core');
+```
+
+The package automatically exports the correct format based on your project's module system.
+
 ## Quick Start
 
 ### Basic Usage
@@ -267,20 +283,58 @@ For local models that expose an OpenAI-compatible API:
 }
 ```
 
-## Health Checks
+## Advanced Configuration
+
+### Retry and Timeout Configuration
+
+```typescript
+const orchestrator = createOrchestrator({
+  providers: [...],
+  strategy: {...},
+  maxRetries: 3, // Maximum retry attempts (default: number of providers)
+  requestTimeout: 30000, // Global timeout in milliseconds (default: 30000)
+  retryDelay: 'exponential', // or number in milliseconds (default: 1000)
+});
+```
+
+### Circuit Breaker
 
-Enable periodic health monitoring:
+Automatically disable providers after consecutive failures:
 
 ```typescript
 const orchestrator = createOrchestrator({
   providers: [...],
   strategy: {...},
-  enableHealthChecks: true,
-  healthCheckInterval: 60000, // every 60 seconds
+  circuitBreaker: {
+    enabled: true,
+    failureThreshold: 5, // Open circuit after 5 failures
+    resetTimeout: 60000, // Reset after 60 seconds
+  },
 });
 ```
 
-Or manually:
+### Health Checks
+
+Enhanced health check configuration:
+
+```typescript
+const orchestrator = createOrchestrator({
+  providers: [...],
+  strategy: {...},
+  healthCheck: {
+    enabled: true,
+    interval: 60000, // Check every 60 seconds
+    timeout: 5000, // Health check timeout (default: 5000ms)
+    maxConsecutiveFailures: 3, // Mark unhealthy after 3 failures (default: 3)
+    latencyThreshold: 10000, // Max latency in ms (default: 10000ms)
+  },
+  // Legacy format still supported:
+  // enableHealthChecks: true,
+  // healthCheckInterval: 60000,
+});
+```
+
+Or manually check health:
 
 ```typescript
 const health = await provider.checkHealth();
@@ -296,9 +350,95 @@ const response = await orchestrator.chat(messages, {
   topP: 0.9,
   topK: 40,
   stopSequences: ['\n\n'],
+  responseLanguage: 'es', // Force response in Spanish
+  frequencyPenalty: 0.5, // Reduce repetition
+  presencePenalty: 0.3, // Encourage new topics
+  seed: 42, // For reproducible outputs
+  timeout: 30000, // Request timeout in milliseconds
+  user: 'user-123', // User identifier for tracking
 });
 ```
 
+### Available Chat Options
+
+- **`temperature`**: Controls randomness (0.0 to 2.0)
+- **`maxTokens`**: Maximum tokens in response
+- **`topP`**: Nucleus sampling threshold
+- **`topK`**: Top-K sampling
+- **`stopSequences`**: Stop generation on these sequences
+- **`responseLanguage`**: Force response language (see below)
+- **`frequencyPenalty`**: Penalize frequent tokens (-2.0 to 2.0)
+- **`presencePenalty`**: Penalize existing tokens (-2.0 to 2.0)
+- **`seed`**: Seed for reproducible outputs
+- **`timeout`**: Request timeout in milliseconds (overrides global timeout)
+- **`user`**: User identifier for tracking/rate limiting
+
+### Forcing Response Language
+
+You can force the AI to respond in a specific language using the `responseLanguage` option:
+
+```typescript
+// Using ISO 639-1 language codes
+const response = await orchestrator.chat(messages, {
+  responseLanguage: 'es', // Spanish
+  // or 'en', 'fr', 'de', 'it', 'pt', 'ja', 'zh', 'ru', etc.
+});
+
+// Using full language names
+const response2 = await orchestrator.chat(messages, {
+  responseLanguage: 'spanish', // Also works
+  // or 'english', 'french', 'german', 'italian', etc.
+});
+```
+
+**How it works**: When `responseLanguage` is specified, the framework automatically prepends a system message instructing the model to respond in the specified language. If you already have a system message, the language instruction will be prepended to it.
+
+**Supported languages**: Spanish, English, French, German, Italian, Portuguese, Japanese, Chinese, Russian, Korean, Arabic, Hindi, Dutch, Polish, Swedish, Turkish (and more via ISO 639-1 codes).
+
+## Metrics and Analytics
+
+Track provider usage, costs, and strategy effectiveness:
+
+```typescript
+const orchestrator = createOrchestrator({
+  providers: [...],
+  strategy: {...},
+  enableMetrics: true, // Enabled by default
+  onMetricsEvent: (event) => {
+    // Optional: Real-time event tracking
+    console.log('Event:', event.type, event.providerId);
+  },
+});
+
+// Make some requests...
+
+// Get overall metrics
+const metrics = orchestrator.getMetrics().getOrchestratorMetrics();
+console.log('Total Requests:', metrics.totalRequests);
+console.log('Total Cost:', metrics.totalCost);
+console.log('Error Rate:', metrics.errorRate);
+
+// Get provider-specific metrics
+const providerMetrics = orchestrator.getMetrics().getProviderMetrics('groq-1');
+console.log('Provider Requests:', providerMetrics?.totalRequests);
+console.log('Provider Cost:', providerMetrics?.totalCost);
+console.log('Success Rate:', providerMetrics?.successfulRequests / providerMetrics?.totalRequests);
+
+// Get strategy metrics
+const strategyMetrics = orchestrator.getMetrics().getStrategyMetrics();
+console.log('Selections by Provider:', strategyMetrics.selectionsByProvider);
+console.log('Average Selection Time:', strategyMetrics.averageSelectionTime);
+```
+
+### Available Metrics
+
+- **Provider Metrics**: Requests, success/failure rates, latency, token usage, costs
+- **Strategy Metrics**: Selection counts, distribution, selection time
+- **Overall Metrics**: Total requests, costs, error rates, requests per minute
+- **Request History**: Detailed history with filtering options
+
+See `examples/metrics.ts` for a complete example.
+
 ## Extensibility
 
 ### Adding a New Provider

package/dist/cjs/core/errors.d.ts

@@ -0,0 +1,21 @@
+/**
+ * Custom error classes for the framework
+ */
+export declare class ProviderError extends Error {
+    readonly providerId: string;
+    readonly cause?: Error | undefined;
+    constructor(message: string, providerId: string, cause?: Error | undefined);
+}
+export declare class StrategyError extends Error {
+    readonly cause?: Error | undefined;
+    constructor(message: string, cause?: Error | undefined);
+}
+export declare class OrchestratorError extends Error {
+    readonly cause?: Error | undefined;
+    constructor(message: string, cause?: Error | undefined);
+}
+export declare class ConfigurationError extends Error {
+    readonly cause?: Error | undefined;
+    constructor(message: string, cause?: Error | undefined);
+}
+//# sourceMappingURL=errors.d.ts.map

package/dist/cjs/core/errors.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"errors.d.ts","sourceRoot":"","sources":["../../../src/core/errors.ts"],"names":[],"mappings":"AAAA;;GAEG;AAEH,qBAAa,aAAc,SAAQ,KAAK;aAGpB,UAAU,EAAE,MAAM;aAClB,KAAK,CAAC,EAAE,KAAK;gBAF7B,OAAO,EAAE,MAAM,EACC,UAAU,EAAE,MAAM,EAClB,KAAK,CAAC,EAAE,KAAK,YAAA;CAKhC;AAED,qBAAa,aAAc,SAAQ,KAAK;aACO,KAAK,CAAC,EAAE,KAAK;gBAA9C,OAAO,EAAE,MAAM,EAAkB,KAAK,CAAC,EAAE,KAAK,YAAA;CAI3D;AAED,qBAAa,iBAAkB,SAAQ,KAAK;aACG,KAAK,CAAC,EAAE,KAAK;gBAA9C,OAAO,EAAE,MAAM,EAAkB,KAAK,CAAC,EAAE,KAAK,YAAA;CAI3D;AAED,qBAAa,kBAAmB,SAAQ,KAAK;aACE,KAAK,CAAC,EAAE,KAAK;gBAA9C,OAAO,EAAE,MAAM,EAAkB,KAAK,CAAC,EAAE,KAAK,YAAA;CAI3D"}

package/dist/cjs/core/errors.js

@@ -0,0 +1,40 @@
+"use strict";
+/**
+ * Custom error classes for the framework
+ */
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.ConfigurationError = exports.OrchestratorError = exports.StrategyError = exports.ProviderError = void 0;
+class ProviderError extends Error {
+    constructor(message, providerId, cause) {
+        super(message);
+        this.providerId = providerId;
+        this.cause = cause;
+        this.name = 'ProviderError';
+    }
+}
+exports.ProviderError = ProviderError;
+class StrategyError extends Error {
+    constructor(message, cause) {
+        super(message);
+        this.cause = cause;
+        this.name = 'StrategyError';
+    }
+}
+exports.StrategyError = StrategyError;
+class OrchestratorError extends Error {
+    constructor(message, cause) {
+        super(message);
+        this.cause = cause;
+        this.name = 'OrchestratorError';
+    }
+}
+exports.OrchestratorError = OrchestratorError;
+class ConfigurationError extends Error {
+    constructor(message, cause) {
+        super(message);
+        this.cause = cause;
+        this.name = 'ConfigurationError';
+    }
+}
+exports.ConfigurationError = ConfigurationError;
+//# sourceMappingURL=errors.js.map

package/dist/cjs/core/errors.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"errors.js","sourceRoot":"","sources":["../../../src/core/errors.ts"],"names":[],"mappings":";AAAA;;GAEG;;;AAEH,MAAa,aAAc,SAAQ,KAAK;IACtC,YACE,OAAe,EACC,UAAkB,EAClB,KAAa;QAE7B,KAAK,CAAC,OAAO,CAAC,CAAC;QAHC,eAAU,GAAV,UAAU,CAAQ;QAClB,UAAK,GAAL,KAAK,CAAQ;QAG7B,IAAI,CAAC,IAAI,GAAG,eAAe,CAAC;IAC9B,CAAC;CACF;AATD,sCASC;AAED,MAAa,aAAc,SAAQ,KAAK;IACtC,YAAY,OAAe,EAAkB,KAAa;QACxD,KAAK,CAAC,OAAO,CAAC,CAAC;QAD4B,UAAK,GAAL,KAAK,CAAQ;QAExD,IAAI,CAAC,IAAI,GAAG,eAAe,CAAC;IAC9B,CAAC;CACF;AALD,sCAKC;AAED,MAAa,iBAAkB,SAAQ,KAAK;IAC1C,YAAY,OAAe,EAAkB,KAAa;QACxD,KAAK,CAAC,OAAO,CAAC,CAAC;QAD4B,UAAK,GAAL,KAAK,CAAQ;QAExD,IAAI,CAAC,IAAI,GAAG,mBAAmB,CAAC;IAClC,CAAC;CACF;AALD,8CAKC;AAED,MAAa,kBAAmB,SAAQ,KAAK;IAC3C,YAAY,OAAe,EAAkB,KAAa;QACxD,KAAK,CAAC,OAAO,CAAC,CAAC;QAD4B,UAAK,GAAL,KAAK,CAAQ;QAExD,IAAI,CAAC,IAAI,GAAG,oBAAoB,CAAC;IACnC,CAAC;CACF;AALD,gDAKC"}

package/dist/cjs/core/interfaces.d.ts

@@ -0,0 +1,166 @@
+/**
+ * Core interfaces for AI providers and strategies
+ */
+import type { ChatMessage, ChatOptions, ChatResponse, ChatChunk, ProviderHealth, ProviderMetadata } from './types.js';
+export type { ChatMessage, ChatOptions, ChatResponse, ChatChunk, ProviderHealth, ProviderMetadata, };
+/**
+ * Main interface that all AI providers must implement
+ */
+export interface AIService {
+    /**
+     * Unique identifier for this provider instance
+     */
+    readonly id: string;
+    /**
+     * Provider metadata
+     */
+    readonly metadata: ProviderMetadata;
+    /**
+     * Check the health status of this provider
+     */
+    checkHealth(): Promise<ProviderHealth>;
+    /**
+     * Perform a chat completion (non-streaming)
+     */
+    chat(messages: ChatMessage[], options?: ChatOptions): Promise<ChatResponse>;
+    /**
+     * Perform a chat completion with streaming
+     */
+    chatStream(messages: ChatMessage[], options?: ChatOptions): Promise<ReadableStream<ChatChunk>>;
+}
+/**
+ * Strategy interface for selecting providers
+ */
+export interface SelectionStrategy {
+    /**
+     * Select a provider from the available list
+     * @param providers List of available providers
+     * @param context Optional context for selection (e.g., request metadata)
+     */
+    select(providers: AIService[], context?: SelectionContext): Promise<AIService | null>;
+    /**
+     * Update strategy state (e.g., after a successful/failed request)
+     */
+    update?(provider: AIService, success: boolean, metadata?: unknown): void;
+}
+/**
+ * Context passed to selection strategies
+ */
+export interface SelectionContext {
+    messages?: ChatMessage[];
+    options?: ChatOptions;
+    previousAttempts?: string[];
+    [key: string]: unknown;
+}
+/**
+ * Configuration for a provider instance
+ */
+export interface ProviderConfig {
+    id: string;
+    type: string;
+    apiKey?: string;
+    baseURL?: string;
+    model?: string;
+    enabled?: boolean;
+    priority?: number;
+    weight?: number;
+    metadata?: Record<string, unknown>;
+    [key: string]: unknown;
+}
+/**
+ * Configuration for a selection strategy
+ */
+export interface StrategyConfig {
+    type: string;
+    [key: string]: unknown;
+}
+/**
+ * Circuit breaker configuration
+ */
+export interface CircuitBreakerConfig {
+    /**
+     * Enable circuit breaker functionality
+     */
+    enabled?: boolean;
+    /**
+     * Number of consecutive failures before opening the circuit
+     */
+    failureThreshold?: number;
+    /**
+     * Time in milliseconds before attempting to reset the circuit
+     */
+    resetTimeout?: number;
+}
+/**
+ * Health check configuration
+ */
+export interface HealthCheckConfig {
+    /**
+     * Interval between health checks in milliseconds
+     */
+    interval?: number;
+    /**
+     * Timeout for individual health checks in milliseconds
+     */
+    timeout?: number;
+    /**
+     * Maximum consecutive failures before marking provider as unhealthy
+     */
+    maxConsecutiveFailures?: number;
+    /**
+     * Maximum latency threshold in milliseconds. Providers exceeding this will be marked unhealthy.
+     */
+    latencyThreshold?: number;
+    /**
+     * Enable periodic health checks
+     */
+    enabled?: boolean;
+}
+/**
+ * Main orchestrator configuration
+ */
+export interface OrchestratorConfig {
+    providers: ProviderConfig[];
+    strategy: StrategyConfig;
+    defaultOptions?: ChatOptions;
+    /**
+     * Maximum number of retry attempts before giving up.
+     * Defaults to the number of providers if not specified.
+     */
+    maxRetries?: number;
+    /**
+     * Global timeout for all requests in milliseconds.
+     * Defaults to 30000 (30 seconds) if not specified.
+     */
+    requestTimeout?: number;
+    /**
+     * Delay between retries in milliseconds, or 'exponential' for exponential backoff.
+     * Defaults to 1000 (1 second) if not specified.
+     */
+    retryDelay?: number | 'exponential';
+    /**
+     * Circuit breaker configuration
+     */
+    circuitBreaker?: CircuitBreakerConfig;
+    /**
+     * Health check configuration
+     */
+    healthCheck?: HealthCheckConfig;
+    /**
+     * Enable metrics collection (default: true)
+     */
+    enableMetrics?: boolean;
+    /**
+     * Callback for metrics events
+     */
+    onMetricsEvent?: import('./metrics.js').MetricsCallback;
+    /**
+     * @deprecated Use healthCheck.enabled instead
+     */
+    enableHealthChecks?: boolean;
+    /**
+     * @deprecated Use healthCheck.interval instead
+     */
+    healthCheckInterval?: number;
+}
+//# sourceMappingURL=interfaces.d.ts.map

package/dist/cjs/core/interfaces.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"interfaces.d.ts","sourceRoot":"","sources":["../../../src/core/interfaces.ts"],"names":[],"mappings":"AAAA;;GAEG;AAEH,OAAO,KAAK,EACV,WAAW,EACX,WAAW,EACX,YAAY,EACZ,SAAS,EACT,cAAc,EACd,gBAAgB,EACjB,MAAM,YAAY,CAAC;AAGpB,YAAY,EACV,WAAW,EACX,WAAW,EACX,YAAY,EACZ,SAAS,EACT,cAAc,EACd,gBAAgB,GACjB,CAAC;AAEF;;GAEG;AACH,MAAM,WAAW,SAAS;IACxB;;OAEG;IACH,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IAEpB;;OAEG;IACH,QAAQ,CAAC,QAAQ,EAAE,gBAAgB,CAAC;IAEpC;;OAEG;IACH,WAAW,IAAI,OAAO,CAAC,cAAc,CAAC,CAAC;IAEvC;;OAEG;IACH,IAAI,CAAC,QAAQ,EAAE,WAAW,EAAE,EAAE,OAAO,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC,YAAY,CAAC,CAAC;IAE5E;;OAEG;IACH,UAAU,CACR,QAAQ,EAAE,WAAW,EAAE,EACvB,OAAO,CAAC,EAAE,WAAW,GACpB,OAAO,CAAC,cAAc,CAAC,SAAS,CAAC,CAAC,CAAC;CACvC;AAED;;GAEG;AACH,MAAM,WAAW,iBAAiB;IAChC;;;;OAIG;IACH,MAAM,CACJ,SAAS,EAAE,SAAS,EAAE,EACtB,OAAO,CAAC,EAAE,gBAAgB,GACzB,OAAO,CAAC,SAAS,GAAG,IAAI,CAAC,CAAC;IAE7B;;OAEG;IACH,MAAM,CAAC,CAAC,QAAQ,EAAE,SAAS,EAAE,OAAO,EAAE,OAAO,EAAE,QAAQ,CAAC,EAAE,OAAO,GAAG,IAAI,CAAC;CAC1E;AAED;;GAEG;AACH,MAAM,WAAW,gBAAgB;IAC/B,QAAQ,CAAC,EAAE,WAAW,EAAE,CAAC;IACzB,OAAO,CAAC,EAAE,WAAW,CAAC;IACtB,gBAAgB,CAAC,EAAE,MAAM,EAAE,CAAC;IAC5B,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED;;GAEG;AACH,MAAM,WAAW,cAAc;IAC7B,EAAE,EAAE,MAAM,CAAC;IACX,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACnC,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED;;GAEG;AACH,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED;;GAEG;AACH,MAAM,WAAW,oBAAoB;IACnC;;OAEG;IACH,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB;;OAEG;IACH,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B;;OAEG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;CACvB;AAED;;GAEG;AACH,MAAM,WAAW,iBAAiB;IAChC;;OAEG;IACH,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB;;OAEG;IACH,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB;;OAEG;IACH,sBAAsB,CAAC,EAAE,MAAM,CAAC;IAChC;;OAEG;IACH,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B;;OAEG;IACH,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB;AAED;;GAEG;AACH,MAAM,WAAW,kBAAkB;IACjC,SAAS,EAAE,cAAc,EAAE,CAAC;IAC5B,QAAQ,EAAE,cAAc,CAAC;IACzB,cAAc,CAAC,EAAE,WAAW,CAAC;IAC7B;;;OAGG;IACH,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB;;;OAGG;IACH,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB;;;OAGG;IACH,UAAU,CAAC,EAAE,MAAM,GAAG,aAAa,CAAC;IACpC;;OAEG;IACH,cAAc,CAAC,EAAE,oBAAoB,CAAC;IACtC;;OAEG;IACH,WAAW,CAAC,EAAE,iBAAiB,CAAC;IAChC;;OAEG;IACH,aAAa,CAAC,EAAE,OAAO,CAAC;IACxB;;OAEG;IACH,cAAc,CAAC,EAAE,OAAO,cAAc,EAAE,eAAe,CAAC;IACxD;;OAEG;IACH,kBAAkB,CAAC,EAAE,OAAO,CAAC;IAC7B;;OAEG;IACH,mBAAmB,CAAC,EAAE,MAAM,CAAC;CAC9B"}

package/dist/cjs/core/interfaces.js

@@ -0,0 +1,6 @@
+"use strict";
+/**
+ * Core interfaces for AI providers and strategies
+ */
+Object.defineProperty(exports, "__esModule", { value: true });
+//# sourceMappingURL=interfaces.js.map

package/dist/cjs/core/interfaces.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"interfaces.js","sourceRoot":"","sources":["../../../src/core/interfaces.ts"],"names":[],"mappings":";AAAA;;GAEG"}

package/dist/cjs/core/metrics.d.ts

@@ -0,0 +1,161 @@
+/**
+ * Metrics and analytics system for the AI Orchestration Framework
+ */
+import type { AIService, ChatResponse } from './interfaces.js';
+import type { ProviderHealth } from './types.js';
+/**
+ * Metrics for a single provider
+ */
+export interface ProviderMetrics {
+    providerId: string;
+    providerName: string;
+    model?: string;
+    totalRequests: number;
+    successfulRequests: number;
+    failedRequests: number;
+    totalTokens: {
+        prompt: number;
+        completion: number;
+        total: number;
+    };
+    totalCost: number;
+    averageLatency: number;
+    lastUsed?: Date;
+    lastSuccess?: Date;
+    lastFailure?: Date;
+    healthHistory: Array<{
+        timestamp: Date;
+        healthy: boolean;
+        latency?: number;
+    }>;
+}
+/**
+ * Strategy selection metrics
+ */
+export interface StrategyMetrics {
+    totalSelections: number;
+    selectionsByProvider: Map<string, number>;
+    selectionsByStrategy: Map<string, number>;
+    averageSelectionTime: number;
+}
+/**
+ * Overall orchestrator metrics
+ */
+export interface OrchestratorMetrics {
+    totalRequests: number;
+    successfulRequests: number;
+    failedRequests: number;
+    totalCost: number;
+    providerMetrics: Map<string, ProviderMetrics>;
+    strategyMetrics: StrategyMetrics;
+    averageRequestLatency: number;
+    requestsPerMinute: number;
+    errorRate: number;
+}
+/**
+ * Event types for metrics tracking
+ */
+export type MetricsEvent = {
+    type: 'provider_selected';
+    providerId: string;
+    strategy: string;
+    timestamp: Date;
+    selectionTime?: number;
+} | {
+    type: 'request_started';
+    providerId: string;
+    timestamp: Date;
+} | {
+    type: 'request_success';
+    providerId: string;
+    response: ChatResponse;
+    latency: number;
+    timestamp: Date;
+} | {
+    type: 'request_failure';
+    providerId: string;
+    error: Error;
+    latency?: number;
+    timestamp: Date;
+} | {
+    type: 'health_check';
+    providerId: string;
+    health: ProviderHealth;
+    timestamp: Date;
+};
+/**
+ * Callback function for metrics events
+ */
+export type MetricsCallback = (event: MetricsEvent) => void;
+/**
+ * Metrics collector for tracking orchestrator usage
+ */
+export declare class MetricsCollector {
+    private providerMetrics;
+    private strategyMetrics;
+    private requestHistory;
+    private callbacks;
+    private selectionTimes;
+    /**
+     * Register a callback for metrics events
+     */
+    onEvent(callback: MetricsCallback): () => void;
+    /**
+     * Emit a metrics event
+     */
+    private emit;
+    /**
+     * Record provider selection
+     */
+    recordSelection(provider: AIService, strategy: string, selectionTime?: number): void;
+    /**
+     * Record request start
+     */
+    recordRequestStart(provider: AIService): void;
+    /**
+     * Record successful request
+     */
+    recordSuccess(provider: AIService, response: ChatResponse, latency: number): void;
+    /**
+     * Record failed request
+     */
+    recordFailure(provider: AIService, error: Error, latency?: number): void;
+    /**
+     * Record health check
+     */
+    recordHealthCheck(provider: AIService, health: ProviderHealth): void;
+    /**
+     * Get or create provider metrics
+     */
+    private getOrCreateProviderMetrics;
+    /**
+     * Get metrics for a specific provider
+     */
+    getProviderMetrics(providerId: string): ProviderMetrics | undefined;
+    /**
+     * Get all provider metrics
+     */
+    getAllProviderMetrics(): Map<string, ProviderMetrics>;
+    /**
+     * Get strategy metrics
+     */
+    getStrategyMetrics(): StrategyMetrics;
+    /**
+     * Get overall orchestrator metrics
+     */
+    getOrchestratorMetrics(): OrchestratorMetrics;
+    /**
+     * Reset all metrics
+     */
+    reset(): void;
+    /**
+     * Get request history (optionally filtered)
+     */
+    getRequestHistory(options?: {
+        providerId?: string;
+        startTime?: Date;
+        endTime?: Date;
+        limit?: number;
+    }): typeof this.requestHistory;
+}
+//# sourceMappingURL=metrics.d.ts.map

package/dist/cjs/core/metrics.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"metrics.d.ts","sourceRoot":"","sources":["../../../src/core/metrics.ts"],"names":[],"mappings":"AAAA;;GAEG;AAEH,OAAO,KAAK,EAAE,SAAS,EAAE,YAAY,EAAE,MAAM,iBAAiB,CAAC;AAC/D,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,YAAY,CAAC;AAEjD;;GAEG;AACH,MAAM,WAAW,eAAe;IAC9B,UAAU,EAAE,MAAM,CAAC;IACnB,YAAY,EAAE,MAAM,CAAC;IACrB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,EAAE,MAAM,CAAC;IACtB,kBAAkB,EAAE,MAAM,CAAC;IAC3B,cAAc,EAAE,MAAM,CAAC;IACvB,WAAW,EAAE;QACX,MAAM,EAAE,MAAM,CAAC;QACf,UAAU,EAAE,MAAM,CAAC;QACnB,KAAK,EAAE,MAAM,CAAC;KACf,CAAC;IACF,SAAS,EAAE,MAAM,CAAC;IAClB,cAAc,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,EAAE,IAAI,CAAC;IAChB,WAAW,CAAC,EAAE,IAAI,CAAC;IACnB,WAAW,CAAC,EAAE,IAAI,CAAC;IACnB,aAAa,EAAE,KAAK,CAAC;QACnB,SAAS,EAAE,IAAI,CAAC;QAChB,OAAO,EAAE,OAAO,CAAC;QACjB,OAAO,CAAC,EAAE,MAAM,CAAC;KAClB,CAAC,CAAC;CACJ;AAED;;GAEG;AACH,MAAM,WAAW,eAAe;IAC9B,eAAe,EAAE,MAAM,CAAC;IACxB,oBAAoB,EAAE,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAC1C,oBAAoB,EAAE,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAC1C,oBAAoB,EAAE,MAAM,CAAC;CAC9B;AAED;;GAEG;AACH,MAAM,WAAW,mBAAmB;IAClC,aAAa,EAAE,MAAM,CAAC;IACtB,kBAAkB,EAAE,MAAM,CAAC;IAC3B,cAAc,EAAE,MAAM,CAAC;IACvB,SAAS,EAAE,MAAM,CAAC;IAClB,eAAe,EAAE,GAAG,CAAC,MAAM,EAAE,eAAe,CAAC,CAAC;IAC9C,eAAe,EAAE,eAAe,CAAC;IACjC,qBAAqB,EAAE,MAAM,CAAC;IAC9B,iBAAiB,EAAE,MAAM,CAAC;IAC1B,SAAS,EAAE,MAAM,CAAC;CACnB;AAED;;GAEG;AACH,MAAM,MAAM,YAAY,GACpB;IACE,IAAI,EAAE,mBAAmB,CAAC;IAC1B,UAAU,EAAE,MAAM,CAAC;IACnB,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,IAAI,CAAC;IAChB,aAAa,CAAC,EAAE,MAAM,CAAC;CACxB,GACD;IACE,IAAI,EAAE,iBAAiB,CAAC;IACxB,UAAU,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,IAAI,CAAC;CACjB,GACD;IACE,IAAI,EAAE,iBAAiB,CAAC;IACxB,UAAU,EAAE,MAAM,CAAC;IACnB,QAAQ,EAAE,YAAY,CAAC;IACvB,OAAO,EAAE,MAAM,CAAC;IAChB,SAAS,EAAE,IAAI,CAAC;CACjB,GACD;IACE,IAAI,EAAE,iBAAiB,CAAC;IACxB,UAAU,EAAE,MAAM,CAAC;IACnB,KAAK,EAAE,KAAK,CAAC;IACb,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,IAAI,CAAC;CACjB,GACD;IACE,IAAI,EAAE,cAAc,CAAC;IACrB,UAAU,EAAE,MAAM,CAAC;IACnB,MAAM,EAAE,cAAc,CAAC;IACvB,SAAS,EAAE,IAAI,CAAC;CACjB,CAAC;AAEN;;GAEG;AACH,MAAM,MAAM,eAAe,GAAG,CAAC,KAAK,EAAE,YAAY,KAAK,IAAI,CAAC;AAE5D;;GAEG;AACH,qBAAa,gBAAgB;IAC3B,OAAO,CAAC,eAAe,CAA2C;IAClE,OAAO,CAAC,eAAe,CAKrB;IACF,OAAO,CAAC,cAAc,CAOd;IACR,OAAO,CAAC,SAAS,CAAmC;IACpD,OAAO,CAAC,cAAc,CAAgB;IAEtC;;OAEG;IACH,OAAO,CAAC,QAAQ,EAAE,eAAe,GAAG,MAAM,IAAI;IAK9C;;OAEG;IACH,OAAO,CAAC,IAAI;IAUZ;;OAEG;IACH,eAAe,CACb,QAAQ,EAAE,SAAS,EACnB,QAAQ,EAAE,MAAM,EAChB,aAAa,CAAC,EAAE,MAAM,GACrB,IAAI;IA6BP;;OAEG;IACH,kBAAkB,CAAC,QAAQ,EAAE,SAAS,GAAG,IAAI;IAY7C;;OAEG;IACH,aAAa,CACX,QAAQ,EAAE,SAAS,EACnB,QAAQ,EAAE,YAAY,EACtB,OAAO,EAAE,MAAM,GACd,IAAI;IA6DP;;OAEG;IACH,aAAa,CAAC,QAAQ,EAAE,SAAS,EAAE,KAAK,EAAE,KAAK,EAAE,OAAO,CAAC,EAAE,MAAM,GAAG,IAAI;IAsBxE;;OAEG;IACH,iBAAiB,CAAC,QAAQ,EAAE,SAAS,EAAE,MAAM,EAAE,cAAc,GAAG,IAAI;IAqBpE;;OAEG;IACH,OAAO,CAAC,0BAA0B;IAwBlC;;OAEG;IACH,kBAAkB,CAAC,UAAU,EAAE,MAAM,GAAG,eAAe,GAAG,SAAS;IAInE;;OAEG;IACH,qBAAqB,IAAI,GAAG,CAAC,MAAM,EAAE,eAAe,CAAC;IAIrD;;OAEG;IACH,kBAAkB,IAAI,eAAe;IAIrC;;OAEG;IACH,sBAAsB,IAAI,mBAAmB;IAuC7C;;OAEG;IACH,KAAK,IAAI,IAAI;IAYb;;OAEG;IACH,iBAAiB,CAAC,OAAO,CAAC,EAAE;QAC1B,UAAU,CAAC,EAAE,MAAM,CAAC;QACpB,SAAS,CAAC,EAAE,IAAI,CAAC;QACjB,OAAO,CAAC,EAAE,IAAI,CAAC;QACf,KAAK,CAAC,EAAE,MAAM,CAAC;KAChB,GAAG,OAAO,IAAI,CAAC,cAAc;CAuB/B"}