@hardlydifficult/worker-server 1.0.4 → 1.0.6

package/README.md

@@ -1,6 +1,6 @@
 # @hardlydifficult/worker-server
 
-A WebSocket-based worker server that manages remote worker connections, health monitoring, request routing, and load balancing through a clean TypeScript interface.
+WebSocket-based worker server with health monitoring, request routing, and load balancing.
 
 ## Installation
 
@@ -15,218 +15,388 @@ import { WorkerServer } from "@hardlydifficult/worker-server";
 
 const server = new WorkerServer({
   port: 8080,
-  heartbeatTimeoutMs: 60000,  // 60 seconds
-  healthCheckIntervalMs: 10000,  // 10 seconds
+  authToken: "my-secret-token", // optional
+  logger: console, // optional, defaults to no-op
 });
 
+// Handle worker registrations and messages
 server.onWorkerConnected((worker) => {
-  console.log(`Worker connected: ${worker.name} (${worker.id})`);
+  console.log(`Worker ${worker.name} (${worker.id}) connected`);
 });
 
-server.onWorkerDisconnected((worker, pendingRequests) => {
-  console.log(`Worker disconnected: ${worker.name} (${worker.id})`);
-  if (pendingRequests.size > 0) {
-    console.log(`Rescheduling ${pendingRequests.size} pending requests`);
-  }
+server.onWorkerDisconnected((worker, pendingRequestIds) => {
+  console.log(`Worker ${worker.id} disconnected with ${pendingRequestIds.size} pending requests`);
 });
 
 server.onWorkerMessage("work_complete", (worker, message) => {
-  console.log(`Work complete: ${message.requestId}`);
+  console.log(`Work completed: ${message.requestId}`);
 });
 
-server.addHttpHandler(async (req, res) => {
-  if (req.url === "/health") {
-    res.writeHead(200, { "Content-Type": "application/json" });
-    res.end(JSON.stringify({ status: "ok" }));
-    return true;
-  }
-  return false;
+// Start the server
+await server.start();
+
+// Get an available worker supporting a model
+const worker = server.getAvailableWorker("gpt-4");
+if (worker) {
+  server.send(worker.id, { type: "work_request", requestId: "req-1" });
+}
+
+// Stop when done
+await server.stop();
+```
+
+## Core Concepts
+
+### Worker Lifecycle Management
+
+The server handles worker connections, registrations, and disconnections with automatic health monitoring.
+
+#### Registration and Authentication
+
+Workers connect via WebSocket and send a `worker_registration` message with an optional `authToken`. If the server is configured with `authToken`, the worker must provide a matching token.
+
+```typescript
+import { WorkerServer } from "@hardlydifficult/worker-server";
+
+const server = new WorkerServer({
+  port: 8080,
+  authToken: "secret",
+});
+
+server.onWorkerConnected((worker) => {
+  console.log(`Worker registered: ${worker.id}`);
 });
 
 await server.start();
-console.log("Worker server running on port 8080");
 ```
 
-## Core Concepts
+#### Heartbeat and Health Monitoring
 
-### Worker Management
+Workers must send periodic `heartbeat` messages. The server tracks the last heartbeat timestamp and closes connections that miss the timeout.
 
-Workers connect via WebSocket, register with capabilities (supported models, concurrency limits), and send periodic heartbeats. The server tracks their status (available, busy, draining, unhealthy) and makes routing decisions based on load and model compatibility.
+```typescript
+const server = new WorkerServer({
+  port: 8080,
+  heartbeatTimeoutMs: 60_000,      // Missed for 60s → unhealthy
+  healthCheckIntervalMs: 10_000,   // Check every 10s
+  heartbeatIntervalMs: 15_000,     // Communicate 15s interval to workers
+});
+```
 
-### Request Tracking
+| Option | Default | Description |
+|--------|---------|-------------|
+| `heartbeatTimeoutMs` | 60000 | Time before worker is marked unhealthy |
+| `healthCheckIntervalMs` | 10000 | Frequency of health checks |
+| `heartbeatIntervalMs` | 15000 | Heartbeat interval communicated to workers |
 
-Requests are tracked per-worker and can be released when complete. Optional categories enable per-category concurrency limits when workers declare `concurrencyLimits` in their capabilities.
+#### Disconnection Handling
 
-### Load Balancing
+When a worker disconnects, the `onWorkerDisconnected` handler is called with its pending request IDs.
 
-Workers are selected based on:
-- Model support (exact or substring match)
-- Current load (least-loaded algorithm)
-- Category-specific limits (when `category` is provided)
+```typescript
+server.onWorkerDisconnected((worker, pendingRequestIds) => {
+  // Reassign pending requests as needed
+  console.log(`Pending: ${[...pendingRequestIds].join(", ")}`);
+});
+```
 
-## API Reference
+### Request Tracking and Load Balancing
 
-### WorkerServer
+Requests are tracked per-worker to avoid overloading and to support category-specific limits.
 
-Main entry point for managing worker connections.
+#### Request Lifecycle
 
-#### Constructor
+Track a request as assigned to a worker, then release it when complete.
 
 ```typescript
-constructor(options: WorkerServerOptions)
+// Assign request to worker
+const worker = server.getAvailableWorker("gpt-4");
+if (worker) {
+  server.trackRequest(worker.id, "req-1");
+  server.send(worker.id, { type: "work_request", requestId: "req-1" });
+}
+
+// Worker sends completion message
+server.onWorkerMessage("work_complete", (worker, message) => {
+  server.releaseRequest(message.requestId, { incrementCompleted: true });
+});
 ```
 
-| Option | Type | Default | Description |
-|--------|------|---------|-------------|
-| `port` | `number` | — | HTTP + WebSocket server port |
-| `authToken` | `string` | `undefined` | Optional token required for worker registration |
-| `heartbeatTimeoutMs` | `number` | `60000` | Milliseconds before a missed heartbeat marks a worker unhealthy |
-| `healthCheckIntervalMs` | `number` | `10000` | Interval (ms) for health checks |
-| `heartbeatIntervalMs` | `number` | `15000` | Heartbeat interval communicated to workers |
-| `logger` | `WorkerServerLogger` | `undefined` | Logger instance (defaults to no-op) |
+#### Available Worker Selection
 
-#### Lifecycle Events
+Workers are selected based on capacity and model support.
 
 ```typescript
-// Called when a worker successfully registers
-onWorkerConnected(handler: WorkerConnectedHandler): () => void;
+// Get least-loaded worker supporting a model
+const worker = server.getAvailableWorker("gpt-4");
+// → least-loaded worker that supports "gpt-4"
+
+// Get any available worker (model-agnostic)
+const anyWorker = server.getAnyAvailableWorker();
+// → any worker (Available or Busy status)
+```
+
+Workers are marked `Busy` when `activeRequests >= maxConcurrentRequests`.
+
+#### Per-Category Concurrency Limits
+
+Workers can define per-category limits in their capabilities. The pool enforces these when `trackRequest` is called with a category.
 
-// Called when a worker disconnects with pending request IDs
-onWorkerDisconnected(handler: WorkerDisconnectedHandler): () => void;
+```typescript
+// Worker capabilities include:
+{
+  models: [{ modelId: "gpt-4", ... }],
+  maxConcurrentRequests: 5,
+  concurrencyLimits: {
+    inference: 2,
+    embedding: 4,
+  }
+}
 
-// Register a handler for a specific message type (dispatched by 'type' field)
-onWorkerMessage<T = Record<string, unknown>>(
-  type: string,
-  handler: WorkerMessageHandler<T>
-): () => void;
+// Track request in a category
+server.trackRequest(worker.id, "req-1", "inference");
 ```
 
-#### Message Operations
+### Message Routing
+
+Messages are routed by the `type` field to registered handlers.
 
 ```typescript
-// Send a JSON message to a specific worker (false if failed)
-send(workerId: string, message: Record<string, unknown>): boolean;
+server.onWorkerMessage("work_complete", (worker, message) => {
+  console.log(`Worker ${worker.id} completed ${message.requestId}`);
+});
 
-// Broadcast to all connected workers
-broadcast(message: Record<string, unknown>): void;
+server.onWorkerMessage("metrics", (worker, message) => {
+  console.log(`Worker ${worker.id} metrics:`, message.metrics);
+});
 ```
 
-#### Pool Queries
+Returns an unsubscribe function:
 
 ```typescript
-// Get least-loaded worker supporting the given model
-getAvailableWorker(model: string, category?: string): WorkerInfo | null;
+const unsubscribe = server.onWorkerMessage("status", handler);
+// later...
+unsubscribe();
+```
 
-// Get any available worker (model-agnostic)
-getAnyAvailableWorker(): WorkerInfo | null;
+### Sending Messages
 
-// Total connected worker count
-getWorkerCount(): number;
+#### Targeted Send
+
+```typescript
+const success = server.send(workerId, { type: "ping" });
+// Returns false if worker not found or WebSocket not open
+```
 
-// Available worker count
-getAvailableWorkerCount(): number;
+#### Broadcast
 
-// Get info about all connected workers
-getWorkerInfo(): WorkerInfo[];
+```typescript
+server.broadcast({ type: "shutdown" });
+// Sends to all connected workers with open sockets
 ```
 
-#### Request Tracking
+### Server Extensibility
+
+#### Additional WebSocket Endpoints
 
 ```typescript
-// Track a request assigned to a worker (optional category)
-trackRequest(workerId: string, requestId: string, category?: string): void;
+// Create a custom WebSocket endpoint
+server.addWebSocketEndpoint("/ws/metrics", (ws) => {
+  ws.on("message", (data) => {
+    console.log("Metrics client message:", data.toString());
+  });
+});
 
-// Release a tracked request (optionally increment completed count)
-releaseRequest(
-  requestId: string,
-  options?: { incrementCompleted?: boolean }
-): void;
+// Clients connect to ws://localhost:8080/ws/metrics
 ```
 
-#### HTTP & WebSocket Extensibility
+#### HTTP Handlers
 
 ```typescript
-// Add an HTTP handler (called in order until one returns true)
-addHttpHandler(handler: HttpRequestHandler): void;
+server.addHttpHandler(async (req, res) => {
+  if (req.url === "/health") {
+    res.writeHead(200, { "Content-Type": "application/json" });
+    res.end(JSON.stringify({ status: "ok" }));
+    return true;
+  }
+  return false; // continue to next handler or 404
+});
 
-// Add an additional WebSocket endpoint at a path
-addWebSocketEndpoint(
-  path: string,
-  handler: WebSocketConnectionHandler
-): void;
+// Custom HTTP responses take precedence over 404
 ```
 
-#### Server Lifecycle
+## Public API
+
+### `WorkerServer`
+
+Main server class for managing worker connections.
+
+| Method | Description |
+|--------|-------------|
+| `onWorkerConnected(handler)` | Register handler for worker registration events |
+| `onWorkerDisconnected(handler)` | Register handler for worker disconnection events |
+| `onWorkerMessage(type, handler)` | Register handler for a specific message type |
+| `send(workerId, message)` | Send a JSON message to a specific worker |
+| `broadcast(message)` | Broadcast a JSON message to all workers |
+| `getAvailableWorker(model, category?)` | Get least-loaded worker supporting model |
+| `getAnyAvailableWorker()` | Get any available/Busy worker |
+| `getWorkerCount()` | Total connected worker count |
+| `getAvailableWorkerCount()` | Available worker count |
+| `getWorkerInfo()` | Get public info about all workers |
+| `trackRequest(workerId, requestId, category?)` | Track request as in-progress |
+| `releaseRequest(requestId, options?)` | Release tracked request |
+| `addHttpHandler(handler)` | Add HTTP request handler |
+| `addWebSocketEndpoint(path, handler)` | Add custom WebSocket endpoint |
+| `start()` | Start HTTP + WebSocket server |
+| `stop()` | Stop server and close all connections |
+
+### `WorkerPool`
+
+Internal pool manager with public helpers.
+
+| Method | Description |
+|--------|-------------|
+| `add(worker)` | Add a connected worker to the pool |
+| `remove(id)` | Remove worker by ID |
+| `get(id)` | Get worker by ID |
+| `has(id)` | Check if worker is in pool |
+| `getAvailableWorker(model, category?)` | Get available worker by model |
+| `getAnyAvailableWorker()` | Get any available/Busy worker |
+| `getCount()` | Total worker count |
+| `getAvailableCount()` | Available worker count |
+| `getWorkerInfoList()` | Get public info for all workers |
+| `checkHealth(timeoutMs)` | Check worker health and return dead IDs |
+| `send(workerId, message)` | Send message to worker |
+| `broadcast(message)` | Broadcast to all workers |
+| `closeAll()` | Close all worker connections |
+
+### `toWorkerInfo(worker)`
+
+Converts internal `ConnectedWorker` to public `WorkerInfo`.
 
 ```typescript
-// Start the HTTP + WebSocket server
-start(): Promise<void>;
+import { toWorkerInfo, type ConnectedWorker } from "@hardlydifficult/worker-server";
 
-// Stop the server and close all connections
-stop(): Promise<void>;
+const internal: ConnectedWorker = /* ... */;
+const publicInfo = toWorkerInfo(internal);
+// No websocket reference in publicInfo
 ```
 
-### WorkerInfo
+### Types
+
+| Type | Description |
+|------|-------------|
+| `WorkerStatus` | `available`, `busy`, `draining`, `unhealthy` |
+| `ModelInfo` | Model capabilities and metadata |
+| `WorkerCapabilities` | Worker capacity, models, and concurrency limits |
+| `WorkerInfo` | Public worker state |
+| `ConnectedWorker` | Internal state (includes WebSocket) |
+| `WorkerServerOptions` | Configuration for `WorkerServer` |
+| `WorkerServerLogger` | Logger interface |
+| `HttpRequestHandler` | HTTP request handler type |
+| `WorkerMessageHandler<T>` | Typed message handler |
+| `WorkerConnectedHandler` | Worker connected event handler |
+| `WorkerDisconnectedHandler` | Worker disconnected event handler |
+| `WebSocketConnectionHandler` | Custom WebSocket endpoint handler |
+
+### Constants and Defaults
 
-Public interface representing a connected worker:
+Default timeouts (milliseconds):
 
 ```typescript
-interface WorkerInfo {
-  readonly id: string;
-  readonly name: string;
-  readonly status: WorkerStatus;  // "available" | "busy" | "draining" | "unhealthy"
-  readonly capabilities: WorkerCapabilities;
-  readonly sessionId: string;
-  readonly connectedAt: Date;
-  readonly lastHeartbeat: Date;
-  readonly activeRequests: number;
-  readonly completedRequests: number;
-  readonly pendingRequestIds: ReadonlySet<string>;
-  readonly categoryActiveRequests: ReadonlyMap<string, number>;
+{
+  heartbeatTimeoutMs: 60_000,
+  healthCheckIntervalMs: 10_000,
+  heartbeatIntervalMs: 15_000,
 }
 ```
 
-### WorkerCapabilities
+### Utility Functions
 
-Describes a worker's capabilities:
+#### `safeCompare(a, b)`
+
+Timing-safe string comparison.
 
 ```typescript
-interface WorkerCapabilities {
-  models: ModelInfo[];
-  maxConcurrentRequests: number;
-  metadata?: Record<string, unknown>;
-  concurrencyLimits?: Record<string, number>;
+import { safeCompare } from "@hardlydifficult/worker-server";
+
+const isValid = safeCompare(inputToken, secretToken);
+```
+
+## Appendix
+
+### Worker Registration Protocol
+
+Workers send:
+
+```json
+{
+  "type": "worker_registration",
+  "workerId": "worker-1",
+  "workerName": "My Worker",
+  "capabilities": {
+    "models": [{
+      "modelId": "gpt-4",
+      "displayName": "GPT-4",
+      "maxContextTokens": 8192,
+      "maxOutputTokens": 4096,
+      "supportsStreaming": true
+    }],
+    "maxConcurrentRequests": 5,
+    "concurrencyLimits": {
+      "inference": 2,
+      "embedding": 4
+    }
+  },
+  "authToken": "optional"
 }
 ```
 
-### ModelInfo
+Server responds:
 
-Describes a supported model:
+```json
+{
+  "type": "worker_registration_ack",
+  "success": true,
+  "sessionId": "uuid",
+  "heartbeatIntervalMs": 15000
+}
+```
 
-```typescript
-interface ModelInfo {
-  modelId: string;
-  displayName: string;
-  maxContextTokens: number;
-  maxOutputTokens: number;
-  supportsStreaming: boolean;
-  supportsVision?: boolean;
-  supportsTools?: boolean;
+### Worker Heartbeat Protocol
+
+Workers send:
+
+```json
+{
+  "type": "heartbeat",
+  "workerId": "worker-1",
+  "timestamp": "2024-01-01T00:00:00.000Z"
 }
 ```
 
-### WorkerStatus
+Server responds:
 
-Worker state enumeration:
+```json
+{
+  "type": "heartbeat_ack",
+  "timestamp": "2024-01-01T00:00:00.000Z",
+  "nextHeartbeatDeadline": "2024-01-01T00:01:00.000Z"
+}
+```
 
-| Status | Description |
-|--------|-------------|
-| `available` | Worker is idle and can accept new requests |
-| `busy` | Worker is at max concurrent requests but may accept model-agnostic tasks |
-| `draining` | Worker is being gracefully decommissioned |
-| `unhealthy` | Worker has missed heartbeats and is presumed degraded |
+### Status Transitions
 
-## Worker Protocol
+- `Available` → `Busy` when `activeRequests >= maxConcurrentRequests`
+- `Busy` → `Available` when `activeRequests < maxConcurrentRequests`
+- Any → `Unhealthy` on heartbeat timeout
+- `Unhealthy` → `Available/Busy` on heartbeat recovery
+
+### Concurrent Request Tracking
+
+The pool tracks requests per-worker and per-category (if provided). It automatically decrements the category count when releasing a tracked request.
+
+### Worker Protocol Summary
 
 Workers communicate using JSON messages with a `type` field:
 
@@ -237,10 +407,4 @@ Workers communicate using JSON messages with a `type` field:
 | `heartbeat` | Worker → Server | Periodic health check |
 | `heartbeat_ack` | Server → Worker | Acknowledgment with next deadline |
 
-All other message types are routed to registered handlers via `onWorkerMessage()`.
-
-## Health Monitoring
-
-- Workers missing heartbeats for `heartbeatTimeoutMs` are marked `unhealthy`
-- Workers missing heartbeats for `3 × heartbeatTimeoutMs` are disconnected
-- Health checks run every `healthCheckIntervalMs`
+All other message types are routed to registered handlers via `onWorkerMessage()`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hardlydifficult/worker-server",
-  "version": "1.0.4",
+  "version": "1.0.6",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
   "files": [