@hardlydifficult/worker-server 1.0.10 → 1.0.12

package/README.md

@@ -1,6 +1,6 @@
 # @hardlydifficult/worker-server
 
-A WebSocket-based server for managing remote worker connections with health monitoring, message routing, and load balancing.
+WebSocket-based remote worker server with health monitoring, message routing, and load balancing.
 
 ## Installation
 
@@ -13,565 +13,293 @@ npm install @hardlydifficult/worker-server
 ```typescript
 import { WorkerServer } from "@hardlydifficult/worker-server";
 
-// Create and start the server
-const server = new WorkerServer({ port: 3000 });
-
-server.start().then(() => {
-  console.log("Worker server running on port 3000");
-  
-  // Listen for worker connections
-  server.onWorkerConnected((worker) => {
-    console.log(`Worker ${worker.name} (${worker.id}) connected`);
-  });
-
-  server.onWorkerDisconnected((worker) => {
-    console.log(`Worker ${worker.name} (${worker.id}) disconnected`);
-  });
-
-  // Route messages by type
-  server.onWorkerMessage("work_complete", (worker, message) => {
-    console.log(`Worker ${worker.id} completed request: ${message.requestId}`);
-  });
+const server = new WorkerServer({
+  port: 19100,
+  authToken: "secret-token", // optional
 });
-```
-
-## Core Concepts
-
-### Worker Registration & Lifecycle
-
-Workers connect via WebSocket and register with authentication (optional). The server tracks their status, heartbeat, and request load.
-
-```typescript
-import { WorkerServer } from "@hardlydifficult/worker-server";
 
-const server = new WorkerServer({ 
-  port: 3000,
-  authToken: "secret-token" // Optional
+server.onWorkerConnected((worker) => {
+  console.log(`Worker ${worker.name} connected`);
 });
 
-server.start();
-server.onWorkerConnected((worker) => {
-  console.log("Worker connected:", worker.id, worker.name);
+server.onWorkerMessage("work_complete", (worker, message) => {
+  console.log(`Worker ${worker.id} completed:`, message);
 });
-```
 
-Workers send a `worker_registration` message:
+await server.start();
+console.log("Server listening on port", server.port);
 
-```json
-{
-  "type": "worker_registration",
-  "workerId": "worker-1",
-  "workerName": "GPU Worker",
-  "capabilities": {
-    "models": [
-      {
-        "modelId": "gpt-4",
-        "displayName": "GPT-4",
-        "maxContextTokens": 32768,
-        "maxOutputTokens": 4096,
-        "supportsStreaming": true
-      }
-    ],
-    "maxConcurrentRequests": 2
-  },
-  "authToken": "secret-token"
+// Send a message to a worker
+const worker = server.getAvailableWorker("sonnet");
+if (worker) {
+  server.send(worker.id, { type: "work_request", requestId: "req-123" });
 }
 ```
 
-The server responds with a registration acknowledgment:
-
-```json
-{
-  "type": "worker_registration_ack",
-  "success": true,
-  "sessionId": "uuid-here",
-  "heartbeatIntervalMs": 15000
-}
-```
+## Core Concepts
 
-### Message Routing
+### WorkerServer
 
-Messages are routed by the `type` field. Handlers receive the worker info and message payload.
+Main entry point for managing worker connections via WebSocket.
 
 ```typescript
-server.onWorkerMessage("status_update", (worker, message) => {
-  console.log(`Worker ${worker.id} status: ${message.statusText}`);
-});
+import { WorkerServer, WorkerStatus } from "@hardlydifficult/worker-server";
 
-// Send messages to workers
-server.send(workerId, {
-  type: "execute",
-  requestId: "req-1",
-  prompt: "Hello, world!"
+const server = new WorkerServer({
+  port: 19100,
+  heartbeatTimeoutMs: 60_000,
+  healthCheckIntervalMs: 10_000,
+  heartbeatIntervalMs: 15_000,
 });
 
-// Broadcast to all workers
-server.broadcast({ type: "shutdown" });
+await server.start();
+await server.stop();
 ```
 
-### Worker Selection & Load Balancing
+#### Lifecycle Events
 
-Select workers by model support or use any available worker. Workers are automatically assigned least-loaded.
+| Method | Description |
+|--------|-------------|
+| `onWorkerConnected(handler)` | Called when a worker registers successfully |
+| `onWorkerDisconnected(handler)` | Called when a worker disconnects; includes pending request IDs |
 
 ```typescript
-// Get the least-loaded worker that supports a specific model
-const worker = server.getAvailableWorker("gpt-4");
-if (worker) {
-  server.send(worker.id, { type: "execute", prompt: "..." });
-}
+server.onWorkerConnected((worker) => {
+  console.log(`Connected: ${worker.id} (${worker.name})`);
+});
 
-// Get any available worker (model-agnostic)
-const anyWorker = server.getAnyAvailableWorker();
+server.onWorkerDisconnected((worker, pendingRequestIds) => {
+  console.log(`Disconnected: ${worker.id} with ${pendingRequestIds.size} pending requests`);
+});
 ```
 
-Request tracking ensures accurate load reporting:
+#### Message Routing
+
+Register handlers for message types sent by workers:
 
 ```typescript
-// Track when a request is assigned
-server.trackRequest(worker.id, requestId);
+server.onWorkerMessage("work_complete", (worker, message) => {
+  const { requestId, result } = message;
+  console.log(`Worker ${worker.id} completed ${requestId}`);
+});
 
-// Release when the response is received (optionally increment completed count)
-server.releaseRequest(requestId, { incrementCompleted: true });
+// Send messages to workers
+const success = server.send(workerId, { type: "work_request", requestId: "req-1" });
+server.broadcast({ type: "shutdown" });
 ```
 
-### Health Monitoring
+#### Worker Selection & Pool Queries
 
-Workers must send periodic heartbeats. Unresponsive workers are marked unhealthy and eventually removed.
+| Method | Description |
+|--------|-------------|
+| `getAvailableWorker(model, category?)` | Least-loaded worker supporting the model |
+| `getAnyAvailableWorker()` | Any available or busy worker (model-agnostic) |
+| `getAvailableSlotCount(model, category?)` | Total free slots across all available workers |
+| `getWorkerCount()` | Total connected workers |
+| `getAvailableWorkerCount()` | Available workers count |
+| `getWorkerInfo()` | Public info for all workers |
 
 ```typescript
-// Heartbeat message format (worker → server)
-{
-  "type": "heartbeat",
-  "workerId": "worker-1",
-  "timestamp": "2024-01-01T00:00:00.000Z"
-}
-
-// Server response
-{
-  "type": "heartbeat_ack",
-  "timestamp": "2024-01-01T00:00:00.000Z",
-  "nextHeartbeatDeadline": "2024-01-01T00:01:00.000Z"
+// Get least-loaded worker supporting a model
+const worker = server.getAvailableWorker("sonnet");
+if (worker) {
+  server.trackRequest(worker.id, "req-123", "local");
 }
-```
 
-Health checks run automatically at the configured interval (default: 10s). Workers missing heartbeats for >3× timeout are removed.
+// Slot counts with category-aware limits
+console.log("Available slots:", server.getAvailableSlotCount("sonnet", "local"));
 
-```typescript
-const server = new WorkerServer({
-  port: 3000,
-  heartbeatTimeoutMs: 60_000,      // 60 seconds before unhealthy
-  healthCheckIntervalMs: 10_000,  // Check every 10 seconds
-});
+// View all workers
+for (const info of server.getWorkerInfo()) {
+  console.log(`${info.name}: ${info.status} (${info.activeRequests}/${info.capabilities.maxConcurrentRequests})`);
+}
 ```
 
-### Category-Aware Concurrency
+#### Request Tracking
 
-Workers can specify per-category concurrency limits for fine-grained control.
+Track and release requests for accurate availability:
 
 ```typescript
-const server = new WorkerServer({ port: 3000 });
-
-// Worker registration includes concurrency limits
-{
-  "capabilities": {
-    "models": [.],
-    "maxConcurrentRequests": 4,
-    "concurrencyLimits": {
-      "chat": 2,
-      "embedding": 3,
-      "tool_use": 1
-    }
-  }
-}
-
-// Track with category
-server.trackRequest(worker.id, requestId, "chat");
+// When assigning a request to a worker
+server.trackRequest(workerId, requestId, "local");
 
-// Release without specifying category (looked up automatically)
-server.releaseRequest(requestId);
+// When the request completes
+server.releaseRequest(requestId, { incrementCompleted: true });
 ```
 
-## HTTP & WebSocket Extensibility
-
-### Custom HTTP Endpoints
+#### Extensibility
 
-Add HTTP handlers that return `true` when they handle the request.
+Add HTTP endpoints and custom WebSocket paths:
 
 ```typescript
+// HTTP handler
 server.addHttpHandler(async (req, res) => {
   if (req.url === "/health") {
     res.writeHead(200, { "Content-Type": "application/json" });
-    res.end(JSON.stringify({ status: "ok" }));
+    res.end(JSON.stringify({ ok: true }));
     return true;
   }
   return false;
 });
-```
 
-### Additional WebSocket Endpoints
-
-Register additional WebSocket paths for non-worker connections.
-
-```typescript
-server.addWebSocketEndpoint("/ws/admin", (ws) => {
-  ws.on("message", (data) => {
-    // Handle admin messages
-  });
+// Custom WebSocket endpoint
+server.addWebSocketEndpoint("/ws/dashboard", (ws) => {
+  ws.send(JSON.stringify({ type: "hello" }));
 });
 ```
 
-### Worker Info
-
-Public worker info (without WebSocket reference):
-
-```typescript
-const worker = server.getAvailableWorker("gpt-4");
-if (worker) {
-  console.log("Active requests:", worker.activeRequests);
-  console.log("Completed requests:", worker.completedRequests);
-  console.log("Pending request IDs:", [...worker.pendingRequestIds]);
-  console.log("Per-category active requests:", worker.categoryActiveRequests);
-}
-```
-
-## Core Components
-
-### WorkerServer
-
-Main server class managing WebSocket connections, HTTP endpoints, and worker pool.
-
-#### Constructor
-
-| Parameter | Type | Default | Description |
-|-----------|------|---------|-------------|
-| `port` | `number` | — | HTTP + WebSocket server port |
-| `authToken` | `string` (optional) | — | Token required for worker registration |
-| `heartbeatTimeoutMs` | `number` | 60000 | Timeout before marking worker unhealthy |
-| `healthCheckIntervalMs` | `number` | 10000 | Interval for health checks |
-| `heartbeatIntervalMs` | `number` | 15000 | Heartbeat interval communicated to workers |
-| `logger` | `WorkerServerLogger` (optional) | No-op | Logger instance |
-
-#### Lifecycle Management
-
-```typescript
-const server = new WorkerServer({ port: 8080, authToken: "secret" });
-
-// Start the server
-await server.start();
-
-// Stop the server gracefully
-await server.stop();
-```
-
-#### Registration Handlers
-
-```typescript
-// Called when a worker successfully registers
-const unsubscribeConnected = server.onWorkerConnected((worker) => {
-  console.log(`Worker connected: ${worker.name}`);
-});
-
-// Called when a worker disconnects
-const unsubscribeDisconnected = server.onWorkerDisconnected((worker, pending) => {
-  console.log(`Worker disconnected with ${pending.size} pending requests`);
-});
-```
-
-#### Message Handling
-
-```typescript
-// Register handlers for domain-specific messages by type
-server.onWorkerMessage("work_request", (worker, message) => {
-  // Process work request from worker
-});
-
-server.onWorkerMessage("status_update", (worker, message) => {
-  // Handle status updates from worker
-});
-```
-
-#### Sending Messages
-
-```typescript
-// Send to a specific worker
-const success = server.send("worker-1", { type: "stop", reason: "shutdown" });
-
-// Broadcast to all connected workers
-server.broadcast({ type: "maintenance_start" });
-```
+### WorkerPool
 
-#### Pool Queries
+Low-level pool manager for worker state and selection.
 
 ```typescript
-// Get least-loaded worker supporting a specific model
-const worker = server.getAvailableWorker("sonnet-3.5");
+import { WorkerPool, toWorkerInfo, WorkerStatus } from "@hardlydifficult/worker-server";
 
-// Get any available worker (model-agnostic)
-const anyWorker = server.getAnyAvailableWorker();
+const pool = new WorkerPool(logger);
 
-// Get all worker info
-const workers = server.getWorkerInfo(); // Returns WorkerInfo[]
+// Add/remove workers
+pool.add(worker);
+pool.remove(workerId);
+const worker = pool.get(workerId);
 ```
 
-#### Request Tracking
-
-```typescript
-// Track a request assigned to a worker
-server.trackRequest("worker-1", "req-123", "inference");
-
-// Release a completed request
-server.releaseRequest("req-123", { incrementCompleted: true });
-```
-
-#### Extensibility
+#### Selection Logic
 
 | Method | Description |
 |--------|-------------|
-| `addHttpHandler(handler)` | Add an HTTP handler (called in order until one returns `true`) |
-| `addWebSocketEndpoint(path, handler)` | Add a WebSocket endpoint at a custom path |
-
-#### Event Handlers
+| `getAvailableWorker(model, category?)` | Least-loaded worker supporting the model, respecting per-category concurrency limits |
+| `getAnyAvailableWorker()` | Any available or busy worker (model-agnostic) |
+| `getAvailableSlotCount(model, category?)` | Total free slots across all available workers for the model |
+| `getCount()` | Total connected workers |
+| `getAvailableCount()` | Available workers count |
+| `getWorkerInfoList()` | Public info for all workers |
 
-| Method | Return | Description |
-|--------|--------|-------------|
-| `onWorkerConnected(handler)` | `() => void` | Called when worker registers |
-| `onWorkerDisconnected(handler)` | `() => void` | Called when worker disconnects (includes pending requests) |
-| `onWorkerMessage(type, handler)` | `() => void` | Register handler for a message type |
+#### Request Management
 
-### WorkerPool
+| Method | Description |
+|--------|-------------|
+| `trackRequest(workerId, requestId, category?)` | Marks request as in-flight and updates status |
+| `releaseRequest(requestId, options?)` | Decrements active count, optionally increments completed count |
 
-Internal class managing worker state and selection. Exposed via `WorkerServer`.
+#### Health Monitoring
 
 | Method | Description |
 |--------|-------------|
-| `getAvailableWorker(model, category?)` | Get least-loaded available worker supporting model |
-| `getAnyAvailableWorker()` | Get any available/busy worker |
-| `trackRequest(workerId, requestId, category?)` | Mark request as in-progress |
-| `releaseRequest(requestId, { incrementCompleted? })` | Release tracked request |
-| `getWorkerInfoList()` | Get public info for all workers |
-| `checkHealth(timeoutMs)` | Return IDs of dead workers (heartbeat > 3x timeout) |
-| `send(workerId, message)` | Send message to specific worker |
-| `broadcast(message)` | Broadcast to all workers |
-| `closeAll()` | Close all worker connections |
+| `checkHealth(timeoutMs)` | Returns IDs of workers exceeding `3x` timeout; marks unhealthy ones |
 
 ### ConnectionHandler
 
-Handles WebSocket connection lifecycle and protocol message routing.
-
-#### Message Routing
-
-```typescript
-import { ConnectionHandler } from "@hardlydifficult/worker-server";
-
-const handler = new ConnectionHandler(pool, config, logger);
-
-// Register handlers for custom message types
-const unregister = handler.onMessage("custom_type", (worker, message) => {
-  console.log(`Received from ${worker.id}:`, message);
-});
-```
-
-#### Event Handlers
-
-```typescript
-handler.onWorkerConnected((worker) => {
-  console.log("Worker connected:", worker.id);
-});
-
-handler.onWorkerDisconnected((worker, pending) => {
-  console.log("Worker disconnected with pending:", pending.size);
-});
-```
-
-## Advanced Features
-
-### HTTP Endpoints
-
-Custom HTTP handlers can be added:
-
-```typescript
-server.addHttpHandler(async (req, res) => {
-  if (req.url === "/health") {
-    res.writeHead(200, { "Content-Type": "application/json" });
-    res.end(JSON.stringify({ status: "ok" }));
-    return true;
-  }
-  return false; // continue to next handler
-});
-```
-
-### Custom WebSocket Endpoints
-
-Additional WebSocket paths can be handled:
-
-```typescript
-server.addWebSocketEndpoint("/ws/admin", (ws) => {
-  ws.on("message", (data) => {
-    // Handle admin WebSocket messages
-  });
-});
-```
-
-### Authentication
+Handles WebSocket lifecycle, registration, heartbeats, and message routing. Most consumers use `WorkerServer`, which encapsulates this.
 
-Optionally require authentication tokens from workers:
+### Message Protocol
 
-```typescript
-const server = new WorkerServer({
-  port: 8080,
-  authToken: "your-secret-token"
-});
-
-// Workers must send registration with matching authToken
-```
-
-### Load Balancing with Category Limits
-
-Workers can declare per-category concurrency limits:
+Workers send JSON messages with a `type` field:
 
-```typescript
-const capabilities = {
-  models: [{ modelId: "sonnet", ... }],
-  maxConcurrentRequests: 10,
-  concurrencyLimits: {
-    inference: 5, // max 5 concurrent inference requests
-    embeddings: 2 // max 2 concurrent embedding requests
-  }
-};
-```
+- `worker_registration` — Register with capabilities and optional `authToken`
+- `heartbeat` — Send periodically to confirm liveness
 
-Requests are then tracked by category:
+The server responds with:
+- `worker_registration_ack` — Success/failure with `sessionId` and `heartbeatIntervalMs`
+- `heartbeat_ack` — Acknowledgment with `nextHeartbeatDeadline`
 
-```typescript
-server.trackRequest("worker-1", "req-1", "inference");
-server.releaseRequest("req-1"); // category looked up automatically
-```
-
-## Type Definitions
+### Types & Interfaces
 
-### WorkerStatus
+#### `WorkerStatus`
 
 | Value | Description |
 |-------|-------------|
 | `available` | Worker can accept new requests |
-| `busy` | Worker is at max concurrent requests |
-| `draining` | Worker is shutting down |
-| `unhealthy` | Worker heartbeat has timed out |
+| `busy` | Worker at capacity, but can accept model-agnostic tasks |
+| `draining` | Worker finishing current work before shutdown |
+| `unhealthy` | Worker failed heartbeat checks |
+
+#### `WorkerInfo`
 
-### ModelInfo
+Public worker metadata (excludes raw WebSocket):
 
 ```typescript
-interface ModelInfo {
-  modelId: string;
-  displayName: string;
-  maxContextTokens: number;
-  maxOutputTokens: number;
-  supportsStreaming: boolean;
-  supportsVision?: boolean;
-  supportsTools?: boolean;
+interface WorkerInfo {
+  readonly id: string;
+  readonly name: string;
+  readonly status: WorkerStatus;
+  readonly capabilities: WorkerCapabilities;
+  readonly sessionId: string;
+  readonly connectedAt: Date;
+  readonly lastHeartbeat: Date;
+  readonly activeRequests: number;
+  readonly completedRequests: number;
+  readonly pendingRequestIds: ReadonlySet<string>;
+  readonly categoryActiveRequests: ReadonlyMap<string, number>;
 }
 ```
 
-### WorkerCapabilities
+#### `WorkerCapabilities`
 
 ```typescript
 interface WorkerCapabilities {
   models: ModelInfo[];
   maxConcurrentRequests: number;
   metadata?: Record<string, unknown>;
-  concurrencyLimits?: Record<string, number>;
+  concurrencyLimits?: Record<string, number>; // per-category limits
 }
 ```
 
-### WorkerInfo
+#### `ModelInfo`
 
 ```typescript
-interface WorkerInfo {
-  id: string;
-  name: string;
-  status: WorkerStatus;
-  capabilities: WorkerCapabilities;
-  sessionId: string;
-  connectedAt: Date;
-  lastHeartbeat: Date;
-  activeRequests: number;
-  completedRequests: number;
-  pendingRequestIds: ReadonlySet<string>;
-  categoryActiveRequests: ReadonlyMap<string, number>;
+interface ModelInfo {
+  modelId: string;
+  displayName: string;
+  maxContextTokens: number;
+  maxOutputTokens: number;
+  supportsStreaming: boolean;
+  supportsVision?: boolean;
+  supportsTools?: boolean;
 }
 ```
 
-## Logging
+### Secure Authentication
 
-The server accepts a logger implementing `WorkerServerLogger`:
+Authentication tokens are compared using timing-safe comparison to prevent brute-force attacks:
 
 ```typescript
-interface WorkerServerLogger {
-  debug(message: string, context?: Record<string, unknown>): void;
-  info(message: string, context?: Record<string, unknown>): void;
-  warn(message: string, context?: Record<string, unknown>): void;
-  error(message: string, context?: Record<string, unknown>): void;
-}
+import { safeCompare } from "@hardlydifficult/worker-server";
+// Internally used by ConnectionHandler; exposed for testing
+const valid = safeCompare("a", "b"); // false
 ```
 
-Default is a no-op logger. To use a custom logger:
-
-```typescript
-const server = new WorkerServer({
-  port: 3000,
-  logger: {
-    debug: console.debug,
-    info: console.info,
-    warn: console.warn,
-    error: console.error,
-  },
-});
-```
-
-## Appendix
-
-### Protocol Messages
-
-**Worker Registration (worker → server)**
+Workers must send the token in registration:
 
 ```json
 {
   "type": "worker_registration",
-  "workerId": "string",
-  "workerName": "string",
-  "capabilities": WorkerCapabilities,
-  "authToken?": "string"
+  "workerId": "worker-1",
+  "workerName": "My Worker",
+  "capabilities": { ... },
+  "authToken": "secret-token"
 }
 ```
 
-**Registration Acknowledgment (server → worker)**
+### Heartbeat Protocol
 
-```json
-{
-  "type": "worker_registration_ack",
-  "success": "boolean",
-  "error?": "string",
-  "sessionId?": "string",
-  "heartbeatIntervalMs?": "number"
-}
-```
-
-**Heartbeat (worker → server)**
+Workers must send periodic heartbeat messages:
 
 ```json
 {
   "type": "heartbeat",
-  "workerId": "string",
-  "timestamp": "string"
+  "workerId": "worker-1",
+  "timestamp": "2024-01-01T00:00:00.000Z"
 }
 ```
 
-**Heartbeat Acknowledgment (server → worker)**
+The server responds with:
 
 ```json
 {
   "type": "heartbeat_ack",
-  "timestamp": "string",
-  "nextHeartbeatDeadline": "string"
+  "timestamp": "2024-01-01T00:00:00.000Z",
+  "nextHeartbeatDeadline": "2024-01-01T00:01:15.000Z"
 }
-```
+```
+
+A worker is considered unhealthy if its heartbeat exceeds `heartbeatTimeoutMs`. It is marked dead and disconnected after `3x` the timeout.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hardlydifficult/worker-server",
-  "version": "1.0.10",
+  "version": "1.0.12",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
   "files": [