agent-relay 3.1.1 → 3.1.2

package/packages/openclaw/src/__tests__/SPEC-ws-client-testing.md

@@ -0,0 +1,192 @@
+# Spec: OpenClawGatewayClient WebSocket Testing
+
+## Problem
+
+The `OpenClawGatewayClient` class in `gateway.ts` handles WebSocket connection, Ed25519 challenge-response auth, RPC message delivery, and automatic reconnection. It currently has 0% test coverage because it opens real WebSocket connections that are hard to mock.
+
+## Approach: In-process Mock WebSocket Server
+
+Use the `ws` package (already a dependency) to spin up a lightweight `WebSocketServer` on a random port within the test process. The mock server implements just enough of the OpenClaw gateway protocol to exercise all client code paths.
+
+## Test Infrastructure
+
+### MockOpenClawServer
+
+```typescript
+import WebSocket, { WebSocketServer } from 'ws';
+import { AddressInfo } from 'node:net';
+
+class MockOpenClawServer {
+  private wss: WebSocketServer;
+  port: number;
+  connections: WebSocket[] = [];
+  receivedMessages: Record<string, unknown>[] = [];
+
+  /** Control flags — tests toggle these to simulate server behavior. */
+  rejectAuth = false;
+  skipChallenge = false;
+  rpcDelay = 0;      // ms delay before responding to RPCs
+  rpcError = false;   // respond to chat.send with an error
+
+  constructor() {
+    this.wss = new WebSocketServer({ port: 0 }); // random port
+    this.port = (this.wss.address() as AddressInfo).port;
+    this.wss.on('connection', (ws) => this.handleConnection(ws));
+  }
+
+  private handleConnection(ws: WebSocket): void {
+    this.connections.push(ws);
+
+    // Step 1: Send connect.challenge
+    if (!this.skipChallenge) {
+      ws.send(JSON.stringify({
+        type: 'event',
+        event: 'connect.challenge',
+        payload: { nonce: 'test-nonce-123', ts: Date.now() },
+      }));
+    }
+
+    ws.on('message', (data) => {
+      const msg = JSON.parse(data.toString());
+      this.receivedMessages.push(msg);
+
+      // Step 2: Handle connect request
+      if (msg.method === 'connect') {
+        ws.send(JSON.stringify({
+          type: 'res',
+          id: msg.id,
+          ok: !this.rejectAuth,
+          ...(this.rejectAuth ? { error: 'auth rejected' } : {}),
+        }));
+        return;
+      }
+
+      // Step 3: Handle chat.send RPC
+      if (msg.method === 'chat.send') {
+        setTimeout(() => {
+          ws.send(JSON.stringify({
+            type: 'res',
+            id: msg.id,
+            ok: !this.rpcError,
+            ...(this.rpcError
+              ? { error: 'delivery failed' }
+              : { payload: { runId: 'run_1', status: 'ok' } }),
+          }));
+        }, this.rpcDelay);
+        return;
+      }
+    });
+  }
+
+  /** Forcibly close all connections (simulates server crash). */
+  disconnectAll(): void {
+    for (const ws of this.connections) ws.close(1006);
+    this.connections = [];
+  }
+
+  async close(): Promise<void> {
+    this.disconnectAll();
+    return new Promise((resolve) => this.wss.close(() => resolve()));
+  }
+}
+```
+
+## Test Cases
+
+### 1. Connection & Authentication
+
+| Test | What it validates |
+|---|---|
+| `should connect and authenticate via challenge-response` | Full happy path: challenge → sign → connect response |
+| `should reject when server denies auth` | `connect()` rejects when server returns `ok: false` |
+| `should timeout if no challenge arrives` | `connect()` rejects after `CONNECT_TIMEOUT_MS` when `skipChallenge=true` |
+| `should resolve immediately if already connected` | Second `connect()` call is a no-op |
+
+### 2. Message Delivery (chat.send RPC)
+
+| Test | What it validates |
+|---|---|
+| `should send chat.send and resolve true on success` | `sendChatMessage()` returns `true` |
+| `should send idempotencyKey when provided` | Verify `params.idempotencyKey` in received message |
+| `should resolve false when RPC returns error` | `rpcError=true` → returns `false` |
+| `should resolve false on RPC timeout` | `rpcDelay=20000` → hits 15s timeout, returns `false` |
+| `should reconnect and retry if not connected` | Disconnect, call `sendChatMessage`, verify reconnection |
+
+### 3. Reconnection
+
+| Test | What it validates |
+|---|---|
+| `should reconnect after server disconnects` | `disconnectAll()` → client reconnects within ~3s |
+| `should not reconnect after stop()` | `disconnect()` then `disconnectAll()` → no reconnection |
+| `should reject pending RPCs on disconnect` | In-flight `sendChatMessage` resolves `false` on disconnect |
+
+### 4. Ed25519 Signature Verification
+
+| Test | What it validates |
+|---|---|
+| `should produce valid Ed25519 signature` | Mock server verifies the signature using the client's public key from the connect payload |
+| `should include correct v3 payload fields` | Verify clientId, clientMode, platform, role, scopes, nonce |
+
+## Implementation Notes
+
+- Each test creates its own `MockOpenClawServer` and `OpenClawGatewayClient` for full isolation.
+- The `OpenClawGatewayClient` class is currently not exported. Either:
+  - (a) Export it (simplest), or
+  - (b) Test indirectly through `InboundGateway` with a real mock WS server (heavier but no API changes).
+- Recommended: export the class with a `@internal` JSDoc tag.
+- Tests should use `afterEach` to close both the mock server and client to prevent port leaks.
+
+## E2E Integration Tests
+
+Separate from the WS unit tests, create integration tests following the broker harness pattern in `tests/integration/broker/`:
+
+### Test: Full gateway message flow with real Relaycast
+
+```
+1. Create ephemeral Relaycast workspace (RelayCast.createWorkspace)
+2. Register two agents: "sender" and "viewer-test-claw"
+3. Start InboundGateway with the workspace key
+4. Post a message to #general via sender agent
+5. Assert the gateway's relaySender.sendMessage was called with correct format
+6. Post a DM from sender to viewer-test-claw
+7. Assert DM delivery with [relaycast:dm] format
+8. Add a reaction via sender
+9. Assert reaction soft notification delivery
+10. Cleanup: stop gateway, workspace is ephemeral
+```
+
+### Test: Gateway reconnection resilience
+
+```
+1. Start gateway with real Relaycast connection
+2. Force-disconnect the SDK WebSocket (call relayAgentClient.disconnect())
+3. Wait for reconnection
+4. Post a message
+5. Assert message is still delivered
+```
+
+### Prerequisites
+
+These tests require network access to `api.relaycast.dev` and should:
+- Use `checkPrerequisites()` pattern from broker harness
+- Be skippable via `skipIfMissing()`
+- Have generous timeouts (120s)
+- Use unique channel/agent names with timestamp suffixes
+
+## File Locations
+
+```
+packages/openclaw/src/__tests__/
+  gateway-threads.test.ts         # Existing unit tests (vitest)
+  ws-client.test.ts               # NEW: WebSocket client unit tests (vitest)
+
+tests/integration/openclaw/
+  gateway-e2e.test.ts             # NEW: Full integration tests (node:test)
+  utils/gateway-harness.ts        # NEW: Gateway test harness
+```
+
+## Estimated Effort
+
+- WS client unit tests: ~2-3 hours
+- E2E integration tests: ~3-4 hours
+- Total: ~1 day

package/packages/openclaw/src/__tests__/gateway-control.test.ts

@@ -0,0 +1,288 @@
+import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
+import { createServer, type Server as HttpServer, type IncomingMessage, type ServerResponse } from 'node:http';
+
+// ---------------------------------------------------------------------------
+// Mocks
+// ---------------------------------------------------------------------------
+
+const eventHandlers: Record<string, Array<(...args: unknown[]) => void>> = {};
+
+function registerHandler(event: string) {
+  return (handler: (...args: unknown[]) => void) => {
+    if (!eventHandlers[event]) eventHandlers[event] = [];
+    eventHandlers[event].push(handler);
+    return () => {
+      eventHandlers[event] = eventHandlers[event].filter((h) => h !== handler);
+    };
+  };
+}
+
+const mockAgentClient = {
+  connect: vi.fn(),
+  disconnect: vi.fn().mockResolvedValue(undefined),
+  subscribe: vi.fn(),
+  channels: {
+    join: vi.fn().mockResolvedValue({ ok: true }),
+    create: vi.fn().mockResolvedValue({ name: 'general' }),
+  },
+  on: {
+    connected: registerHandler('connected'),
+    messageCreated: registerHandler('messageCreated'),
+    threadReply: registerHandler('threadReply'),
+    dmReceived: registerHandler('dmReceived'),
+    groupDmReceived: registerHandler('groupDmReceived'),
+    commandInvoked: registerHandler('commandInvoked'),
+    reactionAdded: registerHandler('reactionAdded'),
+    reactionRemoved: registerHandler('reactionRemoved'),
+    reconnecting: registerHandler('reconnecting'),
+    disconnected: registerHandler('disconnected'),
+    error: registerHandler('error'),
+    any: registerHandler('any'),
+  },
+};
+
+vi.mock('@relaycast/sdk', () => ({
+  RelayCast: vi.fn().mockImplementation(() => ({
+    agents: {
+      registerOrGet: vi.fn().mockResolvedValue({ name: 'viewer-test-claw', token: 'tok_test' }),
+    },
+    channels: { join: vi.fn().mockResolvedValue({ ok: true }) },
+    messages: { list: vi.fn().mockResolvedValue([]) },
+    as: vi.fn().mockReturnValue(mockAgentClient),
+  })),
+}));
+
+const mockSpawnManager = {
+  size: 0,
+  spawn: vi.fn(),
+  release: vi.fn(),
+  releaseByName: vi.fn(),
+  releaseAll: vi.fn().mockResolvedValue(undefined),
+  list: vi.fn().mockReturnValue([]),
+  get: vi.fn(),
+};
+
+vi.mock('../spawn/manager.js', () => ({
+  SpawnManager: vi.fn().mockImplementation(() => mockSpawnManager),
+}));
+
+vi.mock('node:fs/promises', () => ({
+  readFile: vi.fn().mockResolvedValue('{"spawns":[]}'),
+  writeFile: vi.fn().mockResolvedValue(undefined),
+  mkdir: vi.fn().mockResolvedValue(undefined),
+}));
+
+vi.mock('node:fs', () => ({
+  existsSync: vi.fn().mockReturnValue(false),
+}));
+
+// We do NOT mock node:http — we want a real HTTP server for these tests.
+// But we need to intercept createServer in InboundGateway so we can control the port.
+// Strategy: let gateway start its own server, then hit it via fetch().
+
+// We need to override RELAYCAST_CONTROL_PORT to use port 0 (random)
+// Actually, we can't use port 0 because the gateway hardcodes the listen call.
+// Instead, let's mock node:http to capture the request handler, then run a real server.
+
+let capturedHandler: ((req: IncomingMessage, res: ServerResponse) => void) | null = null;
+let realServer: HttpServer | null = null;
+let controlPort = 0;
+
+vi.mock('node:http', async (importOriginal) => {
+  const actual = await importOriginal<typeof import('node:http')>();
+  return {
+    ...actual,
+    createServer: vi.fn((handler: (req: IncomingMessage, res: ServerResponse) => void) => {
+      capturedHandler = handler;
+      // Create a real HTTP server with the captured handler
+      realServer = actual.createServer(handler);
+      return {
+        listen: vi.fn((_port: number, _host: string, cb: () => void) => {
+          // Bind to random port
+          realServer!.listen(0, '127.0.0.1', () => {
+            const addr = realServer!.address() as { port: number };
+            controlPort = addr.port;
+            cb();
+          });
+        }),
+        close: vi.fn((cb?: () => void) => {
+          realServer?.close(() => cb?.());
+        }),
+        address: vi.fn(() => realServer?.address()),
+        on: vi.fn((_event: string, _handler: (...args: unknown[]) => void) => {}),
+      };
+    }),
+  };
+});
+
+import { InboundGateway } from '../gateway.js';
+
+// ---------------------------------------------------------------------------
+// Helpers
+// ---------------------------------------------------------------------------
+
+async function fetchControl(method: string, path: string, body?: unknown): Promise<Response> {
+  const url = `http://127.0.0.1:${controlPort}${path}`;
+  const opts: RequestInit = { method };
+  if (body !== undefined) {
+    opts.body = JSON.stringify(body);
+    opts.headers = { 'Content-Type': 'application/json' };
+  }
+  return fetch(url, opts);
+}
+
+// ---------------------------------------------------------------------------
+// Tests
+// ---------------------------------------------------------------------------
+
+describe('Gateway control HTTP server', () => {
+  let gateway: InboundGateway;
+
+  beforeEach(async () => {
+    vi.clearAllMocks();
+    for (const key of Object.keys(eventHandlers)) {
+      eventHandlers[key] = [];
+    }
+    // Reset mock spawn manager state
+    mockSpawnManager.size = 0;
+    mockSpawnManager.spawn.mockReset();
+    mockSpawnManager.release.mockReset();
+    mockSpawnManager.releaseByName.mockReset();
+    mockSpawnManager.list.mockReturnValue([]);
+
+    gateway = new InboundGateway({
+      config: {
+        apiKey: 'rk_live_test',
+        clawName: 'test-claw',
+        baseUrl: 'https://api.relaycast.dev',
+        channels: ['general'],
+      },
+      relaySender: { sendMessage: vi.fn().mockResolvedValue({ event_id: 'evt_1' }) },
+    });
+    await gateway.start();
+  });
+
+  afterEach(async () => {
+    await gateway.stop();
+    if (realServer) {
+      realServer.close();
+      realServer = null;
+    }
+  });
+
+  it('GET /health returns 200', async () => {
+    const res = await fetchControl('GET', '/health');
+    expect(res.status).toBe(200);
+    const data = await res.json() as Record<string, unknown>;
+    expect(data.ok).toBe(true);
+    expect(data.status).toBe('running');
+    expect(typeof data.uptime).toBe('number');
+  });
+
+  it('POST /spawn with name returns 200', async () => {
+    mockSpawnManager.spawn.mockResolvedValue({
+      id: 'spawn-1',
+      displayName: 'worker-1',
+      agentName: 'claw-ws-worker-1',
+      gatewayPort: 18800,
+    });
+    mockSpawnManager.size = 1;
+
+    const res = await fetchControl('POST', '/spawn', {
+      name: 'worker-1',
+      role: 'researcher',
+    });
+    expect(res.status).toBe(200);
+    const data = await res.json() as Record<string, unknown>;
+    expect(data.ok).toBe(true);
+    expect(data.name).toBe('worker-1');
+    expect(data.agentName).toBe('claw-ws-worker-1');
+    expect(data.id).toBe('spawn-1');
+  });
+
+  it('POST /spawn without name returns 400', async () => {
+    const res = await fetchControl('POST', '/spawn', { role: 'worker' });
+    expect(res.status).toBe(400);
+    const data = await res.json() as Record<string, unknown>;
+    expect(data.ok).toBe(false);
+    expect(data.error).toMatch(/name/i);
+  });
+
+  it('POST /spawn error returns 500', async () => {
+    mockSpawnManager.spawn.mockRejectedValue(new Error('Docker unavailable'));
+
+    const res = await fetchControl('POST', '/spawn', { name: 'worker-1' });
+    expect(res.status).toBe(500);
+    const data = await res.json() as Record<string, unknown>;
+    expect(data.ok).toBe(false);
+    expect(data.error).toContain('Docker unavailable');
+  });
+
+  it('GET /list returns 200 with empty list', async () => {
+    mockSpawnManager.list.mockReturnValue([]);
+
+    const res = await fetchControl('GET', '/list');
+    expect(res.status).toBe(200);
+    const data = await res.json() as Record<string, unknown>;
+    expect(data.ok).toBe(true);
+    expect(data.active).toBe(0);
+    expect(data.claws).toEqual([]);
+  });
+
+  it('GET /list returns handles', async () => {
+    mockSpawnManager.list.mockReturnValue([
+      { id: 's1', displayName: 'alpha', agentName: 'claw-alpha', gatewayPort: 18801 },
+    ]);
+
+    const res = await fetchControl('GET', '/list');
+    expect(res.status).toBe(200);
+    const data = await res.json() as { claws: Array<{ name: string }> };
+    expect(data.claws).toHaveLength(1);
+    expect(data.claws[0].name).toBe('alpha');
+  });
+
+  it('POST /release by name returns 200', async () => {
+    mockSpawnManager.releaseByName.mockResolvedValue(true);
+    mockSpawnManager.size = 0;
+
+    const res = await fetchControl('POST', '/release', { name: 'worker-1' });
+    expect(res.status).toBe(200);
+    const data = await res.json() as Record<string, unknown>;
+    expect(data.ok).toBe(true);
+  });
+
+  it('POST /release by id returns 200', async () => {
+    mockSpawnManager.release.mockResolvedValue(true);
+    mockSpawnManager.size = 0;
+
+    const res = await fetchControl('POST', '/release', { id: 'spawn-1' });
+    expect(res.status).toBe(200);
+    const data = await res.json() as Record<string, unknown>;
+    expect(data.ok).toBe(true);
+  });
+
+  it('POST /release without name or id returns 400', async () => {
+    const res = await fetchControl('POST', '/release', {});
+    expect(res.status).toBe(400);
+    const data = await res.json() as Record<string, unknown>;
+    expect(data.ok).toBe(false);
+    expect(data.error).toMatch(/name.*id|id.*name/i);
+  });
+
+  it('POST /release error returns 500', async () => {
+    mockSpawnManager.release.mockRejectedValue(new Error('Process kill failed'));
+
+    const res = await fetchControl('POST', '/release', { id: 'spawn-1' });
+    expect(res.status).toBe(500);
+    const data = await res.json() as Record<string, unknown>;
+    expect(data.ok).toBe(false);
+    expect(data.error).toContain('Process kill failed');
+  });
+
+  it('GET /unknown returns 404', async () => {
+    const res = await fetchControl('GET', '/nonexistent');
+    expect(res.status).toBe(404);
+    const data = await res.json() as Record<string, unknown>;
+    expect(data.error).toBe('Not found');
+  });
+});