@zyno-io/ts-server-foundation 26.712.36

package/docs/content/srpc.md

@@ -0,0 +1,292 @@
+# SRPC
+
+SRPC is a typed bidirectional RPC layer over WebSocket. It uses generated protobuf message codecs, request/response prefixes, optional metadata, HMAC client authentication, reconnect handling, and multiplexed byte streams.
+
+## Message Shape
+
+SRPC message types must include shared envelope fields plus request/response payload fields.
+
+```protobuf
+syntax = "proto3";
+
+message ClientMessage {
+    string requestId = 1;
+    bool reply = 2;
+    optional string error = 3;
+    optional bool userError = 4;
+    optional TraceContext trace = 5;
+
+    oneof request {
+        PingPong pingPong = 50;
+        ByteStreamOperation byteStreamOperation = 51;
+        UEchoRequest uEchoRequest = 100;
+        DNotifyResponse dNotifyResponse = 200;
+    }
+}
+
+message ServerMessage {
+    string requestId = 1;
+    bool reply = 2;
+    optional string error = 3;
+    optional bool userError = 4;
+    optional TraceContext trace = 5;
+
+    oneof response {
+        PingPong pingPong = 50;
+        ByteStreamOperation byteStreamOperation = 51;
+        UEchoResponse uEchoResponse = 100;
+        DNotifyRequest dNotifyRequest = 200;
+    }
+}
+
+message PingPong {}
+message TraceContext {
+    string traceId = 1;
+    string spanId = 2;
+    int32 traceFlags = 3;
+}
+message WriteByteStream { bytes chunk = 1; }
+message FinishByteStream {}
+message DestroyByteStream { optional string error = 1; }
+message ByteStreamOperation {
+    int32 streamId = 1;
+    oneof operation {
+        WriteByteStream write = 2;
+        FinishByteStream finish = 3;
+        DestroyByteStream destroy = 4;
+    }
+}
+
+message UEchoRequest {
+    string message = 1;
+}
+
+message UEchoResponse {
+    string message = 1;
+}
+
+message DNotifyRequest {
+    string event = 1;
+}
+
+message DNotifyResponse {
+    bool acknowledged = 1;
+}
+```
+
+The prefix is the method name without `Request` or `Response`. A client can invoke prefixes where its message type has `<prefix>Request` and the server message type has `<prefix>Response`. The server can invoke prefixes in the opposite direction.
+
+Generated codecs satisfy `SrpcMessageFns<T>`: an `encode()` function returning bytes (or an object with `finish()`) and a `decode()` function accepting bytes. `ClientMessage` and `ServerMessage` generated by `ts-proto` provide this shape directly.
+
+## Proto Generation
+
+```bash
+corepack yarn tsf-gen-proto resources/proto/service.proto src/generated/proto
+```
+
+Options:
+
+| Flag             | Description                                  |
+| ---------------- | -------------------------------------------- |
+| `--only-types`   | Generates TypeScript type declarations only. |
+| `--use-date`     | Uses `Date` for `google.protobuf.Timestamp`. |
+| `--use-map-type` | Uses `Map` for proto maps.                   |
+
+`PROTOC` can point to a custom `protoc` binary. `ts-proto` must be installed.
+
+## Server
+
+```ts
+import { SrpcServer } from '@zyno-io/ts-server-foundation';
+import { ClientMessage, ServerMessage } from './generated/proto/service';
+
+const server = new SrpcServer({
+    logger,
+    clientMessage: ClientMessage,
+    serverMessage: ServerMessage,
+    wsPath: '/srpc'
+});
+
+server.registerConnectionHandler(async stream => {
+    logger.info('client connected', { clientId: stream.clientId, streamId: stream.id });
+});
+
+server.registerMessageHandler('uEcho', async (_stream, data) => {
+    return { message: data.message };
+});
+
+server.registerDisconnectHandler((stream, cause) => {
+    logger.info('client disconnected', { clientId: stream.clientId, cause });
+});
+```
+
+`SrpcServer` registers its WebSocket upgrade handler through the current app's `app.http` runtime unless an explicit `httpServer` is provided.
+
+Server options:
+
+| Option          | Description                                                |
+| --------------- | ---------------------------------------------------------- |
+| `logger`        | Logger with `info`, `warn`, `error`, and `debug` methods.  |
+| `clientMessage` | Generated codec for client-to-server envelope messages.    |
+| `serverMessage` | Generated codec for server-to-client envelope messages.    |
+| `wsPath`        | WebSocket path.                                            |
+| `debug`         | Compatibility option; currently declared but not read.     |
+| `logLevel`      | `info`, `debug`, or `false`.                               |
+| `httpServer`    | Optional Node HTTP server for direct upgrade registration. |
+
+Handlers may also be zero-argument classes with a `handle(stream, data)` method. A new class instance is created for each request; SRPC does not resolve handler classes through application DI.
+
+Connection setup is deliberately ordered. After authentication, the server creates a pending stream and sends the initial `pingPong`. Registered connection handlers then run in registration order and are awaited. Only after they complete does the stream become active and queued client requests begin. A connection-handler failure disconnects the stream before activation. Disconnect handlers run in registration order but are synchronous callbacks; returning a promise does not delay cleanup.
+
+## Server-To-Client Calls
+
+```ts
+const stream = server.streamsByClientId.get('worker-1');
+
+if (stream) {
+    const result = await server.invoke(stream, 'dNotify', { event: 'reload' }, 5000);
+    result.acknowledged;
+}
+```
+
+`SrpcServer.createInvoke(() => server)` creates a stable invoke function for DI or callbacks that need to resolve the current server lazily.
+
+## Client
+
+```ts
+import { SrpcClient, SrpcConflictError } from '@zyno-io/ts-server-foundation';
+import { ClientMessage, ServerMessage } from './generated/proto/service';
+
+const client = new SrpcClient(logger, 'wss://api.example.com/srpc', ClientMessage, ServerMessage, 'worker-1', { role: 'worker' }, 'shared-secret', {
+    enableReconnect: true,
+    protocolVersion: 2
+});
+
+client.registerConnectionHandler(() => {
+    logger.info('connected');
+});
+
+client.registerMessageHandler('dNotify', async data => {
+    return { acknowledged: true };
+});
+
+client.registerDisconnectHandler(cause => {
+    logger.warn('disconnected', { cause });
+});
+
+try {
+    await client.connect();
+} catch (error) {
+    if (error instanceof SrpcConflictError) {
+        await client.connect({ supersede: true });
+    }
+}
+
+const echo = await client.invoke('uEcho', { message: 'hello' });
+client.disconnect();
+```
+
+Client options:
+
+| Option            | Default | Description                              |
+| ----------------- | ------- | ---------------------------------------- |
+| `enableReconnect` | `true`  | Reconnects after unexpected disconnects. |
+| `protocolVersion` | `2`     | Protocol version sent during handshake.  |
+
+The exported `SrpcClientOptions` type describes this object. `connect({ supersede?: boolean })` controls only that connection attempt and is separate from the constructor options.
+
+With protocol version 2, duplicate `clientId` connections are rejected unless `connect({ supersede: true })` is used.
+Protocol version 1 retains the legacy behavior: a new connection replaces the existing connection with the same client ID even without the supersede flag. Prefer version 2 when duplicate ownership must be explicit.
+
+After the initial ping handshake, the client sends a ping every 55 seconds. A connection with no pong for 75 seconds closes with the `timeout` cause. The server checks the same 75-second inactivity window every 15 seconds. Unexpected client disconnects reconnect after one second when `enableReconnect` is true; `disconnect()`, conflicts, and an explicit `enableReconnect: false` suppress reconnection. `triggerConnectionCheck()` forces an immediate ping-based liveness check.
+
+## Authentication
+
+Clients sign connection metadata with HMAC-SHA256. The client sends `authv`, `appv`, `ts`, `id`, `cid`, `signature`, `_v`, optional `_supersede`, and custom metadata under `m--<key>` WebSocket query parameters.
+
+By default the server verifies signatures with `SRPC_AUTH_SECRET`. Provide per-client secrets with:
+
+```ts
+server.setClientKeyFetcher(async clientId => {
+    return await lookupSecret(clientId);
+});
+```
+
+Replace the built-in HMAC/key-fetcher authentication with a custom authorizer using:
+
+```ts
+server.setClientAuthorizer(async (query, request) => {
+    if (!(await verifyCustomHandshake(query, request))) return false;
+    if (query['m--role'] !== 'worker') return false;
+    return { authorizedRole: 'worker' };
+});
+```
+
+The callback receives the raw query map, including signed fields and `m--`-prefixed custom metadata. Once configured, it is responsible for all authentication; the default HMAC validation and `setClientKeyFetcher()` path are not also run. Returning `false` rejects the connection, `true` accepts it, and an object accepts the connection and merges that object into normalized `stream.meta` alongside custom metadata with the `m--` prefix removed.
+
+Clock drift is controlled by `SRPC_AUTH_CLOCK_DRIFT_MS`, defaulting to 30 seconds.
+
+## Streams
+
+Connected streams expose metadata:
+
+| Field             | Description                                          |
+| ----------------- | ---------------------------------------------------- |
+| `id`              | Server-side stream ID.                               |
+| `clientStreamId`  | Client-generated stream ID.                          |
+| `clientId`        | Client identity.                                     |
+| `appVersion`      | Client app version query value.                      |
+| `protocolVersion` | Handshake protocol version.                          |
+| `supersede`       | Whether the connection requested supersede behavior. |
+| `meta`            | Query metadata plus authorizer metadata.             |
+| `connectedAt`     | Connection timestamp.                                |
+| `lastPingAt`      | Last ping timestamp.                                 |
+
+`streamsById` contains authenticated pending and active streams, while `streamsByClientId` contains only the active stream for each client ID. Connection handlers run while the stream is still pending.
+
+The exported `SrpcStream<TMeta>` type describes these server-side streams. It also exposes the underlying WebSocket and request queue through `$ws` and `$queue`; treat those fields as low-level protocol surfaces and prefer `SrpcServer.invoke()`, handlers, and `SrpcByteStream` for application work.
+
+Established-stream disconnect callbacks receive `disconnect`, `supersede`, `timeout`, or `badArg`. A rejected protocol-v2 duplicate closes with the `conflict` wire cause and rejects the connecting client with `SrpcConflictError` before activation, so connection and disconnect callbacks do not run for that rejected stream. `supersede` identifies a replaced connection, `timeout` identifies ping inactivity, and `badArg` identifies malformed or out-of-sequence messages. Outstanding invocations reject when their stream disconnects.
+
+## Byte Streams
+
+`SrpcByteStream` multiplexes Node `Duplex` streams over an SRPC connection.
+
+```ts
+import { SrpcByteStream } from '@zyno-io/ts-server-foundation';
+
+const sender = SrpcByteStream.createSender(stream);
+sender.write(Buffer.from('chunk'));
+sender.end();
+
+const receiver = SrpcByteStream.createReceiver(stream, senderId);
+receiver.on('data', chunk => {
+    // handle chunk
+});
+```
+
+Byte stream operations are carried in the SRPC envelope `byteStreamOperation` field. Pending receiver data is bounded and expires if a receiver is never attached.
+
+Data that arrives before `createReceiver()` is buffered for at most five seconds. A pending receiver is limited to 2 MiB, all pending receivers on one parent stream share a 2 MiB total limit, and at most 1,024 pending receiver IDs are retained. Exceeding a byte limit turns that pending receiver into an error; exceeding the count limit drops data for additional unknown IDs. These are protocol safety limits, not configurable application buffering.
+
+## Observers
+
+```ts
+import { registerSrpcObserver } from '@zyno-io/ts-server-foundation';
+
+const stop = registerSrpcObserver(entry => {
+    console.log(entry.type, entry.at);
+});
+```
+
+Observers receive connection entries with `{ type, stream, at }`, disconnection entries with `{ type, stream, cause, at }`, and message entries with `{ type, stream, direction, data, at }`. Message direction is relative to the server. Ping, byte-stream, request, and reply envelopes are observable. Observer exceptions are isolated from SRPC behavior. Call the returned function to unregister the observer.
+
+## Errors And Timeouts
+
+On the server, `SrpcError(message, true)` sets `userError` on the reply envelope. The current client rejects that reply as a plain `Error`, so callers can observe the message but not the user-error flag or `SrpcError` class. Errors thrown by client-side handlers are likewise sent without `userError`. Other thrown values are serialized as ordinary remote errors.
+
+`invoke()` defaults to a 30-second timeout on both client and server; a per-call timeout overrides it. The server closes with `badArg` for unknown or missing request IDs and decode failures. After the initial handshake, the client also closes for unknown or missing request IDs, while an undecodable server payload is logged and ignored.
+
+## Telemetry Boundary
+
+The shared envelope reserves a `trace` object with `traceId`, `spanId`, and `traceFlags`, but the current SRPC client and server do not automatically populate, continue, or export spans from that field. Applications that need cross-SRPC trace propagation must place trace context in their message contract and wrap handlers with `withRemoteSpan()` explicitly. HTTP/WebSocket connection establishment can still be observed by the installed HTTP instrumentation.

package/docs/content/telemetry.md

@@ -0,0 +1,173 @@
+# Telemetry
+
+TSF provides OpenTelemetry setup helpers and Sentry error reporting helpers.
+
+## OpenTelemetry Bootstrap
+
+Import the OpenTelemetry bootstrap from the dedicated subpath before importing modules you want instrumented.
+
+```ts
+import { init } from '@zyno-io/ts-server-foundation/otel';
+
+init();
+
+const { createApp } = await import('@zyno-io/ts-server-foundation');
+```
+
+`init()` is idempotent.
+
+## `init(options?)`
+
+```ts
+init({
+    serviceName: 'api',
+    serviceVersion: '1.2.3',
+    enableMetricsEndpoint: true,
+    httpIncomingRequestAttributeHook: request => ({
+        'http.request.host': request.headers.host ?? ''
+    })
+});
+```
+
+Options:
+
+| Option                             | Description                                                  |
+| ---------------------------------- | ------------------------------------------------------------ |
+| `serviceName`                      | Overrides package name for telemetry resource attributes.    |
+| `serviceVersion`                   | Overrides package version for telemetry resource attributes. |
+| `disabled`                         | Skips initialization when true.                              |
+| `instrumentations`                 | Additional OpenTelemetry instrumentations.                   |
+| `httpIncomingRequestAttributeHook` | Adds attributes to incoming HTTP spans.                      |
+| `enableMetricsEndpoint`            | Enables the Prometheus `/metrics` endpoint.                  |
+| `spanProcessors`                   | Custom trace span processors.                                |
+| `metricReaders`                    | Custom metric readers.                                       |
+
+## Environment
+
+| Key                                   | Description                               |
+| ------------------------------------- | ----------------------------------------- |
+| `OTEL_SDK_DISABLED`                   | Set to `true` to disable telemetry setup. |
+| `OTEL_DEBUG`                          | Enables OpenTelemetry diagnostic logging. |
+| `OTEL_EXPORTER_OTLP_ENDPOINT`         | OTLP endpoint for traces and metrics.     |
+| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`  | OTLP endpoint for traces.                 |
+| `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | OTLP endpoint for metric push.            |
+| `OTEL_METRICS_ENDPOINT_ENABLED`       | Enables or disables `/metrics`.           |
+
+Trace providers are installed when an OTLP trace endpoint or custom span processors are configured. Metric providers are installed when metric push, `/metrics`, or custom metric readers are configured.
+
+## Resources And Export
+
+TSF adds these resource attributes to installed trace and metric providers:
+
+| Attribute | Source |
+| --- | --- |
+| `service.name` | `serviceName`, then the working package name, then `unknown`. |
+| `service.version` | `serviceVersion`, then the working package version, then `unknown`. |
+| `deployment.environment` | `APP_ENV`. |
+| `host.name` | Operating-system hostname. |
+| `process.pid` | Current process ID. |
+
+The default OTLP trace exporter uses a `SimpleSpanProcessor` in development and a `BatchSpanProcessor` otherwise. OTLP metrics use a periodic reader with a 10-second export interval. Passing custom `spanProcessors` or `metricReaders` replaces the corresponding default processor/reader choice while retaining any separately enabled Prometheus endpoint.
+
+## Built-In Instrumentations
+
+Default instrumentation includes:
+
+- HTTP
+- Undici
+- DNS
+- ioredis
+- mysql2
+- pg
+
+HTTP context is connected to TSF request handling so request logs and Sentry events can include trace context.
+
+## Metrics Endpoint
+
+When enabled, `createApp()` registers:
+
+```http
+GET /metrics
+```
+
+The endpoint serves Prometheus text output from the OpenTelemetry Prometheus exporter. It authorizes the transport socket's peer address against private/local IP ranges and deliberately ignores `x-real-ip` and `x-forwarded-for`, even when normal request handling trusts proxy headers.
+
+| Status | Meaning                                                              |
+| ------ | -------------------------------------------------------------------- |
+| `200`  | Metrics collected successfully.                                      |
+| `403`  | Caller is not from an allowed private/local address.                 |
+| `503`  | Metrics endpoint is enabled but no Prometheus exporter is available. |
+| `500`  | The exporter failed while collecting or rendering metrics.           |
+
+## Span Helpers
+
+Span helpers are exported from the package root.
+
+```ts
+import {
+    disableActiveTrace,
+    getActiveSpan,
+    getTraceContext,
+    getTracer,
+    isTelemetryInitialized,
+    isTracingInstalled,
+    setSpanAttributes,
+    withLinkedRootSpan,
+    withRemoteSpan,
+    withRootSpan,
+    withSpan
+} from '@zyno-io/ts-server-foundation';
+
+await withSpan('work', { itemId: '123' }, async () => {
+    setSpanAttributes({ result: 'ok' });
+});
+
+await withRootSpan('job', async () => runJob());
+
+await withRemoteSpan('rpc', { traceparent }, undefined, async () => handleRpc());
+```
+
+Helpers are no-ops when tracing is not installed.
+
+`withSpan()` creates a child of the active span, `withRootSpan()` starts without a parent, `withLinkedRootSpan()` starts a root span with explicit links, and `withRemoteSpan()` continues a valid remote context. Helper callbacks may be synchronous or asynchronous. Thrown errors are recorded on the span, set its status to error, end it, and are rethrown unchanged. `setSpanAttributes()` and `disableActiveTrace()` affect only the active span/context.
+
+`SpanInfo`, accepted by `withRemoteSpan()`, is either `{ traceId, spanId, traceFlags? }`, `{ traceparent }`, or `undefined`. Invalid `traceparent` input falls back to a normal child span rather than throwing. `SpanLinkRef`, accepted by `withLinkedRootSpan()`, adds optional link attributes as well as trace/span identifiers.
+
+`isTelemetryInitialized()` reports whether initialization was attempted. `isTracingInstalled()` reports whether a tracer provider is actually active; initialization can be complete without tracing when no trace exporter or custom processor was configured.
+
+## Shutdown And Tests
+
+```ts
+import { resetTelemetryForTests, shutdownTelemetry } from '@zyno-io/ts-server-foundation/otel';
+
+await shutdownTelemetry();
+resetTelemetryForTests();
+```
+
+`shutdownTelemetry()` attempts to shut down every installed provider and clears global state even when shutdown fails. It then rethrows the single provider failure or an `AggregateError` containing multiple failures.
+
+`disabled: true` and `OTEL_SDK_DISABLED=true` return before initialization. Custom `spanProcessors`, `metricReaders`, and `instrumentations` are installed alongside the appropriate provider; registered instrumentations are disabled again during shutdown/reset. Passing `enableMetricsEndpoint: false` overrides the endpoint environment flag but does not disable OTLP push metrics or custom metric readers.
+
+## Sentry
+
+Sentry helpers are exported from the package root.
+
+```ts
+import { flushSentry, installSentry, isSentryInstalled } from '@zyno-io/ts-server-foundation';
+
+installSentry({ dsn: 'https://example@sentry.io/1' });
+
+if (isSentryInstalled()) {
+    await flushSentry();
+}
+```
+
+`createApp()` installs Sentry automatically when `SENTRY_DSN` is configured. Sentry events include the active OpenTelemetry trace context when tracing is already installed at the time Sentry is initialized, so initialize OTel before creating the app or calling `installSentry()`.
+
+Relevant environment keys:
+
+| Key                        | Description                                   |
+| -------------------------- | --------------------------------------------- |
+| `SENTRY_DSN`               | Sentry project DSN.                           |
+| `APP_ENV`                  | Sentry environment.                           |
+| `ALERTS_SLACK_WEBHOOK_URL` | Optional Slack webhook used by alert logging. |