@observertc/observer-js 1.0.0-beta.4 → 1.0.0-beta.6

package/README.md

@@ -15,8 +15,9 @@ and emits a single, unified stream of typed events the application can react to.
 > agent) should be able to integrate the library, or develop it further, from this file alone.
 > A companion doc, [`docs/logging.md`](./docs/logging.md), covers logging integration in depth.
 
-> **Packaging:** the package is **ESM-only** and **server-side** (Node.js ≥ 16). Everything —
-> including the built-in file sink — is exported from the single `@observertc/observer-js` entry.
+> **Packaging:** server-side, **Node.js ≥ 22**, shipped as a **dual ESM + CommonJS** build — so it
+> works whether your project uses `import` (ESM) or `require()` (CommonJS). Everything — including
+> the built-in file sink — is exported from the single `@observertc/observer-js` entry.
 
 ---
 
@@ -51,22 +52,20 @@ npm install @observertc/observer-js
 yarn add @observertc/observer-js
 ```
 
-**ESM-only, server-side.** The package ships ES Modules (`import`, not `require()`) and targets
-**Node.js ≥ 16**. Use it from ESM code (`"type": "module"`, or `.mjs`), or from TypeScript
-compiled to ESM. Everything is exported from the single `@observertc/observer-js` entry:
+**Server-side, Node.js ≥ 22, dual ESM + CommonJS.** The package ships both module formats, so it
+works the same whether your project is ESM or CommonJS — your import line is unchanged either way:
 
 ```ts
 import { Observer, ClientSample, createJsonlFileSinkFactory } from '@observertc/observer-js';
 ```
 
-Written in TypeScript; ships type declarations alongside the build (`dist/index.d.mts`). Runtime
+In an ESM project this resolves to the `.mjs` build; in a CommonJS project (where TypeScript
+compiles your `import` down to `require()`) it resolves to the `.js` build. Everything is exported
+from the single `@observertc/observer-js` entry. Written in TypeScript; ships type declarations for
+both formats (`dist/index.d.ts` for `require`, `dist/index.d.mts` for `import`). Runtime
 dependencies: `@bufbuild/protobuf`, `events`, `uuid`. The library does **not** bundle a logger or
 any transport — see [Logging](#logging).
 
-> **Note on `require()`.** Being ESM-only, the package can't be loaded with CommonJS
-> `require('@observertc/observer-js')`; consume it with `import` (or `await import()` from a CJS
-> module). If you need a CommonJS build, a dual ESM+CJS output is a small change — ask.
-
 `ClientSample` and friends are re-exported from this package, and are also published as the
 shared schema in [`@observertc/schemas`](https://github.com/observertc/schemas); samples
 produced on the client (e.g. by `@observertc/client-monitor-js`) conform to the same shape.
@@ -105,9 +104,8 @@ import { Observer, ClientSample } from '@observertc/observer-js';
 
 // 1. Create an observer.
 const observer = new Observer({
-  // how often the observer aggregates call/client metrics:
-  updatePolicy: 'update-on-interval',
-  updateIntervalInMs: 5000,
+  // when the observer aggregates call/client metrics:
+  updatePolicy: 'update-when-all-call-updated',
   // default policy applied to calls created automatically by accept():
   defaultCallUpdatePolicy: 'update-on-any-client-updated',
   // optional auto-teardown:
@@ -178,7 +176,7 @@ client getStats()  ──►  ClientSample  ──►  observer.accept(sample, c
 |-------|-----------|------------------------|-------|
 | `Observer` | `new Observer(config?)` | — (root) | `observedCalls: Map<string, ObservedCall>`, global counters, the event bus |
 | `ObservedCall` | `observer.createObservedCall(settings)` / lazily by `accept` | `observedCalls` | `observedClients: Map<string, ObservedClient>`, call-wide metrics, `detectors`, `scoreCalculator` |
-| `ObservedClient` | `call.createObservedClient(settings)` / lazily | `observedClients` | `observedPeerConnections: Map<string, ObservedPeerConnection>`, per-client metrics, `report` |
+| `ObservedClient` | `call.createObservedClient(settings)` / lazily | `observedClients` | `observedPeerConnections: Map<string, ObservedPeerConnection>`, per-client metrics |
 | `ObservedPeerConnection` | lazily, from `sample.peerConnections[]` | `observedPeerConnections` | the 15 sub-stat maps below, transport/RTT/bitrate metrics |
 | Sub-stats | lazily, from the `PeerConnectionSample` | maps on the PC | individual WebRTC stat objects |
 
@@ -207,12 +205,49 @@ fields from the schema plus derived fields (deltas, bitrates).
 
 The single entry point. It:
 
-1. drops + emits `sample-rejected` if the observer is closed, or `callId`/`clientId` is missing;
-2. gets or lazily creates the `ObservedCall` and `ObservedClient`;
-3. shallow-merges `context` into the created/looked-up entity `appData`;
-4. delegates to `client.accept(sample, context)`, which fans out to each
+1. drops + emits `sample-rejected` if the observer is closed;
+2. runs the sample through the **global accept-middleware chain** (see below);
+3. (chain terminal) drops + emits `sample-rejected` if `callId`/`clientId` is missing;
+4. gets or lazily creates the `ObservedCall` and `ObservedClient` (their `appData` comes from the
+   configured factories, never from `context`);
+5. delegates to `client.accept(sample, context)`, which fans out to each
    `ObservedPeerConnection.accept(pcSample, context)`.
 
+### Accept middlewares (global pre-dispatch hook)
+
+`observer.addAcceptMiddleware(...)` registers middlewares run on **every** sample inside
+`accept()`, in order, **before** the sample is dispatched to any call or client. Each middleware
+gets a `{ sample, context }` payload; it can inspect or mutate the sample (set/normalize
+`callId`/`clientId`, enrich, redact) or the context, then call `next(payload)` to continue.
+**Not calling `next` drops the sample** — nothing is created and no event fires. A throwing
+middleware is caught and warns (the sample is dropped), never crashing `accept()`.
+
+```ts
+import { Observer, AcceptMiddleware } from '@observertc/observer-js';
+
+const observer = new Observer();
+
+// derive callId/clientId from the app's own attachment, before dispatch
+const route: AcceptMiddleware = ({ sample }, next) => {
+  sample.callId ??= sample.attachments?.roomId as string;
+  sample.clientId ??= sample.attachments?.peerId as string;
+  next({ sample });
+};
+
+// drop samples from a blocklisted client (never dispatched)
+const filter: AcceptMiddleware = (payload, next) => {
+  if (blocked.has(payload.sample.clientId)) return;   // no next() => dropped
+  next(payload);
+};
+
+observer.addAcceptMiddleware(route, filter);
+// observer.removeAcceptMiddleware(route);
+```
+
+This is a lightweight global injection point, distinct from the larger (not-yet-built)
+`ClientSampleProcessor` pipeline in the roadmap. When no middleware is registered, `accept()`
+dispatches directly with no overhead.
+
 ### `context` (the `AcceptContext`)
 
 ```ts
@@ -260,9 +295,9 @@ These return `undefined` (and warn) when the parent is closed; `createObservedCa
 ## Update policies
 
 "Update" means *recompute aggregated metrics and emit the `*-updated` event* at that level.
-Both the observer and each call have a configurable trigger. The `update-on-interval` policy
-**requires** an interval and this is enforced at compile time (a discriminated union); at
-runtime a missing interval warns and falls back rather than throwing.
+Both the observer and each call have a configurable trigger. Updates are **event-driven** — there
+is no built-in timer. An app that wants a fixed cadence can call `observer.update()` /
+`call.update()` from its own `setInterval`.
 
 **Observer-level** (`ObserverConfig.updatePolicy`, default `update-when-all-call-updated`):
 
@@ -270,7 +305,6 @@ runtime a missing interval warns and falls back rather than throwing.
 |--------|-------------------------------------|
 | `update-on-any-call-updated` | any call updates |
 | `update-when-all-call-updated` | every call has updated since the last observer update |
-| `update-on-interval` | a timer fires (`updateIntervalInMs` required) |
 
 **Call-level** (`ObservedCallSettings.updatePolicy`, defaulted from
 `ObserverConfig.defaultCallUpdatePolicy`):
@@ -279,7 +313,6 @@ runtime a missing interval warns and falls back rather than throwing.
 |--------|--------------------------------|
 | `update-on-any-client-updated` | any client in the call updates |
 | `update-when-all-client-updated` | every client has updated since the last call update |
-| `update-on-interval` | a timer fires (`updateIntervalInMs` required) |
 
 ---
 
@@ -354,7 +387,6 @@ additional field(s) on top of that scope.
 | `client-metadata` | `{ metaData: ClientMetaData }` | a client meta item arrived |
 | `client-extension-stats` | `{ extensionStats: ExtensionStat }` | an app-defined extension stat arrived |
 | `client-event` | `{ event: ClientEvent }` | any client event was processed |
-| `client-track-report` | `{ report: TrackReport }` | a track was removed; final per-track report |
 
 #### Peer-connection level — scope `{ observer, observedCall, observedClient, observedPeerConnection }`
 
@@ -404,12 +436,9 @@ listen to them, but prefer the bus equivalents above for application logic.
 ```ts
 new Observer<AppData>(config?: ObserverConfig<AppData>)
 
-type ObserverConfig<AppData = Record<string, unknown>> =
-  ( { updatePolicy: 'update-on-interval'; updateIntervalInMs: number }
-  | { updatePolicy?: 'update-on-any-call-updated' | 'update-when-all-call-updated'; updateIntervalInMs?: number }
-  ) & {
+type ObserverConfig<AppData = Record<string, unknown>> = {
+    updatePolicy?: 'update-on-any-call-updated' | 'update-when-all-call-updated';
     defaultCallUpdatePolicy?: ObservedCallSettings['updatePolicy'];
-    defaultCallUpdateIntervalInMs?: number;
     appData?: AppData;
     closeClientIfIdleForMs?: number;
     closeCallIfEmptyForMs?: number;
@@ -419,6 +448,8 @@ type ObserverConfig<AppData = Record<string, unknown>> =
     createClientAppData?: (p: { clientId: string; observedCall: ObservedCall }) => Record<string, unknown>;
     // sink factory — produces a per-client sink that receives every accepted sample (see Sinks).
     createClientSink?: (p: { clientId: string; observedCall: ObservedCall }) => ClientSampleSink | undefined;
+    // track-resolver factory — produces a call's RemoteTrackResolver (see Remote track resolution).
+    createTrackResolver?: (observedCall: ObservedCall) => RemoteTrackResolver | undefined;
   };
 ```
 
@@ -439,6 +470,7 @@ const observer = new Observer({
 Key members:
 
 - `accept(sample: ClientSample, context?: AcceptContext): void`
+- `addAcceptMiddleware(...mw: AcceptMiddleware[]): this` / `removeAcceptMiddleware(...mw): this` — global pre-dispatch sample hooks (see [Accept middlewares](#accept-middlewares-global-pre-dispatch-hook))
 - `getObservedCall<T>(callId): ObservedCall<T> | undefined`
 - `createObservedCall<T>(settings): ObservedCall<T> | undefined`
 - `getOrCreateObservedCall<T>(settings): ObservedCall<T> | undefined`
@@ -455,13 +487,10 @@ Key members:
 ### `ObservedCall`
 
 ```ts
-type ObservedCallSettings<AppData = Record<string, unknown>> =
-  ( { updatePolicy: 'update-on-interval'; updateIntervalInMs: number }
-  | { updatePolicy?: 'update-on-any-client-updated' | 'update-when-all-client-updated'; updateIntervalInMs?: number }
-  ) & {
+type ObservedCallSettings<AppData = Record<string, unknown>> = {
+    updatePolicy?: 'update-on-any-client-updated' | 'update-when-all-client-updated';
     callId: string;
     appData?: AppData;
-    remoteTrackResolvePolicy?: 'p2p' | 'mediasoup-sfu' | 'none';
     closeCallIfEmptyForMs?: number;
   };
 ```
@@ -474,7 +503,7 @@ Key members:
 - `addIssue(issue: ClientIssue): void` — raise a **call-level** issue → emits `call-issue`
 - `readonly detectors: Detectors` — server-side detector registry (empty by default; see [Detectors](#detectors-server-side-extension-point))
 - `scoreCalculator: ScoreCalculator`, `get score()`, `readonly calculatedScore`
-- `remoteTrackResolver?: RemoteTrackResolver`
+- `remoteTrackResolver?: RemoteTrackResolver` — set from `ObserverConfig.createTrackResolver` at call creation (see [Remote track resolution](#remote-track-resolution-mediasoup--sfu))
 - aggregates: `numberOfIssues`, `numberOfPeerConnections`, `numberOfInboundRtpStreams`,
   `numberOfOutboundRtpStreams`, `numberOfDataChannels`, `maxNumberOfClients`,
   `clientsUsedTurn: Set<string>`, `startedAt?`, `endedAt?`, `closedAt?`, `closed`
@@ -508,7 +537,6 @@ Key members:
 - Per-tick deltas: `deltaReceivedAudioBytes`, `deltaSentAudioBytes`, … (see source for the full set)
 - Lifecycle: `joinedAt?`, `leftAt?`, `closedAt?`, `closed`, `get score()`
 - Metadata: `browser?`, `engine?`, `platform?`, `operationSystem?`, `mediaDevices`, `mediaConstraints`
-- `readonly report: ClientReport` — cumulative report (byte/packet totals + RTT & score distributions)
 - `accept(sample, context?)`, `close()`
 
 ### `ObservedPeerConnection`
@@ -589,9 +617,6 @@ mediasoup set `PRODUCER_*` / `CONSUMER_*` / `DATA_PRODUCER_*` / `DATA_CONSUMER_*
 `MEDIA_DEVICES_SUPPORTED_CONSTRAINTS`, `USER_MEDIA_ERROR`, `LOCAL_SDP`, `OPERATION_SYSTEM`,
 `ENGINE`, `PLATFORM`, `BROWSER`.
 
-`Reports` exported: `ClientReport` (cumulative per-client totals + RTT/score distributions) and
-`TrackReport` (per-track final report, delivered on `client-track-report`).
-
 ---
 
 ## Detectors (server-side extension point)
@@ -644,29 +669,47 @@ class Detectors {
 ## Remote track resolution (mediasoup / SFU)
 
 In an SFU, one participant's **outbound** track is delivered to other participants as **inbound**
-tracks. To correlate them server-side, set `remoteTrackResolvePolicy: 'mediasoup-sfu'` on the
-call settings. The built-in `MediasoupRemoteTrackResolver` subscribes to the bus (filtered to
-its call) and maps producer↔consumer using track `attachments` (`producerId`, `consumerId`):
+tracks (one **publisher** → many **subscribers**). Correlation is **opt-in** per observer: set
+`ObserverConfig.createTrackResolver`, a factory invoked when each call is created that returns the
+call's `RemoteTrackResolver` (or `undefined` for none).
+
+`RemoteTrackResolver` is a generic, strategy-driven class. It subscribes to the bus (filtered to
+its call) and links tracks by **publisher id** — the link key — maintaining the links directly on
+the tracks: `inboundTrack.remoteOutboundTrack` and `outboundTrack.remoteInboundTracks: Set`.
 
 ```ts
-const call = observer.createObservedCall({ callId, remoteTrackResolvePolicy: 'mediasoup-sfu' });
-// later, given an inbound track:
-const source = inboundTrack.getRemoteOutboundTrack();     // the producing ObservedOutboundTrack
-const consumers = outboundTrack.getRemoteInboundTracks(); // the consuming ObservedInboundTrack[]
+import { Observer, createDefaultMediasoupRemoteTrackResolverFactory } from '@observertc/observer-js';
+
+const observer = new Observer({
+  createTrackResolver: createDefaultMediasoupRemoteTrackResolverFactory(),
+});
+
+// later, given tracks (links are kept up to date as tracks come and go):
+const source    = inboundTrack.remoteOutboundTrack;       // the publishing ObservedOutboundTrack
+const receivers = [ ...outboundTrack.remoteInboundTracks ]; // the subscribing ObservedInboundTrack[]
 ```
 
-Custom topologies can implement the `RemoteTrackResolver` interface and assign
-`call.remoteTrackResolver`:
+Two built-in factories ship: `createDefaultMediasoupRemoteTrackResolverFactory()` (publisher =
+`attachments.producerId`, subscriber = `attachments.consumerId`) and
+`createP2pRemoteTrackResolverFactory()` (matches by RTP **SSRC**, preserved end-to-end in p2p).
+
+For any other topology, build a `RemoteTrackResolver` with your own key resolvers — the publisher
+id is just whatever links a subscribed track to the published one:
 
 ```ts
-interface RemoteTrackResolver {
-  resolveRemoteOutboundTrack(inboundTrack: ObservedInboundTrack): ObservedOutboundTrack | undefined;
-  resolveRemoteInboundTracks(outboundTrack: ObservedOutboundTrack): ObservedInboundTrack[] | undefined;
-}
+import { Observer, RemoteTrackResolver } from '@observertc/observer-js';
+
+const observer = new Observer({
+  createTrackResolver: (observedCall) => new RemoteTrackResolver(observedCall, {
+    resolveOutboundTrackPublisherId: (out) => out.attachments?.mediaId as string | undefined,
+    resolveInboundTrackPublisherId:  (inb) => inb.attachments?.mediaId as string | undefined,
+    resolveInboundTrackSubscriberId: (inb) => inb.attachments?.subId  as string | undefined, // optional
+  }),
+});
 ```
 
-The application is expected to put `direction` (`'send'`/`'recv'`), `producerId`, `consumerId`,
-and `label` into `PeerConnectionSample.attachments` / track `attachments`.
+For the mediasoup factory, the application puts `producerId` / `consumerId` (and optionally
+`direction`, `label`) into the track `attachments`.
 
 ---
 
@@ -733,12 +776,32 @@ observer.on('client-sink-created', ({ observedClient, sink }) => {
 |--------|-----------|-------|
 | `createJsonlFileSinkFactory` | `({ directory, flags?, getFileName?, serializeSample? }) => ClientSampleSinkFactory` | per-client JSONL files; path defaults to `${callId}__${clientId}.jsonl` under `directory` (which **must exist**) |
 | `createJsonlFileSink` | `({ path, flags?, serializeSample? }) => ClientSampleSink` | a single JSONL file; wraps `fs.WriteStream` and re-emits its `close`/`finish`/`drain`/`error` |
-| `JsonlFileSink` | `class extends ClientSampleSink` | the underlying class, if you want to construct it directly |
+| `JsonlFileSink` | `class extends ClientSampleSink` | the underlying class; exposes `readonly path` so a `close` handler knows which file is ready |
 | `createInMemorySink` / `InMemorySink` | `(samples?: ClientSample[]) => InMemorySink` | collects the accepted **sample objects** into `.samples: ClientSample[]`; emits `close` on `end()` |
 
 `serializeSample?: (sample: ClientSample) => string` overrides the default `JSON.stringify` for
 the JSONL sinks (e.g. to redact or reshape before writing).
 
+### Reading sink-specific info (e.g. the file path)
+
+The bus hands you the sink as the base `ClientSampleSink`. To read information specific to a sink
+type — for a file sink, where it was written — **narrow with `instanceof`** and read the sink's
+public fields. `JsonlFileSink` exposes `path`:
+
+```ts
+import { JsonlFileSink } from '@observertc/observer-js';
+
+observer.on('client-sink-created', ({ observedClient, sink }) => {
+  if (sink instanceof JsonlFileSink) {
+    const { path } = sink;                          // the file this client's samples go to
+    sink.once('close', () => uploadFile(path));     // close = flushed & fd closed → ready
+  }
+});
+```
+
+The general pattern: each concrete sink exposes whatever it wants as `public readonly` fields, and
+consumers narrow (`instanceof YourSink`) to read them. Your own sinks do the same.
+
 ### Writing your own sink
 
 Subclass `ClientSampleSink` and emit the lifecycle events yourself — for any non-file
@@ -800,14 +863,15 @@ filtering, per-module routing, and full silencing.
 
 The library **warns and degrades; it does not throw** on operational problems:
 
-- Bad config (`update-on-interval` without an interval) → warn + safe fallback policy.
 - `createObservedCall` / `createObservedClient` on a closed parent → warn + return `undefined`.
 - Duplicate id → warn + return the **existing** instance.
 - `accept()` on a closed client → warn + no-op.
 - Sample missing `callId`/`clientId`, or observer closed → `sample-rejected` event.
+- A throwing accept-middleware → warn + drop that sample (never crashes `accept()`).
 
-Therefore `create*` and `getOrCreate*` return `T | undefined`; **guard the result.** The only
-remaining `throw`s are internal invariants in the unused `Middleware` utility.
+Therefore `create*` and `getOrCreate*` return `T | undefined`; **guard the result.** The
+`Middleware` utility's internal invariants (e.g. calling `next()` twice) throw, but those throws
+are caught by `accept()` and surfaced as a warning.
 
 ---
 
@@ -844,7 +908,11 @@ export type { JsonlFileSinkOptions, JsonlFileSinkFactoryOptions } from './sinks/
 export { InMemorySink, createInMemorySink } from './sinks/InMemorySink';
 
 export { Middleware } from './common/Middleware';
-export type { TrackReport, ClientReport } from './Reports';
+
+// remote track correlation
+export { RemoteTrackResolver } from './utils/RemoteTrackResolver';
+export type { RemoteTrackResolvers, RemoteTrackResolverFactory } from './utils/RemoteTrackResolver';
+export { createDefaultMediasoupRemoteTrackResolverFactory, createP2pRemoteTrackResolverFactory } from './utils/RemoteTrackResolverFactories';
 ```
 
 ---
@@ -853,23 +921,23 @@ export type { TrackReport, ClientReport } from './Reports';
 
 ```bash
 yarn install
-yarn build       # tsup → dist/ (ESM: index + sinks, with .d.mts types & sourcemaps)
+yarn build       # tsup → dist/ (dual ESM .mjs + CJS .js, single entry, .d.ts/.d.mts + sourcemaps)
 yarn lint        # eslint -c .eslintrc.json "src/**/*.ts"
 yarn typecheck   # tsc --noEmit
 yarn test        # jest
 ```
 
 The build is driven by [`tsup`](https://tsup.egoist.dev) (config in `tsup.config.ts`): a single
-entry (`src/index.ts`), ESM output to `dist/` with `.d.mts` types and sourcemaps. CI
-(`.github/workflows/ci.yml`) runs lint + typecheck + **build** + test on every push/PR.
+entry (`src/index.ts`), dual ESM + CommonJS output to `dist/` (`index.mjs` / `index.js`) with
+`.d.mts` / `.d.ts` types and sourcemaps, targeting Node 22. CI (`.github/workflows/ci.yml`) runs
+lint + typecheck + **build** + test on every push/PR.
 
 **Project layout** (`src/`): `Observer.ts`, `ObservedCall.ts`, `ObservedClient.ts`,
 `ObservedPeerConnection.ts`, the `Observed*` sub-stat classes, `ObserverEvents.ts` (the typed
 event map + scope types), `detectors/` (`Detector`, `Detectors`), `scores/`, `updaters/`
 (update-policy strategies), `utils/` (remote-track resolvers), `common/` (`logger`, `utils`,
-`Middleware`), `schema/` (sample/event/meta types), and `sinks/` (the import-safe
-`ClientSampleSink` base + the Node-only `JsonlFileSink` / `InMemorySink`, re-exported via the
-`@observertc/observer-js/sinks` subpath).
+`Middleware`), `schema/` (sample/event/meta types), and `sinks/` (the `ClientSampleSink` base +
+`JsonlFileSink` / `InMemorySink`, re-exported from the package root).
 
 **Conventions to follow when developing further:**