@qencode/calls 0.2.0

package/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

package/README.md

@@ -0,0 +1,242 @@
+# @qencode/calls
+
+Low-latency 1:1 audio and video calls inside your own UI. The SDK takes a short-lived
+participant credential minted by **your backend**, connects, renders media into views you
+place, and reports call quality. It ships no screens and no theme. Version 0.2.0, pre-release.
+
+## Quickstart
+
+### 1. Your backend mints a credential
+
+Never put a Qencode API key in a browser. Your server calls the Qencode calls API and hands
+the token response to the page unchanged:
+
+```
+POST /v1/access_token/{api_key}          -> { access_token }              (project JWT, 24 h)
+POST /v1/calls                           -> { call: { id, ... } }         Authorization: Bearer <access_token>
+POST /v1/calls/{id}/tokens               -> { token, url, regions, identity, room_name, expires_at }
+     body { "identity": "alice", "name": "Alice", "role": "A", "ttl": 600 }
+```
+
+Return that last object to the browser (optionally add `call_id` and `api_base`). See
+`examples/backend-node/server.js` for a complete 60-line version.
+
+### 2. The page joins
+
+```html
+<script type="module">
+  import { Call } from 'https://unpkg.com/@qencode/calls/dist/qencode-calls.esm.bundle.js';
+
+  const cred = await fetch('/api/calls/join?as=alice').then(r => r.json());
+  const call = Call.create(cred, { videoProfile: 'p540_60' });
+
+  call.on('remoteVideo', track => { document.querySelector('#remote').track = track; });
+  call.on('stateChanged', (state, reason) => console.log(state, reason ?? ''));
+  call.on('credentialExpiring', async () => {
+    call.updateCredential(await fetch('/api/calls/refresh').then(r => r.json()));
+  });
+
+  await call.connect();
+  document.querySelector('#self').track = call.localVideo;
+</script>
+
+<qencode-video id="remote"></qencode-video>
+<qencode-video id="self" mirror></qencode-video>
+```
+
+`npm install @qencode/calls` gives the same API as an ES module with TypeScript types.
+
+## Builds and size
+
+| File | Use | Size, gzipped |
+|---|---|---|
+| `dist/qencode-calls.esm.js` | `import` through a bundler; the engine (`livekit-client`) is a dependency your bundler resolves and dedupes | ~11 KB |
+| `dist/qencode-calls.esm.bundle.js` | `<script type="module">` with no bundler; engine included | ~154 KB |
+| `dist/qencode-calls.umd.js` | classic `<script>`; exposes `window.QencodeCalls`; engine included | ~154 KB |
+
+The SDK itself is about 11 KB gzipped. The self-contained builds are dominated by the engine,
+which is about 145 KB gzipped on its own.
+
+## Call
+
+| Member | Meaning |
+|---|---|
+| `Call.create(credential, options)` | Builds a call. No network yet. |
+| `connect()` | Probes regions when the credential lists several, connects, publishes camera and microphone per options. Rejects with a `CallError`. |
+| `leave()` | Unpublishes, disconnects, releases devices. Idempotent. |
+| `state` | `idle`, `connecting`, `connected`, `reconnecting`, `ended`; `endReason` says why it ended. |
+| `localVideo`, `remoteVideo`, `remoteAudio` | Track handles with `attach(element)` and `detach()`. Remote audio is played by the SDK; the handle is for metering or custom output. |
+| `setMicrophoneEnabled(bool)`, `setCameraEnabled(bool)` | Mute keeps the track published, so unmute is instant. |
+| `setVideoProfile(name)` | Switches tier mid-call. Simulcast changes republish; other changes restart capture in place. |
+| `setLatencyMode('lowest' \| 'smooth')` | Jitter buffer target, degradation preference and audio framing in one switch. Clears a `setJitterBufferTarget` pin. |
+| `setJitterBufferTarget(ms)` | Pins the receiver's jitter buffer target regardless of the mode. Returns which property took it: `jitterBufferTarget`, `playoutDelayHint`, or `unsupported` (Firefox). |
+| `setVideoEncoding({ codec, simulcast })` | Switches codec or simulcast mid-call; the video track is republished. |
+| `setVideoSource(source \| null)` | Replaces the published video with a custom track or factory, or returns to the camera. See below. |
+| `devices` | `list()`, `setCamera(id)`, `setMicrophone(id)`, `setSpeaker(id)`, `onChange(handler)`, `canSelectSpeaker`. |
+| `sendMessage(payload, reliable = true)` | Up to 15 KB, 30 per second, to the peer. Strings and JSON objects arrive as sent, `Uint8Array` as bytes. |
+| `stats` | Latest quality snapshot, refreshed every second. |
+| `peer` | Identity, display name and mute state of the other human, or `null`. |
+| `videoCodec`, `simulcast`, `jitterBufferTargetMs`, `jitterBufferSupport`, `regionProbe` | What is in force: codec, simulcast, jitter target in ms, which receiver property took it, and the probe results by region name (`null` when no probe ran). |
+| `updateCredential(credential)` | Fresh token for the same call and identity, used by the next reconnect. |
+| `on(event, handler)` returns an unsubscribe function; `off`, `once`. |
+
+### Options
+
+`adaptiveStream` (default `true`) lets the engine match what is sent and received to how the
+video is displayed: a phone rendering a desktop's 720p pulls the 540p layer, and a hidden or
+background video element pauses that track on the server until it is visible again. Two tabs
+on one machine therefore show a black remote picture in whichever tab is in the background.
+Set it to `false` to always send and receive the full profile, which is what a measurement
+bench wants and what a customer UI rarely does.
+
+| Option | Default | Notes |
+|---|---|---|
+| `audio`, `video` | `true` | Publish on connect. |
+| `videoProfile` | `p540_60` | See profiles. |
+| `latencyMode` | `lowest` | See latency modes. |
+| `region` | auto | Pin a region name from the credential. |
+| `telemetry` | `true` | Post quality stats to Qencode every 5 s. |
+| `autoReconnect` | `true` | Resume for up to 60 s after a network change, then end with reason `network`. |
+| `endOnPeerLeft` | `true` | 1:1 semantics: the call ends when the peer leaves. |
+| `cameraId`, `microphoneId` | system default | Initial devices. |
+| `apiBase` | `https://api.qencode.com` | Telemetry target; `credential.api_base` also works. |
+| `logLevel` | `warn` | Never logs media or credentials. |
+| `videoSource` | camera | A `MediaStreamTrack` or a factory `(profile) => MediaStreamTrack`; see custom video source. |
+| `codec` | `h264` | `h264`, `vp8`, `vp9`, `av1`. The engine falls back to VP8 when a peer cannot decode the choice. |
+| `simulcast` | per profile | Force simulcast layers on or off. |
+| `jitterBufferTargetMs` | per latency mode | Pin the receiver's jitter buffer target; `0` asks for the browser's floor. |
+| `telemetryExtra` | none | `(direction) => fields` appended to every telemetry row; see telemetry. |
+| `forceRelay` | `false` | Connect through TURN only, to measure the relay path. |
+
+### Custom video source
+
+Pass `videoSource` to publish something other than the raw camera: a canvas with an overlay or a
+timestamp, a processed camera track (virtual background, filters), or a screen.
+
+```ts
+const call = Call.create(cred, {
+  videoSource: (profile) => {                 // called on every publish and republish
+    canvas.width = profile.width; canvas.height = profile.height;
+    return canvas.captureStream(profile.fps).getVideoTracks()[0];
+  },
+});
+```
+
+A factory is called with the profile so it can size its output, and is called again on
+`setVideoProfile`, `setVideoEncoding` and `setVideoSource`; the SDK stops the track it returned
+when it is replaced or the call ends. A `MediaStreamTrack` passed directly is published as is,
+gets new encoding parameters on a profile switch, and is never stopped by the SDK. Encoding
+caps, codec, simulcast and mute work the same either way. `devices.setCamera` is refused while a
+custom source is published: switch cameras in the code that produces the track.
+
+### Events
+
+| Event | Payload |
+|---|---|
+| `stateChanged` | `(state, endReason \| null)` |
+| `peerJoined`, `peerLeft` | `(peer)` |
+| `remoteVideo`, `remoteAudio` | `(track \| null)`; fires again after a reconnect with the replacement track |
+| `peerMuted` | `('audio' \| 'video', muted)` |
+| `stats` | `(snapshot)` once per second |
+| `qualityChanged` | `('good' \| 'fair' \| 'poor', 'recv' \| 'send')`, with hysteresis |
+| `message` | `(payload, reliable)` |
+| `credentialExpiring` | `(expiresAt)` two minutes before expiry, and on reconnect if already expired |
+| `devicesChanged` | `(list)` |
+| `moderation` | reserved; no-op until the moderation agent ships |
+| `error` | `(CallError)` non-fatal errors; fatal ones end the call |
+
+### Errors
+
+`CallError` has `code`, `message`, `retryable` and `cause`. Codes: `credentialInvalid`,
+`credentialExpired`, `roomFull`, `roomClosed`, `permissionDenied`, `deviceUnavailable`,
+`network`, `unsupported`, `internal`.
+
+A media server refuses a third participant by closing the websocket without a reason a
+browser can read. `connect()` then asks the server's validate endpoint whether the credential
+is still accepted; if it is, the rejection was capacity and the error is `roomFull`, otherwise
+it stays `network`. That is one extra request, on that failure path only.
+
+## Profiles
+
+| Profile | Resolution | fps | Cap |
+|---|---|---|---|
+| `audioOnly` | none | | 48 kbps |
+| `p360_30` | 640×360 | 30 | 450 kbps |
+| `p540_30` | 960×540 | 30 | 800 kbps |
+| `p540_60` (default) | 960×540 | 60 | 1.2 Mbps |
+| `p720_30` | 1280×720 | 30 | 1.2 Mbps |
+| `p720_60` | 1280×720 | 60 | 1.8 Mbps |
+| `p1080_30` | 1920×1080 | 30 | 2.2 Mbps |
+
+Hardware H.264 everywhere it exists, VP8 as the fallback. Simulcast is on at 720p and above.
+The captured frame is cover-fitted to the profile's aspect ratio, never stretched.
+
+## Latency modes
+
+`lowest` (default) asks the browser for its minimum jitter buffer, drops resolution before
+frame rate under pressure, and uses 10 ms audio frames where the platform allows. `smooth`
+lets the buffer grow to 150 ms and drops frame rate before resolution. Switch mid-call.
+
+## Stats
+
+Every second, `call.stats` and the `stats` event carry:
+
+| Field | Source |
+|---|---|
+| `recv.rttMs`, `send.rttMs` | selected ICE candidate pair, round trip to the media node |
+| `recv.jitterBufferMs` | Δ`jitterBufferDelay` / Δ`jitterBufferEmittedCount` |
+| `recv.decodeMs`, `send.encodeMs` | Δ`totalDecodeTime` / Δ`framesDecoded`, Δ`totalEncodeTime` / Δ`framesEncoded` |
+| `recv.fps`, `width`, `height`, `kbps`, `codec` | inbound-rtp; `send.*` from outbound-rtp |
+| `recv.lossPct`, `jitterMs`, `freezes`, `freezeMs` | inbound-rtp deltas |
+| `recv.processingMs` | Δ`totalProcessingDelay` / Δ`framesDecoded`, receive to render |
+| `recv.absCaptureLatencyMs` | sender capture to now through the abs-capture-time extension; Chromium, and only when the media node forwards it. A cross-check for the estimate. |
+| `send.lossPct` | remote-inbound-rtp `fractionLost` |
+| `send.targetKbps` | the encoder's current bitrate target |
+| `recv.transport`, `candidateType` | `udp`, `tcp`, `relay-udp`, `relay-tcp`; `host`, `srflx`, `prflx`, `relay` |
+| `peerRttMs` | the peer's own RTT, exchanged over the data channel |
+| `estimatedLatencyMs` | `rtt/2 + peerRtt/2 + jitterBuffer + decode + one frame interval`. An estimate; it excludes capture and display. |
+| `quality.recv`, `quality.send` | `good`, `fair`, `poor` |
+| `region`, `nodeId`, `serverVersion`, `joinMs` | where the call landed and how long connect took |
+
+## Telemetry and privacy
+
+With `telemetry: true` and a credential that carries `call_id`, the SDK posts the last
+snapshots every 5 s to `POST {apiBase}/v1/calls/{call_id}/stats` using the participant token.
+The rows are the fields above plus SDK version, platform and user agent. Nothing identifying the
+user is sent, media never is, and a failed post is dropped silently. Manual credentials without a
+`call_id` post nothing.
+
+`telemetryExtra` adds your own fields to every row. It is called once per row with `'recv'` or
+`'send'`. Three keys are stored as columns because the API knows them: `g2g_p50`, `g2g_p95` and
+`g2g_samples`, an app's own measured glass-to-glass latency in ms over its window, as the Qencode
+bench posts. Every other key lands in the row's `extra` object next to the SDK's keys, which win
+on a name clash.
+
+```ts
+Call.create(cred, { telemetryExtra: (direction) => ({ session: mySessionId, screen: direction === 'recv' ? 'room' : undefined }) });
+```
+
+## Browser support
+
+Chromium 110+, Safari 16.4+, Firefox 115+. A secure context is required. Device labels are
+empty until the first permission grant. Firefox has no `jitterBufferTarget`, so `lowest` mode
+runs with the browser's default buffer there and `stats` still reports the size. Speaker
+selection needs `setSinkId`, which Safari lacks; `devices.canSelectSpeaker` says so.
+
+## Running the example
+
+```bash
+cd examples/backend-node && npm install
+QENCODE_API_KEY=<your project API key> npm start   # API_BASE defaults to https://api.qencode.com
+# open http://localhost:8787/ in two tabs or on two devices, join as alice and bob
+```
+
+The backend creates one call, mints a credential per identity, and serves `examples/vanilla`.
+
+## Development
+
+```bash
+npm install
+npm test          # vitest, jsdom, engine mocked
+npm run build     # typecheck, ESM + UMD bundles, declarations in dist/
+```