@vidtreo/recorder 1.7.13 → 1.8.1

package/README.md

@@ -8,6 +8,32 @@ Vidtreo SDK for browser-based video recording and transcoding. Similar to Ziggeo
 npm install @vidtreo/recorder
 ```
 
+## Direct Upload Transport
+
+Starting with `@vidtreo/recorder` 1.8.0, video bytes upload directly from the browser to Vidtreo-managed R2 storage through a resumable multipart session. The part size scales with the file (`calculateDirectUploadPartSize`: about 64 parts, whole MiB, clamped to 5–64 MiB) and the backend returns the authoritative geometry plus the parts already verified by R2. The SDK uploads up to four missing parts at once, computes a SHA-256 manifest checksum plus an R2-supported MD5 transport checksum, obtains a short-lived URL whose Signature V4 headers bind `Content-MD5` to that part, and confirms only after all parts succeed.
+
+- The API key is sent only to the configured Vidtreo backend and is never sent to R2.
+- Upload progress reaches `100%` only after the backend accepts confirmation.
+- `onProgress` is a phased, eased value: preparation and initiation fill 0–5%, transferred bytes fill 5–90%, confirmation eases from 90% toward 99%. It never moves backwards. `onProgressDetails` keeps the raw `loadedBytes`/`totalBytes` and reports the current `phase` (`UploadProgressPhase`: `"preparing" | "transferring" | "finalizing" | "completed"`).
+- Each part is retried up to five times with exponential backoff on network errors, inactivity, and HTTP 408/429/5xx. Retries pause while the browser is offline, and an expired part URL is re-signed once.
+- Pending uploads keep one idempotency identity across offline recovery and retries.
+- Retries resume provider-verified parts and never store multipart URLs or checksums in IndexedDB.
+- Aggregate progress remains monotonic across parts and reaches `100%` only after confirmation.
+- If confirmation is retried after the backend already accepted the upload, the SDK returns the existing video without uploading the Blob again.
+- Signed upload URLs remain in memory for one attempt and are not stored in IndexedDB or included in SDK errors.
+
+### Persistent Upload Queue and Resume
+
+Recordings and native-camera uploads are written to an IndexedDB upload queue before they are sent. If the page is reloaded, the tab crashes, or the network drops, the next recorder instance on the same origin picks the upload up again:
+
+- The queued upload keeps its idempotency key, so the backend returns the same video and the SDK uploads only the parts R2 has not verified yet.
+- Failed attempts are retried by the queue with exponential backoff (up to 10 attempts) on top of the per-part retries. Retries wait while the browser is offline.
+- Uploads authenticated with `apiKey` store the key with the queued item. Uploads authenticated with `getAccessToken` never persist a token: the provider is called again when the upload resumes, so the page must create the recorder (or native-camera handler) with the same `getAccessToken` after a reload.
+- Errors that cannot succeed on retry (such as `upload.insufficient-credits`) are reported immediately and are not retried.
+- When IndexedDB is unavailable (private browsing, blocked storage) the SDK uploads directly without persistence.
+
+Version 1.8.0 does not fall back to the legacy Worker-proxied upload endpoint. Applications that need the previous transport must remain on an earlier SDK version while the backend continues to support those clients. Custom Vidtreo backend deployments must expose initiation, part authorization, and confirmation endpoints. Managed R2 CORS must allow browser `PUT` plus the signed `Content-MD5` header and expose `ETag`.
+
 ## Quick Start
 
 ```typescript
@@ -46,7 +72,8 @@ Creates a new recorder instance with the specified configuration.
 
 **Parameters:**
 
-- `config.apiKey` (required): Your API key for backend authentication
+- `config.apiKey` (required unless `getAccessToken` is set): Your API key for backend authentication
+- `config.getAccessToken` (optional): `() => Promise<string>` that returns a short-lived desktop access token. Use it instead of `apiKey` when your server mints temporary tokens, so no long-lived key is exposed in the page. The SDK calls it whenever it needs a fresh token, including when a queued upload resumes after a reload. Provide exactly one of `apiKey` or `getAccessToken`
 - `config.apiUrl` (optional): Your backend API URL endpoint. Defaults to `https://core.vidtreo.com` if not provided
 - `config.enableSourceSwitching` (optional): Enable switching between camera and screen during recording. Default: `false`
 - `config.enableMute` (optional): Enable mute/unmute functionality. When disabled, `muteAudio()`, `unmuteAudio()`, and `toggleMute()` will throw errors. Default: `true`
@@ -55,15 +82,15 @@ Creates a new recorder instance with the specified configuration.
 - `config.maxRecordingTime` (optional): Maximum recording duration in milliseconds. When the maximum time is reached, recording automatically stops. If not set, recording can continue indefinitely until manually stopped. Examples: `300000` (5 minutes), `600000` (10 minutes), `1800000` (30 minutes)
 - `config.countdownDuration` (optional): Countdown duration in milliseconds before recording starts. Default: `0` (no countdown)
 - `config.userMetadata` (optional): Custom metadata object to include with uploads
-- `config.onUploadComplete` (optional): Callback invoked when upload completes successfully
+- `config.onUploadComplete` (optional): Callback invoked when upload completes successfully. Receives a [`RecordingUploadResult`](#recordinguploadresult)
 - `config.onUploadProgress` (optional): Callback invoked during upload with progress value (0-1)
-- `config.onUploadError` (optional): Callback invoked when upload fails
+- `config.onUploadError` (optional): Callback invoked when upload fails. Read the error code with [`getUploadErrorCode`](#upload-errors)
 - `config.onAudioWarning` (optional): Callback invoked when an audio issue is detected during recording. Receives an `AudioWarning` object. See [Audio Warnings](#audio-warnings) for details
 - `config.onRecordingStart` (optional): Callback invoked when recording starts
 - `config.onRecordingStop` (optional): Callback invoked when recording stops
 - `config.onError` (optional): Callback invoked when a stream error occurs
 
-**Throws:** `Error` if `apiKey` is missing
+**Throws:** `Error` if neither `apiKey` nor `getAccessToken` is provided. If both are provided it throws the shared configuration error `"Provide either apiKey or getAccessToken, not both"` with `code: CONFIG_ERROR_CREDENTIAL_CONFLICT` (`"config.credential-conflict"`). The same error is thrown by `RecorderController.initialize`, `ConfigManager`, `ConfigService`, `NativeCameraHandler` and `VideoUploadService`, and by the React, Vue and Web Component packages. Use `isCredentialConflictError(error)`, `hasCredentialConflict(apiKey, getAccessToken)` or `assertSingleCredential(apiKey, getAccessToken)` to check it yourself.
 
 #### Methods
 
@@ -123,6 +150,14 @@ Switches the recording source between camera and screen during an active recordi
 
 **Throws:** `Error` if source switching is not enabled or if not currently recording
 
+##### `reselectScreenSource(): Promise<boolean>`
+
+Opens the screen picker again while the source is `'screen'` or `'camera-screen'` and swaps the newly selected screen or window into the running preview or recording. The source type, the microphone and the camera bubble are kept.
+
+**Returns:** `Promise<boolean>` - `true` when a new screen was applied, `false` when the picker was cancelled
+
+**Throws:** `Error` if source switching is not enabled or no screen capture is active
+
 ##### `getAvailableDevices(): Promise<AvailableDevices>`
 
 Retrieves the list of available camera and microphone devices.
@@ -200,7 +235,8 @@ Cleans up all resources, stops active streams, and cancels any pending operation
 
 ```typescript
 interface VidtreoRecorderConfig {
-  apiKey: string;
+  apiKey?: string;
+  getAccessToken?: () => Promise<string>;
   apiUrl?: string;
   enableSourceSwitching?: boolean;
   enableMute?: boolean;
@@ -211,10 +247,8 @@ interface VidtreoRecorderConfig {
   userMetadata?: Record<string, unknown>;
   enableTabVisibilityOverlay?: boolean;
   tabVisibilityOverlayText?: string;
-  onUploadComplete?: (result: {
-    recordingId: string;
-    uploadUrl: string;
-  }) => void;
+  captureSystemAudio?: boolean;
+  onUploadComplete?: (result: RecordingUploadResult) => void;
   onUploadProgress?: (progress: number) => void;
   onUploadError?: (error: Error) => void;
   onRecordingStart?: () => void;
@@ -238,6 +272,32 @@ interface RecordingStartOptions {
 }
 ```
 
+### RecordingUploadResult
+
+Payload passed to every upload-complete callback (`VidtreoRecorder` `onUploadComplete`, `UploadCallbacks.onSuccess`, the upload queue, native-camera uploads, and the React/Vue/Web Component packages). It contains every field of the backend `VideoUploadResult` plus the backward-compatible `recordingId` and `uploadUrl` fields.
+
+```typescript
+type RecordingUploadResult = {
+  recordingId: string;           // same value as id
+  uploadUrl: string;             // "" when the backend has no URL yet
+  id: string;
+  publicId: string;
+  filename: string;
+  fileSize: number;
+  mimeType: string;
+  duration: number | null;
+  status: string;
+  createdAt: string;
+  transferStatus: "pending" | "transferring" | "transferred" | "failed" | null;
+  transferError: string | null;
+  transferredAt: string | null;
+  r2DeleteAfter: string | null;
+  r2DeletedAt: string | null;
+};
+```
+
+Use `createRecordingUploadResult(videoUploadResult)` to build this payload from a raw `VideoUploadService` result.
+
 ### RecordingStopResult
 
 ```typescript
@@ -251,9 +311,11 @@ interface RecordingStopResult {
 ### SourceType
 
 ```typescript
-type SourceType = 'camera' | 'screen';
+type SourceType = 'camera' | 'screen' | 'camera-screen';
 ```
 
+`'camera-screen'` is opt-in and only used when passed explicitly. See [Screen-only capture and camera + screen](#screen-only-capture-and-camera--screen-advanced).
+
 ### RecordingState
 
 ```typescript
@@ -661,6 +723,34 @@ try {
 }
 ```
 
+### Upload Errors
+
+Upload failures that need user-facing handling carry a stable code. Read it with `getUploadErrorCode`:
+
+```typescript
+import {
+  getUploadErrorCode,
+  UPLOAD_ERROR_INSUFFICIENT_CREDITS,
+  type UploadErrorCode,
+} from '@vidtreo/recorder';
+
+const recorder = new VidtreoRecorder({
+  apiKey: 'your-api-key',
+  onUploadError: (error) => {
+    const code: UploadErrorCode | null = getUploadErrorCode(error);
+    if (code === UPLOAD_ERROR_INSUFFICIENT_CREDITS) {
+      showMessage('This video could not be uploaded. Please contact the site administrator.');
+    }
+  },
+});
+```
+
+| Code | Meaning |
+|------|---------|
+| `upload.insufficient-credits` | The backend rejected the upload with HTTP 402 because the account has no credits left. The upload is not retried. |
+
+`getUploadErrorCode` returns `null` for every other error (network failures, timeouts, and so on). Those errors are retried automatically before they are reported.
+
 ### Recording with Disabled Features
 
 ```typescript
@@ -701,6 +791,91 @@ For web component installation, usage, and examples, please refer to the [@vidtr
 
 ## Advanced Usage
 
+### Screen-only capture and camera + screen (advanced)
+
+These capabilities are opt-in. Without the options below the recorder behaves exactly as before: screen capture starts from the camera preview and `'camera-screen'` is never used.
+
+#### Screen-only capture
+
+Set `startScreenWithoutCamera: true` to start `startPreview('screen')` / `startRecording({}, 'screen')` with `getDisplayMedia` for video and a microphone-only `getUserMedia` request. The camera is never opened. Mute, audio levels, microphone changes and stopping work as with the camera. Switching to `'camera'` or `'camera-screen'` opens the camera at that moment. If the user cancels the screen picker, the start rejects with an error whose `code` is `ERROR_SCREEN_SELECTION_CANCELLED` (`'screen.selection-cancelled'`).
+
+#### Camera + screen (`'camera-screen'`)
+
+`'camera-screen'` records the screen as the full frame with the camera as a picture-in-picture bubble. Preview and recording use the same composed video track; audio is the microphone.
+
+```typescript
+import { isPictureInPictureSupported, VidtreoRecorder } from '@vidtreo/recorder';
+
+const recorder = new VidtreoRecorder({
+  apiKey: 'your-api-key',
+  enableSourceSwitching: true,
+  startScreenWithoutCamera: true,
+  pictureInPicture: { position: 'bottom-right', shape: 'circle', size: 0.22 },
+});
+
+if (isPictureInPictureSupported()) {
+  await recorder.startRecording({}, 'camera-screen');
+  await recorder.switchSource('screen');
+  await recorder.switchSource('camera-screen');
+  const cameraOnly = recorder.getCameraStream();
+}
+```
+
+`pictureInPicture` options:
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `position` | `'bottom-right' \| 'bottom-left' \| 'top-right' \| 'top-left' \| { x: number; y: number }` | `'bottom-right'` | Bubble corner, or the normalized top-left of the bubble in the output frame (0 to 1 each) |
+| `shape` | `'circle' \| 'rounded'` | `'circle'` | Bubble shape |
+| `size` | `number` (0.05 to 0.5) | `0.22` | Bubble diameter as a fraction of the frame height |
+
+Invalid values throw during initialization. The bubble has a 3% margin, a white border of about 2% of its diameter and a soft shadow. The camera image is center-cropped into the bubble and is not mirrored.
+
+- Switching between `'camera'`, `'screen'` and `'camera-screen'` works in preview and while recording, without stopping the recording.
+- If the user stops sharing from the browser or OS, the recorder switches to the camera and keeps recording.
+- Stopping, `stopPreview` and `cleanup()` stop the screen, camera and microphone tracks and the compositor.
+- `getCameraStream()` returns a video-only stream with the live camera track (in `'camera'` and `'camera-screen'`), or `null`.
+- Composition runs in a dedicated Web Worker (`MediaStreamTrackProcessor` → `OffscreenCanvas` → `MediaStreamTrackGenerator`). New frames drive it, not timers or `requestAnimationFrame`, so it keeps running while the page is hidden or minimized. Output resolution is the screen resolution capped to the recording resolution. Frame rate follows the screen capture, capped at 30 fps.
+- `'camera-screen'` requires `MediaStreamTrackProcessor`, `MediaStreamTrackGenerator` and `OffscreenCanvas` (Chromium-based browsers and Electron). Check with `isPictureInPictureSupported()`. On other browsers the switch reports an error with code `ERROR_PICTURE_IN_PICTURE_UNSUPPORTED` (`'screen.picture-in-picture-unsupported'`) and the current source stays active.
+
+#### Free position and live layout updates
+
+A free `position` places the bubble's top-left corner at `{ x, y }` as fractions of the output frame width and height. The point is clamped so the whole bubble stays inside the frame for its size and the frame aspect ratio; corner presets keep their 3% margin, free positions have none.
+
+`setPictureInPicture(update)` merges a partial update into the current configuration, validates it and applies it to the running compositor without restarting it, in preview and while recording. The next composed frame eases toward the new rect over about 120 ms; pass `smooth: false` to jump. Without a running compositor the update is stored and used the next time `'camera-screen'` starts. Invalid values throw and keep the previous layout.
+
+```typescript
+recorder.setPictureInPicture({ position: { x: 0.72, y: 0.64 }, size: 0.25 });
+recorder.setPictureInPicture({ position: 'top-left', smooth: false });
+const layout = recorder.getPictureInPictureConfig();
+```
+
+`RecorderController` exposes the same `setPictureInPicture(update)` and `getPictureInPictureConfig()`.
+
+The React hook `useVidtreoRecorder` accepts the same `startScreenWithoutCamera` and `pictureInPicture` options (`pictureInPicture` is the initial value) and exposes `actions.setPictureInPicture(update)`. It exposes `state.sourceType` and `state.cameraStream`. The prebuilt UI components do not offer these sources.
+
+### System audio (advanced)
+
+Set `captureSystemAudio: true` (also accepted by `useVidtreoRecorder` and `RecorderController.configureSources`) to record the computer's sound (calls, videos, other apps) together with the microphone when recording `'screen'` or `'camera-screen'`. Default `false`: the display stream's audio is requested with `audio: false` and any audio track is stopped, exactly as before.
+
+- Screen pickers are requested with `audio: true`. Whether audio comes back depends on the browser and the user's choice (Chrome only offers it for tabs, or for the whole screen on Windows/ChromeOS; Electron apps return it from `setDisplayMediaRequestHandler` with `audio: 'loopback'`). Without a display audio track the recording falls back to the microphone, without an error.
+- While recording, the microphone and the system audio are mixed into one audio track with Web Audio (`AudioContext` → one `MediaStreamAudioSourceNode` and `GainNode` per input → `MediaStreamAudioDestinationNode`). Web Audio resamples both inputs to the context rate. The context is created when the recording starts and closed when it stops.
+- Mute only silences the microphone; the system audio keeps recording. Audio levels (`getAudioStreamForAnalysis`) keep showing the microphone only.
+- Reselecting the screen, or switching between camera and screen sources, swaps the system audio input without stopping the recording. A recording started from the camera picks up system audio once it switches to a screen source.
+- The preview never plays the system audio. If the microphone picks up the speakers, the system audio is recorded twice (once directly, once through the room). Browser echo cancellation (`echoCancellation`, on by default) only cancels audio the page itself plays, so recommend headphones during calls.
+
+### Recording title
+
+Call `setRecordingTitle(title)` before or during a recording to name the uploaded file `<title>.<ext>` instead of `recording-<timestamp>.<ext>`. The title is trimmed, whitespace is collapsed, control characters and `/ \ : * ? " < > |` are removed, leading dots are stripped and the result is capped at 120 characters. An empty result keeps the default name. The title is cleared once the recording's upload is queued, so set it again for the next recording. Pass `null` to clear it.
+
+```typescript
+recorder.setRecordingTitle('Demo – Q4 ventas');
+await recorder.startRecording({}, 'camera');
+const result = await recorder.stopRecording(); // uploads "Demo – Q4 ventas.mp4"
+```
+
+The same method exists on `RecorderController`, the React hook exposes it as `actions.setRecordingTitle(title)`, and `sanitizeRecordingTitle(title)` is exported to preview the sanitized name.
+
 ### Telemetry
 
 The recorder automatically sends telemetry events to monitor SDK usage, detect issues, and improve the product. **Telemetry is always active** and cannot be disabled through configuration.
@@ -893,6 +1068,55 @@ await controller.initialize({
 });
 ```
 
+#### Native Camera Uploads
+
+`NativeCameraHandler` validates a video picked with the device camera or gallery (`<input type="file" capture>`), transcodes it, and uploads it through the same persistent queue as recorded videos. UI packages use this single API for their native-camera mode.
+
+```typescript
+import {
+  ConfigService,
+  NativeCameraHandler,
+  VideoUploadService,
+  getUploadErrorCode,
+} from '@vidtreo/recorder';
+
+const handler = new NativeCameraHandler(
+  {
+    apiKey: 'your-api-key',            // or getAccessToken: () => fetchToken()
+    backendUrl: 'https://core.vidtreo.com',
+    maxRecordingTime: 300000,
+    maxFileSize: 500 * 1024 * 1024,
+    userMetadata: { interviewId: '42' },
+  },
+  ConfigService.getInstance({ apiKey: 'your-api-key', backendUrl: 'https://core.vidtreo.com' }),
+  new VideoUploadService()
+);
+
+await handler.startUploadQueue({
+  onUploadProgress: (progress) => updateProgress(progress),
+  onUploadProgressDetails: (details) => updatePhase(details.phase),
+  onUploadRetry: (details) => showRetry(details.attempt),
+  onUploadComplete: (result) => saveVideo(result),
+  onUploadError: (error) => showError(getUploadErrorCode(error)),
+});
+
+const file = await handler.handleFileSelection(input.files[0]);
+const result = await handler.processAndUpload(
+  (transcodeProgress) => updateTranscode(transcodeProgress),
+  (uploadProgress) => updateProgress(uploadProgress),
+  (details) => updatePhase(details.phase),
+  (retry) => showRetry(retry.attempt)
+);
+console.log(result.recordingId, result.transferStatus, result.blob);
+
+handler.destroy();
+```
+
+- `startUploadQueue(callbacks?)` opens the IndexedDB queue and immediately resumes uploads left over from a previous page load (same idempotency key, only missing parts are sent). Its callbacks receive events for those resumed uploads and for queue-level errors. It resolves to `false` when persistent storage is unavailable; uploads then go directly to the backend.
+- `processAndUpload(onTranscodeProgress, onUploadProgress, onUploadProgressDetails?, onUploadRetry?)` queues the transcoded file and resolves with a `NativeCameraUploadResult` (`RecordingUploadResult & { blob: Blob }`) once the upload is confirmed. It rejects only with a terminal error, after the queue has exhausted its retries or immediately for `upload.insufficient-credits`. Events for this upload go to the per-call callbacks, not to the `startUploadQueue` callbacks.
+- Calling `processAndUpload` without `startUploadQueue` opens the queue on demand.
+- `destroy()` stops the queue timers. An upload that is still pending stays in IndexedDB and resumes the next time a handler or recorder starts the queue.
+
 #### Stream Management
 
 ```typescript