@banou/media-player 0.6.2 → 0.7.0

package/README.md

@@ -1,6 +1,151 @@
-## Media Player
+# @banou/media-player
 
-A player that somehow manages to playback MKV files
+A React video player for files the browser cannot open on its own. It takes a `read(offset, size)` and
+a byte length, remuxes into fragmented MP4 as it plays through [libav-wasm](https://github.com/Banou26/libav-wasm),
+and renders ASS/SSA subtitles with jassub. Nothing is downloaded up front, so it plays a 4 GB MKV over
+HTTP range requests, out of a torrent, off a local disk, or out of anything else that can answer for a
+byte range.
 
+Playback state runs on [video.js v10](https://github.com/videojs/v10). None of its skin is used: the
+chrome here is its own.
 
-Todo: add support for multi audio tracks
+## Usage
+
+```tsx
+import MediaPlayer from '@banou/media-player'
+
+<MediaPlayer
+  read={(offset, size) => Promise<ArrayBuffer>}
+  size={fileByteLength}
+  publicPath="/"
+  libavWorkerUrl="/libav-worker.js"
+  jassubWorkerUrl={jassubWorkerUrl}
+  jassubWasmUrl="/jassub-worker-modern.wasm"
+  defaultFontUrl="/default.woff2"
+  title="episode.mkv"
+  autoplay
+/>
+```
+
+`read` and `size` travel together: pass both or neither. With neither, the player renders its chrome
+over a black frame and waits, which is the empty state.
+
+`inputToRemuxerInput` builds the pair from a `Blob`/`File`, a URL (probed for its length over a range
+request), or your own reader:
+
+```ts
+import { inputToRemuxerInput } from '@banou/media-player'
+
+const source = await inputToRemuxerInput({ blob: file })
+const source = await inputToRemuxerInput({ url: 'https://example.com/episode.mkv' })
+const source = await inputToRemuxerInput({ length, read })
+```
+
+`usePlayer` reads and drives playback state from anywhere inside a `MediaPlayer`. It is the only hook
+the chrome uses: the built-in video.js state and this player's own source state (tracks, thumbnails,
+indexes, readiness) live on one store, so there is no second context to reach for.
+
+```tsx
+import { usePlayer } from '@banou/media-player'
+
+const paused = usePlayer((state) => state.paused)          // subscribes to that field
+const player = usePlayer()                                  // no selector: the store, no subscription
+player.play()
+```
+
+`useSeekThumbnails` and `usePictureInPicture` are exported for reuse outside the bundled chrome.
+
+`downloadedRanges` paints byte spans you already hold onto the seekbar, mapped through the keyframe
+index rather than by percentage, because a file's download progress is not its playback progress:
+containers carry headers, fonts and attachments that occupy no time at all.
+
+### The assets your app has to serve
+
+Nothing is bundled: the workers and the wasm are fetched at runtime from urls you provide, so they have
+to be copied out of `node_modules` and hosted. `src/asset-urls.ts` is a worked example, and the
+`copy-assets` script is what puts them in `public/`.
+
+`publicPath` is the directory **libav's two wasm files** are served from, and both have to be there:
+
+| file | from | when it is used |
+| --- | --- | --- |
+| `libav.wasm` | `libav-wasm/build/` | browsers without JSPI: Safari, and every browser on iOS |
+| `libav-jspi.wasm` | `libav-wasm/build/` | Chrome and Edge 137+, Firefox 153+ |
+
+libav-wasm picks between them at runtime on `typeof WebAssembly.Suspending === 'function'`, so serving
+only one does not fail everywhere: it fails on exactly the browsers that pick the missing file, which
+reads as a browser bug rather than a missing asset. Serve both.
+
+The rest are named individually: `libavWorkerUrl` (`libav-wasm/build/worker.js`) and the optional
+`defaultFontUrl`. jassub also has two builds, and the same warning applies:
+
+| option | file | when it is used |
+| --- | --- | --- |
+| `jassubWasmUrl` | `jassub/dist/jassub-worker-modern.wasm` | wherever WebAssembly SIMD exists |
+| `jassubLegacyWasmUrl` | `jassub/dist/jassub-worker.wasm` | Safari before 16.4, and anything else without SIMD |
+
+`jassubLegacyWasmUrl` is optional in the type and not in practice: jassub falls back to a bare
+`'jassub-worker.wasm'`, which it resolves against the `blob:` url its worker is built from, and that
+throws. Leaving it unset does not fall back to the slower build, it loses subtitles entirely.
+
+jassub ships a classic worker script, so wrap it:
+
+```ts
+const jassubWorkerUrl = URL.createObjectURL(
+  new Blob([`importScripts("/jassub-worker.js")`], { type: 'application/javascript' }),
+)
+```
+
+The chrome is sized in `rem` against a **62.5% root font size**. Set `html { font-size: 62.5% }` or
+every control renders 1.6x too large.
+
+## What it does
+
+Play and pause, seek with a preview thumbnail and a keyframe-accurate scrub, volume on a log curve,
+mute, playback speed, audio track selection, subtitle track selection, picture in picture, fullscreen,
+and keyboard shortcuts. Nothing is persisted: volume, speed and track choices start at their defaults
+every load.
+
+### Picture in picture keeps the subtitles
+
+Subtitles are painted by jassub onto a canvas over the video, and picture in picture takes a video
+element and nothing else, so the browser has no way to composite the two: a plain
+`requestPictureInPicture()` puts the bare video in the window and leaves the subtitles on the page.
+
+So the player composites them itself. Every presented frame is drawn to an offscreen canvas with the
+subtitle canvas on top, and `captureStream()` turns that into a MediaStream backing a hidden video
+element, which is the one that enters the window. The original element keeps playing and stays the
+only audio source, since a canvas stream carries no audio track.
+
+The mirror element is never paused. A paused video stops rendering its MediaStream, so pausing it to
+reflect the real element froze the window: seeking while paused left the old scene on screen. The
+transport state is carried by the Media Session instead, which is what the window reads for its
+play/pause button.
+
+If anything in that path is unavailable the player falls back to handing the browser the bare video,
+which plays without subtitles rather than not at all.
+
+## Layout
+
+One package. `src/` is the demo app, `src/lib` is the library it publishes, and the app imports it by
+its published name so it stays an honest consumer.
+
+- `src/` the app: `main.tsx`, `routes/home.tsx`.
+  Built by `vite.config.ts` into `build/`.
+- `src/lib/engine/` the pipeline, with no React in it: MediaSource feeding, remux, jassub, thumbnails.
+  Published as `@banou/media-player/engine`.
+- `src/lib/react/` the player component, its chrome, and the hooks.
+
+Built by `vite.lib.config.ts` into `dist/`, which is what npm publishes.
+
+## Development
+
+```sh
+npm install
+npm run dev        # the demo app on port 4560
+npm run build      # the app, into build/
+npm run build-lib  # the library, into dist/
+```
+
+The app opens on an empty player: black, with the chrome and nothing else. Drop a file anywhere, click
+to pick one, or paste a URL.

package/dist/engine/index.d.ts

@@ -0,0 +1,10 @@
+export { startPlayback, terminateRemuxer, DEFAULT_BUFFER_SIZE } from './playback';
+export type { PlaybackOptions, PlaybackController, MediaIndex, AudioStream } from './playback';
+export { createSubtitleRenderer, SUBTITLES_OFF } from './subtitles';
+export type { SubtitleRenderer, SubtitleRendererOptions, SubtitleStream } from './subtitles';
+export { createThumbnailGenerator } from './thumbnails';
+export type { ThumbnailGenerator, ThumbnailGeneratorOptions, ThumbnailImage } from './thumbnails';
+export { getTimeRanges, updateSourceBuffer } from './source-buffer';
+export type { TimeRange } from './source-buffer';
+export { createPictureInPicture } from './picture-in-picture';
+export type { PictureInPictureController, PictureInPictureOptions } from './picture-in-picture';

package/dist/engine/index.js

@@ -0,0 +1,2 @@
+import { a as e, c as t, i as n, l as r, n as i, o as a, r as o, s, t as c } from "../engine-Dmc7JSd9.js";
+export { o as DEFAULT_BUFFER_SIZE, a as SUBTITLES_OFF, c as createPictureInPicture, s as createSubtitleRenderer, i as createThumbnailGenerator, t as getTimeRanges, n as startPlayback, e as terminateRemuxer, r as updateSourceBuffer };

package/dist/engine/picture-in-picture.d.ts

@@ -0,0 +1,27 @@
+/**
+ * Picture in picture with the subtitles burned in.
+ *
+ * The window takes a video element and nothing else, so subtitles painted on a canvas over the video
+ * cannot come along. Each frame is composited with the subtitle canvas onto an offscreen canvas, and
+ * `captureStream()` backs a hidden mirror element, which is the one that enters the window. The
+ * original element keeps playing and stays the only audio source.
+ */
+export type PictureInPictureOptions = {
+    video: HTMLVideoElement;
+    /** The subtitle canvas. jassub sizes it to the video's content rect, so it maps 1:1. */
+    canvas: HTMLCanvasElement;
+    maxWidth?: number;
+    /** Where the hidden mirror is mounted. It must be in the document. Defaults to the video's parent. */
+    container?: HTMLElement;
+    /**
+     * Toggle used when compositing is unavailable or fails. The React layer passes the player store's
+     * own action, which handles Safari and exits fullscreen first; the default here does neither.
+     */
+    fallback?: () => Promise<void>;
+};
+export type PictureInPictureController = {
+    toggle: () => Promise<void>;
+    destroy: () => void;
+};
+/** Takes over the document's Media Session play/pause handlers while a session is open. */
+export declare const createPictureInPicture: (options: PictureInPictureOptions) => PictureInPictureController;

package/dist/engine/playback.d.ts

@@ -0,0 +1,45 @@
+import type { AudioStream } from 'libav-wasm/build/worker';
+import type { SubtitleStream } from './subtitles';
+export type { AudioStream };
+/** A keyframe index entry: the byte offset a keyframe starts at, and the time it plays at. */
+export type MediaIndex = {
+    pos: number;
+    timestamp: number;
+};
+export type PlaybackOptions = {
+    videoElement: HTMLVideoElement;
+    canvasElement: HTMLCanvasElement;
+    read: (offset: number, size: number) => Promise<ArrayBuffer>;
+    length: number;
+    /** serves BOTH `libav.wasm` and `libav-jspi.wasm`; libav-wasm picks one on `WebAssembly.Suspending` */
+    publicPath: string;
+    libavWorkerUrl: string;
+    jassubWorkerUrl: string;
+    jassubWasmUrl: string;
+    /** jassub's non-SIMD build; without it subtitles fail rather than degrade on Safari before 16.4 */
+    jassubLegacyWasmUrl?: string;
+    defaultFontUrl?: string;
+    bufferSize?: number;
+    audioStreamIndex?: number;
+    onReady?: () => void;
+    onError?: (error: unknown) => void;
+    onRecovered?: () => void;
+    onSeek?: (fraction: number) => void;
+    onSubtitleStreams?: (streams: SubtitleStream[]) => void;
+    onAudioStreams?: (streams: AudioStream[], selected: number) => void;
+};
+export type PlaybackController = {
+    destroy: () => void;
+    selectSubtitleStream: (streamIndex: number | undefined) => void;
+    /** Keyframe index of the input, which is what maps a downloaded byte range onto the timeline. */
+    indexes: MediaIndex[];
+    duration: number;
+    videoMimeType: string;
+    audioMimeType: string;
+};
+export declare const DEFAULT_BUFFER_SIZE = 2500000;
+export declare const terminateRemuxer: (remuxer: {
+    worker: Worker;
+    destroy: () => Promise<void>;
+}) => void;
+export declare const startPlayback: (options: PlaybackOptions) => Promise<PlaybackController>;

package/dist/engine/source-buffer.d.ts

@@ -0,0 +1,12 @@
+export type TimeRange = {
+    index: number;
+    start: number;
+    end: number;
+};
+export declare const getTimeRanges: (sourceBuffer: SourceBuffer) => TimeRange[];
+export declare const updateSourceBuffer: (sourceBuffer: SourceBuffer, mediaSource: MediaSource) => {
+    appendBuffer: (buffer: ArrayBuffer | Uint8Array) => Promise<void>;
+    unbufferRange: (start: number, end: number) => Promise<void>;
+    updateTimestampOffset: (timestampOffset: number) => Promise<void>;
+    endOfStream: () => Promise<void>;
+};

package/dist/engine/subtitles.d.ts

@@ -0,0 +1,36 @@
+import type { Attachment, SubtitleFragment } from 'libav-wasm/build/worker';
+export type SubtitleStream = {
+    streamIndex: number;
+    title: string;
+    language: string;
+};
+/** -1 turns subtitles off. It frees the track and matches no header, so nothing is set. */
+export declare const SUBTITLES_OFF = -1;
+export type SubtitleRendererOptions = {
+    video: HTMLVideoElement;
+    canvas: HTMLCanvasElement;
+    workerUrl: string;
+    /** The SIMD build, `jassub-worker-modern.wasm`. Used wherever WebAssembly SIMD is available. */
+    wasmUrl: string;
+    /**
+     * The non-SIMD build, `jassub-worker.wasm`, for Safari before 16.4 and anything else without SIMD.
+     *
+     * Not optional in practice, only in the type. jassub picks `wasmUrl ?? 'jassub-worker.wasm'` when SIMD
+     * is missing, and that bare relative name resolves against the blob: url the worker is built from,
+     * which throws. So leaving this unset does not degrade to the slower build, it fails outright.
+     */
+    legacyWasmUrl?: string;
+    /** Fallback face for `liberation sans`. Without it jassub falls back to whatever the wasm build embeds. */
+    defaultFontUrl?: string;
+    onStreams?: (streams: SubtitleStream[]) => void;
+};
+export type SubtitleRenderer = ReturnType<typeof createSubtitleRenderer>;
+export declare const createSubtitleRenderer: (options: SubtitleRendererOptions) => {
+    pushAttachments: (incoming: Attachment[]) => void;
+    pushFragments: (fragments: SubtitleFragment[]) => void;
+    selectStream: (streamIndex: number | undefined) => void;
+    getStreams: () => SubtitleStream[];
+    getSelectedStream: () => number | undefined;
+    setOnStreams: (cb: (streams: SubtitleStream[]) => void) => void;
+    destroy: () => void;
+};

package/dist/engine/thumbnails.d.ts

@@ -0,0 +1,22 @@
+export type ThumbnailImage = {
+    url: string;
+    startTime: number;
+    endTime: number;
+};
+export type ThumbnailGeneratorOptions = {
+    publicPath: string;
+    workerUrl: string;
+    length: number;
+    /** The same reader playback uses, so generation shares its fetch order rather than competing. */
+    read: (offset: number, size: number) => Promise<ArrayBuffer>;
+    onThumbnails: (thumbnails: ThumbnailImage[]) => void;
+    /** Seconds between thumbnails. Widened automatically on long content to stay under MAX_THUMBNAILS. */
+    interval?: number;
+    width?: number;
+};
+export type ThumbnailGenerator = {
+    /** Report which byte ranges are readable. Called with no argument when the whole file is. */
+    update: (ranges?: [number, number][]) => void;
+    destroy: () => void;
+};
+export declare const createThumbnailGenerator: (options: ThumbnailGeneratorOptions) => Promise<ThumbnailGenerator>;