shaka-player 4.6.3 → 4.7.0

package/docs/design/offline.md

@@ -1,153 +0,0 @@
-# Shaka Player v2 Offline Design
-
-last update: 2016-05-24
-
-by: [joeyparrish@google.com](mailto:joeyparrish@google.com)
-
-
-## Overview
-
-Offline storage in Shaka Player v2 will be represented by an instance of
-`shaka.offline.Storage`.  The API for offline storage will not be on
-`shaka.Player` so that offline capabilities can be trivially omitted from the
-build.  Internal classes used to implement offline storage/playback will live in
-the `shaka.offline` namespace.  Everything in the `shaka.offline` namespace will
-be listed in `build/types/offline` for quick exclusion of the feature.
-
-`Storage` will store streams for offline playback, allow selection of which
-tracks to store, provide progress information while storing streams, allow the
-listing of stored content, and allow the removal of stored content.  To reduce
-the number of parameters in the `store()` method, we will offer a `configure()`
-interface to set callbacks.  We will also provide a default track-selection
-callback to make it easier to get started.
-
-Backward compatibility with stored content from v1 is an explicit non-goal.  If
-it turns out that there is a need in the community to migrate stored content
-from v1 to v2, a simple migration utility can be written later.
-
-
-#### `Storage` API sketch
-
-```js
-new shaka.offline.Storage(player)
-
-shaka.offline.Storage.support => boolean
-
-// NOTE: Not connected to player.configure().
-shaka.offline.Storage.prototype.configure({
-  trackSelectionCallback: function(allTracks) {
-    // The default callback selects:
-    //   1. highest bandwidth video track with height <= 480,
-    //   2. middle bandwidth audio track with best audio pref language match,
-    //   3. all text tracks with any text pref language match.
-    // Return an array of selected Tracks.
-  },
-  progressCallback: function(storedContent, percentComplete) {}
-});
-
-// Store content and metadata.
-shaka.offline.Storage.prototype.store(manifestUri, appMetadata, manifestParser) => Promise.<StoredContent>
-
-// Remove stored content and metadata.
-shaka.offline.Storage.prototype.remove(storedContent) => Promise
-
-// List stored content.
-shaka.offline.Storage.prototype.list() => Promise.<Array.<StoredContent>>
-
-StoredContent:
-  offlineUri: string  // at which the stored content can be accessed
-  originalManifestUri: string  // the original manifest URI of the content we stored
-  duration: number  // length of the stored content in seconds
-  size: number  // size of the stored content in bytes
-  tracks: !Array.<shaka.extern.Track>  // the tracks we stored
-  appMetadata: object  // arbitrary format, provided by the application to store()
-```
-
-
-#### Implementation
-
-Like v1, v2 will use IndexedDB for storage.
-
-In v1, applications were required to persist the list of stored content and any
-app metadata in window.localStorage.  In v2, we will provide an API to list
-stored content and we will store app metadata along-side the content.
-
-In v1, progress was inaccurate.  It was calculated in terms of segments, even
-though segment sizes are very different between audio and video.  In v2,
-progress can be calculated in terms of bytes, which will provide a more accurate
-percentage.  If the index does not have byte ranges for segments, presumed
-segment sizes can be calculated based on average bitrate and duration.  This
-will be accurate enough on average to provide a smooth progress bar.  (Smooth
-relative to v1.)  Actual bytes transferred will be tracked in the
-`StoredContent` structure.
-
-In v1, segments were combined into larger chunks to reduce the overhead of
-reading from the disk.  This was premature optimization, as we had no data to
-suggest this was necessary.  In v2, all segments will be stored in their
-original form.
-
-`store()` will transfer segment types (audio, video) in parallel, since some
-CDNs limit download rate based on the bandwidth requirements of the stream.
-
-`Storage` will have to load a manifest using the correct parser, filter the
-tracks as `Player` does, and invoke a callback to select streams.  This should
-involve a factoring of the load and filtering logic of `Player`, perhaps into
-`ManifestParser` or `PlayerUtil`.  These methods will not be exported.  A
-`Player` instance is required to instantiate a `Storage`, and `Storage` will get
-manifest configuration from that instance.
-
-`Storage` will also need to use `DrmEngine` directly to persist licenses and get
-the session IDs.  The session IDs will be stored in the database as part of the
-content.  Removing a piece of content should result in any associated persistent
-sessions being released.
-
-A scheme plugin will register itself with `NetworkingEngine` to implement the
-`offline` scheme.  Manifest URIs will be of the form `offline:123`, where `123`
-is an automatically-assigned ID representing the stored content.  Segment URIs
-will be of the form `offline:123/7/99`, where `7` is an auto-ID for the track
-and `99` is an auto-ID for the segment.
-
-The scheme handler will handle `HEAD` requests and return a `Content-Type`
-header with the value `application/x-offline-manifest`.  This type will be
-registered with `ManifestParser` in order to "parse the manifest" (load it from
-the database).
-
-To include offline support info in `Player.probeSupport()` but still make it
-possible to exclude offline from the build, `Player.probeSupport()` will now
-have a plugin interface. `Storage` will register its `support()` method with
-`Player` under the key `offline`.  Overall library support is not contingent on
-the `basic` support of any of the plugins, including offline.  Support plugins
-are synchronous, for simplicity.
-
-It's okay if the app selects multiple streams of a type.  For text, this may be
-necessary, for example, to store subtitles for all languages.  For audio it may
-be necessary as well, for similar reasons.  For video it makes little sense, but
-we won't restrict it.  Apps can make whatever decision they want about tracks to
-store, but we should document best practice.
-
-We could also issue a warning if multiple tracks of same type, kind, and
-language are selected, since offline adaptation is currently an uncommon
-scenario and may indicate a mistake on the part of the app developer.  For
-example, this would trigger on two video with different bitrate or size, or two
-audio of same language but different bitrate, or two text of same language and
-kind.
-
-`Storage` will need good unit and integration tests.  Integration tests should
-store a stream created by `StreamGenerator` and verify that the `StoredContent`
-structure is correct.  A test should also load the stored data and verify that
-it matches the generated segments.
-
-
-#### Error conditions
-
-Errors during `store()`:
-  - No IndexedDB support
-  - No persistent license support
-  - Out of storage space
-  - Can't store a live stream
-  - Any error that could occur during player.load()
-
-Errors while loading content:
-  - Content not found
-  - Persistent license cannot be loaded (may have been cleared by browser)
-  - Any error that could occur during player.load()

package/docs/design/ownership.gv

@@ -1,76 +0,0 @@
-# Generate png with: dot -Tpng -O ownership.gv
-
-digraph shaka2_ownership {
-  label=<<u>Shaka 2 Ownership Diagram</u>>
-  labelloc=t
-  node [ shape=rectangle ]
-
-  MediaSource [ shape=oval ]
-  SourceBuffer [ shape=oval ]
-  TextTrack [ shape=oval ]
-
-  AbrManager [ shape=trapezium ]
-  HttpFetchPlugin [ shape=trapezium ]
-  ManifestParser [ shape=trapezium ]
-  TextDisplayer [ shape=trapezium ]
-  TextParser [ shape=trapezium ]
-
-  Player -> AbrManager
-  Player -> NetworkingEngine
-  Player -> ManifestParser
-  Player -> MediaSourceEngine
-  Player -> StreamingEngine
-  Player -> DrmEngine
-  Player -> Manifest
-  Player -> TextDisplayer
-
-  TextDisplayer -> TextTrack
-
-  NetworkingEngine -> HttpFetchPlugin
-
-  ManifestParser -> NetworkingEngine [ style=dotted ]
-
-  DrmEngine -> NetworkingEngine [ style=dotted ]
-
-  StreamingEngine -> MediaSourceEngine [ style=dotted ]
-  StreamingEngine -> Manifest [ style=dotted ]
-  StreamingEngine -> NetworkingEngine [ style=dotted ]
-
-  MediaSourceEngine -> MediaSource
-  # MediaSourceEngine owns multiple SourceBuffers, so we represent that here
-  # with two edges:
-  MediaSourceEngine -> SourceBuffer
-  MediaSourceEngine -> SourceBuffer
-  MediaSourceEngine -> Transmuxer
-  MediaSourceEngine -> TextEngine
-
-  TextEngine -> TextDisplayer [ style=dotted ]
-  TextEngine -> TextParser
-
-  # Player does not own MediaSource since v2.4, but without this link, the graph
-  # becomes unbalanced and harder to read.  This is a hack.
-  Player -> MediaSource [ style=invisible arrowhead=none ]
-
-  subgraph cluster_legend {
-    style=rounded
-    color="#444444"
-    label=<<u>Legend</u>>
-    fontsize=12
-    fontcolor="#444444"
-
-    node [ shape=plaintext color="#444444" fontsize=10 fontcolor="#444444" ]
-    edge [ color="#444444" ]
-
-    blank1 [ label="" fixedsize=true height=0 ]
-    blank2 [ label="" fixedsize=true height=0 ]
-    "weak reference" -> blank2 [ style=dotted ]
-    ownership -> blank1
-
-    plugin [ shape=trapezium ]
-    "browser built-in" [ shape=oval ]
-    "internal class" [ shape=rectangle ]
-  }
-
-  # Hack to move the legend beneath the main graph:
-  TextParser -> "internal class" [ style=invisible arrowhead=none ]
-}

package/docs/design/preload.md

@@ -1,299 +0,0 @@
-# Shaka Player Preload API Design
-
-Last updated: 2020-03-01
-
-by: [michellezhuo@google.com](mailto:michellezhuo@google.com)
-
-
-## Objective
-
-Enable Shaka Player to pre-load content before involving MediaSource or a
-`<video>` element.
-
-This allows an application using Shaka Player to:
-- Pre-load the beginning of several pieces of video content speculatively, while
-  the user is making a decision on which video to play.
-- Start to play the selected video almost immediately.
-
-[Github issue](https://github.com/shaka-project/shaka-player/issues/880)
-
-
-## Background & Context
-
-### Media Source
-
-The [MediaSource][] API, formally known as Media Source Extensions (MSE),
-provides functionality enabling plugin-free web-based streaming media. Using
-MSE, media streams can be created via JavaScript, and played using `<audio>` and
-`<video>` elements.
-
-The MediaSource interface of the [Media Source Extensions API][] represents a
-source of media data for an HTMLMediaElement object. A MediaSource object can be
-attached to an HTMLMediaElement to be played.
-
-The SourceBuffer interface represents a chunk of media to be passed into an
-HTMLMediaElement and played, via a MediaSource object. This can be made up of
-one or several media segments.
-
-[Media Source Extensions API]:https://developer.mozilla.org/en-US/docs/Web/API/Media_Source_Extensions_API
-[MediaSource]: https://developer.mozilla.org/en-US/docs/Web/API/MediaSource
-
-### StreamingEngine & MediaSourceEngine
-
-The StreamingEngine is the streaming component in Shaka Player. It’s responsible
-for fetching the content, and pipelines the content to MediaSourceEngine.
-The MediaSourceEngine manages the MediaSource and the SourceBuffers.
-
-
-## Design Overview
-
-The design will address two major requirements:
-1. Enable preloading a manifest in Player before attaching to a video element
-2. Enable StreamingEngine to fetch and store a segment in memory without a video
-   element
-
-
-## Design Goals
-
-### Compatibility
-
-Currently, the Shaka Player loads a manifest and streams a video via its Load
-API. The Preload API should be backward compatible with the existing Load API.
-
-An application should be able to:
-- Attach a video element and load a manifest, and start playback with Shaka
-  Player
-- Preload manifests and attach a video element later to start playback with
-  Shaka Player
-
-### Reusability
-
-The Preload API implementation should reuse the current Load API logic as much
-as possible.
-
-
-## Design Details
-
-### Player
-Currently,  with the Load API, the Player:
-- must attach a `<video>` element before loading a manifest
-- can load only one manifest at one time.
-
-With the new Preload API enabled, the Player should be able to:
-- pre-load a manifest, and attach a video element later to start playback.
-- pre-load more than one manifest, and finish loading one of them at a time.
-
-#### APIs
-
-**Existing: Load API**
-
-```js
-const player = new shaka.Player();
-player.attach(video);
-await player.load('foo.mpd');
-```
-The Player constructor takes a `<video>` element (HTMLMediaElement) as an
-optional parameter. If the `<video>` element is available, we attach the
-`<video>` element to the player, and set up the MediaSource.
-
-A `<video>` element must be attached to the Player before `load()` gets called.
-The Player can load only one video at one time.
-`load()` loads the manifest, initializes the playback components and starts
-playback, including:
-- Parsing the manifest
-- Initializing the DRM Engine
-- Attaching the playback-related listeners to the video element
-- Initializing playback and observers
-- Initializing ABR Manager and StreamingEngine
-- Starting playback
-
-**New: Preload API**
-
-```js
-const player = new shaka.Player();
-
-const loader1 = player.preload('foo1.mpd');
-const loader2 = player.preload('foo2.mpd');
-
-player.attach(video);
-
-player.load(loader1); // The buffered data in loader1 is transferred to
-                      // the Player and the MediaSource. loader1 is
-                      // invalidated.
-loader2.destroy();    // loader2 is invalidated.
-```
-
-The new Preload API doesn’t require a `<video>` element before the Player
-preloads a manifest.  The `<video>` element must be attached to the Player
-before `load()` gets called.
-
-`preload()` parses a manifest, and pipelines the manifest to the
-StreamingEngine. StreamingEngine fetches the beginning piece of the content,
-and stores it in memory.
-
-The Player can preload multiple manifests in parallel, and store each of them as
-a loader.
-
-
-After `preload()`, `load(loader)` will start playback by opening the MediaSource
-and pipelining the fetched content to MediaSourceEngine.
-
-`load()` invalidates the loader instance once the playback starts.
-
-After the playback starts, the application can invalidate the other loaders when
-they are no longer needed by calling `destroy()`.
-
-When the Player is destroyed, the remaining loaders will also be destroyed.
-
-The `load()` function will take either an asset Uri or a loader instance as the
-first parameter.
-```js
-    load(assetUri | loader, startTime, mimeType) {}
-```
-
-If `load()` gets an uri as the input, it goes to the original loading process.
-
-If `load()` gets a loader instance, it means the content is preloaded, and the
-Player goes to the following steps after preload to finish loading and start
-playback.
-
-
-### the Load Graph
-
-The Player uses a state machine (the “load graph”) to go through the steps of
-attaching a video element, parsing a manifest, initializing components, etc.
-to start the video playback.
-
-Currently the Player can only have one load graph, so it can only load one
-manifest.
-
-To enable preloading multiple manifests in parallel, we’ll extract the load
-graph and the loading process to a Loader class. The Player can have multiple
-Loader instances when preloading.
-
-Once the user decides to load one of the manifests, the Player will invalidate
-the chosen Loader passed to load(), and the Loader will be responsible for the
-life cycle of loading and playback. The application can use other preloaded
-Loader instances later, or invalidate them if they are no longer needed.
-
-The load graph contains the following states (nodes):
-- Attach Node
-- Media Source Node
-- Manifest Parser Node
-- Manifest Node
-- Drm Engine Node
-- Load Node
-- Unloading Node
-- Src Equals Node
-
-To enable the Preload API, we’ll separate the original Load Node into two nodes,
-one “Preload node” and a “Load Node”. We’ll add a new path of “preload manifest
--> attach video -> load” in the graph.
-
-![Shaka preload and load diagram](preload.png)
-
-#### Implementation steps
-
-1. Extract the load graph from the Player into a Loader class.
-2. The Player’s APIs should have the same behavior as before. The Player should
-forward the API calls to the Loader class.
-3. Add a Preload node in the Load Graph.
-4. Make a path of preload manifest -> attach video -> load in the graph. We can
-check if the video element is available to see which path we are on.
-5. Attach the video element to the MediaSource when `load()` is called.
-
-#### Alternative approach
-
-To enable preloading multiple manifests, another approach is to have two load
-graphs, one in the Player class and another one in the Loader class.
-
-The load graph in Player is responsible for the overall life cycle, including
-attaching / detaching the video, and parsing / loading / unloading the manifest.
-the load graph in Loader is responsible for a part of the overall graph,
-the parsing / loading manifest process.
-
-However, with that approach, the life cycle of the video playback, and the
-components needed for playback may be managed and accessed in two places. It may
-be more complex to implement.
-
-
-### StreamingEngine & MediaSourceEngine
-
-When loading a media content, the Player initializes the MediaSource when the
-StreamingEngine starts to stream a piece of content. The MediaSource gets open
-once a video gets attached to it. We can append buffers to the MediaSource once
-it’s open.
-
-With the Preload API, the StreamingEngine should be able to download an
-audio/video/text stream without the video element attached. The
-MediaSourceEngine should be able to start without the video element, and store
-the downloaded content in memory.
-
-This way, the initialization steps of StreamingEngine, MediaSourceEngine,
-MediaSource when preloading remain almost the same with the original loading
-process. The state of MediaSourceEngine remains the same, and StreamingEngine
-doesn't need to know the state of MediaSource. The downloaded content is kept
-in and managed by MediaSourceEngine, whether stored in memory or appended to the
-SourceBuffer. This approach requires the least changes in StreamingEngine and
-MediaSourceEngine.
-
-New field added to MediaSourceEngine:
-
-`preloadBuffers_` map
-- key: `contentType`
-- value: an object of `{ bufferedData, mimeType }`
-
-However, the Player can also have a MediaSourceEngine attached to a video
-element, before content is loaded or preloaded.  So we won't always be able to
-attach the Loader's MediaSourceEngine to a video element.
-
-In that case, we need to replace the Loader's MediaSourceEngine with the
-Player's copy, and transfer the stored buffered data.
-
-And since MediaSourceEngine operations are sync, and StreamingEngine runs on its
-own timers, we need to handle new segments being streamed while this process of
-transferring buffered segments is in progress. We can pause StreamingEngine
-update timers while you do the transfer.
-
-
-#### Implementation Steps
-
-When preloading:
-1. When the Player initializes the MediaSourceEngine:
-   1. In MediaSourceEngine, skip attaching the video to the media source for
-   now. The MediaSource’s readyState is closed.
-   2. In Player.onInitializeMediaSourceEngine_(), do not wait for the
-   MediaSource to be open.
-   3. Add a preloadBuffers map in the MediaSourceEngine.
-2. When the Player initializes the StreamingEngine, and starts streaming:
-   1. In `MediaSourceEngine.init()`, skip adding the SourceBuffer for now.
-   Instead, store a `<contentType, { bufferedData, mimeType } >` key-value pair
-   in a preloadBuffers map.
-
-When loading with the loader instance:
-1. If the Player hasn't got an attached MediaSource already, attach the video
-element to the MediaSource, and the MediaSource gets open.
-2. Append the buffered data to the SourceBuffers.
-
-
-### Constraints
-The Preload API can't work with src=. Preloading hinges on using MediaSource to
-keep segments in memory and parse them later, and this can't be done with src=.
-
-
-## Test Plans
-
-### Existing use case
-
-StreamingEngine fetches a piece of content and appends to MediaSourceEngine.
-When preloading, the MediaSourceEngine appends the data to the SourceBuffer.
-
-### New use cases
-
-1. When preloading, the StreamingEngine fetches a piece of content and appends
-   to MediaSourceEngine, and the MediaSourceEngine stores the data in memory.
-2. When the MediaSource is ready, StreamingEngine pipelines the fetched content
-   to the MediaSource.
-3. For a live stream video, StreamingEngine can keep fetching the latest piece
-   of the content. The MediaSourceEngine stores the latest data, and abandons
-   the older data.

package/docs/design/talking-about-languages.md

@@ -1,46 +0,0 @@
-# Shaka Player - Talking About Language
-
-## Summary
-This document outlines the standard set of terms and definitions used by Shaka Player when working with language.
-
-## Locales
-A __locale__ is the top level "object" and is composed of three __components__:
- - __language__, a lowercase 2-character code. Preferably from [ISO 639](http://www.loc.gov/standards/iso639-2/php/code_list.php).
- - __region__, an uppercase 2-character code. Preferably from ISO 3166.
- - __dialect__, a lowercase n-character code.
-
-A locale is not required to use every component but must follow one of three patterns:
- - language (e.g. "en")
- - language-REGION (e.g. "en-US")
- - language-REGION-dialect (e.g. "en-US-wa")
-
-## Relationships
-Locales follow a tree-like structure, so we use tree-like terms to talk about the relationships between locales:
-
-![Locale Tree](locales_tree.svg)
-
- - "en" is the parent of "en-US".
- - "en-US" is the parent of "en-US-tx".
- - "en" is the grandparent of "en-US-tx".
- - "en-US" and "en-CA" are siblings.
- - "en-US-tx" and "en-US-wa" are siblings.
-
-## Compatibility
-There are three types of compatibility:
- - Locale Compatible - When two locales share the same language, region, and dialect.
-  - "en" and "en"
-  - "en-US" and "en-US"
-  - "en-US-wa" and "en-US-wa"
- - Region Compatible - When two locales share the same language and region.
-  - "en-US" and "en-US"
-  - "en-US" and "en-US-wa"
-  - "en-US-wa" and "en-US-tx"
- - Language Compatible - When two locales share the same language.
-  - "en-US" and "en-US"
-  - "en-US" and "en-CA"
-  - "en-US-wa" and "en-US-tx"
-  - "en-US-wa" and "en-CA-mb"
-  - "en" and "en-US-wa"
-  - "en-US" and "en-US-wa"
-
-![Locale Compatibility](locale_compatibility.svg)