Initial Author of this Specification was Ian Hickson, Google Inc., with the following copyright statement:
© Copyright 2004-2011 Apple Computer, Inc., Mozilla Foundation, and Opera Software ASA. You are granted a license to use, reproduce and create derivative works of this document.
All subsequent changes since 26 July 2011 done by the W3C WebRTC Working Group (and previously the Device APIs Working Group) are under the following Copyright © 2011-2023 World Wide Web Consortium. W3C® liability, trademark and permissive document license rules apply.
This document defines a set of JavaScript APIs that allow local media, including audio and video, to be requested from a platform.
This document is not complete. The API is based on preliminary work done in the WHATWG.
Before this document proceeds to Proposed Recommendation, the WebRTC Working Group intends to address issues that emerged from wide review.
This document defines APIs for requesting access to local multimedia devices, such as microphones or video cameras.
This document also defines the MediaStream API, which provides the means to control where multimedia stream data is consumed, and provides some control over the devices that produce the media. It also exposes information about devices able to capture and render media.
This specification defines conformance criteria that apply to a single product: the User Agent that implements the interfaces that it contains.
Conformance requirements phrased as algorithms or specific steps may be implemented in any manner, so long as the end result is equivalent. (In particular, the algorithms defined in this specification are intended to be easy to follow, and not intended to be performant.)
Implementations that use ECMAScript [[ECMA-262]] to implement the APIs defined in this specification must implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [[WEBIDL]], as this specification uses that specification and terminology.
A source is the "thing" providing the source of a media stream track. The source is the broadcaster of the media itself. A source can be a physical webcam, microphone, local video or audio file from the user's hard drive, network resource, or static image. Note that this document describes the use of microphone and camera type sources only, the use of other source types is described in other documents.
An application that has no prior authorization regarding sources is only given the number of available sources, their type and any relationship to other devices. Additional information about sources can become available when applications are authorized to use a source (see ).
Sources do not have constraints — tracks have constraints. When a source is connected to a track, it must produce media that conforms to the constraints present on that track, to that track. Multiple tracks can be attached to the same source. [=User Agent=] processing, such as downsampling, MAY be used to ensure that all tracks have appropriate media.
Sources have constrainable properties which have [=capabilities=] and [=settings=] exposed on tracks. While the constrainable properties are "owned" by the source, sources MAY be able to accommodate different demands at once. For this reason, capabilities are common to any (multiple) tracks that happen to be using the same source, whereas settings MAY differ per track (e.g., if two different track objects bound to the same source query capability and settings information, they will get back the same capabilities, but may get different settings that are tailored to satisfy their individual constraints).
Setting (Source Setting)A setting refers to the immediate, current value of the source's constrainable properties. Settings are always read-only.
A source conditions may dynamically change, such as when a camera switches to a lower frame rate due to low light conditions. In these cases the tracks related to the affected source might not satisfy the set constraints any longer. The platform SHOULD try to minimize such excursions as far as possible, but will continue to deliver media even when a temporary or permanent condition exists that prevents satisfying the constraints.
Although settings are a property of the source, they are only exposed to the application through the tracks attached to the source. This is exposed via the ConstrainablePattern interface.
CapabilityFor each constrainable property, there is a capability that describes whether it is supported by the source and if so, the range of supported values. As with settings, capabilities are exposed to the application via the ConstrainablePattern interface.
The values of the supported capabilities must be normalized to the ranges and enumerated types defined in this specification.
A {{MediaStreamTrack/getCapabilities()}} call on a track returns the same underlying per-source capabilities for all tracks connected to the source.
This API is intentionally simplified. Capabilities are not capable of describing interactions between different values. For instance, it is not possible to accurately describe the capabilities of a camera that can produce a high resolution video stream at a low frame rate and lower resolutions at a higher frame rate. Capabilities describe the complete range of each value. Interactions between constraints are exposed by attempting to apply constraints.
ConstraintsConstraints provide a general control surface that allows applications to both select an appropriate source for a track and, once selected, to influence how a source operates.
Constraints limit the range of operating modes that a source can use when providing media for a track. Without provided track constraints, implementations are free to select a source's settings from the full ranges of its supported capabilities. Implementations may also adjust source settings at any time within the bounds imposed by all applied constraints.
{{MediaDevices/getUserMedia()}} uses constraints to help select an appropriate source for a track and configure it. Additionally, the ConstrainablePattern interface on tracks includes an API for dynamically changing the track's constraints at any later time.
A track will not be connected to a source using {{MediaDevices/getUserMedia()}} if its initial constraints cannot be satisfied. However, the ability to meet the constraints on a track can change over time, and constraints can be changed. If circumstances change such that constraints cannot be met, the ConstrainablePattern interface defines an appropriate error to inform the application. [[[#the-model-sources-sinks-constraints-and-settings]]] explains how constraints interact in more detail.
For each constrainable property, a constraint exists whose name corresponds with the relevant source setting name and capability name.
A constraint falls into one of three groups, depending on its place in the constraints structure. The groups are:
In general, [=User Agents=] will have more flexibility to optimize the media streaming experience the fewer constraints are applied, so application authors are strongly encouraged to use required constraints sparingly.
The two main components in the MediaStream API are the {{MediaStreamTrack}} and {{MediaStream}} interfaces. The {{MediaStreamTrack}} object represents media of a single type that originates from one media source in the [=User Agent=], e.g. video produced by a web camera. A {{MediaStream}} is used to group several {{MediaStreamTrack}} objects into one unit that can be recorded or rendered in a media element.
Each {{MediaStream}} can contain zero or more {{MediaStreamTrack}} objects. All tracks in a {{MediaStream}} are intended to be synchronized when rendered. This is not a hard requirement, since it might not be possible to synchronize tracks from sources that have different clocks. Different {{MediaStream}} objects do not need to be synchronized.
While the intent is to synchronize tracks, it could be better in some circumstances to permit tracks to lose synchronization. In particular, when tracks are remotely sourced and real-time [[?WEBRTC]], it can be better to allow loss of synchronization than to accumulate delays or risk glitches and other artifacts. Implementations are expected to understand the implications of choices regarding synchronization of playback and the effect that these have on user perception.
A single {{MediaStreamTrack}} can represent multi-channel content, such as stereo or 5.1 audio or stereoscopic video, where the channels have a well defined relationship to each other. Information about channels might be exposed through other APIs, such as [[?WEBAUDIO]], but this specification provides no direct access to channels.
A {{MediaStream}} object has an input and an output that represent the combined input and output of all the object's tracks. The output of the {{MediaStream}} controls how the object is rendered, e.g., what is saved if the object is recorded to a file or what is displayed if the object is used in a [^video^] element. A single {{MediaStream}} object can be attached to multiple different outputs at the same time.
A new {{MediaStream}} object can be created from existing media streams or tracks using the {{MediaStream/MediaStream()}} constructor. The constructor argument can either be an existing {{MediaStream}} object, in which case all the tracks of the given stream are added to the new {{MediaStream}} object, or an array of {{MediaStreamTrack}} objects. The latter form makes it possible to compose a stream from different source streams.
Both {{MediaStream}} and {{MediaStreamTrack}} objects can be cloned. A cloned {{MediaStream}} contains clones of all member tracks from the original stream. A cloned {{MediaStreamTrack}} has a set of constraints that is independent of the instance it is cloned from, which allows media from the same source to have different constraints applied for different consumers. The {{MediaStream}} object is also used in contexts outside {{MediaDevices/getUserMedia}}, such as [[?WEBRTC]].
The MediaStream constructor composes a new stream out of existing tracks. It takes an optional argument of type {{MediaStream}} or an array of {{MediaStreamTrack}} objects. When the constructor is invoked, the User Agent must run the following steps:
Let stream be a newly constructed {{MediaStream}} object.
Initialize stream.{{MediaStream/id}} attribute to a newly generated value.
If the constructor's argument is present, run the following steps:
Construct a set of tracks tracks based on the type of argument:
A {{MediaStream}} object:
Let tracks be a set containing all the {{MediaStreamTrack}} objects in the {{MediaStream}} track set.
A sequence of {{MediaStreamTrack}} objects:
Let tracks be a set containing all the {{MediaStreamTrack}} objects in the provided sequence.
For each {{MediaStreamTrack}}, track , in tracks, run the following steps:
If track is already in stream's [=track set=], skip track.
Otherwise, add track to stream's [=track set=].
Return stream.
The tracks of a {{MediaStream}} are stored in a track set. The track set MUST contain the {{MediaStreamTrack}} objects that correspond to the tracks of the stream. The relative order of the tracks in the set is User Agent defined and the API will never put any requirements on the order. The proper way to find a specific {{MediaStreamTrack}} object in the set is to look it up by its {{MediaStreamTrack/id}}.
An object that reads data from the output of a {{MediaStream}} is referred to as a {{MediaStream}} consumer. The list of {{MediaStream}} consumers currently include media elements (such as [^video^] and [^audio^]) [[HTML]], Web Real-Time Communications (WebRTC; {{RTCPeerConnection}}) [[?WEBRTC]], media recording (MediaRecorder) [[?mediastream-recording]], image capture (ImageCapture) [[?image-capture]], and web audio ({{MediaStreamAudioSourceNode}}) [[?WEBAUDIO]].
{{MediaStream}} consumers must be able to handle tracks being added and removed. This behavior is specified per consumer.
A {{MediaStream}} object is said to be active when it has at least one {{MediaStreamTrack}} that has not [=MediaStreamTrack/ended=]. A {{MediaStream}} that does not have any tracks or only has tracks that are [= MediaStreamTrack/ended =] is inactive.
A {{MediaStream}} object is said to be audible when it has at least one {{MediaStreamTrack}} whose {{MediaStreamTrack/[[Kind]]}} is "audio" that has not [=MediaStreamTrack/ended=]. A {{MediaStream}} that does not have any audio tracks or only has audio tracks that are [=MediaStreamTrack/ended=] is inaudible.
The [=User Agent=] may update a {{MediaStream}}'s [=track set=] in response to, for example, an external event. This specification does not specify any such cases, but other specifications using the MediaStream API may. One such example is the WebRTC 1.0 [[?WEBRTC]] specification where the [=track set=] of a {{MediaStream}}, received from another peer, can be updated as a result of changes to the media session.
To add a track track to a {{MediaStream}} stream, the [=User Agent=] MUST run the following steps:
If track is already in stream's [=track set=], then abort these steps.
Add track to stream's [=track set=].
[= Fire a track event=] named {{addtrack}} with track at stream.
To remove a track track from a {{MediaStream}} stream, the [=User Agent=] MUST run the following steps:
If track is not in stream's [=track set=], then abort these steps.
[=set/Remove=] track from stream's [=track set=].
[= Fire a track event =] named {{removetrack}} with track at stream.
See the MediaStream constructor algorithm
See the MediaStream constructor algorithm
{{MediaStream}}See the MediaStream constructor algorithm
The id attribute MUST return the value to which it was initialized when the object was created.
When a {{MediaStream}} is created, the User Agent MUST generate an identifier string, and MUST initialize the object's {{id}} attribute to that string, unless the object is created as part of a special purpose algorithm that specifies how the stream id must be initialized. A good practice is to use a UUID [[rfc4122]], which is 36 characters long in its canonical form. To avoid fingerprinting, implementations SHOULD use the forms in section 4.4 or 4.5 of RFC 4122 when generating UUIDs.
An example of an algorithm that specifies how the stream id must be initialized is the algorithm to associate an incoming network component with a {{MediaStream}} object. [[?WEBRTC]]
active of type {{boolean}}, readonlyThe {{active}} attribute MUST return true if this {{MediaStream}} is [= stream/active =] and false otherwise.
onaddtrack of type {{EventHandler}}The event type of this event handler is {{addtrack}}.
onremovetrack of type {{EventHandler}}The event type of this event handler is {{removetrack}}.
Returns a sequence of {{MediaStreamTrack}} objects representing the audio tracks in this stream.
The {{getAudioTracks}} method MUST return a sequence that represents a snapshot of all the {{MediaStreamTrack}} objects in this stream's [=track set=] whose {{MediaStreamTrack/[[Kind]]}} is equal to "audio". The conversion from the [=track set=] to the sequence is [=User Agent=] defined and the order does not have to be stable between calls.
getVideoTracks()Returns a sequence of {{MediaStreamTrack}} objects representing the video tracks in this stream.
The {{getVideoTracks}} method MUST return a sequence that represents a snapshot of all the {{MediaStreamTrack}} objects in this stream's [=track set=] whose {{MediaStreamTrack/[[Kind]]}} is equal to "video". The conversion from the [=track set=] to the sequence is [=User Agent=] defined and the order does not have to be stable between calls.
getTracks()Returns a sequence of {{MediaStreamTrack}} objects representing all the tracks in this stream.
The {{getTracks}} method MUST return a sequence that represents a snapshot of all the {{MediaStreamTrack}} objects in this stream's [=track set=], regardless of {{MediaStreamTrack/[[Kind]]}}. The conversion from the [=track set=] to the sequence is User Agent defined and the order does not have to be stable between calls.
getTrackById()The {{getTrackById}} method MUST return either a {{MediaStreamTrack}} object from this stream's [=track set=] whose {{MediaStreamTrack/[[Id]]}} is equal to trackId, or null, if no such track exists.
addTrack()Adds the given {{MediaStreamTrack}} to this {{MediaStream}}.
When the {{addTrack}} method is invoked, the [=User Agent=] MUST run the following steps:
Let track be the methods argument and stream the {{MediaStream}} object on which the method was called.
If track is already in stream's [=track set=], then abort these steps.
[=MediaStream/Add a track|Add=] track to stream's [=track set=].
Removes the given {{MediaStreamTrack}} object from this {{MediaStream}}.
When the {{removeTrack}} method is invoked, the [=User Agent=] MUST run the following steps:
Let track be the methods argument and stream the {{MediaStream}} object on which the method was called.
If track is not in stream's [=track set=], then abort these steps.
[=MediaStream/Remove a track|Remove=] track from stream's [=track set=].
Clones the given {{MediaStream}} and all its tracks.
When the {{clone()}} method is invoked, the User Agent MUST run the following steps:
Let streamClone be a newly constructed {{MediaStream}} object.
Initialize streamClone.{{MediaStream.id}} to a newly generated value.
Clone each track in this {{MediaStream}} object and add the result to streamClone's track set.
User agent code that expects to fire the addtrack or removetrack events is expected to keep the target {{MediaStream}} objects alive by reference from another object. The presence of addtrack or removetrack event listeners does not need to be taken into account when trying to garbage collect {{MediaStream}} objects.
A {{MediaStreamTrack}} object represents a media source in the [=User Agent=]. An example source is a device connected to the [=User Agent=]. Other specifications may define sources for {{MediaStreamTrack}} that override the behavior specified here. Several {{MediaStreamTrack}} objects can represent the same media source, e.g., when the user chooses the same camera in the UI shown by two consecutive calls to {{MediaDevices/getUserMedia()}}.
A {{MediaStreamTrack}} source defines the following properties:
The data from a {{MediaStreamTrack}} object does not necessarily have a canonical binary form; for example, it could just be "the video currently coming from the user's video camera". This allows [=User Agents=] to manipulate media in whatever fashion is most suitable on the user's platform.
A script can indicate that a {{MediaStreamTrack}} object no longer needs its source with the {{MediaStreamTrack/stop()}} method. When all tracks using a source have been stopped or ended by some other means, the source is stopped. If the source is a device exposed by {{MediaDevices/getUserMedia()}}, then when the source is stopped, the [=User Agent=] MUST run the following steps:
Let mediaDevices be the {{MediaDevices}} object in question.
Let deviceId be the source device's {{MediaDeviceInfo/deviceId}}.
Set mediaDevices.{{MediaDevices/[[devicesLiveMap]]}}[deviceId] to false.
If the [=permission state=] of the permission associated with the device's kind and deviceId for mediaDevices's [=relevant settings object=], is not {{PermissionState/"granted"}}, then set mediaDevices.{{MediaDevices/[[devicesAccessibleMap]]}}[deviceId] to false.
To create a MediaStreamTrack with an underlying source, and a mediaDevicesToTieSourceTo, run the following steps:
Let track be a new object of type source's [=MediaStreamTrack source type=].
Initialize track with the following internal slots:
[[\Source]], initialized to source.
[[\Id]], initialized to a newly generated unique identifier string. See {{MediaStream.id}} attribute for guidelines on how to generate such an identifier.
[[\Kind]], initialized to "audio" if source is an audio source, or "video" if source is a video source.
[[\Label]], initialized to source's label, if provided by the User Agent, or "" otherwise. [=User Agents=] MAY label audio and video sources (e.g., "Internal microphone" or "External USB Webcam").
[[\ReadyState]], initialized to {{MediaStreamTrackState/"live"}}.
[[\Enabled]], initialized to true.
[[\Muted]], initialized to true if source is [= source/muted =], and false otherwise.
[[\Restrictable]], initialized to false.
If mediaDevicesToTieSourceTo is not null, [=tie track source to `MediaDevices`=] with source and mediaDevicesToTieSourceTo.
Run source's [=MediaStreamTrack source-specific construction steps=] with track as parameter.
Return track.
To initialize the underlying source of track to source, run the following steps:
Initialize track.{{MediaStreamTrack/[[Source]]}} to source.
Initialize track's [[\Capabilities]], [[\Constraints]], and [[\Settings]], as specified in the {{ConstrainablePattern}}.
To tie track source to `MediaDevices`, given source and mediaDevices, run the following steps:
Add source to mediaDevices.{{MediaDevices/[[mediaStreamTrackSources]]}}.
To stop all sources of a [=global object=], named globalObject, the [=User Agent=] MUST run the following steps:
For each {{MediaStreamTrack}} object track whose relevant global object is globalObject, set track's {{MediaStreamTrack/[[ReadyState]]}} to {{MediaStreamTrackState/"ended"}}.
If globalObject is a {{Window}}, then for each source in globalObject's [=associated `MediaDevices`=].{{MediaDevices/[[mediaStreamTrackSources]]}}, [= source/stopped | stop =] source.
The [=User Agent=] MUST [=stop all sources=] of a globalObject in the following conditions:
If globalObject is a {{Window}} object and the [=unloading document cleanup steps=] are executed for its [=associated document=].
If globalObject is a {{WorkerGlobalScope}} object and its closing flag is set to true.
An implementation may use a per-source reference count to keep track of source usage, but the specifics are out of scope for this specification.
To clone a track the [=User Agent=] MUST run the following steps:
Let track be the {{MediaStreamTrack}} object to be cloned.
Let source be track's {{MediaStreamTrack/[[Source]]}}.
Let trackClone be the result of [=create a MediaStreamTrack | creating a MediaStreamTrack=] with source and null.
Set trackClone's {{MediaStreamTrack/[[ReadyState]]}} to track's {{MediaStreamTrack/[[ReadyState]]}} value.
Set trackClone's [[\Capabilities]] to a clone of track's [[\Capabilities]].
Set trackClone's [[\Constraints]] to a clone of track's [[\Constraints]].
Set trackClone's [[\Settings]] to a clone of track's [[\Settings]].
Run source [=MediaStreamTrack source-specific clone steps=] with track and trackClone as parameters.
Return trackClone.
There are two dimensions related to the media flow for a {{MediaStreamTrackState/"live"}} {{MediaStreamTrack}} : muted / not muted, and enabled / disabled.
Muted refers to the input to the {{MediaStreamTrack}}. A {{MediaStreamTrack}} is [= MediaStreamTrack/muted =] when its source is muted, i.e. temporarily unable to provide the track with data. Live samples MUST NOT be made available to a {{MediaStreamTrack}} while it is [=MediaStreamTrack/muted=].
The [=MediaStreamTrack/muted=] state is outside the control of web applications, but can be observed by the application by reading the {{MediaStreamTrack/muted}} attribute and listening to the associated events {{mute}} and {{unmute}}. The reasons for a {{MediaStreamTrack}} to be muted are defined by its source.
For camera and microphone sources, the reasons to [=source/muted|mute=] are [=implementation-defined=]. This allows user agents to implement privacy mitigations in situations like: the user pushing a physical mute button on the microphone, the user closing a laptop lid with an embedded camera, the user toggling a control in the operating system, the user clicking a mute button in the [=User Agent=] chrome, the [=User Agent=] (on behalf of the user) mutes, etc.
On some operating systems, microphone access may get stolen from the [=User Agent=] when another application with higher-audio priority gets access to it, for instance in case of an incoming phone call on mobile OS. The [=User Agent=] SHOULD provide this information to the web application through {{MediaStreamTrack/muted}} and its associated events.
Whenever the [=User Agent=] initiates such an [= implementation-defined=] change for camera or microphone sources, it MUST queue a task, using the user interaction task source, to [=MediaStreamTrack/set a track's muted state=] to the state desired by the user.
To set a track's muted state to newState, the [=User Agent=] MUST run the following steps:
Let track be the {{MediaStreamTrack}} in question.
If track.{{MediaStreamTrack/[[Muted]]}} is already newState, then abort these steps.
Set track.{{MediaStreamTrack/[[Muted]]}} to newState.
If newState is true let eventName be {{mute}}, otherwise {{unmute}}.
[=Fire an event=] named eventName on track.
Enabled/disabled on the other hand is available to the application to control (and observe) via the {{MediaStreamTrack/enabled}} attribute.
The result for the consumer is the same in the sense that whenever {{MediaStreamTrack}} is muted or disabled (or both) the consumer gets zero-information-content, which means silence for audio and black frames for video. In other words, media from the source only flows when a {{MediaStreamTrack}} object is both unmuted and enabled. For example, a video element sourced by a {{MediaStream}} containing only muted or disabled {{MediaStreamTrack}}s for audio and video, is playing but rendering black video frames in silence.
For a newly created {{MediaStreamTrack}} object, the following applies: the track is always enabled unless stated otherwise (for example when cloned) and the muted state reflects the state of the source at the time the track is created.
A {{MediaStreamTrack}} has two states in its life-cycle: live and ended. A newly created {{MediaStreamTrack}} can be in either state depending on how it was created. For example, cloning an ended track results in a new ended track. The current state is reflected by the object's {{MediaStreamTrack/readyState}} attribute.
In the live state, the track is active and media (or zero-information-content if the {{MediaStreamTrack}} is [= MediaStreamTrack/muted =] or [= MediaStreamTrack/enabled | disabled =]) is available for use by consumers.
If the source is a device exposed by `navigator.mediaDevices.`{{MediaDevices/getUserMedia()}}, then when a track becomes either muted or disabled, and this brings all tracks connected to the device to be either muted, disabled, or stopped, then the UA MAY, using the device's {{MediaDeviceInfo/deviceId}}, deviceId, set `navigator.mediaDevices.`{{MediaDevices/[[devicesLiveMap]]}}[deviceId] to false, provided the UA sets it back to true as soon as any unstopped track connected to this device becomes un-muted or enabled again.
When a {{MediaStreamTrackState/"live"}}, [= MediaStreamTrack/muted | unmuted =], and [= MediaStreamTrack/enabled =] track sourced by a device exposed by {{MediaDevices/getUserMedia()}} becomes either [= MediaStreamTrack/muted =] or [= MediaStreamTrack/enabled | disabled =], and this brings all tracks connected to the device (across all [=navigables=] the user agent operates) to be either muted, disabled, or stopped, then the UA SHOULD relinquish the device within 3 seconds while allowing time for a reasonably-observant user to become aware of the transition. The UA SHOULD attempt to reacquire the device as soon as any live track sourced by the device becomes both [= MediaStreamTrack/muted | unmuted =] and [= MediaStreamTrack/enabled =] again, provided that track's [=relevant global object=]'s [=associated `Document`=] [=Document/is in view=] at that time. If the document is not [=Document/is in view|in view=] at that time, the UA SHOULD instead queue a task to [=MediaStreamTrack/muted|mute=] the track, and not queue a task to [=MediaStreamTrack/muted|unmute=] it until the document comes [=Document/is in view|into view=]. If reacquiring the device fails, the UA MUST [= track ended by the User agent | end the track =] (The UA MAY end it earlier should it detect a device problem, like the device being physically removed).
The intent is to give users the assurance of privacy that having physical camera (and microphone) hardware lights off brings, by aligning physical and logical “privacy indicators”, at least while the current document is the sole user of a device.
While other applications and documents using the device simultaneously may interfere with this intent at times, they do not interfere with the rules laid forth.
A {{MediaStreamTrack}} object is said to end when the source of the track is disconnected or exhausted.
If all {{MediaStreamTrack}}s that are using the same source are [= MediaStreamTrack/ended =], the source will be [= source/stopped =].
After the application has invoked the {{MediaStreamTrack/stop()}} method on a {{MediaStreamTrack}} object, or once the [=source=] of a {{MediaStreamTrack}} permanently ends production of live samples to its tracks, whichever is sooner, a {{MediaStreamTrack}} is said to be ended.
For camera and microphone sources, the reasons for a source to [=MediaStreamTrack/ended|end=] besides {{MediaStreamTrack/stop()}} are [=implementation-defined=] (e.g., because the user rescinds the permission for the page to use the local camera, or because the User Agent has instructed the track to end for any reason).
When a {{MediaStreamTrack}} track ends for any reason other than the {{MediaStreamTrack/stop()}} method being invoked, the [=User Agent=] MUST queue a task that runs the following steps:
If track's {{MediaStreamTrack/[[ReadyState]]}} has the value {{MediaStreamTrackState/"ended"}} already, then abort these steps.
Set track's {{MediaStreamTrack/[[ReadyState]]}} to {{MediaStreamTrackState/"ended"}}.
Notify track's {{MediaStreamTrack/[[Source]]}} that track is [= MediaStreamTrack/ended =] so that the source may be [= source/stopped =], unless other {{MediaStreamTrack}} objects depend on it.
[=Fire an event=] named ended at the object.
If the end of the track was reached due to a user request, the event source for this event is the user interaction event source.
To invoke the device permission revocation algorithm with permissionName, run the following steps:
Let tracks be the set of all currently {{MediaStreamTrackState/"live"}} MediaStreamTracks whose permission associated with this kind of track ("camera" or "microphone") matches permissionName.
For each track in tracks, end the track.
{{MediaStreamTrack}} is a constrainable object as defined in the Constrainable Pattern section. Constraints are set on tracks and may affect sources.
Whether Constraints were provided at track initialization time or need to be established later at runtime, the APIs defined in the ConstrainablePattern Interface allow the retrieval and manipulation of the constraints currently established on a track.
Once ended, a track will continue exposing a list of inherent constrainable track properties. This list contains deviceId, facingMode and groupId.
The kind attribute MUST return [=this=].{{MediaStreamTrack/[[Kind]]}}.
{{id}} of type {{DOMString}}, readonlyThe id attribute MUST return [=this=].{{MediaStreamTrack/[[Id]]}}.
{{label}} of type {{DOMString}}, readonlyThe label attribute MUST return [=this=].{{MediaStreamTrack/[[Label]]}}.
{{enabled}} of type {{boolean}}The enabled attribute controls the [= MediaStreamTrack/enabled =] state for the object.
On getting, [=this=].{{MediaStreamTrack/[[Enabled]]}} MUST be returned. On setting, [=this=].{{MediaStreamTrack/[[Enabled]]}} MUST be set to the new value.
Thus, after a {{MediaStreamTrack}} has [= MediaStreamTrack/ended =], its {{MediaStreamTrack/enabled}} attribute still changes value when set; it just doesn't do anything with that new value.
{{muted}} of type {{boolean}}, readonlyThe muted attribute reflects whether the track is [= MediaStreamTrack/muted =]. It MUST return [=this=].{{MediaStreamTrack/[[Muted]]}}.
onmute of type {{EventHandler}}The event type of this event handler is mute.
onunmute of type {{EventHandler}}The event type of this event handler is unmute.
{{readyState}} of type {{MediaStreamTrackState}}, readonlyOn getting, the readyState attribute MUST return [=this=].{{MediaStreamTrack/[[ReadyState]]}}.
onended of type {{EventHandler}}The event type of this event handler is ended.
When the {{clone()}} method is invoked, the [=User Agent=] MUST return the result of [=clone a track=] with [=this=].
stopWhen a {{MediaStreamTrack}} object's {{stop()}} method is invoked, the User Agent MUST run following steps:
Let track be the current {{MediaStreamTrack}} object.
If track's {{MediaStreamTrack/[[ReadyState]]}} is {{MediaStreamTrackState/"ended"}}, then abort these steps.
Notify track's source that track is [= MediaStreamTrack/ended =].
A source that is notified of a track ending will be [= source/stopped =], unless other {{MediaStreamTrack}} objects depend on it.
Set track's {{MediaStreamTrack/[[ReadyState]]}} to {{MediaStreamTrackState/"ended"}}.
Returns the capabilites of the source that this {{MediaStreamTrack}}, the constrainable object, represents.
See ConstrainablePattern Interface for the definition of this method.
Since this method gives likely persistent, cross-origin information about the underlying device, it adds to the fingerprint surface of the device.
getConstraintsSee ConstrainablePattern Interface for the definition of this method.
getSettingsWhen a {{MediaStreamTrack}} object's {{MediaStreamTrack.getSettings()}} method is invoked, the [=User Agent=] MUST run following steps:
Let track be the current {{MediaStreamTrack}} object.
If track's {{MediaStreamTrack/[[ReadyState]]}} is {{MediaStreamTrackState/"ended"}}, run the following sub steps:
Let settings be a new {{MediaTrackSettings}} dictionary.
For each property of the list of inherent constrainable track properties, add a corresponding property to settings if track had such property at the time it was ended, with the value at the time track was ended.
Return settings.
Return the current settings of the track as defined in ConstrainablePattern Interface.
When a {{MediaStreamTrack}} object's {{applyConstraints()}} method is invoked, the User Agent MUST run following steps:
Let track be the current {{MediaStreamTrack}} object.
If track's {{MediaStreamTrack/[[ReadyState]]}} is {{MediaStreamTrackState/"ended"}}, run the following sub steps:
Let p be a new promise.
[= resolve =] p with undefined.
Return p.
Invoke and return the result of the applyConstraints template method where:
Other specifications may define constrainable properties that at times must not be exposed.
For every [=settings dictionary=] with resizeMode set to "none", the [=User Agent=] MUST include another otherwise identical [=settings dictionary=] with resizeMode set to "crop-and-scale". Constraining around non-native modes is not supported.
The net effect is to reflect that crop-and-scale is a superset of none.
| live |
The track is active (the track's underlying media source is making a best-effort attempt to provide data in real time). The output of a track in the {{MediaStreamTrackState/"live"}} state can be switched on and off with the {{MediaStreamTrack/enabled}} attribute. |
| ended |
The track has [= MediaStreamTrack/ended =] (the track's underlying media source is no longer providing data, and will never provide more data for this track). Once a track enters this state, it never exits it. For example, a video track in a {{MediaStream}} ends when the user unplugs the USB web camera that acts as the track's media source. |
{{MediaTrackSupportedConstraints}} represents the list of constraints recognized by a [=User Agent=] for controlling the Capabilities of a {{MediaStreamTrack}} object. This dictionary is used as a function return value, and never as an operation argument.
Future specifications can extend the {{MediaTrackSupportedConstraints}} dictionary by defining a partial dictionary with dictionary members of type {{boolean}}.
The constraints specified in this specification apply only to instances of {{MediaStreamTrack}} generated by {{MediaDevices.getUserMedia()}}, unless stated otherwise in other specifications.
{{MediaTrackCapabilities}} represents the Capabilities of a {{MediaStreamTrack}} object.
Future specifications can extend the MediaTrackCapabilities dictionary by defining a partial dictionary with dictionary members of appropriate type.
For historical reasons, {{MediaTrackCapabilities/deviceId}} and {{MediaTrackCapabilities/groupId}} are {{DOMString}} instead of the `sequence<DOMString>` expected by {{Capabilities}} in the ConstrainablePattern.
A camera can report multiple facing modes. For example, in a high-end telepresence solution with several cameras facing the user, a camera to the left of the user can report both {{VideoFacingModeEnum/"left"}} and {{VideoFacingModeEnum/"user"}}. See facingMode for additional details.
resizeMode of type sequence<{{DOMString}}>The [=User Agent=] MAY use cropping and downscaling to offer more resolution choices than this camera naturally produces. The reported sequence MUST list all the means the UA may employ to derive resolution choices for this camera. The value {{VideoResizeModeEnum/"none"}} MUST be present, indicating the ability to constrain the UA from cropping and downscaling. See resizeMode for additional details.
sampleRate of type {{ULongRange}} See sampleRate for details. sampleSize of type {{ULongRange}} See sampleSize for details. echoCancellation of type sequence<{{boolean}}>If the source cannot do echo cancellation a single false MUST be the only element in the list. If the source can do echo cancellation, then true MUST be included in the list. If the script can control the feature, the list MUST include at least both true and false. Additionally, if the source allows controlling which audio sources will be cancelled, it must include any supported values from the {{EchoCancellationModeEnum}} enum. If true or false are included in the list, they must appear before any value from {{EchoCancellationModeEnum}}. See echoCancellation for additional details.
autoGainControl of type sequence<{{boolean}}>If the source cannot do auto gain control a single false is reported. If auto gain control cannot be turned off, a single true is reported. If the script can control the feature, the source reports a list with both true and false as possible values. See autoGainControl for additional details.
noiseSuppression of type sequence<{{boolean}}>If the source cannot do noise suppression a single false is reported. If noise suppression cannot be turned off, a single true is reported. If the script can control the feature, the source reports a list with both true and false as possible values. See noiseSuppression for additional details.
latency of type {{DoubleRange}} See latency for details. channelCount of type {{ULongRange}} See channelCount for details. deviceId of type {{DOMString}} See deviceId for details. groupId of type {{DOMString}} See groupId for details. backgroundBlur of type sequence<{{boolean}}> If the source does not have built-in background blurring, a single false is reported. If background blurring cannot be turned off, a single true is reported. If the script can control the feature, the source reports a list with both true and false as possible values. See backgroundBlur for details.See Constraints and ConstraintSet for the definition of this element.
Future specifications can extend the MediaTrackConstraintSet dictionary by defining a partial dictionary with dictionary members of appropriate type.
{{MediaTrackSettings}} represents the Settings of a {{MediaStreamTrack}} object.
Future specifications can extend the MediaTrackSettings dictionary by defining a partial dictionary with dictionary members of appropriate type.
The names of the initial set of constrainable properties for MediaStreamTrack are defined below.
The following constrainable properties are defined to apply to both video and audio {{MediaStreamTrack}} objects:
| deviceId | {{DOMString}} | The identifier of the device generating the content of the {{MediaStreamTrack}}. It conforms with the definition of {{MediaDeviceInfo.deviceId}}. Note that the setting of this property is uniquely determined by the source that is attached to the {{MediaStreamTrack}}. In particular, {{MediaStreamTrack/getCapabilities()}} will return only a single value for deviceId. This property can therefore be used for initial media selection with {{MediaDevices/getUserMedia()}}. However, it is not useful for subsequent media control with {{MediaStreamTrack/applyConstraints()}}, since any attempt to set a different value will result in an unsatisfiable ConstraintSet. If a string of length 0 is used as a deviceId value constraint with {{MediaDevices/getUserMedia()}}, it MAY be interpreted as if the constraint is not specified. |
| groupId | {{DOMString}} | The [=document=]-unique group identifier for the device generating the content of the {{MediaStreamTrack}}. It conforms with the definition of {{MediaDeviceInfo.groupId}}. Note that the setting of this property is uniquely determined by the source that is attached to the {{MediaStreamTrack}}. In particular, {{MediaStreamTrack/getCapabilities()}} will return only a single value for groupId. Since this property is not stable between browsing sessions, its usefulness for initial media selection with {{MediaDevices/getUserMedia()}} is limited. It is not useful for subsequent media control with {{MediaStreamTrack/applyConstraints()}}, since any attempt to set a different value will result in an unsatisfiable ConstraintSet. |
The following constrainable properties are defined to apply only to video {{MediaStreamTrack}} objects:
| width | {{unsigned long}} | The width, in pixels. As a capability, its valid range should span the video source's pre-set width values with min being equal to 1 and max being the largest width. The [=User Agent=] MUST support downsampling to any value between the min width range value and the native resolution width. |
| height | {{unsigned long}} | The height, in pixels. As a capability, its valid range should span the video source's pre-set height values with min being equal to 1 and max being the largest height. The [=User Agent=] MUST support downsampling to any value between the min height range value and the native resolution height. |
| frameRate | {{double}} | The frame rate (frames per second). If video source's pre-set can determine frame rates, then, as a capability, its valid range should span the video source's pre-set frame rate values with min being equal to 0 and max being the largest frame rate. The [=User Agent=] MUST support frame rates obtained from integral decimation of the native resolution frame rate. If frame rate cannot be determined (e.g. the source does not natively provide a frame rate, or the frame rate cannot be determined from the source stream), then the capability values MUST refer to the [=User Agent=]'s vsync display rate. As a setting, this value represents the configured frame rate. If decimation is used, this is that value rather than the native frame rate. For example, if the setting is 25 frames per second via decimation, the native frame rate of the camera is 30 frames per second but due to lighting conditions only 20 frames per second is achieved, {{frameRate}} reports the setting: 25 frames per second. |
| aspectRatio | {{double}} | The exact aspect ratio (width in pixels divided by height in pixels, represented as a double rounded to the tenth decimal place) or aspect ratio range. |
| facingMode | {{DOMString}} | This string is one of the members of {{VideoFacingModeEnum}}. The members describe the directions that the camera can face, as seen from the user's perspective. Note that getConstraints may not return exactly the same string for strings not in this enum. This preserves the possibility of using a future version of WebIDL enum for this property. |
| resizeMode | {{DOMString}} |
This string is one of the
members of {{VideoResizeModeEnum}}. The members
describe the means by which the resolution can be derived by
the UA. In other words, whether the UA is allowed to use
cropping and downscaling on the camera output.
The UA MAY disguise concurrent use of the camera, by downscaling, upscaling, and/or cropping to mimic native resolutions when "none" is used, but only when the camera is in use in another application outside the [=User Agent=]. Note that getConstraints may not return exactly the same string for strings not in this enum. This preserves the possibility of using a future version of WebIDL enum for this property. |
| backgroundBlur | {{boolean}} | Some platforms or User Agents may provide built-in support for background blurring of video frames, in particular for camera video streams. Web applications may either want to control or at least be aware that background blur is applied at the source level. This may for instance allow the web application to update its UI or to not apply background blur on its own. |
On systems where it's desirable to sometimes automatically flip the X and Y axis of the resulting captured video in response to ongoing environmental factors, the {{width}}, {{height}} and {{aspectRatio}} constraints and capabilities MUST remain unaffected in all algorithms and be considered in the primary orientation only, except for the {{MediaStreamTrack/getSettings()}} algorithm where settings for these constrainable properties MUST be flipped if necessary to match the returned dimensions of the captured video at any point in time.
The primary orientation of a system that supports flipping the X and Y axis of resulting captured video is defined by the User Agent for the particular system.
On systems that support automatic switching between landscape and portrait mode, [=User Agents=] are encouraged to make landscape mode the primary orientation.
| user |
The source is facing toward the user (a self-view camera). |
| environment |
The source is facing away from the user (viewing the environment). |
| left |
The source is facing to the left of the user. |
| right |
The source is facing to the right of the user. |
Below is an illustration of the video facing modes in relation to
the user.
| none |
This resolution and frame rate is offered by the camera, its driver, or the OS. Note: The UA MAY report this value to disguise concurrent use, but only when the camera is in use in another [=navigable=]. |
| crop-and-scale |
This resolution is downscaled and/or cropped from a higher camera resolution by the [=User Agent=], or its frame rate is decimated by the [=User Agent=]. The media MUST NOT be upscaled, stretched or have fake data created that did not occur in the input source, except as noted below. Note: The UA MAY upscale to disguise concurrent use, but only when the camera is in use in another application outside the [=User Agent=]. |
The following constrainable properties are defined to apply only to audio {{MediaStreamTrack}} objects:
| sampleRate | {{unsigned long}} | The sample rate in samples per second for the audio data. |
| sampleSize | {{unsigned long}} | The linear sample size in bits. As a constraint, it can only be satisfied for audio devices that produce linear samples. |
| echoCancellation | {{boolean}} or {{DOMString}} | This is either false, true, or one of the members of {{EchoCancellationModeEnum}}. When one or more audio streams are being played in the processes of various microphones, it is often desirable to attempt to remove sound being played from the input signals recorded by the microphones. This is referred to as echo cancellation. There are cases where it is not needed and it is desirable to turn it off so that no audio artifacts are introduced. This allows applications to control this behavior. |
| autoGainControl | {{boolean}} | Automatic gain control is often desirable on the input signal recorded by the microphone. There are cases where it is not needed and it is desirable to turn it off so that the audio is not altered. This allows applications to control this behavior. |
| noiseSuppression | {{boolean}} | Noise suppression is often desirable on the input signal recorded by the microphone. There are cases where it is not needed and it is desirable to turn it off so that the audio is not altered. This allows applications to control this behavior. |
| latency | {{double}} | The latency or latency range, in seconds. The latency is the time between start of processing (for instance, when sound occurs in the real world) to the data being available to the next step in the process. Low latency is critical for some applications; high latency may be acceptable for other applications because it helps with power constraints. The number is expected to be the target latency of the configuration; the actual latency may show some variation from that. |
| channelCount | {{unsigned long}} | The number of independent channels of sound that the audio data contains, i.e. the number of audio samples per sample frame. |
| {{EchoCancellationModeEnum/"all"}} |
The system MUST attempt to remove all sound being played by the system from the input signal of the microphone. This option is meant to provide maximum privacy, as it prevents the transmission of local audio such as notifications or screen readers. |
| {{EchoCancellationModeEnum/"remote-only"}} |
The system MUST attempt to remove the sound from incoming audio {{MediaStreamTrack}}s sourced from WebRTC {{RTCPeerConnection}}s. This option is useful for cases where it is desirable to transmit locally played audio. One example is a remote music class, where a student plays an instrument together with some accompaniment produced by a local application. In this case, the application requires audio coming from the remote participant (i.e., the teacher) to be cancelled in order to avoid echo, but also requires that the accompaniment not be cancelled since the music teacher on the remote side needs to hear it together with the sound from the instrument.It is up to the UA to decide which {{RTCPeerConnection}}s to cancel, but the ones being played out by the browsing context capturing microphone SHOULD be among those cancelled. |
In addition to the values from {{EchoCancellationModeEnum}}, the {{echoCancellation}} constrainable property also accepts the values true and false. false means that no echo cancellation will take place. true means that the UA decides what audio will be removed from the signals recorded by the microphone. true MUST attempt to cancel at least as much as {{EchoCancellationModeEnum/"remote-only"}} and SHOULD attempt to cancel as much as {{EchoCancellationModeEnum/"all"}}.
A {{MediaStreamTrack}} object MUST NOT be garbage collected if it is not [=MediaStreamTrack/ended=] and there are any event listeners registered for mute, unmute or ended events. Each source type can further refine the garbage collection rules as sources may never fire a particular event.
The {{addtrack}} and {{removetrack}} events use the {{MediaStreamTrackEvent}} interface.
The {{addtrack}} and {{removetrack}} events notify the script that the [=track set=] of a {{MediaStream}} has been updated by the [=User Agent=].
Firing a track event named e with a {{MediaStreamTrack}} track means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the {{MediaStreamTrackEvent}} interface with the {{MediaStreamTrackEvent/track}} attribute set to track, MUST be created and dispatched at the given target.
Constructs a new {{MediaStreamTrackEvent}}.
The track attribute represents the {{MediaStreamTrack}} object associated with the event.
[=User Agents=] provide a media pipeline from sources to sinks. In a [=User Agent=], sinks are the <[^img^]>, <[^video^]>, and <[^audio^]> tags. Traditional sources include streamed content, files, and web resources. The media produced by these sources typically does not change over time - these sources can be considered to be static.
The sinks that display these sources to the user (the actual tags themselves) have a variety of controls for manipulating the source content. For example, an <[^img^]> tag scales down a huge source image of 1600x1200 pixels to fit in a rectangle defined with width="400" and height="300".
Sources have a lifetime. By default, a source lifetime is tied to the context that created it. For instance, sources created by {{MediaDevices.getUserMedia()}} are considered as created by its navigator.{{mediaDevices}} context. Similarly, sources of {{RTCRtpReceiver}} objects are bound to the {{RTCPeerConnection}} itself, which is bound to its creation context. Except if stated explicitly in the definition of specific sources, a source is always [= source/stopped =] when its creation context goes away. It should be noted that two sources of different contexts may use the same capture device at the same time. One source can be stopped independently of the other one.
The getUserMedia API adds dynamic sources such as microphones and cameras - the characteristics of these sources can change in response to application needs. These sources can be considered to be dynamic in nature. A <[^video^]> element that displays media from a dynamic source can either perform scaling or it can feed back information along the media pipeline and have the source produce content more suitable for display.
Note: This sort of feedback loop is obviously just enabling an "optimization", but it's a non-trivial gain. This optimization can save battery, allow for less network congestion, etc...
Note that {{MediaStream}} sinks (such as <[^video^]>, <[^audio^]>, and even {{RTCPeerConnection}}) will continue to have mechanisms to further transform the source stream beyond that which the Settings, Capabilities, and Constraints described in this specification offer. (The sink transformation options, including those of {{RTCPeerConnection}}, are outside the scope of this specification.)
The act of changing or applying a track constraint may affect the [=settings=] of all tracks sharing that source and consequently all down-level sinks that are using that source. Many sinks may be able to take these changes in stride, such as the <[^video^]> element or {{RTCPeerConnection}}. Others like the Recorder API may fail as a result of a source setting change.
The {{RTCPeerConnection}} is an interesting object because it acts simultaneously as both a sink and a source for over-the-network streams. As a sink, it has source transformational capabilities (e.g., lowering bit-rates, scaling-up / down resolutions, and adjusting frame-rates), and as a source it could have its own settings changed by a track source.
To illustrate how changes to a given source impact various sinks, consider the following example. This example only uses width and height, but the same principles apply to all of the Settings exposed in this specification. In the first figure a home client has obtained a video source from its local video camera. The source's width and height settings are 800 pixels and 600 pixels, respectively. Three {{MediaStream}} objects on the home client contain tracks that use this same