Media Service

The media service provides interfaces for controlling playback and recording of streaming media. The service is capable of handling a broad range of media content in different formats from numerous sources. It is very flexible in many ways:

• Stream Sources The media service can fetch media streams from the network using many popular networking protocols.
• Stream Formats The media player knows all stream formats in common use for MPEG1 and MPEG2 encoded content: system streams, program streams, and transport streams. It also supports elementary streams (like MP3) and RTP, which is sometimes used for MPEG4.
• Conditional Access The media service can be configured to handle conditional access systems.
• Content Encodings Decoding of video and audio content, regardless of encoder used, is handled transparently.

The media service is capable of handling multiple media player instances concurrently. Each instance has the capacity of handling one media stream. Two player instances running on the same system are completely separated from each other. Regardless what happens to one player, the other players on the system are unaffected.

Overview

The media service provides a number of streaming media resources to applications:

• Media Players Media players connect to streaming media resources, detect the type of media content, and use the appropriate hardware resources to decode and optionally decrypt the media stream. Applications use media players to control the playback of video music.
• Picture in Picture Media Players Picture in picture media players connect to streaming media resources and decode and render video. At the same time the regular media players are able to decode and render video from another sources.
• Media Recorders Media recorders connect to streaming media resources, and write the media streams to permanent data storage areas, usually a disk. Applications may use recorders to implement DVR functionality.
• Media Session Interfaces Media sessions is the collective name of media players, recorders and distributors. An application can get an interface to any media player, recorder or distributor independent on what application that created it. The player, recorder or distributor is then accessed through a media session interface, where the application can only use a subset of the functionality available in the player, recorder or distributor interface. The application can also get an interface to the session class via the ToiMediaService::createSessionInstance() method for content distribution purposes. See Distributor Properties and Usage for more details on the SetProperties parameters and high level description of the distributor.

In general, an application may have a number of concurrent instances of the different media resources. Since the hardware limits what is possible to do, the KreaTV IP-STB platform performs the resource management necessary to keep the right media player instance visible to the user.

Media Players

Media Player States

A media player instance has a well defined state in every moment of time. Depending on the state, the player responds differently to commands from the application.

Two Fundamental State Machines

The player state is a combination of the states in two separate state machines. One of the state machines is primarily controlled by the client application, and the other is primarily controlled by events on the network. They are not completely independent since application actions may affect the network and the network may affect what the application is allowed to do, but for the sake of simplicity, it helps to think of them as independent of each other.

The media player state machine controlled by the client application.

The media player state machine controlled by the client application has the following states:

• Idle In the Idle state the media player does not handle a stream, and consequently has no connection to a media stream source.
• Connecting In the Connecting state the media player is in the process of establishing a connection to a media stream source.
• Paused In the Paused state the media player has a connection to a media source but playback is halted.
• Playing In the Playing state the media player has a connection to a media source and plays the stream from that source in normal pace.
• Fast Forwarding In the Fast Forwarding state the media player has a connection to a media source and plays the stream from that source forwards at a pace above normal playback.
• Rewinding In the Rewinding state the media player has a connection to a media source and plays the stream from that source backwards at a pace above normal playback.
• Failed In the Failed state the media player may or may not have a connection to a media source but the process of getting the stream from that source has failed in some fatal way.

The Paused, Playing, Rewinding, and Fast Forwarding states are collectively referred to as Playback states. Depending on the media source used, some of the state transitions between the playback states may not be supported. See the section Limitations Imposed by Stream Clients below for more information.

The state machine controlled by events on the network has two states only:

• No Data In this state there is no data to read. There are several possible reasons for this: no stream has been started, playback of the selected stream cannot be started for some reason, problems on the network cause the stream to be delivered late or not at all. Whatever happens, intentionally or unintentionally, that cause interrupts in the stream delivery triggers a transition to this state.
• Data Available In this state stream data is available at the expected pace.

For example, assume that the application instructs the media player to start playing a stream from a multicast channel. When stream data arrives to the player it makes a state transition from No Data to Data Available. If there is serious congestion on the network so that IP packets occasionally do not reach the destination and the user experience is affected, the player temporarily goes to No Data. It returns to Data Available when the stream feed is stable again. However, if the packet loss is not serious enough to affect the user, the player should remain in Data Available.

In addition to the two states described above, the player keeps track of its position in the media stream. Depending on the stream source the position has slightly different semantics.

Sessions

The media player is said to have an active session when it is in any other state than the Idle state. Under normal conditions, having an active session means that the media player is processing a media stream. However, that is not always the case. The media player will never assume the Idle state without being explicitly told by the application to do so. This is a very important property of the media player state machine. Events from the network can never put the player in the Idle state and thereby terminating the session. If an error occurs with the network connection, and that error is determined to be fatal to the communication with the server, the media player puts itself in the Failed state, which gives the application an opportunity to clean up on its side before ending the session and possibly starting a new one.

Limitations Imposed by Stream Clients

The properties of the access method used to receive the media stream may have impact on the media player state machine. For example, a media stream received over a multicast channel cannot be put in the Rewinding state, since the media player has no control of the transmission of the stream. Its only option is to play the stream as it is received from the network, or stop playing it completely.

The part of the media service responsible for receiving stream data from the network or other sources is designed for extensibility. The design allows for stream client plugins to be hooked in to the media service. Depending on the set of stream client plugins the media service is configured with, it will understand different network protocols. Consequently, depending on the stream client used, some of the state transitions in the media player state machine may not be possible to perform. In the previous example, the multicast stream client was unable to rewind the stream. In particular, transitions to all playback states (Playing, Paused, Fast Forwarding, and Rewinding) are not possible to perform for all media stream clients.

When a state transition is not possible to perform, the application is notified about this fact in one of two ways, depending on the latency involved to get that information from the stream source. The application must always be prepared to handle both mechanisms:

• Operation Not Supported Exception thrown from the Play() function. For example, it is likely that the Play() function with a fast forwarding argument throws this exception when using a multicast stream client, because it does not require any additional processing to conclude that these operations are not supported by this particular stream client. In contrast, if these functions return without any exception being thrown, the state transition has actually taken place from the application's point of view, but the media player internally may still be in the process of actually performing the transition towards the stream source.
• OnStateChanged event received at a later stage when the media player has found out that the state transition was not possible to perform. In this case the media player state jumps back to the last state that was valid. As an example, a stream client maintaining a private session with a video server may be denied a request to rewind the stream because of some problem on the server side which the client could not possibly know about before actually asking the server about it.

Media Player Parameters

Adjusting media player parameters is a tool for fine-tuning the operation of a media player instance. Each player instance has a private set of parameters.

The value of parameters are modified either by using the SetParameter function or by specifying an extended URL to the Start function. If the SetParameter function is called in the Idle state, the change affects the default value of the parameter for that particular media player instance. The default value is the value assigned to the parameter when starting a new session, in case the parameter is not explicitly defined in the URL argument to the Start method. If the SetParameter function is called in any other state but the Idle state, the change affects the current session only.

Some parameters can be changed in the configuration file as well. This is described in the Media Service Configuration document.

Stream Information

The amount of information available about the stream the media player currently is playing depends on the stream format. The media player differentiates between the following stream formats: elementary, system, program, transport, and RTP streams. Elementary streams are the simplest stream format, which can only contain one audio stream or one video stream. The other stream formats can hold several audio and video streams as well as other information streams such as subtitles and teletext.

The OnStreamInfoChanged Event

The OnStreamChanged event is emitted to observers/listeners at three different occasions: Right after start of playing a stream, when the stream infomation changes and when changing active video or audio stream. The following table describes when the event is emitted and what stream information is made available:

Media Player Capabilities

A session has a number of capabilities. A capability can for example indicate that the current media player can rewind or that it supports timeshift. See the API documentation for the media player for a list of available capabilities. The capabilities change depending on the currently played media source. They can also change when starting timeshift buffering or when switching to the timeshift buffer.

The OnCapabilitiesChanged Event

The OnCapabilitiesChanged event is emitted to observers/listeners when capabilities have changed. See the API documentation for the media player.

Timeshift

If the current media source has the timeshift capability, it is possible to start timeshift buffering. When timeshift buffering has been started, the media player will get pause and rewind capabilities. After a rewind or pause, the media player starts playing from the timeshift buffer instead of from the live stream and it becomes possible to trick play in the timeshift buffer.

When the boot image is configured, it is possible to select whether the timeshift buffer should be circular and how much data it should be able to hold. A circular buffer is a fixed-size space that keeps the most recently written data; old data that doesn't fit inside the configured size is overwritten as new data arrives.

Media Recorders

Media recorders do not decode the media stream. Instead, they store the incoming media stream to media assets. See the Asset Manager Service for information about media assets.

Just like media players, media recorders have a state machine that is mostly controlled by the application and sessions. However, the set of states and commands differ.

The media recorder state machine controlled by the client application.

The media recorder state machine controlled by the client application has the following states:

• Idle In the Idle state the media recorder does not handle a stream, and consequently has no connection to a media stream source.
• Connecting In the Connecting state the media recorder is in the process of establishing a connection to a media stream source.
• Paused In the Paused state the media recorder has a connection to a media source but recording is halted.
• Recording In the Recording state the media recorder has a connection to a media source and records the stream from that source.
• Failed In the Failed state the media recorder may or may not have a connection to a media source, but the process of getting the stream from that source has failed in some fatal way.

See also: TOI Media Service Interface