HLS Support

Introduction to HLS

The Apple HTTP Live Streaming (HLS) protocol defines an adaptive-bitrate (ABR) streaming solution using regular web servers. The stream is divided into multiple short media segments, indexed by playlists. A playlist is regarded as a media playlist or a master playlist. A media playlist lists the Uniform Resource Identifier (URI) of media segments, and a master playlist contains a set of variant streams, which represents different versions of the same content for different bitrates. A variant stream can also specifiy a set of renditions; alternate versions of the content such as audio and WebVTT subtitles in different languages. Alternate renditions of video, such as video recorded from different angles, is also possible (note that this is not supported in KreaTV).

Support in KreaTV

There are three types of HLS playlists; live, video on demand (VOD) and event, and KreaTV supports regular playback of all of them. Trickplay features such as seek and pause are supported while fast forward, rewind and slow motion are not. Unencrypted and AES–128 CBC encrypted streams can be played, and currently the supported Digital Rights Management (DRM) systems are Verimatrix ViewRight and Secure Media HLS+, read more about encryption.

KreaTV continuously measures available bandwidth and selects the stream variant with the best quality for the current network conditions in order to maintain uninterrupted playback.

HLS Protocol

The HLS protocol version 7 is described in HTTP Live Streaming draft-pantos-http-live-streaming-17 written by R. Pantos and W. May at Apple Inc. KreaTV follows this specification unless anything else is specified in this documentation.

The HLS tags, and information whether they are supported in KreaTV or not, are listed in the table below. The version number specifies in which protocol version the tag was introduced. If tag has been revised in later protocol versions those are specified enclosed by brackets.

Tag	Supported	HLS version	Remarks
EXTM3U	Yes	1
EXT-X-VERSION	Yes	2
EXTINF	Yes	1 (3)
EXT-X-BYTERANGE	Yes	4
EXT-X-DISCONTINUITY	Yes	1
EXT-X-KEY	Partial	1 (2, 5)	Attribute METHOD only supports the value AES-128 and NONE, i.e SAMPLE-AES is not supported. Attribute URI must reference a static resource, i.e. the key returned by a URI must never change. Attributes KEYFORMAT and KEYFORMATVERSION are not supported.
EXT-X-MAP	No	5
EXT-X-PROGRAM-DATE-TIME	No	1
EXT-X-TARGETDURATION	Yes	1
EXT-X-MEDIA-SEQUENCE	Yes	1
EXT-X-DISCONTINUITY-SEQUENCE	Yes	6
EXT-X-ENDLIST	Yes	1
EXT-X-PLAYLIST-TYPE	Yes	3
EXT-X-I-FRAMES-ONLY	No	4
EXT-X-MEDIA	Partial	4 (5, 6, 7)	Attribute TYPE only supports the types AUDIO and SUBTITLES. Attribute ASSOC-LANGUAGE, INSTREAM-ID, DEFAULT, AUTOSELECT and FORCED are not supported. Attribute CHARACTERISTICS does not support value “public.easy-to-read”.
EXT-X-STREAM-INF	Partial	1 (2, 4, 6, 7)	Attributes VIDEO, CLOSED_CAPTIONS and CODECS are not supported.
EXT-X-I-FRAME-STREAM-INF	No	4 (6)
EXT-X-SESSION-DATA	No	7
EXT-X-INDEPENDENT-SEGMENTS	No	6
EXT-X-START	Partial	6	Attribute PRECISE is not supported, i.e. playback will start at the beginning of the closest media segment.
EXT-X-ALLOW_CACHE	No	1 (7)	Not supported in KreaTV. Removed from HLS protocol version 7.

Note that unsupported tags are ignored if present in the playlist.

Media Segments

Section three in HTTP Live Streaming draft-pantos-http-live-streaming-17 specifies the streaming format of the media segments. Below is a list of clarifications and additional requirements placed on the streams for compatibility with KreaTV. Streams violating these requirements may work but are not officially supported.

Note that additional requirements for main stream, external audio and external subtitles are listed in the corresponding sections.

Generic considerations for MPEG-TS

The following requirements are imposed on the MPEG-TS format:

• The Transport Stream (TS) MUST NOT contain more than one program.
• A media segment MUST NOT contain partial TS packets, i.e. the chunker may only end a media segment at the end of a complete TS packet.
• A media segment MUST NOT contain partial Packetized Elementary Stream (PES) packets, i.e. the chunker may only end a media segment at the end of a complete PES packet.
• A media segment containing h264 encoded content MUST start with an IDR-frame to guarantee a smooth transition between variants.
• A media segment containing mpeg2 encoded content MUST start with an I-frame to guarantee a smooth transition between variants.
• The transport stream may be encrypted according to the HLS protocol using the AES-128 method. No other method is supported.
• The transport stream MUST NOT have any additional encryption, i.e. the use of a traditional MPEG-TS based conditional access system in addition to the HLS DRM is not supported.

Generic considerations for ADTS

The following requirements are imposed on the Audio Data Transport Stream (ADTS) format:

• Only Advanced Audio Coding (AAC) codec is supported.

Main stream

The main stream is the variant playlist that is used if no alternative rendition has been selected. The main stream is the only stream containing video; alternate renditions of video, such as video recorded from different angles, is not supported. A video stream must be in MPEG-TS format. If switching to an audio-only variant, the audio segments will be considered the main stream. A main audio stream may be in either MPEG-TS or ADTS format.

For the main stream these additional requirements are imposed to the MPEG-TS container format:

• The stream MUST NOT contain more than one video stream.
• The stream may contain multiple audio and subtitle streams.

Audio and subtitles

In HLS, variant streams are used to specify sets of renditions; alternate versions of the content such as audio and WebVTT subtitles in different language. Alternate audio and subtitles are also entitled external audio and subtitles. When an alternate rendition of any of those types is selected, the secondary stream will be downloaded from a separate media playlist since it is not part of the main stream.

Audio

The main stream may contain audio, but the master playlist may also specify external audio in a separate media playlist. External audio streams must be in MPEG-TS or ADTS format.

In the case of external audio, the renditions are defined by the tag EXT-X-MEDIA in the master playlist, with the TYPE attribute set to AUDIO. The attribute LANGUAGE specifies the Identifying Language for the rendition. If the attribute CHARACTERISTICS value “public.accessibility.describe-video” is set, the rendition is interpreted as visually impaired.

If the optional attribute URI is omitted for the tag EXT-X-MEDIA when TYPE is set to AUDIO the rendition is assumed to be part of the main stream, i.e not external audio. The audio PID must be part of the PMT. Language, characteristics and other metadata from the PMT will be used rather than from the information from the playlist.

Settings in KreaTV for audio are defined by the following attributes in ToiInformationService:

Attribute	Value	Description
CFG_MEDIA_AUDIO_AUTOSELECTION	TRUE or FALSE	Defines if audio track should be automatically selected or not.
CFG_MEDIA_AUDIO_FORMATPRIORITY	“ac3,mpeg”	Defines the priority of audio formats.
CFG_MEDIA_AUDIO_LANGUAGEPRIORITY	Quoted-string containing one of the tags for Identifying Language, e.g. “eng,swe”	Defines the priority order for languages.
CFG_MEDIA_AUDIO_PRIORITYORDER	“language,type,format”	Defines the priority order for the attributes specified for audio.
CFG_MEDIA_AUDIO_TYPEPRIORITY	“normal,hearing_impaired,visually_impaired”	Defines the priority order for type of audio. Note that hearing_impaired is not supported for external audio in HLS.

For external audio data these additional requirements are imposed on the MPEG-TS format:

• The transport stream MUST NOT contain more than one audio stream.
• The transport stream MUST NOT contain any other media streams.

Subtitles

The main stream may contain subtitles in traditional broadcast formats, e.g. DVB and/or TeleText subtitles and/or ID3/PNG subtitles embedded as Timed Metadata for HLS compatible with the Envivio's spec for "DVB Bitmap Subtitles in HLS". In addition the master playlist may also specify external subtitles in WebVTT format in a separate media playlist. See WebVTT: The Web Video Text Tracks Format for details about the WebVTT format.

In the case of WebVTT subtitles, the renditions are defined by the tag EXT-X-MEDIA in the master playlist, with the TYPE attribute set to SUBTITLES. The attribute LANGUAGE specifies the Identifying Language for the rendition.

Note that the tag EXT-X-MEDIA attribute CHARACTERISTICS value “public.easy-to-read” is currently not supported. If any of the values “public.accessibility.transcribe-spoken-dialog” and “public.accessibility.describe-music-and-sound” is specified for the same attribute, the rendition is interpreted as hearing impaired.

Settings in KreaTV for subtitles are defined by the following attributes in ToiInformationService:

Attribute	Value	Description
CFG_MEDIA_SUBTITLING_AUTOSELECTION	TRUE or FALSE	Defines if subtitle track should be automatically selected or not.
CFG_MEDIA_SUBTITLING_LANGUAGEPRIORITY	Quoted-string containing one of the tags for Identifying Language, e.g. “eng,swe”	Defines the priority order for languages.
CFG_MEDIA_SUBTITLING_MODEPRIORITY	“WebVTT,DVB,Teletext”	Defines the priority order for subtitle types.
CFG_MEDIA_SUBTITLING_TYPEPRIORITY	normal,hearing_impaired	Defines the priority order for type of subtitles.

If autoselection is set to true, the best matching subtitle track will be selected. If autoselection is set to false, no subtitles will be automatically displayed, but the available subtitle tracks can still be selected manually.

For external subtitle data these additional requirements are imposed to the WebVTT format:

• Encryption is not supported in any form. A subtitle media playlist containing the EXT-X-KEY tag will be ignored.

Playback

Three types of playlists are supported; live, event and VOD. If a playlist is a VOD or event playlist, it is specified by the tag EXT-X-PLAYLIST-TYPE. This tag is omitted for live playlists.

The tag EXT-X-START attribute TIME-OFFSET indicates the point at which the playback should be started. If this tag is omitted, the playback starts from the default position. For live playlis the default position is three target durations (specified by the tag EXT-X-TARGETDURATION) from the end. For all other playlist types the default position is at the beginning of the playlist. Note that the attribute PRECISE is not supported, i.e. playback will start at the beginning of the closest media segment.

The current position is reported to the application with the ToiMediaPlayerBase.ToiPositionInfo structure. This structure also contains the duration of the playlist. The position information is reported at playback and when the playlist is loaded.

Live

A live playlist is also called a sliding window playlist, and means that the server inserts new media segments at the end of the playlist when they are available, and removes the oldest segments from the start of the playlist. The playlist is reloaded by KreaTV at regular intervals to catch these changes. The interval is specified by the tag EXT-X-TARGETDURATION. If the loading of the playlist fails for some reason, the reload interval is halved. EXT-X-ENDLIST tag is not present in the playlist.

When a live playlist is loaded, the valid playback range is between zero and the playlist duration. However, the duration presented to the application will be reduced by three target durations (specified by the tag EXT-X-TARGETDURATION) to limit the playback from starting to close to the live position. This is done to prevent possible playback stalls and is according to HTTP Live Streaming draft-pantos-http-live-streaming-17. Playback would normally start three target durations from the end unless otherwise specified by the EXT-X-START tag.

Event

An event playlist means that the server is not allowed to change or delete any part of the playlist, it can only be updated with new media segments at the end of the file. When the last media segment has been added by the server, the EXT-X-ENDLIST tag is also added. As for the live playlist, the playlist is reloaded at regular intervals to catch these changes, but only until the EXT-X-ENDLIST tag has been added. The interval is specified by the tag EXT-X-TARGETDURATION. If the loading of the playlist fails for some reason, the reload interval is halved.

As for the live playlist, the valid playback range is between zero and the playlist duration. The duration of the event playlist is reduced by three target durations until the EXT-X-ENDLIST tag is added. This means that it is not possible for the user to start playback less than three target durations from the end of the playlist until all media segments have been added.

VOD

A VOD playlist is never updated, it contains the EXT-X-ENDLIST tag and all segments are listed in the playlist from the start. In this case, the playlist is only loaded once by KreaTV, and the valid playback range is the whole playlist duration.

Trickplay

Two trick play operations are supported, seek (ToiMediaPlayer.playFromPosition) and pause (ToiMediaPlayer.play with pace = 0). Fast forward, slow motion and rewind are not supported.

The valid range is specified by the ToiMediaPlayerBase.ToiPositionInfo structure. For all types of playlists the range will initially be equal to the total playlist duration with lower bound of zero and upper bound of the durations. In the case of a live playlist, the range will be updated as segments are added and/or removed by the server. The lower bound will be increased by the duration of the removed segments. The upper bound will be increased by the duration of the newly added segments. For event playlists, the upper bound will be increased by the duration of newly added segments. The valid range for a VOD playlist is not updated during playback.

For live playlists, there is a special case of being in paused state and the current position becomes invalid, i.e. falls outside the lower boundary. In this scenario, the picture will be frozen until the user resumes playback, and the playback will start from the live position, i.e. a jump forward in the stream.

Encryption

KreaTV supports playing AES–128 CBC encrypted streams, i.e. streams with the tag EXT-X-KEY with attribute METHOD set to the value AES-128. When this encryption method is selected, the media segments are completely encrypted using the Advanced Encryption Standard with a 128-bit key, Cipher Block Chaining (CBC), and PKCS7 padding. The CBC is restarted on each media segment and the initialization vector (IV) may be specified by the attribute IV. If the attribute IV is omitted, the media segment number is used as IV.

The attribute URI specifies the path to the key, and it must reference a static resource, i.e. the key returned by a URI must never change. The value SAMPLE-AES for the attribute METHOD is not supported, nor the attributes KEYFORMAT and KEYFORMATVERSION.

Note that encryption for external subtitles is not supported in any form. A subtitle media playlist containing the EXT-X-KEY tag will be ignored.

Verimatrix ViewRight and Secure Media HLS+ Digital Rights Management (DRM) systems are supported. The DRM system is specified using an IIP, see Configuration. Note that use of multiple DRM systems is not supported, i.e. it is not supported to specify more than one DRM system IIP.

Bandwidth variant selection

By default KreaTV selects the first variant in the master playlist when connecting to a new HLS asset in accordance with Apple's recommendations (the behavior can be modified with the initial_variant_selection_strategy IIP parameter). Thereafter KreaTV continuously selects the variant with the best quality that can be maintained without playback interruptions based on current network bandwidth and amount of buffered data. Switching between quality variants will most commonly take place in the transition between segments and will appear seamless given that requirements for media segments are fulfilled.

If the network bandwidth drops suddenly, a variant switch can be performed in the middle of a segment also and may also be accompanied by suspension of the playback until sufficient data from the lower quality variant has been downloaded. Playback will remain suspended as long as the network bandwidth is not sufficient (or until the buffers are full).

Configuration

HLS support is sold separately. A valid license must be included in the bootimage configuration in order to unlock the functionality, it is thus not enabled by default.

#Example of licence IIP for HLS
kreatv-license-hls

Apart from the licence IIP, the basic option for HLS also has to be included.

#Required configuration for HLS.
kreatv-option-mediasource-protocol-hls

In the default HLS setup, the HLS playback will be configured to reserve 21 MB of memory.

There are also some advanced configuration options available that can be used to fine tune the playback experience and performance of the HLS stack.

#Enable advanced memory configuration options (default is false)
kreatv-option-mediasource-protocol-hls::advanced_config_enabled=true
kreatv-option-mediasource-protocol-hls::max_target_duration=10
kreatv-option-mediasource-protocol-hls::max_bitrate=20
#kreatv-option-mediasource-protocol-hls::memory_size=19000
kreatv-option-mediasource-protocol-hls::max_network_latency=100

Note that there are two mutually exclusive ways to specify how much memory should be reserved for HLS playback, either specify memory_size parameter or max_target_duration and max_birate parameters. See the IIP description for details about the advanced settings.

There are two different strategies for doing the initial variant selection. The default strategy is to select the first variant in the master playlist. Alternatively, the variant that is closest below, or equal to, the previously played variant's bandwidth can be selected. This latter strategy is selected by setting:

kreatv-option-mediasource-protocol-hls::initial_variant_selection_strategy=maintain_bandwidth

The IIP for Verimatrix or SecureMedia is required to enable playback of encrypted content, these are not required for playback of unencrypted streams. Note that only one DRM system IIP should be included in the boot image configuration, using multiple DRM systems is not supported in KreaTV.

#Example of IIP for Verimatrix active (SecureMedia inactive)
kreatv-option-hls-verimatrix
#kreatv-option-ca-securemediaservice

References

HTTP Live Streaming draft-pantos-http-live-streaming-17

Tags for Identifying Languages

Timed Metadata for HTTP Live Streaming

WebVTT: The Web Video Text Tracks Format