Video Output Service

Adaptive Video Mode (AVM)

KreaTV can fit incoming content to the video outputs dynamically by applying different geometric transforms and other actions that depend on media and display properties. We call this Adaptive Video Mode (AVM).

AVM is configured by providing a set of rules that match properties of the incoming media, like resolution or aspect ratio, and video window conditions, like full-screen mode, and specify what to do in each case.

The default behavior, when no customized rules are provided, is to honor content aspect ratio, i.e., to adapt the content to the video window in such a way that the picture is not distorted.

AVM Rules

Individual rules in the AVM interface consist of a name, an output to apply the rule to, and two sets of data: conditions and actions.

Conditions are used to determine which rules should be considered active. They are divided into two categories: content conditions, which include resolution, aspect ratio, framerate etc of the main window content, and window conditions, which is the fullscreen status of the main video window. Rules that contain no conditions are always active, which is useful for specifying a default behaivor.

Actions specify what to do when a certain rule becomes active. They are transforms that affect the way content is stretched or zoomed in the main video window.

Rules are organized into rule sets where each set is meant to describe an overall strategy. For example, there may be a "Pillarboxing of 4/3 content" set or a "Fullscreen video in native resolution" set, each containing any number of rules. Only one rule in the set is active at a given time; when more than one rule is applicable to the current playback situation, KreaTV chooses the first one in the order they have been specified.

Rule sets are specified in XML-format files that must comply with this schema. Each rule set contains a name which is used to load these sets into the system via the TOI API, please refer to Using the AVM API. The XML files themselves must be provided at build time of boot image via the kreatv-option-adaptive-videomode-rules.

Several rule sets can be loaded into the system at the same time. If several of those sets have matching rules, the highest priority rule in each set will become active and the actions in each rule will be applied. The order in which the actions are applied is the same as the order in which the rule sets were loaded into the system. The order of application is only relevant if there are several sets whose active rules are in conflict with each other, such as two rules whose actions zoom the content on an HD output. In such an event, the action of the latter rule will override the action of the previous one, meaning that the end result will reflect the action of the active rule in the set which was loaded last.

Triggers

In normal circumstances, you program AVM with a set of rules to adapt the stream to the display geometry automatically. AVM reacts to changes in media and video window properties and applies the rules that match the situation. All of this happens automatically with no interaction from the application other than loading the rules themselves.

AVM allows an additional mechanism that might be useful in some cases, that is to trigger specific rules programmatically from the TOI Javascript interface.

To accomplish this, all you need to do is to add a Trigger condition to the rule. Triggers have a name that you can pass to ToiVideoOutputService::setAdaptiveTrigger() to change the trigger status and activate or deactivate the rule at will from the application. There is an example here.

List of Rule Conditions

A rule must contain one or more conditions. Rules become active when all their conditions are met. The list of possible conditions are as follows:

Content Conditions

These conditions are matched to the content shown in the main video window.

Dimension

Contains the elements Width, Height, WidthDelta and HeightDelta, all of which must be specified when using this condition. If the content width is within Width +/- WidthDelta and its height is within Height +/- HeightDelta, the condition is considered to be satisfied.

AspectRatio

Contains single value, which is one of "4_3", "16_9" or "UNKNOWN". This condition matches the aspect ratio signaled within the stream, not the ratio between horizontal and vertical resolution in pixels.

Since many streams do not match either of these values, it is generally recommended to rely on PixelAspectRatio instead.

DeltaFrameRate

Contains two elements, FrameRate and Delta. If the framerate specified in the content stream lies within FrameRate +/- Delta, this condition is considered to be satisfied.

Progressive

Contains a single boolean value which is compared to whether the content stream is progressive or not.

Afd

Short for Active Format Description, this condition matches the AFD of the content. If the content AFD is equal to the one given in this condition, the condition is considered to be satisfied. Possible values are:

• UNKNOWN
• 16_9_TOP
• 14_9_TOP
• MORE_THAN_16_9_CENTRE
• SAME_AS_CODED
• 4_3_CENTRE
• 16_9_CENTRE
• 14_9_CENTRE
• 4_3_SHOOT_AND_PROTECT_14_9_CENTRE
• 16_9_SHOOT_AND_PROTECT_14_9_CENTRE
• 16_9_SHOOT_AND_PROTECT_4_3_CENTRE

PixelAspectRatio

Contains two integers, Numerator and Denominator, which describe the pixel aspect ratio of the stream. This is the aspect ratio of each individual pixel that is required to display the video frame at the original resolution with no aspect ratio distortion. It is described as Source Aspect Ratio (SAR) by the MPEG2 standard (see Video Demystified p.597) and can be found in the Aspect_ratio_information field.

The PixelAspectRatio, together with the video frame dimensions, determine the final aspect ratio of the video window. There is an Action described below for using this information to handle aspect ratio in an optimal way, but if there is a need for exceptional cases the PixelAspectRatio condition can be used.

Window Conditions

These conditions are matched to the main video window itself, regardless of content.

FullScreen

Contains a single boolean value. It is compared to whether the main video window covers the entire screen or not. Note that SetVideoTransform does not necessarily affect this condition.

Trigger Conditions

These conditions are matched if given adaptive triggers are set through video output service.

Name

Contains a string value which denotes the name of the adaptive trigger.

Value

Contains a string value which denotes the value of the adaptive trigger.

List of Rule Actions

A rule must contain one or more actions. When the rule becomes active, these actions are applied. The list of possible actions are as follows:

DestinationTransform

Contains either a transform expressed with four elements X, Y, Width, Height or a FollowPixelAspectRatio element.

The transform is expressed in floating point coordinates, which means that they will result in the same visual effect regardless of what content or video mode is shown. The video window is considered to be of size (1.0, 1.0), so a transform with values X=0.0, Y=0.0, Width=1.0 and Height=1.0 would represent the whole window. This transform is applied to the video window itself, reshaping it according to the supplied coordinates.

If FollowPixelAspectRatio is present the video window transform is set to display the video with correct aspect ratio based on the pixel aspect ratio coded in the stream and the aspect ratio of the display as set by setDisplayInfo.

SourceTransform

Contains either a source transform with six elements: X, Y, Width, Height, AdjustX and AdjustY or a FollowDisplayExtension element. The first four elements of the source transform are expressed in relative floating point coordinates. They are similar to the DestinationTransform values, except that they are applied to the content instead of the video window. The four coordinates describe a rectangle on the content stream which is to be displayed in the window, which means that the source transform enables the application developer to cut out parts of the content.

The AdjustX and AdjustY values are expressed in integer pixel coordinates. These allow the application developer to cut away an exact number of pixels from the top and bottom or left and right sides of the content. This is typically useful for removing overscan pixels.

If FollowDisplayExtension is present the source transform is set according to the MPEG sequence display extension height and width coded in the stream.

WideScreenSignal

Sets the WSS in the vertical blanking interval (VBI) of the analog video signal. The value can be any of 4_3, 14_9_LETTERBOX, 14_9_LETTERBOX_TOP, 16_9_LETTERBOX, 16_9_LETTERBOX_TOP, MORE_THAN_16_9_LETTERBOX, 14_9, 16_9 or FOLLOW_SCREEN_ASPECT_RATIO. The latter is default, which makes the WSS follow the aspect ratio for each screen as set via setDisplayInfo.

ScartFunctionSelect

Contains the value signalled in pin 8 on the SCART output. The value can be either OFF, 16_9 or 4_3.

Using the AVM API

The operative part of the API is the ToiVideoOutputService::loadAdaptiveRuleSets() function, which takes as argument a sequence of strings. This sequence is the sequence of rule sets mentioned above, where each string is the name of one particular rule set. Calling this function will immediately remove any previously loaded rule sets and load the provided rule set names instead. If no names are provided to the function, any existing rule sets are simply cleared.

The result of LoadAdaptiveRuleSets is persistent, which means that when rebooting the box, the same rule sets will be automatically loaded again - assuming the corresponding XML files have not been removed from the bootimage between reboots.

Two additional functions exist in the AVM API: ToiVideoOutputService::getAvailableAdaptiveRuleSetNames() and ToiVideoOutputService::getLoadedAdaptiveRuleSetNames(). The former function returns the sequence of names of the rule sets which are available for loading, and the latter does the same for the rule sets which are currently loaded.

initAdaptiveRuleSets: function() {
  try {
    toi.videoOutputService.loadAdaptiveRuleSets(["KreaTV HD boxing"]);
  }
  catch (e) {
    console.log("Exception when loading AVM RuleSets: " + e);
  }
},

Default rule sets available

• KreaTV HD Pillarboxing
Pillarbox 4:3 content on 16:9 HD output.
• KreaTV SD Letterboxing
Letterbox 16:9 content on 4:3 SD output.

Pillarboxing Example

When playing 4:3 content, the application or portal might want the content to be pillarboxed when displayed on HDMI. This can achieved by configuring the adaptive mode to display the video using only the center 4:3 region of the display.

To implement this example, a rule set with one single rule needs to be created. It might look as follows:

This rule will match any content with an aspect ratio of 4/3. It will move the left side of the image 1/8th of the screen to the right and squeeze it to 4/3rds of the screen width. Below is an example of a real XML rule doing the above.

<RuleSet display="HD" name="Pillarbox 4:3 content">
  <Rule name="Pillarbox 4:3">
    <Conditions>
      <Content>
        <AspectRatio>4_3</AspectRatio>
      </Content>
    </Conditions>
    <Actions>
      <DestinationTransform>
        <X>0.125</X>
        <Y>0.0</Y>
        <Width>0.75</Width>
        <Height>1.0</Height>
      </DestinationTransform>
    </Actions>
  </Rule>
</RuleSet>

Triggers Example

<RuleSet name="MyTriggers" display="HD">
  <Rule name="TriggerRule1">
    <Conditions>
      <Trigger>
        <Name>MyTrigger</Name>
        <Value>Rule1</Value>
      </Trigger>
    </Conditions>
    <Actions>
      <SourceTransform>
        <X>0.5</X>
        <Y>0.5</Y>
        <Width>0.5</Width>
        <Height>0.5</Height>
        <AdjustX>200</AdjustX>
        <AdjustY>200</AdjustY>
      </SourceTransform>
    </Actions>
  </Rule>
  <Rule name="TriggerRule2">
    <Conditions>
      <Trigger>
        <Name>MyTrigger</Name>
        <Value>Rule2</Value>
      </Trigger>
    </Conditions>
    <Actions>
      <DestinationTransform>
        <X>0.5</X>
        <Y>0.5</Y>
        <Width>0.5</Width>
        <Height>0.5</Height>
      </DestinationTransform>
    </Actions>
  </Rule>
</RuleSet>

This ruleset has two trigger rules. If you load the ruleset MyTriggers, and use ToiVideoOutputService.setAdaptiveTrigger("MyTrigger", "Rule2") to activate Rule2, you can get the action of transform of destination.

See also: TOI Video Output Service Interface

See also: Adaptive video mode rule sets schema