Remix

Learning Outcomes

• Understand the use cases and their benefits
• Understand the Remix workflow
• Understand how Remix can be deployed and integrated

Diagram

Schematically

All components are HTTP based. Results are easily proxied and cached. Above setup presents three new elements when compared to VOD or Live:

• SMIL origin
• Remix
• Transcode

The SMIl origin is the playlist generator, the customer integration point: here the business rules are implemented (which playlist is generated).

The Remix component transforms XML into a so-called ‘remix’ (or dref) MP4. It is a small MP4 containing metadata an references (URLs) to actual media.

Transcode may be required for Ad insertion uses cases (to fit the ad with the content as ads provided by Adinsertion Networks typically do not have the same properties as your main content). How to effectively scale certain use cases (personalisation vs. caching) or how to pass metrics downstream (for instance VAST triggers) is not discussed here.

Use Cases

Unified Remix ‘mixes’ clips, from one or various sources, into a single stream. The player on the client side sees no difference between a stream that has been ‘mixed’ and one that hasn’t.

Remix starts with an XML playlist using SMIL as the markup language, this is the responsibility of the playlist generator or ‘SMIL origin’.

The SMIL origin acts as intermediary between Unified Remix and the other parts of a video streaming setup. The playlist’s construction can be rule based, use the information from an Electronic Programming Guide (EPG) or the VAST response from an ad network. The playlist defines which content should be stitched.

Remix then generates an MP4 file from the SMIL file. This MP4 file then is the input for Unified Origin and allows it to to create a new, continuous stream for all playout formats.

Resolving a playlist server side has these benefits:

• Works everywhere, on all devices, as they see only one stream
• No player development is required, larger audiences can be reached as it ‘just works’
• Flexibility, changes can be made server side without having to update devices or apps
• Ad blockers are circumvented

The Remix playlist has many use cases, it’s a future proof solution.

• Bumper For Everyone (VOD)
• Pre-, Mid-, Post-roll (VOD)
• Virtual Subclips (VOD)
• nPVR (Archive2VOD)
• Dynamic Ad Insertion (VOD)
• Live Scheduling (VOD2Live)
• Live Remix (Live2VOD)
• Bumper For Everyone (Live)
• Dynamic Ad Replacement (Live)
• Overlay (VOD/Live)

See Remix nPVR or Remix VOD for requirements and available functionality. [1] [2]

Note

Please note that not all use cases are available yet, only VOD and VOD2Live so far - to combine Live with VOD is much more complex!

Background

Video playlist generation normally taking place at the broadcaster is moved to OTT provider allowing features broadcasters/OTT providers expect such as promotional videos to market their own channel/brand or different channels. This is part of two trends:

Programmatic TV

Personalised video is synonymous with ‘programmatic tv’, an automated and data-driven method of buying and delivering ads against TV content. It’s not just ads, the content could also be delivered like this.

Push to Pull

There is a current behavioural change in how media is consumed, from push - where viewers sit at home and passively watch what is broadcast, to pull - where viewers request what they wish to watch.

By leveraging the internet’s GET method and creating the playlist upstream, on-demand Unified Remix can deliver this.

Remix nPVR

Remix nPVR enables the creation of a flexible, scalable, performant and reliable catch up or nPVR service while minimizing storage overhead. [3]

Utilising Unified Capture to create a time based segment archive and Unified Remix to prepare playlists based on the archive provides a number of advantages over traditional nPVR solutions:

• No storage duplication, if nPVR ‘recordings’ overlap the content will still only be stored once
• Easy changes without any repackaging or re-encoding, it’s possible to update the start or end time of a recording based on broadcast as-runs just by updating the playlist
• Easy to cut out breaks by creating a playlist which only includes the primary content

Schematically

Remix VOD

Remix VOD enables server side mixing of content to produce a single output stream. Clients and playback devices see the stream coming from a single origin and having a timeline without any discontinuities.

The solution makes use of two Unified Streaming Platform products:

• Remix, to create a single presentation from multiple input sources
• Origin, for Just-In-Time packaging/transmuxing to all supported streaming formats

Remix VOD allows for the following use cases:

• Bumper for Everyone (Warning Slate, Dubcard, Promo)
• Virtual Subclips
• Pre-, Mid-, Post-Roll

The actual flow is depicted in detail in the online Remix documentation. [2]

Note

Please review the Remix documentation, it not only outlines the workflow but also provides background on the SMIL format used, the storage options and how to prepare content.

Remix AVOD

Remix AVOD enables you to build SCTE 35-based ad insertion workflows based on existing VOD content libraries, using third-party ad insertion platforms. [7]

Remix AVOD allows for three different workflows:

• Media includes Timed Metadata and is properly conditioned by encoder, making it ready for direct use with Unified Packager or Unified Origin VOD.
• Media is properly conditioned by encoder, but Unified Remix is used to add the necessary Timed Metadata.
• Media does not contain Timed Metadata nor is it conditioned, and Unified Remix is used to take care of both.

The general workflow of Unified Remix is that it creates a remixed MP4 file from a SMIL Playlist. This remixed MP4 is ISOBMFF compliant.

Schantically this looks like the following:

You will see a general example of how this works in the hands on section.

Note

The Remix AVOD demo shows this working with several platforms:

• The Ad Insertion Platform (AIP)
• AWS Elemental MediaTailor
• Serverside.ai
• Yospace

Deployment and Workflows

Remix is a stateless, microservice architecture.

This allows a lot of flexibility regarding deployment, for instance:

• A single webserver using multiple vhosts for the different components
• Containerised, e.g. using Docker (docker-compose on github), Kubernetes or ECS
• Terraform configration for cloud based instance deployment (AWS, Azure, Google e.a.)

System metrics and webserver logs may be fed into an ELK stack using netdata for monitoring purpose.

Hands-On

Show the contents of a Remix (dref) MP4

The purpose of the hands-on is the get a better understanding of what a dref MP4 file is.

SMIL file creation

In your home directory create a file with the following content using the extension .smil:

<?xml version="1.0" encoding="utf-8"?>
<smil xmlns="http://www.w3.org/2001/SMIL20/Language">
  <head/>
  <body>
    <seq>
      <video src="https://usp-s3-storage.s3.eu-central-1.amazonaws.com/remix/tears-of-steel_600k.mp4"/>
    </seq>
  </body>
</smil>

MP4 creation

Then use the unified_remix executable to create the dref MP4:

#!/bin/bash

unified_remix -o remix.mp4 remix.smil

The result of this command will be a small fragmented MP4 file, a so-called ‘data reference’ or dref MP4 (after the dref box in ISO-14496-12). [4]

It’s contents are fragmented MP4, but there is only metadata, for the actual audio/video samples it will point to the URL referenced in the SMIL file (the S3 bucket in this case):

^@^@^@^Xi
ftypiso6^@^@^@^@iso6rmix^@^@^@(freeUSP by CodeShop^Q^Q^Q^Q^Q^Q^Q^Q^Q^Q^Q^Q^Q^Q^Q^Q^Q^@^A^[^?
moov^@^@^@lmvhd^@^@^@^@^@^@^@^@^@^@^@^@^@^@0^@^@<89>d^@^@^A^@^@^A^@^@^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^@@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^B^@^A^[^
trak^@^@^@\tkhd^@^@^@^G^@^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^@<89>d^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^@@^@^@^@^Aà^@^@^@È^@^@^@^A^Z§
mdia^@^@^@ mdhd^@^@^@^@^@^@^@^@^@^@^@^@^@^@0^@^@<89>d^@UÄ^@^@^@^@^@2hdlr^@^@^@^@^@^@^@^@vide^@^@^@^@^@^@^@^@^@^@^@^@USP Video Handler^@^@^A^ZM
minf^@^@^@^Tvmhd^@^@^@^A^@^@^@^@^@^@^@^@^@^@^@wdinf^@^@^@o
dref^@^@^@^@^@^@^@^A^@^@^@_url ^@^@^@^@https://usp-s3-storage.s3.eu-central-1.amazonaws.com/remix/tears-of-steel_600k.mp4^@^@^A^Yº
stbl^@^@^@ªstsd^@^@^@^@^@^@^@^A^@^@^@<9a>avc1^@^@^@^@^@^@^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^Aà^@È^@H^@^@^@H^@^@^@^@^@^@^@^A
AVC Coding^

Create playout URL

Copy the dref MP4 file to the webserver document root (/var/www/tears-of-steel).

As with a regular MP4, you can create any type of content by requesting a URL. For instance, MPEG-DASH:

#!/bin/bash

curl -v http://${aws_hostname}/remix.mp4/.mpd

More examples

• Adding more bitrates and audio

<?xml version="1.0" encoding="utf-8"?>
<smil xmlns="http://www.w3.org/2001/SMIL20/Language">
  <head/>
  <body>
    <seq>
     <par>
      <video src="https://usp-s3-storage.s3.eu-central-1.amazonaws.com/remix/tears-of-steel_600k.mp4"/>
      <video src="https://usp-s3-storage.s3.eu-central-1.amazonaws.com/remix/tears-of-steel_1200k.mp4"/>
      <video src="https://usp-s3-storage.s3.eu-central-1.amazonaws.com/remix/tears-of-steel_eng.mp4"/>
     </par>
    </seq>
  </body>
</smil>

Note

Please open the remix.mp4 to see multiple URLs.

• Adding DRM is done in two steps: create a dref file and create an ism file (server manifest) with keys that refers to the dref file. For instance using the previously created dref file:

#!/bin/bash

openssl rand 16 > key
openssl rand 16 > init_vector

CEK=`cat key | hexdump -e '16/1 "%02x"'`
KIV=`cat init_vector | hexdump -e '16/1 "%02x"'`

mp4split \
  -o remix-drm.ism \
  --hls.key=:${CEK} \
  --hls.key_iv=${KIV} \
  --hls.license_server_url=http://${aws_hostname}/key \
  --hls.playout=sample_aes \
  remix.mp4

Then do the following to see what manifest is generated:

#!/bin/bash

sudo cp remix.mp4 /var/www/tears-of-steel
sudo cp remix-drm.ism /var/www/tears-of-steel

curl -v http://${aws_hostname}/remix-drm.ism/.m3u8

Remix VOD - Pre-Roll

In this hands-on we will build on the previous one and edit the playlist so we stream a pre-roll ad before the main content.

SMIL file creation

In your home directory create a file with the following content (an ad and a video) using the extension .smil:

<?xml version="1.0" encoding="utf-8"?>
<smil xmlns="http://www.w3.org/2001/SMIL20/Language">
  <head/>
  <body>
    <seq>
      <video src="https://usp-s3-storage.s3.eu-central-1.amazonaws.com/remix/remix01_x264.mp4"/>
      <video src="https://usp-s3-storage.s3.eu-central-1.amazonaws.com/remix/tears-of-steel_600k.mp4"/>
    </seq>
  </body>
</smil>

MP4 creation

Then use the unified_remix executable to create the dref MP4:

#!/bin/bash

unified_remix \
  -o remix-ad.mp4 \
  remix-ad.smil

Create playout URL

Copy the dref MP4 file to the webserver document root:

#!/bin/bash

sudo cp remix-ad.mp4 /var/www/tears-of-steel

As with a regular MP4, you can create any type of content by requesting a URL. For instance, MPEG-DASH:

#!/bin/bash

curl -v http://${aws_hostname}/remix-ad.mp4/.mpd

Microservice Setup using Docker

A fully functional Remix demo can be found in Github. This setup uses Docker and docker-compose [5] to create a set of containers hosting Remix as a containerized micro service. [6]

The playlist generator (the SMIL origin) is setup to demonstrate pre-rolls. However, any kind of SMIL origin can be built to generate SMIL playlists based on the business logic involved: it’s the integration point.

Note

Please run the Docker setup locally (on your laptop) if you’re interested!

Footnotes

[1]	https://docs.unified-streaming.com/documentation/remix/npvr.html

[2]	(1, 2) https://docs.unified-streaming.com/documentation/remix/vod.html

[3]	https://demo.unified-streaming.com/npvr/

[4]	https://www.iso.org/standard/68960.html

[5]	https://docs.docker.com/compose/

[6]	https://github.com/unifiedremix/remix

[7]	https://docs.unified-streaming.com/documentation/remix/avod.html