The Conference for Video Engineers

In this talk we will show how to double the x265 encoding speed while keeping same encoder.

Adaptive streaming implies encoding of the same content in multiple bitrates and spatial resolutions. With that said, the underlying content does not change. This talk presents a simple yet powerful hierarchical encoding technique for improving computational efficiency of multi-resolution and multi-rate encoding. Video content is first encoded at a lower resolution or at a higher bitrate of same resolution. Encoder decisions taken during this encode, such as motion vectors and quadtree block structure, are then used in higher resolution encodes.

The proposed encoding approach is not pure theory. It is integrated into the stable branch of x265 open source HEVC encoder and thus is available to anyone using v2.7 and newer. The authors observed speed-ups of up to 250% for higher resolutions with no penalty in terms of visual quality. Neither is hierarchical encoding an HEVC-specific acceleration -- any motion-compensated codec should benefit from the approach.

A look at how FFmpeg wraps and exposes the hardware encoding and decoding APIs available on modern phones and tablets running iOS or Android

VP9 encoder is deployed by YouTube and Netflix at scale. When Netflix deployed it in 2016, reports showed they were able to save up to 36% bandwidth by using VP9 encoding together with their video chunking approach. While VP9 is proven at scale for VOD, it has not been extensively used for live applications which is still dominated by the earlier H.264 video standard.

In this lecture, we focus on the architecture of an FPGA-based VP9 encoder that can deliver pristine quality video at high resolutions and extremely low bitrates. We shall explain the key coding tools in VP9 that can be leveraged to deliver a consistent quality video across a wide variety of bitrates. We shall also present our analysis of how this compares with a live H.264 encoder including both subjective and objective comparison results.

We will also explain how FPGAs can be used to design a system that can perform heavy computations like motion estimation and complex mode decisions to optimize video quality. This system will be software-like with flexible upgrades and yet high density making it deployable on-prem or in the Cloud.

Per-title encoding is powerful, but it's just a start. Once you know the right adaptive curve for a video, what do you do next?

In this talk, we will discuss how data and machine learning can make video encoding better. After a quick summary of why per-title encoding matters, we will describe how our approach to instant and free per-title encoding. We will demonstrate that this approach is highly scalable, and how to transition from per-title to per-shot encoding.

We will then demonstrate how encoding an optimal bitrate ladder is only half the battle. A per-shot encoded bitrate ladder can ensure the highest quality rendition is being delivered, but it doesn’t ensure the right rendition is being delivered to the right person.

In order to solve that problem, we will demonstrate how AI can intelligently choose renditions to maximize QoE metrics, and how that data creates a positive feedback loop. Instead of just monitoring your QoE, you can use AI to actively improve it over time.

We can conclude by discussing the video delivery paradigm. You start with per-shot encoding using AV1 and optimized VMAF; then you can use AI to maximize experience while optimizing costs; lastly you deliver the video using ABR and super resolution upscaling (maybe).

As part of our FIFA World Cup coverage this year, we've been able to deliver matches live in UHD with HDR10. We had a few short weeks to negotiate source acquisition, choose an encoder, configure our players, and determine the correct experience for our users. This project moved quickly and we were ultimately able to launch our UHD HDR experience on Chromecast, Amazon Fire TV and Android TV just before the semi-final matches started, with Apple TV and Roku support coming days later. This process provided an excellent learning opportunity, introducing us to the new challenges of supporting HEVC, HDR10 and much higher bitrate streams. We're happy to share our story, touching on cloud acquisition, cloud encoding, packaging (determining our bitrate/frame rate/resolution ladders), learnings around device support for HDR content, and the performance of our different players (Shaka, AVF, Exo & Roku).

This presentation examines how we built and scaled an analytics pipeline and metadata exchange to millions of concurrent users with only a handful of engineers. From building a clean dataset and choosing metrics to the impact of these choices on AB tests results, we’ll discuss what went wrong (of course), what went right (even better), and the tools we used. We’ll talk about statistical significance, metric variance and how using rudimentary statistical methods can improve confidence in production tests. Finally, we describe basic data visualization techniques that capture the complexity of each experiment’s results, especially useful when running dozens of experiments at the same time. And of course, a few fun facts from our findings on millions of streams and devices around the world.

This talk will give a first glimpse of the upcoming video coding standard Versatile Video Coding (VVC), scheduled to be released in a first version by the end of 2020. Based on technologies from HEVC, VVC has new coding tools and features in the proposal, which significantly increase the coding performance and ensure continuous innovation of video codecs technologies, while not to directly inheriting the licensing problems of HEVC. We will go into detail on some of these tools, show the performance improvements compared to HEVC and explore the challenges we expect to encounter as we plan for the next codec evolution.

Have you been dating just one CDN for what feels like forever? While monogamy is good for personal relationships it can open a lot of risk when it comes to streaming media delivery. As CDN delivery continues to become more commoditized working through the solution to have an intelligent Multi CDN solution in place is very wise for wide spread critical delivery at scale. This session will dive into the options, providers, and solutions regarding how and what to consider. Then we will present what are some of the gotchas around security and implementation that need to be accounted for in the client and infrastructure as well as how to take it to the next level with advanced concepts.

Lossy encoding is somewhat destructive by definition. Even when encoding with modern optimizations, part of details, grain and spatial contrast are lost making the video flat and less vivid. With special processing and a bit of machine learning it's possible to perceptually reconstruct loss details directly in real time during playback bringing perceptual quality back in time (almost) to the level it had before encoding.

An Updated analysis of HTML5 vs Native playback on mobile devices. This lighting talk will give an updated performance and gap analysis of HTML5 vs Native video playback on iOS and Android. We will show the data around startup time, memory usage, cpu usage, and recently added features. Although the gap is narrowing, we will demonstrate why Native is still leading the race.

Low-latency streaming is a big deal, and the Hls.js team wants to bring it to all video devs. But there's just one problem - how do we build something which works for everyone? In this talk I'll be walking through the ongoing LHLS standard and how you can help.

In the 2015 Demuxed keynote ("What idiot designed this?"), Matt Szatmary talked about the strange history behind things like frame rates, interlacing, and 188 byte packets. Much of digital video is arbitrary, unnecessary, and weird.

This talk will explore the opposite question. What about video is derived from human biology and fundamental physics? A lot, it turns out. In this talk, we'll explore how aspect ratios, color depth, pixel density, sample rates, and more are all directly affected by human physiology. Understanding these things can help us all deliver better video experiences, while avoiding abominations like vertical video.

This talk will briefly cover the history of CEA-608 captions, conceptually how to parse them from MPEG-TS segments and how to do the same with FMP4s.

As more video distribution and consumption mechanisms are created across the video landscape, detailed playback metrics are quickly becoming a video engineers goto tool for validation of platform quality and health. But when quality of experience is key, how can these metrics be used in a way to create actionable solutions to issues? In this presentation Matt Fisher, lead video playback engineer at Vimeo, will discuss some real world use cases where utilizing detailed playback metrics led to quickly identifying and solving complex video playback issues.

Edit Decision Lists (EDLs) have been an integral part of post-production processes for the past three decades. In this talk, we will look at some of the EDL formats and examine the evolution of use cases of EDLs in non-linear editing systems. We will explore what it means to render an EDL in today's cloud based parallel processing architectures.

Brightcove has created an open source tool to simulate HLS livestreams for testing purposes. This allows us to more easily reproduce error conditions and simulate bad data. This talk will focus on how our simulator works and how it has aided in weeding out bugs.

This talk will introduce a new high performance software fabric for managing and distributing video over the existing Internet based on a low latency and Internet-scale storage solution with no shared state, just-in time composition (rendering, personalization, etc.) through dynamic bitcode, blockchain controlled access and versioning, and efficient, high bandwidth content routing. We will show several examples using the fabric to stream video just-in-time to consumer audiences from single master version media and metadata, and various novel monetization solutions through blockchain smart contracts enabled by the fabric.

Video engineers at Netflix want to be immersed in applying algorithms and analyzing the media, and it is ideal if video engineers can do this without worrying about the underlying compute platform. Archer is an easy to use MapReduce style platform for media processing that uses containers so that users can bring their OS-level dependencies. Archer can apply hundreds of thousands of CPUs to make media innovation happen in minutes, join us to see how we do it at Netflix using Archer. In this talk, we will share the key differentiators that shaped Archer.

Per-title encoding is powerful, but it's just a start. Once you know the right adaptive curve for a video, what do you do next?

In this talk, we will discuss how data and machine learning can make video encoding better. After a quick summary of why per-title encoding matters, we will describe how our approach to instant and free per-title encoding. We will demonstrate that this approach is highly scalable, and how to transition from per-title to per-shot encoding.

We will then demonstrate how encoding an optimal bitrate ladder is only half the battle. A per-shot encoded bitrate ladder can ensure the highest quality rendition is being delivered, but it doesn’t ensure the right rendition is being delivered to the right person.

In order to solve that problem, we will demonstrate how AI can intelligently choose renditions to maximize QoE metrics, and how that data creates a positive feedback loop. Instead of just monitoring your QoE, you can use AI to actively improve it over time.

We can conclude by discussing the video delivery paradigm. You start with per-shot encoding using AV1 and optimized VMAF; then you can use AI to maximize experience while optimizing costs; lastly you deliver the video using ABR and super resolution upscaling (maybe).

What is the visual quality of the mobile video generated through your pipeline? How accurate is your measurement?

Using Adaptive Paired Comparison, the quality of a video sample is evaluated against a reference scale. Using previous responses to select the next reference sample, we only need to show each video once. With this approach we quickly and accurately estimate the perceived visual quality on mobile devices across a large set of videos, allowing us to understand impact of future encoding pipeline changes, without the need for a test lab.

Learn how and why Philo built synchronized playback. We'll cover the architecture that allows us to sync up players for both live and recorded content, and some of the use cases that we're exploring, such as watch parties, second-screen, and cross-device control.

Our progress on integrating machine learning with adaptive streaming at YouTube - what worked, what didn't, what results we have seen.

As part of our FIFA World Cup coverage this year, we've been able to deliver matches live in UHD with HDR10. We had a few short weeks to negotiate source acquisition, choose an encoder, configure our players, and determine the correct experience for our users. This project moved quickly and we were ultimately able to launch our UHD HDR experience on Chromecast, Amazon Fire TV and Android TV just before the semi-final matches started, with Apple TV and Roku support coming days later. This process provided an excellent learning opportunity, introducing us to the new challenges of supporting HEVC, HDR10 and much higher bitrate streams. We're happy to share our story, touching on cloud acquisition, cloud encoding, packaging (determining our bitrate/frame rate/resolution ladders), learnings around device support for HDR content, and the performance of our different players (Shaka, AVF, Exo & Roku).

Video quality measurement is an essential step in any media pipeline. There are quite a few objective metrics available today including well known SSIM & PSNR. These objective metrics are not real time to be used in a live encoder workflow, but fast enough to do quick offline video quality analysis. The challenge with such objective metrics though, is that they are not truly representative of video quality as our human eyes perceive it, thus making it hard for optimizations such as bitrate reduction.

To make decisions such as adding or reducing a certain amount of bitrate while maintaining the same subjective video quality, we need to rely on alternative methods. This talk will thus focus on understanding perceptual video quality and evaluating such alternative methods to make reliable decisions. Among many insights that will be shared, some of them will include: BD-Rate curves, existing rate control methods, gaps in using objective video quality metrics, perceptual video quality evaluation tools & benefits, using alternative methods.

In the jungle of solutions for low latency live, there are many current options, ranging from WebRTC, to proprietary UDP protocols to standard segmented media with ever-shortening segments. This session highlights one of these - chunked-encoded chunked-transferred CMAF - as a optimal and practical confluence of both reach and performance. On the technical side we'll investigate the underlying technology, the latency regimes possible, compatibility with legacy players, cachability on delivery networks and player behavior requirements. There will be a live demonstrations of several streams on a production network. This talks brings a standards perspective from DVB and DASH as well as CDN support. As a sweetener, we'll point you at open source code on both the encoder and player side for doing this all yourself. For free. From your tree.

Viewers of Twitch have extremely diversified network conditions. This poses a big challenge when it comes to providing a high-quality live streaming service and deploying new front and back-end features. Therefore, it is essential to identify, understand and build a knowledge base of our community’s typical network conditions in order to optimize the viewing experience for certain user groups and to speed up the software release cycle.

Based on viewer playback metrics collected at Twitch, we propose an unsupervised machine learning approach to generate playback activity clusters. Each cluster represents the playback behavior of a specific kind of network condition. After gathering the cluster information, we are able to compute the network condition parameters in order to reproduce the playback metrics of each cluster through a matching process. The results of these identified network parameters are used in Twitch’s transcoder and player algorithm development. With these parameters, we are able to accelerate the deployment of our new ABR playback algorithm for Twitch’s viewers using low latency streaming on mobile networks.