The High Efficiency Video Coding (HEVC) standard is the latest video coding standard and was finalized in 2013. HEVC is able to reduce the bit rate of the most widely used video coding standard, namely Advance Video Coding (AVC), by 50% while also is able to maintain the same objective and perceptive video quality. However, in the last few years, multimedia content has grown dramatically. Every day, millions of videos are transmitted over the Internet or are broadcast from a TV station. Moreover, with the rapid advance in telecommunication networks and the quick development of High Definition (HD) and Ultra High Definition (UHD) screens the users demand an improved experience. A recent Cisco report showed that in 2017 IP video traffic accounted for 75% of all Internet traffic, and they expect that it will continue growing till over 82% by the year 2022.
In this scenario, it is clear that a further reduction of the video bit rate it is needed. In October 2015, the Joint Video Exploration Team (JVET) was founded with the goal of exploring new video coding tools and analyzing the viability of a new coding standard. The JVET was formed by video expert from both the ITU-T VCEG and ISO/IEC MPEG, and in April 2018 transitioned into the Joint Video Experts Team (also abbreviated to JVET) with the task to develop a new video coding standard. This future video coding standard, named Versatile Video Coding (VVC), promises further bit rate reductions of 30-50% compared with HEVC and it is expected to be finalized by the year 2020. In parallel, in August 2015, several leading Internet companies announced the formation of the Alliance for Open Media (AOM) being their main objective the definition of a new royalty-free and open source video codec. This new codec it is called AV1 and promises up to 30% better video compression when compared with state-of-the-art video encoders such as AVC and HEVC. The improvements in compression efficiency are obtained by means of an increment in the complexity, which is in an increment in the execution time and energy consumption, of the encoding and decoding tools.
The general goal of this research line is to develop libraries that can serve as a reference to all those developers who intend to accelerate current or in-progress/future video codecs in heterogeneous platforms taking into account the real time needs, the cost of the platform, and the energy consumption.
In order to mitigate this complexity and provide real time HEVC encoding for high resolution, we have proposed several algorithms to optimize the HEVC quad-tree partitioning procedure, intra/inter prediction and mode decision by means of H264-based methods and spatial and temporal homogeneity analysis, which is directly applied to the original video. This has been done combining the capabilities of CPUs and GPUs as well as the human visual system (HVS) to provide a better QoE. These ideas in the multimedia line have been developed collaborating with a cutting-edge company such as PRODYS.
A completely different application scope for multimedia applications are mobile devices. In particular, video content should be preferably distributed in a format that is in accordance with the display and memory capabilities, processing power, and computational constraints of consumer electronics appliances as well as with the network bandwidth. In this scope, low-power asymmetric multicore processors (AMPs) have attracted considerable attention due to their appealing performance/power ratio for energy-constrained environments. However, these processors pose a significant programming challenge due to the integration of cores with different performance capabilities, asking for an asymmetry-aware scheduling solution that carefully distributes the workload. In this line, we have presented several architecture-aware implementations of an HEVC decoder that embeds a criticality-aware scheduling strategy tuned for an ARM big.LITTLE AMP.