We’ve recently been talking about a brand new video processor about to join the ARM Mali Multimedia Suite (MMS) of GPU, Video & Display IP. Egil, our next generation Mali video processor due for release later this year takes a step forward in functionality and performance to meet the needs of advancing video content. With more than 500 hours of video uploaded to YouTube every single minute it’s no surprise that optimizing video processing in sync with the full suite of products has been a key focus for us.
The MMS comprises software drivers and hardware optimized to work together right out of the box with the aim to maximize efficiency, enable faster time to market and vastly reduce potential support requirements. It has been designed to optimize performance between the various IP blocks through use of bandwidth saving technologies such as ARM Frame Buffer Compression (AFBC). AFBC can be implemented across the entire range of multimedia IP within an SoC and, depending on the type of content we’re talking about, this can produce bandwidth savings up to 50%. An AFBC-capable display controller or an AFBC-capable GPU can directly read the compressed frames produced by an AFBC-capable video decoder, such as Egil, reducing overall pipeline latency.
ARM approaches video processing in a different way from other IP providers. We believe it is better to provide all the codecs required in a unified Video IP solution, controlled by a single API, making it easier to develop flexible, multi-standard SoCs. To do this we analyse the codecs to be supported, establish which functions are required and develop hardware blocks to address each function - such as motion estimation & compensation, transforms, bitstreams and so on.
The hardware IP is developed as a core to operate at a set performance level, with multiple cores being used to address higher performance points. The ‘base core’ in Egil is designed to operate at 1080p60 frames per second, which will provide two Full HD encode and/or decode streams running simultaneously at 30 frames per second – assuming a 28HPM manufacturing process. To address 4K UHD 2160p at 60 frames per second (as for a 4K UHD TV) would require a four-core implementation.
At the same time as developing the hardware IP we also develop firmware to manage the video IP and interface with the host software. The firmware manages codec implementation, multi-core synchronization and communication requirements as well as additional specialist functions such as error concealment, picture reordering and rate control, saving the hardware and host CPU from getting involved in these steps at all. The result is unified video IP providing an easy to use, multi-standard, scalable solution capable of simultaneous encode and decode of multiple video streams, potentially even using different codecs at different resolutions!
Brand new in Egil is VP9 encode and decode capability, making it the first multi-standard video processor IP to support VP9 encode. We’ve also significantly enhanced HEVC encode and deliver an android reference software driver. Whilst currently OpenMaxIL based, this will be uprated to V4L2 as this is introduced to future versions of Android. This driver takes responsibility for setting up a particular video session, allocating memory, gating power dynamically and dramatically reduces the CPU load. The built in core scheduler manages multiple encode/decode streams and maps single or multiple video streams across multiple cores for maximum performance. This makes the new Mali video processor perfect for video conferencing and allows you to seamlessly share your viewed content with others. Not only that, but it means you can view multiple content streams at once, allowing you to keep one eye on the game throughout your meeting!
Another exciting aspect of ARM’s presence in the video space is our involvement with the Alliance for Open Media. As a founding member we’ve been working with leading internet companies in pursuit of an open and royalty-free AOMedia Video codec. We are heavily involved in this Joint Development Foundation project to define and develop media technologies addressing marketplace demand for a high quality open standard for video compression and delivery over the web. Timeline for the new codec, AV1, is to freeze the AV1 bitstream in Q1 2017, with first hardware support expected in the year that follows.
The multi standard nature of our new processor allows both encode and decode as well as supporting new and legacy codecs, all in a single piece of IP. Scalable to allow for every level of use case this next generation processor provides the perfect balance of efficiency and performance in the low power requirements of the mobile device. Egil is due for launch later in 2016.