Early Implementation of Mobile Real-time EVC Player MPEG Submission October 2020

 

  By Olga Krovyakova - November, 3 2020

Abstract

This contribution describes an early implementation of a Real-time Mobile Video Player designed to decode and playback bitstreams conformance with MPEG-5/Part 1: Essential Video Coding (EVC) International Standard under publication. The Player is based on ETM reference software version 6.1 and optimized for ARM architecture. The Player is developed and tested on Huawei P40 Pro smartphone and demonstrates 1080p real-time playback at 24 fps on a subset of EVC Main profile tools.

 

1. Introduction

The current early implementation is targeted to demonstrate feasibility of real-time playback of EVC bitstreams on top market available mobile devices. Huawei P40 Pro Smartphone with Kirin 990 SoC was chosen as a target device. It is operated under EMUI 10.1 based on Android 10 and has in total 8 CPU cores, among which:

• 2 hi-end Cortex-A76 cores clocked at 2.86GHz;
• 2 mid-end Cortex-A76 cores clocked at 2.36GHz;
• 4 low-end Cortex-A55 cores clocked at 1.95GHz.

As an input material “Kimono1” and “ParkScene” test sequences from JVT-VC test set were selected. Both test sequences have 1080p resolution at 24 fps with 4:2:0 color format and 10 seconds duration.

As an encoder configuration default Random Access configuration of EVC Common Test Condition (CTC) was selected. Quantization Parameter (QP) 32 was chosen as a target QP.

 

2. Supported EVC tools

In order to achieve real-time playback speed on the target device a subset of EVC tools was selected based on profiling information, collected by Android Profiler on the test sequences encoded and decoded with ETM6.1 reference software. Figure 1 and Figure 2 demonstrate obtained profiling data on the target device for these two sequences correspondingly.

  

Figure 1. ETM6.1 Kimono bitstream decoding profiling

Figure 2. ETM6.1 ParkScene bitstream decoding profiling

 

Based on the profiling data a subset of EVC tools was selected as summarized in Table 1. Tools names and abbreviations in the table follow the standard connotations as defined in the EVC specification.

 

Table 1. Selected toolset for the early real-time player implementation

Tool short name	Tool full name	ETM6.1 CTC RA default configuration	Selected toolset
ADMVP	Advanced Motion Vectors Prediction	1	1
AFFINE	Affine prediction	1	1
HTDF	Hadamard Transform Domain Filter	1	1
DMVR	Decoder side Motion Vectors Derivation	1	1
ADCC	Advanced Coefficients Coding	1	1
ADDB	Advanced Deblocking	1	1
ALF	Adaptive Loop Filter	1	0
AMVR	Adaptive Motion Vectors resolution	1	1
ATS	Adaptive Transforms Selection	1	1
BTT	Binary and Ternary Trees	1	0
CM_INIT	Context Modeling Initialization	1	1
DBF	Deblocking Filter	1	1
EIPD	Enhanced Intra Prediction Directions	1	0
HMVP	History Motion vectors Prediction	1	1
iQT	Advanced Quantization and Transforms	1	1
MMVD	Merge with Motion Vectors Difference	1	1
POCS	Advanced Picture Order Count	1	1
RPL	Reference Picture List	1	1
SUCO	Split Unit Coding Order	1	0
IBC	Intra Block Copy	0	0

 

As can be seen from Table 1 most of EVC tools were included into the selected toolset.  

3. Implementation details and playback

In order to achieve real-time playback on the selected toolset the following main modifications were performed on top ETM 6.1 reference SW:

1. ARM SIMD implementation of the most critical functions in MC part
2. Waterfront-like parallel processing of CTU lines inside one frame.
3. Deblocking filter parallel implementation

For implementation simplicity the only 4 most powerful (hi-end and mid-end) cores of Kirin SoC were selected to be used in the Player.

As a result the Player demonstrates 24+ fps playback on the device. The average decoding speed over 10 second playback is summarized in Table 2.

 

Table 2. Average results of the selected toolset playback on the device 

Test sequence	Decoding speed (fps)
ParkScene	24.39
Kimono1	25

In order to check playback speed objectively and subjectively the Player was deployed and tested on Huawei P40 Pro smartphone.

Figure 3 demonstrates a picture of the Player during Kimono bitstream playback on the device.

Figure 3. Picture of the Player working on Huawei P40 Pro

 4. Profiling of the optimized decoder

Figure 4. Optimized decoder Kimono bitstream profiling

Figure 5. Optimized decoder ParkScene bitstream profiling

 

5. CPU Utilization

The current implementation of the Player is optimized for Huawei P40 Pro mobile phone in terms of CPU cores usage, more specificity the SW works only with 4 most powerful Kirin 990 CPU cores (hi-end and mid-end). Figure 6 and Figure 7 give CPU Utilization percentage for each of the 4 powerful cores during 10 second playback.

Figure 6. Hi-end and mid-end CPU cores utilization. Kimono playback.

Figure 7. Hi-end and mid-end CPU cores utilization. ParkScene playback.

 

6. Power Consumption

In order to estimate power consumption an infinite playback loop of test bitstreams was launched at 100% charged device and the process was working until the device switched off due to energy insufficiency. As a result the Player was working for more than 4.5 hours with constant 24 fps speed. Figure 10 and 11 summarize obtained results.

Figure 10. Power consumption and playback speed during infinite playback of Kimono

Figure 11. Power consumption and playback speed during infinite playback of ParkScene

 

7. Conclusion

The early version EVC software Player implemented by Solveig Multimedia demonstrates that real-time playback of EVC bitstreams is feasible and can be used on modern mobile devices with reasonable power consumption.

 

 About the author

Olga Krovyakova is the Technical Support Manager in Solveig Multimedia since 2010.

She is the author of many text and video guidelines of company's products: Video Splitter, HyperCam, WMP Trimmer Plugin, AVI Trimmer+ and TriMP4.

She works with programs every day and therefore knows very well how they work. Сontact Olga via support@solveigmm.com if you have any questions. She will gladly assist you!