Performance Comparison of VVC, AV1, HEVC, and AVC for High Resolutions
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThis paper is simply a comparison of codecs. There are a number of such comparisons and given that the latest codecs are designed for higher resolutions they are expected to give better results. The novelty of this work is therefore low.
Comments on the Quality of English LanguageEnglish needs considerable improvements. The paper has a lot of typos and use of incorrect tense making it sometimes difficult to read.
Author Response
Dear reviewer, I am kindly writing to ask if it would be possible to shorten the duration of the second round of review of my article. I understand that you have a lot of work to do, but I need the article as part of a project and would appreciate your help in expediting the review process. Thank you very much for your understanding.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsWhat is shown in Table 8 is not clear. What is the anchor?
Figure 5-9 plots should be enlarged to have two in a row. There is no page limit in the journal.
Group of Pictures is not specified in the manuscript. There is only the information that Intra frames appear every second. Frame types and profiles (e.g., low delay, random access) should be given. Results should distinguish frame types. Please provide BD-PSNR apart from BD-Rate in Tables 5-9.
Comments on the Quality of English LanguageThere is a typo in line 36.
Author Response
Dear reviewer, I am kindly writing to ask if it would be possible to shorten the duration of the second round of review of my article. I understand that you have a lot of work to do, but I need the article as part of a project and would appreciate your help in expediting the review process. Thank you very much for your understanding.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsIn this study, the authors conducted tests on seven different sequences, clearly demonstrating the compression performance of the latest and most widely used video codecs (H.266/VVC, AV1, H.265/HEVC, and H.264/AVC). This work has certain value, but there are still some shortcomings as follows:
1. Does this test still have some limitations or factors that may influence the results, such as the selection of the test set, adjustments to encoding parameters, etc.? Addressing these aspects would help readers better understand the reliability and applicability of the study.
2. In the paper, we observe that as the resolution increases, newly developed codecs such as H.266/VVC and AV1 exhibit superior performance. If necessary, a more detailed explanation could be provided on why H.266/VVC performs better at different resolutions, including the presence of specific technological features or optimization strategies.
3. If necessary, the "Conclusion" section could benefit from further elaboration, encouraging the authors to highlight key findings and underscore the significance of their work more effectively. Adding more comprehensive content to this section will enhance the overall conclusion, ensuring that readers gain a clear understanding of the critical contributions and implications of the study.
4. The authors have conducted a commendable comparison of the performance of H.266/VVC, AV1, H.265/HEVC, and H.264/AVC. However, I am curious whether differences in the experimental configurations may also impact the performance comparison. Therefore, I believe it is essential for the authors to showcase the experimental setup.
5.In addition to PSNR, SSIM, and VMAF, other evaluation metrics should be considered to comprehensively evaluate the performance of the codec, especially the subjective experience in different application scenarios.
6.Since a subset of test sequences is selected from the entire data set, it is recommended to gain in-depth knowledge of the characteristics of each selected sequence. This can help understand why these sequences were chosen and how they represent the entire data set.
7.For metrics such as PSNR, SSIM and VMAF, it is recommended to emphasize their value ranges and indicate the units of the indicators in the text to help readers better understand the significance of the evaluation results.
8.In the conclusion, possible future research directions can be briefly mentioned, such as further optimizing the codec, considering performance in different application scenarios, etc. This helps guide the reader in thinking about possible extensions and further research directions.
Comments on the Quality of English Language1. Minor editing of English language required。
2. The quality of figures should be improved.
Author Response
Dear reviewer, I am kindly writing to ask if it would be possible to shorten the duration of the second round of review of my article. I understand that you have a lot of work to do, but I need the article as part of a project and would appreciate your help in expediting the review process. Thank you very much for your understanding.
Author Response File: Author Response.pdf
Reviewer 4 Report
Comments and Suggestions for AuthorsThis submission presents a comprehensive analysis of the latest video codecs, including H.264/AVC, H.265/HEVC, H.266/VVC, and AV1, focusing on their compression performance at high resolutions. The key contributions of this study are its objective quality assessment using PSNR, SSIM, and VMAF metrics, and the evaluation of codec performance across various bitrates at 8K, UHD, and FHD resolutions. The study employs seven diverse test video sequences, ensuring a robust and varied analysis. The use of the Bjøntegaard Delta model for calculating bitrate savings and the presentation of rate-distortion curves for each codec at different resolutions are particularly noteworthy, as the paper provides a clear and quantifiable comparison of codec efficiencies. The paper's structure is well-organised. The data is effectively communicated through comprehensive tables and rate-distortion curves, making the findings accessible and understandable.
Here are two suggestions for improvement:
- To enrich the context and provide a clearer understanding of each codec, an introduction by flow charts highlighting the encoding process of each standard would be highly beneficial. These visual aids would help in overviewing the technical complexities and make the paper more accessible to readers who may not be deeply familiar with video encoding technologies.
- While the paper excels in comparing the quality and bitrate efficiency of different codecs, a deeper analysis of their computational efficiency would be beneficial. This could include evaluating the processing time and resource utilisation of each codec, providing a more holistic view of their practical applicability in various scenarios.
Author Response
Dear reviewer, I am kindly writing to ask if it would be possible to shorten the duration of the second round of review of my article. I understand that you have a lot of work to do, but I need the article as part of a project and would appreciate your help in expediting the review process. Thank you very much for your understanding.
Author Response File: Author Response.pdf
Reviewer 5 Report
Comments and Suggestions for AuthorsIn this paper, the authors present the performance comparison of the modern video codecs H.266/VVC, AV1, H265/HEVC and H.264/AVC. Comparison is made for the videos with three different resolutions (8K, Ultra High Definition (UHD), and Full High Definition (FHD)). The listed above codecs were tested on the seven video sequences. As a quality measure the authors considered: peak signal-to-noise ratio (PSNR), SSIM (Similarity Index) and VMAF (Video Multi-Method Assessment Fusion) quality criteria . Comparison of the rate-distortion curves for the video codecs when using different quality criteria is presented .
Remarks.
1. In my opinion, this paper lacks definitions and explanations for terminology, approaches, and quality measures used in the area. In the first five pages, the authors use a large number of abbreviations and terms which are partially explained on page 6 and later.
A non-full list of examples is given below.
· For example, the quality measures: PSNR, SSIM, and VMAF appear already in the abstract, but they are explained only on page 6.
· For resolutions: 240p, 480p, Full HD, Ultra HD, and 8K, frame sizes, frame rates, and number of bits per pixel have to be specified.
· 4:2:0 YUV is not defined
· Abbreviation SI-TI values appears at the beginning of page 3 and is explained only at the bottom of the page.
· Page 2, “…in terms of coding gains and complexity….” Coding gains are not defined.
Typos
1. Page 1, row 36, igh
2. Page 2, row 89 From this reason should be for this reason
3. Page 5, row 156 parameteres should be parameters
4. Page 6, row 168 Bjontegaard Delta should be Bjøntegaard Delta
I cannot recommend the publication of this paper in its present form. I recommend that the authors rewrite the paper to better explain the terminology.
Author Response
Dear reviewer, I am kindly writing to ask if it would be possible to shorten the duration of the second round of review of my article. I understand that you have a lot of work to do, but I need the article as part of a project and would appreciate your help in expediting the review process. Thank you very much for your understanding.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThis paper remains simply a comparison of codecs. There are a number of such comparisons and given that the latest codecs are designed for higher resolutions they are expected to give better results. The novelty of this work is therefore low. Just changing datasets and using the PP8K does not change the trends of other studies and well known conclusions.
Comments on the Quality of English LanguageEnglish needs improvement.
Author Response
Dear reviewer,
Please see the attached file with our responses. I am kindly writing to ask if it would be possible to shorten the duration of this round of review of my article. I understand that you have a lot of work to do, but I need the article as part of a project and would appreciate your help in expediting the review process.
Thank you very much for your understanding.
Author Response File: Author Response.pdf
Reviewer 4 Report
Comments and Suggestions for AuthorsI appreciate the efforts made by the authors in addressing the feedback provided during the previous review round. The 2nd feedback from me about computational efficiency comparisons had been fully addressed.
However, it seems there has been a misunderstanding regarding my 1st comment concerning the visual representation of the encoding process. I had suggested the inclusion of a flow chart for EACH coding standard discussed in the manuscript. The flow chats were expected to highlight the standards' respective key encoding components, those encoding components may refer to motion estimation (ME), motion compensation (MC), etc as examples of relevant encoding components in HEVC, H.264, etc. The objective was to provide a clear visual differentiation and overview of the distinct features and operational flows inherent to EACH standard, thereby elucidating the performance outcomes observed by the authors.
The revised manuscript now includes a general framework outlining the experimental process (encoding, evaluation), which, while valuable, does not align with the given suggestion. Given this, I would kindly ask the authors to provide those flow charts introduced under a minor revision to fill up such information gap.
Comments on the Quality of English LanguageN/A
Author Response
Dear reviewer,
please see the attached file with our responses. I am kindly writing to ask if it would be possible to shorten the duration of this round of review of my article. I understand that you have a lot of work to do, but I need the article as part of a project and would appreciate your help in expediting the review process.
Thank you very much for your understanding.
Author Response File: Author Response.pdf
Round 3
Reviewer 1 Report
Comments and Suggestions for AuthorsThis paper provides very limited novelty and is simply a comparison of codecs.
Comments on the Quality of English LanguageEnglish has been improved.
