Test Coverage in Microservice Systems: An Automated Approach to E2E and API Test Coverage Metrics
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsIn general, the article is well written and the details shown for each of its sections are appreciated.
The most interesting thing about the paper is the presentation of 3 metrics to measure coverage in microservices that can be used for decision making and better use of resources or their planning.
In my opinion, the article has good contributions but they have not been highlighted in the conclusions, therefore this section should be improved, focusing on the analysis of the results and the proof of concept.
In some aspects of form, it can be improved, for example:
Figures 11 and 12 need a little more explanation.
Some of the references could be updated, and the order of their appearance reviewed, I would believe that it is better to present them in alphabetical order.
Author Response
Dear Reviewer,
Thank you for all your supportive feedback and comments. We provide detailed feedback to individual comments as follows. Please note we provide an answer and an action item referencing changes made.
Regards,
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsInteresting and actual topic, text quality acceptable, only one exception - weak work with references
Introduction
- only 3 citations, you should add more citations there, try to put your research in the context of other articles
Related works
- too few related works, for each topic exists much more related works, you can mention them at least by one sentence
Chapter 3
- its OK, easy to read, no mistakes
Case study
- keep only one figure from 8 and 9, merge them to one figure
- tables 1,2,3 increasy font, I am unable read subscripts
- Figure 12, remove (a) part, increase (b) part, remove black background, its hard read text
Discucssion
- add more references, compare your findings with related works
Conclussion
- too short, extend it, add most interesting results, add more about future research
References
- very few referemce, only 16, it is necessary add at least 20 more to total number 35-40
Author Response
Dear Reviewer,
Thank you for all your supportive feedback and comments. We provide detailed feedback to individual comments as follows. Please note we provide an answer and an action item referencing changes made.
Regards,
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for Authors(1) The author should add citations of the related work to the research background in the first chapter to highlight technical challenges and difficulties.
(2) In Chapter 2, the author should list and compare the differences between related work and this study, as well as the position of this study in the field.
(3) Chapter 3 should be a methodology rather than a test coverage indicator from a narrow point of view. The test coverage metric is at the core of this study. However, as a scientific journal paper rather than a purely technical report, the authors should have elaborated more extensively on the technical possibilities and discussed the method.
(4) Although the author gives a coverage formula, how do you obtain data such as the number of endpoints?
(5) Figure 3 shows the proposed method. How long does it take to extract endpoint information from source code and logs? How accurate is this method? Are there differences between different programming languages?
(6) The reviewer does not recommend that case studies appear in manuscripts. If the analysis is only for Java project source code, then this setting should be reflected in the title. If not, the author should also show the results from source code in other programming languages.
(7) The author should consider copyright for different icons in Figure 6, which is unsuitable for direct illustrations.
(8) The data and parameters used by the authors in the results are scattered. The author should first state the purpose and intention in the evaluation chapter.
(9) Figure 11 is invalid. The author should consider another way to illustrate this.
(10) The reviewer appreciates the author's separate chapter for discussion. However, the text in Figure 12b is unclear, and the authors should modify their color. In addition, if there is only 5.1, there is no need to set up this subsection.
(11) The conclusion chapter does not correspond enough to the results. The contributions and shortcomings of this study should be stated separately.
That's all. Thanks.
Author Response
Dear Reviewer,
Thank you for all your supportive feedback and comments. We provide detailed feedback to individual comments as follows. Please note we provide an answer and an action item referencing changes made.
Regards,
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
Comments and Suggestions for AuthorsI am fine with this version of the article, all my recommandations were solved
Author Response
Dear Reviewer,
Thank you for your insightful feedback. It significantly improved the quality of the paper.
Regards,
Reviewer 3 Report
Comments and Suggestions for Authors(1) At the end of the Abstract, the author should state the main results (preferably with numerical values).
(2) The reviewer suggests that the author include a table in Chapter 2 comparing the differences between related work and this study.
(3) The author should conduct a scientific discussion of the method at the end of Chapter 3, which should be different from the discussion of the Case Study in Chapter 5.
(4) What is the purpose of the separate discussion in Chapter 6? Authors should think carefully about the connections between chapter ordering.
The reviewer commends the authors for their high level of revision. The above details need to be further improved. That's all. Thanks.
Author Response
Dear Reviewer,
We appreciate all the supportive feedback and comments. Attached, we provide responses to each individual comment.
Regards,
Author Response File: Author Response.pdf
