From Monolithic Systems to Microservices: A Comparative Study of Performance

Round 1

Reviewer 1 Report

Making a note that this document has been previously submitted to this journal. I think most of the comments from my previous review have been addressed. However, there are some comments that I had on this new iteration. First of all, even if it is a re-submission, I believe that the paper should be accompanied with a cover letter explaining how each comment was addressed by the authors in the new version. Secondly, the paper still fails to highlight what exactly is the contribution of this work. Adding "new knowledge related to the evolution from Monolithic to Microservices architectures" is not sufficient I believe. Especially when the authors already published at least two papers on the same topic.

A major point that I mentioned in my previous review was with regards to the experimental setup. This iteration also lacks several elements that may affect the generalisation of the presented results/method. For instance, are the application used for evaluation open source. Is there some diagram available that explains their exact architecture. Is it Figure 1? Because the way that is presented is in a generic way, not the exact components that were deployed. Is this presented Figure 3 and Figure 4? What are the components then: Service 1 is Calculator Service, Service 2 is Reporting Service etc.

The paper still feels difficult to digest. Would be possible to create some diagram of the flow contribution, method and evaluation flow: we perform step 1, 2 and 3 because of this... With regards to results, in case of microservices approach: are th results aggregated over all microservices or is a single service? Other than that I think that there are several things that improve the paper.

1. Some senteces are ambiguous:
- "time-consuming is complicated because of its limited scalability and maintainability" -> is "time-consuminng" a technique? or there a missing word.
- "a comparative analysis of the performance of hardware resources and connectivity" -> this phrasing is unclear to me. Is the performance of a monolithic and microservices-based application measured on the same hardware resources?
- "However, most importantly, compared to our proposal, some of its results were not determined during the execution of services in both architectures. Even worse, its level of precision lacked contrast with a mathematical model, just as we performed in our research." -> Does this mean that some parameters where not evaluated because it is not possible, and they require a mathematical model to approximate their performance?

2. "Based on the given scenario, this study focuses primarily on a compilation of results from established research about the evaluation of both architectures" -> which scenario is this? This should be clarified.

3. "Then, we evaluated two different scenarios, of which one comprises a web application based on a
monolithic architecture that works on a virtual server with KVM." -> "Then" indicates a consequence or a result, and should go at the end of the group of sentences that created that consquence. It should not be used for starting a paragraph.

4. "When a company is created, all its applications generally start being Monolithic; this is reasonable, as these systems initially work well within limited settings, requiring less of their equipment. However, as companies grow and evolve, so they also need their application architecture. As systems become large and complex, companies turn into Microservices as a long-term infrastructure solution." -> Is this valid for all the companies? Would be possible to provide some references on this?

5. "Each version of the current study is executed with the proposed application in an environment
that matches its purpose. Subsequently, we performed the monolithic application on a Kernel-based
Virtual Machine (KVM) solution that implements full virtualization with Linux distributions. The
other case is the application with Microservices Architecture orchestrated with Docker. Its purpose is
to create lightweight and portable containers that can be executed independently and autonomous
of the Operating System. For both cases, the hardware resources were of the same characteristics,
including Intel Core i7 2.7 GHz CPU, 16 GB memory, HD disk, and virtual network." -> This all we know about the two applications being evaluated. Is there some diagram describing their architecture? Are they open-source?

6. With regards to Table 1 and Table 2:
- Why there is network traffic for the monolithic application if everything is deployed in a single place?
- The elapsed time unites are confusing. Are these results evaluated every millisecond?
- In table 1, are the metrics results aggregated over all containers? or is an average per container?

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper conducted a comparative and quantitative analysis of performance for the monolithic architecture and the microservice architecture and applied the mathematical non-parametric regression model to validate the experimental data. The results in this paper demonstrate more efficiency in respect to hardware resources, cost reduction, and high productivity when using microservices. In general, in the resubmitted version, the authors have addressed most of the review comments in a satisfactory manner so that the quality of the new version have been improved considerably. Only two minor suggestions:
- The formulas of the applied mathematical model are still not easy to follow. It would be better to add more informative descriptions (what and why) for these formulas.
- Section 6.5 is too long (especially in the new version). It would be better to add subsections to discuss important findings or lessons learned or itemize them.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

This sentence does not make sense: "time-consumption is complicated because of its limited scalability and maintainability"

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

The current manuscript only presents a study has been to apply a mathematical model order to validate the relationship between different variables, parameters, and restrictions when comparing applications using monolithic versus microservices architectures. However, this manuscript is just a good academic report, it does not have the scientific requirements for publication. The current work has no scientific contribution or innovation to the area.
Below are listed some flaws that authors should take into account in a future submission.
- The title is not clear, if the work is just a comparative study, then the title could be something like From Monolithic Systems to Microservices - Comparative Study of Performance;
- Authors should present in the introduction what the contribution to the scientific community is with the current work if such contribution exists ...
- The authors present many related works, however, the authors do not explain well the contribution of each one of these works to the current comparative study;
- Part of section 3 is a background (concepts) of the study, so some sub-sections should be presented before the related work, the remaining section we can consider the "Implementation Methodology";
- In section 5, the authors should better explain figure 5 and figure 9;
- In section 6 the authors state - "When we compare both the monolithic with the microservice architecture, we note that either has
527 its merits and demerits, as characterized by the volume and complexity of the entire system. "- this statement is very superficial, the authors must present a list of merits and demerits;
- The future work is very short, if the authors intend to continue with the study or to make new innovative investigations based on the current report, then the authors must be more clear, and thus be able to identify contributions of the current study.
These limitations prevented me from recommending the publication of this manuscript.

Reviewer 2 Report

This paper conducted a comparative and quantitative analysis of performance for the monolithic architecture and the microservice architecture, and applied the mathematical non-parametric regression model to validate the experimental data. In general, the proposed concepts and final findings are fascinating and beneficial for the advance of microservice technology. Nevertheless, there are still several ways in which it could be improved:

- The authors stated that the mathematical model was applied to “to represent the relationship between different variables, parameters, and restrictions”. However, what kinds of variables, parameters, and restrictions are not clearly described in the article. Suggest adding some discussions.

- The related work section is rich in content. It would be better if an analysis was conducted to elicit common understandings and identify issues that need to be addressed by investigating the current literature.

- I believe that the definitions and formulas of the applied mathematical model are not easy to follow for most readers, even the specialists in the domain of microservices. Suggest discussing more “why” and “how” to apply the model.

- The transition from Section 3 to Section 4 is not very strong. Why mathematical models “must” be applied?

- Many words were misspelled, for example, “difculties”, “quanties”, “ndings” in the abstract (and many similar defects in the whole paper). I guess that the authors copied the article content from a PDF file to a Word file. Careful proofreading is required.

 

Other remarks:

- Page 5: “… compared to our proposal, its results were determined during the execution of the services …”: it seems not easy to understand what is “it”. Suggest checking this sentence.

- Suggesting replacing “micro service” by “microservice”, “data base” by “database”.

Reviewer 3 Report

The paper presents a mathematical model that uses nonparametric regression to optimally migrate a monolith to a microservices deployment. The authors conclude that the microservices architecture is applied in large scale deployment with high traffic.
The paper is presenting an important topic. Even though some of the facts about the microservices architecture are established, I think a model would be required to decide when to use such an architecture. In this case I'm wondering why not use this architecture from the beginning to avoid migration? But other than that my main concern is with the presentation of the paper. I think the paper should be spell-checked before the submission. Also, the evaluation section seems incomplete. What are the evaluation scenarios, the experimental settings? some numbers, graphs are shown but they just came from somewhere

----------------------------------

- Abstract; "none of them quanties the differences in performance with precision" -> So what? Why is this important?
- Introduction: "a weak solution choice" -> how do we know which one is the weak or strong?
- Introduction: "Docker, Chef, Puppet, automatic scaling, among others, the implementation of those tools consumes time and resources" -> As opposed to what? I believe these solutions have been developed to save time and resources. Indeed, it takes time to master them, but one of the main purposes is to improve developer experience, and to save resources.
- Introduction: "an absolute contradiction appears to be generated in the available literature when comparing the performance of these architectures" -> which is?
- Introduction: "Therefore, the main aim of the current study has been to apply a mathematical model in
order to validate the relationship between different variables, parameters, and restrictions when
comparing applications using monolithic versus microservices architectures" -> first of all, avoid starting paragraphs with "therefore". Secondly, what are these variables, parameters, restrictions ?
- Introduction: "This new work concerns an expansion of our previous work published in [7]" -> what were the discoveries or conclusions in the previous article, and what is added in this one? Why is not the previous article sufficient? What does it add to the body of knowledge?
- Introduction: "The R language has been the implementation language" -> so what? why is this important?
- Section 2: "calculated the metric price per million requests" -> is this "evaluated the price per million requests metric" ?
- Section 2: "Comparing such efforts with those obtained by the current study, we may state that we developed
a comparative analysis of the performance of hardware resources and connectivity, between monolithic
and microservices architecture, in development and production environments" -> Why is the word "may" used here?
- Section 3: "the monolithic architecture uses a single development technology" -> Can you give some examples?
- Section 3: "The habit based on knowledge gives rise to trust in the same bare solutions, even knowing the risks that can lead to short or long-term problems" -> This is unclear. What does it mean? Which habit? Why is it a habit? and what is a habit in the context of this work?
- Results: Experimental design is missing. Where are these numbers coming from? What are the performed experiments? What are the scenarios?
- Conclusion: "some computational metrics have been evaluated" -> which one?
- Conclusion: "The use of containers allows jobs to be conducted independently" -> as opposed to what? why not with VM?

---------------------------------

1. Abstract: "none of them quanties the differences" -> Is "quanties" is a typo? Should this be "quantifies"?
2. Introduction: "resource-efcient" -> "resource-efficient"
3. Introduction: "efcient resources management" -> "efficient resource management"
4. Introduction: "new ndings" -> "new findings"