CloudOps: Towards the Operationalization of the Cloud Continuum: Concepts, Challenges and a Reference Framework

Round 1

Reviewer 1 Report

This is well written paper that discuss many challenges and solutions for realizing the DevOps concept on Cloud-Edge environment under the cloud continuum requirement.

The discussions are thorough and a reference framework solution is proposed.

The only weakness of this paper is that there is no evaluation or case study to validate the proposed framework. So, it is hard to say how the proposed framework can work or be used in practice. Hence, the reviewer strongly suggest the authors should identify or build a real system that matches to the proposed, and use it to support the claims made in the paper either by experiment evaluation or discussion.

Also, the font size of figure4 is too small to read.

Minor: The caption of figure3 should be in the same page of the figure.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

The author summarizes the research progress and existing problems of the cloud continuum, as well as some challenges in all stages of the operation phase of the application.  The paper puts forward a CloudOps reference framework and its advantages. But the descriptions of the paper are more conceptual with lack of application operations and performance analysis in real-time. The paper could be modified as a  review paper for publishment.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper focuses on the topic of managing computing resources across the IoT-Edge-Cloud continuum, and provides an overview of the main concepts and challenges. The authors also describe a novel CloudOps workflow as a complementary extension to established DevOps practices.

The paper is well written and structured. It is clearly within the scope of the journal special issue. I have a few comments that can help further improve the paper:

• In the abstract, the authors mention lightweight embedded devices as part of the cloud continuum. The rest of the paper, however, only focuses on Cloud and Edge infrastructures, but not the IoT devices, which was a bit disappointing for me because I agreed to review the paper intrigued by this challenge. Personally, I find this the most challenging and interesting part, since constrained devices do not support containerisation/resource isolation, and are not always directly connected to the Internet. I know there are some existing approaches that focus on such last-mile management of software even for non-Internet-connected IoT devices. Maybe the authors could include a discussion on whether/how their approach can be further extended in this direction.
• Section 1 should include end-to-end security as one of the challenges. This is clearly one of the main issues for migrating applications and data out of the cloud. This should include not just the traditional cyber-security challenges, but also the social and physical aspects, since edge infrastructures are more exposed to physical tempering and human interaction. Data privacy and ownership should also be discussed, since many scenarios do not tolerate sending data to the cloud and thus strictly require edge computing.
• Does the whole CloudOps approach assume that microservices are generic computing applications and only non-functional factors are taken into account when placing them in the continuum? For example, in the described smart surveillance scenario, do managed microservices only deal with video/image processing (which can run both at the edge and in the cloud) or also handle video capturing itself (and thus obviously can only run on the device)? 
• For me, it is difficult to evaluate the feasibility of the whole approach without some proof-of-concept demonstration. While the 6 functional blocks described in Section 3 make perfect sense to me, the whole architecture still remains quite high-level and conceptual. It is also difficult to judge the relevance and the value of the proposed framework to the 2 scenarios in Section 4, without having some minimum implementation in place.
• It is not clear to me how the challenges listed in the introduction are then related to Section 2, which is also about challenges. Also, Figure 1 is not referred to in the text, and it is not clear whether it is related to the challenges in Section 1 or Section 2.
• A few typos that I spotted: 
• Page 2, line 52: Full stop missing after ‘needed’.
• Page 3, line 132: some redundant text in parentheses.
• Page 3, Figure 1: what is ‘ns’?
• Page 7, line 338: ‘anther’
• Page 15, line 595: Space between two sentences missing.
• Page 16, line 664: ‘targets users’
• Diagrams should be more readable, and the fonts increased. Figure 4 is especially non-illegible.

Author Response

Please see the attachment

Round 2

Reviewer 1 Report

Thanks the efforts for adding the case study in Section4.3. But it is still difficult to judge or validate the contributions of this work from the example for two reasons.

First, the diagram in figure 7 does not show the differences between the traditional DevOps and the proposed CloudOps. Instead of using the diagram to argue the system is implemented based on the CloudOps framework, the authors should use it to highlight and discuss the values or improvements from using the proposed framework in this particular use case.

Second, Table2 is given to argue the benefits of the proposed framework. But there is no clarification or justification on how these costs (PM/HP) are measured. So, it is hard to convince the readers. More quantitative or qualitative measurements should be given for the validation.

typo: line 706 "figure 3"=>"Figure 3"

Author Response

Dear reviewer,

Please see attachment.

Best regards

Juncal

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper presents a reference framework and discusses some concepts. The research idea is valueable to the reader.

Author Response

Dear reviewer,

Please see attachment.

Best regards

Juncal

Author Response File: Author Response.pdf

Reviewer 3 Report

I can see that the paper has improved with some of my (as well as other reviewers’) comments being addressed. However, I was not provided with a detailed response letter to fully evaluate to what extent my comments have been addressed (or maybe refused with some argued justification). From what I can see now, the main shortcoming of the paper, related to a proof of concept implementation, has not been sufficiently improved:

>>For me, it is difficult to evaluate the feasibility of the whole approach without some proof-of-concept demonstration. While the 6 functional blocks described in Section 3 make perfect sense to me, the whole architecture still remains quite high-level and conceptual. It is also difficult to judge the relevance and the value of the proposed framework to the 2 scenarios in Section 4, without having some minimum implementation in place.

In my opinion, the new additions in the current form do not qualify for a proper implementation and validation. There is a new Figure 7, but the text explanation is insufficient. It is currently difficult to understand how all these components in the diagram map to Fig. 3. I would expect this mapping to be explicitly explained and discussed in the text, rather than having the reader going back and forth comparing the two diagrams. 

I would like to see a more technical description of the individual components - e.g. a JSON snippet of the Application Description, a concrete implementation technology behind each component (there are now just three cloud provider icons and “.NET” mentioned in the text, which do not explain much), information flows between these components, what each of the arrows represents, etc. Please also provide a reference to a public code repository, if it exists.

Although I originally did not request to evaluate the optimisation/operationalisation aspects, I am now wondering how all these PMs were estimated? Is there any justification for this?

Also, as far as I could see from this “track-changes” version, the following comment has not been addressed, unless I am missing something (that is why a response letter would help):

>> Does the whole CloudOps approach assume that microservices are generic computing applications and only non-functional factors are taken into account when placing them in the continuum? For example, in the described smart surveillance scenario, do managed microservices only deal with video/image processing (which can run both at the edge and in the cloud) or also handle video capturing itself (and thus obviously can only run on the device)? 

Lastly, while the original paper was almost perfect in terms of English, the newly added text has much more typos and errors, also the caption of the new Table 2 was copied and pasted. As a reviewer, I am left with a somewhat twisted impression that the changes were made hastily and without much preparation and proof-reading, especially given the short time between the original review and the re-submission. I would recommend spending more time thoroughly preparing the new version this time, not for the sake of addressing the reviewers’ comments but for improving the quality and contents of the paper.

Author Response

Dear reviewer,

Please see attachment.

Best regards

Juncal

Author Response File: Author Response.pdf

Round 3

Reviewer 1 Report

All my questions have been addressed. I have no further comment.