Specification and Verification in Integrated Model of Distributed Systems (IMDS)

Round 1

Reviewer 1 Report

The article discusses an specification and verification considering the IMDS model. Below I am presenting my comments:

a/ We must present in the Abstract what is our motivation really. What is the gap in the literature today?

b/ It is important to present briefly some related work in the introduction to survey us a motivation. Again, what is the gap in the area? What research opportunities we have in the area?

c/ We have a lot of English erros, including The prosed IMDS...

d/ I did not understand your contributions. IMDS was also published right? And, how can I observe the system deadlock? Must I insert traces in the code, or something like that?

e/ How did you select the related work? It is important to present shortly in this section an ANALYSIS, so detaching the gap in the area.

f/ What is the main section of your article? Section 3? But IMDS was published earlier, right?

g/  How hard is to use your system in the user viewpoint? And what is the computational complexity of your solution?

h/ You must reorganize section 3. Is is very hard to understand your ideas. We have much text in the overview subsection.

i/ Figure 1 is not clear. What is happening? I can cut it away from the article without problem.

j/ What is the API for your proposal?

l/ Your solution works fine with produce-consumer applications? Can I use it for P2P or fog? Or yet, can I use your proposal with cloud computing where we can have resource elasticity?

m/ Is your proposal agnostic in terms of programming language?

n/ In the evaluation Sections, section 4 and 5, are there benchmarks in the area for comparison purposes? Why using Dedando environment?

o/   Your conclusion is so big. And much of the text present in the conclusion is not clear along the article.

p/ I observed in the conclusion that your idea is to present a new version for IMDS. But it is not clear the main limitations of the current proposal.

Author Response

Dear Reviewer,

Thank you for a very insightful review. Below are the answers to the your comments. The relevant fragments are marked in the text of the paper with a yellow background. Minor changes are throughout the text because the it has been checked by a native speaker.

Comments and Suggestions for Authors

The article discusses an specification and verification considering the IMDS model. Below I am presenting my comments:

Point a/ We must present in the Abstract what is our motivation really. What is the gap in the literature today?

Response: The abstract has been revised substantially to highlight the gap in the literature, the motivation and the contribution. Three features are highlighted: asynchrony, communication duality, automated total/partial communication/resource deadlock/termination detection.

Point b/ It is important to present briefly some related work in the introduction to survey us a motivation. Again, what is the gap in the area? What research opportunities we have in the area?

Response: Three paragraphs are added to the introduction:

“The most popular formalisms for modeling distributed systems are mentioned in Section 2.1. Majority of them is synchronous, which requires simultaneous activity or access to global/non-local state, which is unrealistic. Such approaches kill locality, autonomy and asynchrony. Even asynchronous formalism assume some ordering of messages on the input of servers (queues or stacks). None of known modeling techniques support communication duality.”

“Many techniques mentioned in Section 2.2 support automated model checking, but for a very limited collection of features, typically total deadlock. The deadlock is typically not distinguishable from termination, because both are discontinuation of processes. Partial deadlock/termination, in which not all processes are involved, can be automatically detected for system with restricted scheme, for example purely cyclic systems. None of the methods identify communication deadlocks and resource deadlocks.”

“The proposed IMDS formalism is a basis on several new ideas and techniques for specification and verification of distributed systems, mentioned in other papers: structural siphon-based deadlock detection in Petri nets equivalent to IMDS, applied to arbitrary shaped systems, dual distributed automata (for servers and for agents) to simulate counterexamples over distributed system components, non-exhaustive partial deadlock detection, timed IMDS specification and others. They exceed the scope of the present paper and therefore they are addressed in conclusions.”

Point c/ We have a lot of English erros, including The prosed IMDS...

Response: Corrected, the paper has been checked by the native speaker.

Point d/ I did not understand your contributions. IMDS was also published right? And, how can I observe the system deadlock? Must I insert traces in the code, or something like that?

Response: New paragraph is added in Section 4:

“The formulas for deadlock detection and termination checking, given in Table 1, do not depend on the structure of the verified system. Therefore, everything a user must do is a system specification as a set of actions over sets of messages and states. Example specification is given in Section 3.6. Then, the verification is run automatically in Dedan in “push the button” style, giving the counterexamples if a deadlock is found.”

Point e/ How did you select the related work? It is important to present shortly in this section an ANALYSIS, so detaching the gap in the area.

Response: In related work in Section 2.1 most popular synchronous and asynchronous formalisms were chosen. The paragraph is added at the end of Section 2.1:

“Summing up, the majority of formalisms are synchronous ones. Several asynchronous formalisms do not support autonomy because the servers react to the messages accordingly to their arrival sequence (FIFO or LIFO). The autonomy requires that a server choses a message to serve on its own. No formalism provides a duality of communication in a unified formalism (message passing/resource sharing).”

In related work in Section 2.2 the examples are chosen to show how the structure of the verified system is limited in various ways. Also, methods not distinguishing between deadlocks different types and deadlock from termination are presented. The paragraph is added at the end of Section 2.2:

“In the context of automatic verification, most methods only identify total deadlocks. They do not differentiate deadlocks from termination. Partial deadlocks are automatically located in a minority of methods, at the expense of limiting the structure of verified systems. None of the methods can distinguish between communication deadlocks and resource deadlocks.”

Point f/ What is the main section of your article? Section 3? But IMDS was published earlier, right?

Response: New sentences are included at the end of Introduction:

“Section 3 is the most important in the paper. IMDS was published earlier, but it was formulated in a different way, difficult to understand. Moreover, the experience of using the formalism for three years and the invention of many new ideas based on IMDS strongly influenced the current definition.”

Point g/ How hard is to use your system in the user viewpoint? And what is the computational complexity of your solution?

Response: First question is addressed in the reply to point j)

The answer to the second question is a new paragraph at the end of section 4:

“The complexity of the verification is typical: it is P-Complete [Schnoebelen, 2002], which means that the evaluation time of the time formula is |LTS|x|phi|, where |LTS| denotes the number of nodes in the Labeled Transition System, and |phi| is the length of a formula phi. Every formula has one or two temporal operators, therefore this factor is constant. In the case of large systems, we use the Uppaal verifier [Behrmann et al., 2011], which is equipped with a symbolic representation of the reachable space.”

Point h/ You must reorganize section 3. Is is very hard to understand your ideas. We have much text in the overview subsection. 

Response: Section 3.1 is shortened. In Sections 3.2 and 3.3 some symbols and equations are eliminated.

Point i/ Figure 1 is not clear. What is happening? I can cut it away from the article without problem. 

Response: It is just a graphical comment to the system action. It can be cut away, but in my opinion graphical comment simplifies the understanding. I would rather leave this drawing.

Point j/ What is the API for your proposal?

Response: New paragraph is added at the end of Section 3.6:

“The presented input of the Dedan program follows IMDS formulation and it is uncomfortable for programmers. Therefore, the Rybu preprocessor for imperative-like programming was developed by the students of ICS, WUT under a supervision of the author. The students of second year studies use Rybu language in verification of their synchronization solutions. The Rybu language exceeds the scope of this paper, it is described in [Daszczuk et al., 2017]. More than 200 students verified their exercises, and in their opinion the Dedan/Rybu program is not difficult to use. Some of the models exceed 6 thousand actions.”

Point l/ Your solution works fine with produce-consumer applications? Can I use it for P2P or fog? Or yet, can I use your proposal with cloud computing where we can have resource elasticity?

Response: New paragraph is added at the end of Section 3.6. I do not deal with cloud computing, therefore I cannot answer your question on cloud computing directly. I listed the projects successfully verified. I believe that we can model any system, except for systems described in the limitations. However, with some effort, we can model even those systems. About cloud computing with resource elasticity: I think that architecture and strategies may be checked against deadlocks, scalability and performance are not subject to model checking. Please note that every system must be expressed as a set of actions (over states and messages) to be verified.

“Most student exercises are in RPC paradigm. However, we modeled systems of several schemes:

-communication protocols

-train scheduling

-production cell

-autonomous vehicle guidance

-patrol robots swarm

-distributed systems for access control 

-business processes“

Point m/ Is your proposal agnostic in terms of programming language?

Response: New paragraph is added at the end of Section 3.6.

“Models can be expressed in different programming languages, but the designer must keep in mind that a typical programming language is based on globally available variables, regardless of whether it is sequential or parallel. The distributed systems specification should not contain global variables or centrally started processes/threads. Individual servers or agents can be defined in traditional languages, but somehow they should be combined into a distributed system.“

Point n/ In the evaluation Sections, section 4 and 5, are there benchmarks in the area for comparison purposes? Why using Dedando environment?

Response: Three new paragraph is added at the end of Section 5.

“The bounded buffer is just an example. Other simple examples can be found in [Dedan Examples]. There are some benchmarks in the literature, mainly synchronic systems or systems having a global state available, such as solitaire game, 4-queen problem, 8-puzzle., dining philosophers, etc. The latter one, converted into a distributed version, is included in [Dedan Examples]. 

The Karlsruhe Production Cell is a serious benchmark [Lewerentz ea al., 1995]. Most of the verification examples are synchronous, but several asynchronous models can be found in the literature. In our solution, the asynchronous model is verified [Daszczuk, 2019], in which distributed devices are modeled as servers, and metal plates moving through the cell are modeled as agents. Our verification finds an expected deadlock, but none of the other solutions can express the deadlock from the perspective of cooperating controllers (server view) and from the perspective of metal plates traveling through the cell (agent view). However, the main benefit comes from automatic verification using Dedan. Most examples of Karlsruhe Production Cell verification use formulas individually designed for the verification.

Similarly, in the automatic vehicle guidance system [Czejdo et al., 2016], the communication deadlock can be observed in road segment controllers cooperation, which are modeled as servers. The resource deadlock is visible in the guided vehicles perspective, which are modeled as agents.”

Point o/ Your conclusion is so big. And much of the text present in the conclusion is not clear along the article. 

Response: The Conclusions were shortened substantially.

Point p/ I observed in the conclusion that your idea is to present a new version for IMDS. But it is not clear the main limitations of the current proposal. 

Response: The paragraph at the end of Section 3.1 presents the limitation:

“The obvious limitation of the modeled system class comes from the IMDS rules: there is one input message and one output message in a regular action (no output message in the agent-terminating action). Certain classes of systems are difficult to model: systems with broadcast communication, consumable resources or pools of dynamically created processes. However, this is not impossible, because a designer can use a pool of "sleeping" agents that are woken up as needed. This pool must be limited. This limitation is common to all static verification methods.”

It is repeated in the Conclusions:

“The structure of the verified system is not restricted, provided that it can be modeled as a set of actions over states and messages. The only limitation is due to the model must be bounded to be statically verified: features such as broadcasting messages, consumable resources or dynamically created processes are difficult to specify.”

Author Response File: Author Response.pdf

Reviewer 2 Report

The revised paper was improved with clarifying several aspects of their figures and language mistakes.

- The resolutions of all figures must be improved because the letters in the figures are not clear.

Author Response

The revised paper was improved with clarifying several aspects of their figures and language mistakes.

- The resolutions of all figures must be improved because the letters in the figures are not clear.

Response: The font in the figures was enlarged.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have improved the article. Thank you for addressing almost all my comments. Below new comments:

a/ Present the abstract in a succinct way: area: problem, open gap, your work, contribution, results. It is so large today.

b/ Present the Introduction in a succinct way: the same: area: problem, open gap, your work, contribution, results. It is so large today.

c/ What are the limitations of the work?

Author Response

Comments and Suggestions for Authors

The authors have improved the article. Thank you for addressing almost all my comments. Below new comments:

Point a/ Present the abstract in a succinct way: area: problem, open gap, your work, contribution, results. It is so large today.

Response: The summary has been rebuilt from scratch, in a suggested manner. It is as short as possible, but it must cover a problem, a gap and a solution for the following functions: locality, autonomy, asynchrony, communication duality, differentiation of termination/deadlock, differentiation of communication/resource deadlock, partial deadlock for an arbitrary shape system.

Point b/ Present the Introduction in a succinct way: the same: area: problem, open gap, your work, contribution, results. It is so large today.

Response: The introduction has been rebuilt substantially and shortened, in a suggested manner. The problem was similar to that in the abstract.

Point c/ What are the limitations of the work?

Response: The text on limitations has been expanded in both places (Section 3.1 and Conclusions)

“The obvious limitation of the modeled system class comes from the IMDS rules: there is exactly one input message and at most one output message of the action. Certain classes of systems are difficult to model: systems with broadcast communication, consumable resources or pools of dynamically created processes. However, this is not impossible, because a designer can use a pool of "sleeping" agents that are woken up as needed. This pool must be limited in order to statically verify the systems.”

“The structure of the verified system is not restricted, provided that it can be modeled as a set of actions over states and messages. The only limitation is due to the model must be bounded to be statically verified: features such as broadcasting messages, consumable resources or dynamically created processes are difficult to specify. This limitation can be overcome with a static pool of agents waiting to be activated. They can be enabled in the splitting of the process, creating a consumable resource, launching a process in a cloud application, or issuing a broadcast message. However, the dynamic creation of processes would be more natural. The obvious limitation is the size of the system, which is to be statically checked, due to the exponential explosion of the reachability space.”

Reviewer 2 Report

The revised paper was improved with clarifying several aspects of their figures and language mistakes.

Author Response

Comments and Suggestions for Authors

The revised paper was improved with clarifying several aspects of their figures and language mistakes.

Response: Thank you for the reviews.