HAZOP Ontology Semantic Similarity Algorithm Based on ACO-GRNN

Round 1

Reviewer 1 Report

The paper is original and in general well written but it misses some key references and comparison with the state of the art.

In the abstract, the authors needs to add some results. E.g. the proposed method is better by % percent. The authors mention in the conclusion “The experimental results show that the 438 comprehensive similarity calculation model proposed in this paper has a great improve-439 ment in accuracy compared with the traditional semantic similarity algorithm, and the 440 pearson correlation coefficient reaches 0.9819, which is very similar to the expert scoring 441 result.” They need to mention something similar about their results in the abstract.

Add a new session 2 where you state the related work and compare the proposed approach with the state of the art. This will help you add some additional references as well. You should try and have at least 30 references if possible.

I propose the addition of a diagram that explains the approach and help others to replicate the experiment.

Please replace figure 7 with a higher resolution figure

Please add limitations and future work where you explain how to address these limitations in the conclusions.

 

Overall this a good well written paper and it can be accepted after the above mention minor concerns have been addressed.

 

 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

The paper tackles the problem of combining different ingredients, such as ontologies, the assessment of semantic similarity, and Deep Learning techniques, to enforce the HAZOP (Hazard and Operability) safety analysis method, widely adopted in safety evaluation in the petrochemical industry.

STRONG POINTS

• The paper focuses on a timely topic, with a compelling approach which could be generalised to different safety-critical scenarios (domains).
• The approach has been fairly experimented.

WEAK POINTS

• The research core is not evident and boils down to the adoption of a Deep Learning algorithm whose objective is to tune the weights of the coefficients used to balance the impact of three different semantic similarity metrics.
• The utility of the approach is not supported by evidence of an implementation through a software tool, to be used by experts.
• In the approach, it is not clear why ontologies are of paramount importance and the benefits they deliver with respect to the adoption of a plain graph database.
• There is a lack of formalism in the presentation of contents in certain parts of the paper.

In the following, detailed comments for each section are reported. Authors are encouraged to syntactically check their writings, since there are several typos scattered along the paper.

INTRODUCTION

• In order for achieving a better degree of comprehension, authors should leverage the introduction only to frame the context, leaving other details to specific sections. The structure of such sections could be:

• • A “Related Work” section, whose aim is two-fold. On the one hand, it is apt to introduce background research definitions (e.g., the ones concerning semantic similarity, the definition of deviation, ...). On the other hand, a critical comparison of authors’ contribution with respect to the existing literature has to be provided, specifying what are the comparison features adopted. Indeed, answering these questions is of utmost importance: what are the contributions of the paper, apart from applying existing and well-known Deep Learning approaches and semantic similarity calculation metrics? What are the strengths and weaknesses of the approach? For helping the reader, at the end of the dissertation, a summary table can be added as well. Globally speaking, References should be expanded including up-to-date works.
    

• • After the “Related Work” section, a “Motivating Example” section would clarify what are the current issues in HAZOP analysis and the value added by an ontology. In this respect, a real case study/example serves to stress the problems and the challenges that are going to be tackled. This is especially advocated to help non-expert readers in understanding the motivation behind authors’ proposals.

• • After a “Motivating Example”, an “Approach Overview” section can be exploited to briefly summarise, in a stepwise way, the phases of the approach, with reference to the “Motivating Example” and the research challenges.

• As a minor remark, at the end of the introduction, an outline of the paper is missing.

CONSTRUCTION OF HAZOP ONTOLOGY

• The description of the ontology suffers from several flaws. This section should be re-engineered by undoubtedly specifying: (i) how the ontology has been built (reuse of concepts/relationships coming from reference ontologies? automated procedures?); (ii) description of the thematic areas of the ontology and their goal.
• It seems that the ontology is composed only of concepts and (object) relationships (which form the T-BOX of the ontology). Is that correct? Are authors also using individuals? (i.e., instances of concepts obtained with the rdf:type property). What is the language used for representing the ontology? # of concepts? # of relationships? Overall, a lack of rigorousness in the notation for expressing ontology concepts and relationships is perceived. 

• A link to the ontology file should be included in the paper (you may upload the ontology in Web tools such as WebProtégé and obtain a shareable link to insert).
• The definition of propagation path should be more emphasised (e.g., with the support of a graphical example), as it represents a crucial aspect in the paper.
• There is an inconsistency between Figure 1 and Figure 2. What is the difference between “deviation_parameter” concept and the dashed box labelled with “deviation event”. If they are not equal, it would be worthy to model also the concept of event through the ontology.

ONTOLOGY SEMANTIC SIMILARITY CALCULATION

• Most of the employed similarity metrics are strongly based on string distances between concepts names, datatypes and so forth. Such metrics are highly dependent on the design and the construction of the ontology (the so-called ontology engineering task), which are both unknown to the reader.
• The steps related to Figure 6 should be “linked” to the figure (e.g., including numbers to provide a visual feedback of the data/execution flow).

EXPERIMENTS AND RESULTS ANALYSIS

• Before delving into experimentations, a section devoted to the description of the architecture, the technologies used, the experimental setup and the statistics about the dataset used are missing.
• Details about the processing of HAZOP analysis reports and, more specifically, on how the ontology is effectively constructed starting from the reports, are not provided (with an extraction algorithm? which kind of algorithm?).
• Another fuzzy aspect regards the expert scoring results. It is claimed that they are assigned by 7 experts, but the assignment procedure is not known to the reader. Are they supported or forced to rely on their expertise only?

• From Table 3, a misleading aspect emerges. Consider line 3 and the two concepts “Too high liquid level” - “Too low liquid level”. They are intuitively opposite concepts. Therefore, how can their semantic similarity be steered by a high correlation?

CONCLUSIONS

• A thorough discussion on the result is only partially given in the Experimentation section. Future research streams are not provided, thus weakening the innovation potential brought by the paper, giving an inherent proof of its application exercise virtue.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

In the revised version of their manuscript, authors tried to address some of the issues raised during the first review round.

I regret to say that, despite the effort employed to tackle my concerns, there are still some critical points inside the newly submitted paper:

• The introduction is too long and very hard to read. Authors must split it into several sections, following the detailed outline I provided in the first review report (which has been slightly neglected, but was of the most important points to be considered). Indeed, the given suggestions about the reengineering of such contents have been mostly ignored.

• The automatic procedure(s) to create the individuals on the HAZOP ontology are not described inside the paper, nor they have been taken into consideration during the experimentation (the focus is on assessing semantic similarity, which is true, but details regarding the automatic generation would help closing the circle and appreciated - e.g. statistics about generation method, time, encountered issues…).
• Some sentences and paragraphs lack scientific presentation rigorousness. For instance, in the Introduction “Take the deviation as a clue…”[...]. This style of writing is quite poor and should be avoided through the paper.
• Authors should insert a very brief example regarding the conversion between a part of an HAZOP report into semantic concepts and relationships (e.g., using a supporting image), to have an idea about the original data.
• Section 3.1 (Tools and Techniques) does not concretely describe the architecture and the software module employed. Did authors use a unique Python script to run all the steps related to ontology automatic population, semantic similarity assessment etc.? What are the libraries employed? Underlying storage solutions for documents and the ontology? Is the ontology stored in a triplestore? From the current description, the reader perceives a hastily organised development strategy.
• After having clarified how the overall experts score is calculated (i.e., by averaging the scores given by experts, belonging to the range [0,1]) it is still hard to imagine a feasible assignment of such score by each single expert. For instance, to rate the similarity between two concepts A and B, on what basis an expert assigns 0.6 or 0.65 or 0.654?. This could be a negligible aspect, but when evaluating the average of experts scores, decimals may oscillate a lot, influencing the comparison with the scores calculated with other algorithms. Perhaps, authors should resort to another type of ground truth to determine the reference scores.
• If a software will be implemented as a future effort, try at least to provide an utilisation scenario with the description/mockup of a prototype GUI. 
• An interesting aspect would be devising a metric to evaluate what is the concrete benefit in conducting the HAZOP analysis with the ontology (e.g., measuring somehow the saved time, the degree of satisfaction, the effectiveness, …). Inside the text it is only stated that it enables to save time and endless analysis sessions.
• The outline of the paper should not contain details about the contributions of the paper (you can add a dedicated paragraph/sub-section for that purpose).
• In this revised version, the research contribution is still very limited. Authors should work more on trying to let the research core emerge more.
• Overall, there are several typos in the text and in the figures. Author should carefully perform again a syntactical check.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 3

Reviewer 2 Report

In the last version of the paper, authors addressed almost all my concerns.

I would like to recommend a final spell checking for words and sentences (as a suggestion, try to read the paper starting from the end, in order to rule out typos inside the text or odd sentences).

All in all, please consider and work on my suggestions as a future research stream, in order to produce a thorough research paper.