Context-Based Support to Enhance Developers’ Learning of Software Security
Round 1
Reviewer 1 Report
This paper is well-written and structured. The research methodology used in this paper is well-defined. However, I think it requires some minor revisions in mainly three parts.
(1) Introduction needs to give more emphasis on the topic being investigated. Instead, it was stated that this paper is an advancement to previous works. Therefore, it needs to show what was missing from those works which is being covered here with more concentrationto the contribution of knowledge in this work. Hence, an in-depth description is required to investigate the gap between the current works and the proposed one.
(2) I suggest adding a table that consists of current and previous works in the field of this paper. Such table needs to illustrate how the contribution in this paper differs from previous work.
(3) Conclusion needs to very concise to address the main contributions of this work.
Overall good but minor editing is required. For example, it's mentioned 'thesis' instead of 'paper' in line 252.
Author Response
Comment 1: Introduction needs to give more emphasis on the topic being investigated. Instead, it was stated that this paper is an advancement to previous works. Therefore, it needs to show what was missing from those works which is being covered here with more concentration to the contribution of knowledge in this work. Hence, an in-depth description is required to investigate the gap between the current works and the proposed one.
Response: Thanks for the comment. In an effort to provide a preliminary analysis of security teaching methodologies, we have identified several drawbacks associated with conventional approaches. Please refer to Line 55-61 for more information on these limitations, which are further elaborated upon in Section 2.1.
Comment 2: I suggest adding a table that consists of current and previous works in the field of this paper. Such table needs to illustrate how the contribution in this paper differs from previous work.
Response: Thanks for the reviewer’s suggestion. Upon thorough examination of related work in the security education and learning, we have discovered a multitude of differences, including the target audience, teaching techniques, design concepts, system architecture, etc. Using tables may be a challenge to present information in a comprehensive manner. Therefore, in our manuscript, we have opted to include a descriptive approach to address the related work. Nevertheless, we describe the features of our approach differentiating from the previous work in Line 247-253.
Comment 3: Conclusion needs to very concise to address the main contributions of this work.
Response: The first paragraph of Conclusion section is rewritten, in which more concrete contributions are addressed.
Comment 4: Overall good but minor editing is required. For example, it's mentioned 'thesis' instead of 'paper' in line 252
Response: Thanks for the correction. The issue is fixed.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
This paper outlines innovative context based learning approach to the field of software security which is very interesting as from the scientific as well as from the practical point of view. It offers novel way of learning, which can be generalized into the other technological fields. Nevertheless, I have following remarks on the paper:
- Authors in abstract on few places mention "software insecurity" term. I think that this is not very appropriate notion, I suggest to them to use just "software security". Also, it would be nice if they use this terminology everywhere in the rest of the paper in order to have consistency in storytelling.
- At the beginning of Introdution part, authors refers to the StackOverflow survey without proper reference. When this survey was carried out? Who participated in it? Put a proper reference.
- Similar situation like previously mentioned, is with Veracode and DevOps.com research authors refer to.
- Sentence in Introduction which starts with "Without a base level of experienced security knowledge..." is a little bit unclear. Please rephrase it.
- In 2nd paragraph of Introduction, authors argue the ways where developers can learn about software security. Apart from books, open literature, internet, another interesting direction is scientific databases, from which I think developers can find many interesting approaches and techniques in ensuring the security of the software they are developing. This must be included in this part of the paper. Some references which should find their place here is:
Kuk, K., Petar, M., Spalevic, M., & Gocic, M. (2019). Algorithm design in Python for cybersecurity. Electrotechnical and Computer Science Conference, ERK, Slovenia.
MELESE, A. (2022). AUTOMATIC SOURCE CODE VULNERABILITY DETECTION, CLASSIFICATION AND PRIORITIZATION USING DEEP LEARNING ALGORITHM (Doctoral dissertation).
- On line 65 remove the page number from the referenced research. It is unnecessary.
- From lines 71 to 76 authors argue the aim of their research with their own thinking on software security issues withouth clear connection with the previous text, which can affect the reader in not getting the purpose of the paper.
- RQ2 is not formulated well. Better option will be "How ontologies can manage contextualized software security knowledge?"
- RQ3 is also not formulated well. Maybe better option is "How contextualized learning system for software security can be constructed and to what extent does the system affect the learning outcome?"
- In line 105 authors mention RQ4 which does not exist in paper. Maybe a typo? Or author maybe refer to the RQ1?
- Paragraph from lines 110 - 116 is unnecessary. Remove it.
- Section 2.1 misses proper discussion about software metrics which also during software development can reveal the possible holes in software which can be exploited by attackers. No attention in this section is paid to learning about software security during software development process. This must be corrected. Some reference which can help in this direction are:
Kuk, K., Milić, P., & Denić, S. (2020). Object-oriented software metrics in software code vulnerability analysis. In 2020 International Conference on INnovations in Intelligent SysTems and Applications (INISTA) (pp. 1-6). IEEE.
Garg, R., & Singh, R. K. (2022). SBFSelector: Analysis of Metrics to Improve Traceability in Collaborative Environments. International Journal of Open Source Software and Processes (IJOSSP), 13(1), 1-19.
S. Jain and M. Ingle, “A Review of Security Metrics in Software Development Process”, International Journal of Computer Science and Information Technology, vol. 2, no. 6, 2011, pp. 2627-2631.
- Section 2.2 uses a mix of citation styles (APA and IEEE). Please correct this, and use the journal recommendations for proper referencing. Check the whole paper as well.
- Sentence on lines 194 - 195 is unclear and not understandable. Please reformulate it.
- Sentence on line 252 is unclear. Reformulate it. Maybe instead of "thesis" you mean "paper"?
- Presentation and description of design of solution proposed in paper, in Section 3 is not performed very well. Try to formulate the description of the design more precisely in order to have smooth storytelling.
- In Section 4 where authors mention artifacts as a novel approach, they do not explain what they refer to. Just simple relation with software security, without proper explanation. This must be corrected.
- In Section 4.1.2 and later on in the text as well, authors use term "security knowledge" which is inappropriate. What does mean "security knowledge"? Is that about security of knowledge or what? Usage of this term must be rephrased so as not to confuse the reader.
- On Figure 4 for Design Cycle 3 there is a type in box for "A web-based learning system...". It should be "Alpha version", not "Alptha version"
- Figure 6 description is badly formatted.
- Try to decrease the font size for Table 2 in order to achieve better visibility of heading cells.
- Line 420 has a language typo.
- In Section 4.2.3 again mix of citation styles. Please refer to the journal instructions to the authors for correction of this.
- When authors depict system architecture on Figure 9, they indicate that via browser user interacts with their system in the way that browser issues request to the Servlet, and receives response via JSP, AJAX. This is wrongly formulated. Either JavaScript or jQuery can communicate with Servlet, JSP or AJAX, so here should be placed arrow like one between Servlet and Apache Jena.
- In Section 4.3.2 authors claim that they used ontology for creating knowledge base for their solution. I don't see anywhere this ontology. What constitutes it? What are main building blocks? This should be added in paper.
- Line 540 has a language typo.
- Thorough out section 4.3 authors describes their solution in comparison with native PHP programming, without considering utilization of software security concepts via popular frameworks such as Laravel and Symfony for PHP or Angular, React, nodeJS for Javascript. It would be interesting to see how these fits with the proposed platform.
- In answer to the RQ3 in Section 5 (Discussion), authors state that they work on contextualized learning system for improved software security training. Maybe this claim is not formulated properly, because authors develop context-based system for learning of security concepts.
- Section 7.1 which outlines further research is very poor. Please try to extend it.
- Keeping in mind all previously said as well as reading the paper, I feel that a title of the paper is not appropriate for the paper in current form. I suggest authors to consider renaming it, for example "Towards Context-Based Learning About Software Security"
Author Response
Comment 1: Authors in abstract on few places mention "software insecurity" term. I think that this is not very appropriate notion, I suggest to them to use just "software security". Also, it would be nice if they use this terminology everywhere in the rest of the paper in order to have consistency in storytelling.
Response: We have thoroughly reviewed and updated the terminology used throughout the document, replacing the term "security insecurity" with the more appropriate designation of "software security.
Comment 2: At the beginning of Introduction part, authors refer to the StackOverflow survey without proper reference. When this survey was carried out? Who participated in it? Put a proper reference.
Response: A citation (Stake Overflow Survey 2021) is included in the corresponding reference (Line 23-27).
Comment 3: Similar situation like previously mentioned, is with Veracode and DevOps.com research authors refer to.
Response: A citation (The DevSecOps Global Skills Survey) is included (Line 33).
Comment 4: Sentence in Introduction which starts with "Without a base level of experienced security knowledge..." is a little bit unclear. Please rephrase it.
Response: The sentence is rephased into “Without fundamental security knowledge,”
Comment 5: In 2nd paragraph of Introduction, authors argue the ways where developers can learn about software security. Apart from books, open literature, internet, another interesting direction is scientific databases, from which I think developers can find many interesting approaches and techniques in ensuring the security of the software they are developing. This must be included in this part of the paper.
Response: Thanks for the comment. We rephased the sentence about the learning sources, where the scientific databases were added.
Comment 6: On line 65 remove the page number from the referenced research. It is unnecessary.
Response: The page number was removed.
Comment 7: From lines 71 to 76 authors argue the aim of their research with their own thinking on software security issues without clear connection with the previous text, which can affect the reader in not getting the purpose of the paper.
Response: We concur with the reviewer's comment that the statement is redundant, a factor that could potentially lead to an uneasy feeling among the audience. In this regard, the corresponding sentences were rewritten (Line 75-80).
Comment 8: RQ2 is not formulated well. Better option will be "How ontologies can manage contextualized software security knowledge?"
Response: We acknowledge the reviewer’s comment in describing RQ2. It is not formulated as following: “What approach can be taken to develop an ontology that effectively manages contextualized knowledge related to software security?” RQ2 in the Discussion section is revised as well.
Comment 9: RQ3 is also not formulated well. Maybe better option is "How contextualized learning system for software security can be constructed and to what extent does the system affect the learning outcome?"
Response: RQ3 is rewritten as following: “What strategies can be employed to build a learning system for software security that incorporates contextualized knowledge content, and what impact does this system have on the overall learning outcomes?” RQ3 in the Discussion section is revised as well.
Comment 10: In line 105 authors mention RQ4 which does not exist in paper. Maybe a typo? Or author maybe refer to the RQ1?
Response: Thanks for the reviewer’s correction. It is mistake. It should be RQ3, instead of RQ4.
Comment 11: Paragraph from lines 110 - 116 is unnecessary. Remove it.
Response: The corresponding sentences are removed.
Comment 12: Section 2.1 misses proper discussion about software metrics which also during software development can reveal the possible holes in software which can be exploited by attackers. No attention in this section is paid to learning about software security during software development process. This must be corrected. Some reference which can help in this direction are: (The reference list is omitted by the author.)
Response: As acknowledged in the review, security metrics play a crucial role in monitoring and enhancing security throughout the software development lifecycle. However, our work focuses on how to effectively deliver and present security knowledge to learners. Nevertheless, the reviewer did make a valid point regarding the implications of our approach. Through our framework, we could equip developers with the necessary understanding of security metrics, including interpreting security testing results, which can be applied in later phases of development. By leveraging this approach, we believe that developers will be better equipped to build secure software systems.
Comment 13: Section 2.2 uses a mix of citation styles (APA and IEEE). Please correct this, and use the journal recommendations for proper referencing. Check the whole paper as well.
Response: The citation styling issues are fixed throughout the paper. Thanks for reviewer’s correction.
Comment 14: Sentence on lines 194 - 195 is unclear and not understandable. Please reformulate it.
Response: The corresponding sentences are rephased.
Comment 15: Sentence on line 252 is unclear. Reformulate it. Maybe instead of "thesis" you mean "paper"?
Response: The corresponding sentences are rephased.
Comment 16: Presentation and description of design of solution proposed in paper, in Section 3 is not performed very well. Try to formulate the description of the design more precisely in order to have smooth storytelling.
Response: The whole Section 3 was rewritten, including reformulating Figure 2.
Comment 17: In Section 4 where authors mention artifacts as a novel approach, they do not explain what they refer to. Just simple relation with software security, without proper explanation. This must be corrected.
Response: The objective of design cycle 1 in Section 4.1 (for designing context-based learning approach) was modified, in which the design artifact is explain briefly.
Comment 18: In Section 4.1.2 and later on in the text as well, authors use term "security knowledge" which is inappropriate. What does mean "security knowledge"? Is that about security of knowledge or what? Usage of this term must be rephrased so as not to confuse the reader.
Response: A footnote is included for "Security Knowledge" when it is first mentioned in the Introduction section.
Comment 19: On Figure 4 for Design Cycle 3 there is a type in box for "A web-based learning system...". It should be "Alpha version", not "Alptha version"
Response: This typo issue is fixed.
Comment 20: Figure 6 description is badly formatted.
Response: A clear version of figure is updated.
Comment 21: Try to decrease the font size for Table 2 in order to achieve better visibility of heading cells.
Response: Table 2 is re-formatted.
Comment 22: Line 420 has a language typo.
Response: The corresponding sentences are rephased.
Comment 23: In Section 4.2.3 again mix of citation styles. Please refer to the journal instructions to the authors for correction of this.
Response: The citation formatting issue is resolved.
Comment 24: When authors depict system architecture on Figure 9, they indicate that via browser user interacts with their system in the way that browser issues request to the Servlet, and receives response via JSP, AJAX. This is wrongly formulated. Either JavaScript or jQuery can communicate with Servlet, JSP or AJAX, so here should be placed arrow like one between Servlet and Apache Jena.
Response: Figure 9 is reformulated.
Comment 25: In Section 4.3.2 authors claim that they used ontology for creating knowledge base for their solution. I don't see anywhere this ontology. What constitutes it? What are main building blocks? This should be added in paper.
Response: The ontology is constructed in Design Cycle 2, described in Section 4.2.
Comment 26: Line 540 has a language typo.
Response: The corresponding sentences are rephased (Line 542-544).
Comment: Thorough out section 4.3 authors describes their solution in comparison with native PHP programming, without considering utilization of software security concepts via popular frameworks such as Laravel and Symfony for PHP or Angular, React, nodeJS for Javascript. It would be interesting to see how these fits with the proposed platform.
Response: Thanks for the reviewer’s interest in the system. We have modeled more language-specific knowledge in the system, which is piloting internally in bachelor courses. We plan to make the whole system as open-source and share out the ontological knowledge base.
Comment 27: In answer to the RQ3 in Section 5 (Discussion), authors state that they work on contextualized learning system for improved software security training. Maybe this claim is not formulated properly, because authors develop context-based system for learning of security concepts.
Response: Following the research objective, the statement is rephased as “This research identified the need for a contextualized learning system that could enhance software security learning.”
Comment 28: Section 7.1 which outlines further research is very poor. Please try to extend it.
Response: Both Section 7.1.1 and Section 7.1.2 are enhanced.
Comment 29: Keeping in mind all previously said as well as reading the paper, I feel that a title of the paper is not appropriate for the paper in current form. I suggest authors to consider renaming it, for example "Towards Context-Based Learning About Software Security"
Response: Thanks for the review’s suggestion. Given that a similar title has been used in other related publications, we made a slight change on the title “Context-Based Support to Enhance Developers’ Learning of Software Security”
Author Response File: Author Response.pdf
Reviewer 3 Report
I suggest adding to the title that this is a preliminary study. A small research sample and the presented statistical evaluation do not guarantee the results obtained (values ​​of the standard deviation and the other statistical measures). Therefore, the certainty of the formulated generalizations is low. It is worth highlighting this point in the article.
There are no research hypotheses in the article.
Author Response
Comment 1: I suggest adding to the title that this is a preliminary study. A small research sample and the presented statistical evaluation do not guarantee the results obtained (values ​​of the standard deviation and the other statistical measures). Therefore, the certainty of the formulated generalizations is low. It is worth highlighting this point in the article.
Response: Thanks for reviewer’s comment regarding the limitation of the research work. We’ve highlighted the issue of generalizability in Section 6, in which it states “This experimental evaluation took place at a university with student volunteers enrolled in the Software Security course. This particular sample is not necessarily representative of the population as a whole, and the limited sample size (36) also hindered its generalizability.”
Comment 2: There are no research hypotheses in the article.
Response: We added the following hypothesis statement in Introduction (Line 78) “We hypothesize that through the integration of context-based learning methods, the application of this learning tool is anticipated to enhance software developers' learning experience in security.”
Author Response File: Author Response.pdf
