ProtectingSmall and Medium Enterprises: A Specialized Cybersecurity Risk Assessment Framework and Tool

Reviewer 3

The article addresses the creation of a cybersecurity risk assessment framework for SMEs.

Common threats and vulnerabilities were identified. The case was made that traditional

frameworks catering to large companies are too complex and don’t scale well for smaller

enterprises like SMEs. The advantages and disadvantages of the popular frameworks, notably

NIST CSF and ISO 27001/2, were identified. A new framework for SMEs was subsequently

developed, and snippets of the framework (e.g. Figure 1 and Figure 2) were shown and briefly

discussed. The framework was implemented as a running system. The framework's user

experience (UX – usability) was tested among stakeholders, and its corresponding advantages

and disadvantages were noted. Participants found the technical language and lack of depth

challenging.

I have the following feedback on the manuscript.

Section 2.2: The research questions are wordy and many for a manuscript this length.

Also, make sure you return to each of these at the appropriate place and consider to

what extent you have answered each one.

We appreciate the reviewer’s feedback regarding the number of research questions. To streamline the manuscript and focus more concisely, we have consolidated the research questions into three key questions that address the main aspects of the study. This revision helps in maintaining clarity and relevance throughout the manuscript. Thank you for your constructive suggestion. We appreciate the reviewer's suggestion to ensure that each research question is addressed appropriately. In response, we have added a new subsection titled "Addressing Research Questions" that explicitly connects each research question to the relevant sections of the manuscript. This subsection reviews how our findings align with and answer each research question, providing clarity on the contributions of our research. Thank you for your constructive feedback, which has helped improve the clarity and completeness of our work.

Section 3.1.1 concerns employee attitudes, but Table 1 indicates different categories of

challenges. Consequently, the contents of Table 1 should be discussed and linked to

employee attitudes, which is what that section is about.

We appreciate your insightful comment regarding the alignment between Section 3.1.1 and the contents of Table 1. To address this, we have introduced a new Section 3.1.2 that specifically discusses the categories of challenges presented in Table 1 and their connection to employee attitudes. This addition ensures a clearer linkage between the table and the section's focus. Thank you for your valuable feedback.

Section 3.2. Review of Popular Frameworks should be fleshed out. Amongst others,

these frameworks should be shown (diagrams, tables, etc.) so the reader can see what

they look like. Then, the authors of this article should conduct their own, first-hand

analyses of these structures instead of simply relying on what others have said about

the frameworks. In the same way, Section 3.3 should be fleshed out, and the new

information in Section 3.2 should be used to provide a more thorough justification for

answering RQ1 and RQ2. To some extent, these comments apply to Section 4.1 as well.

We thank the reviewer for their valuable feedback. The frameworks have been described in more detail in both the text and the table. This allows for a more broader visualization of the similarities and differences between each framework. Their unique focuses and key components can also be observed. 

Picking up on the previous comment (point [3]), how do the frameworks in Section 4.2

link with those mentioned in Section 3.2, Section 3.3, and Section 4.1? The links among

the discussions of these frameworks should be added. This would also strengthen the

justification for RQ3 being answered in Section 4.2.

We thank the reviewer for their feedback. The frameworks in section 3.2 exist to provide background information about popular frameworks that are used in the market. Brief descriptions of them are provided for context and to show how they do not completely cater to SMEs.   Section 3.3 provides an overview of the key findings of the section,  in terms of SME vulnerabilities and popular frameworks. Finally, section 4.1 has been updated to provide a clear link between the three sections. 

Section 5, Figure 1: Who developed this figure? If not the authors, give a reference.

Figure 1 illustrates how to use the framework and tools effectively, as well as provides basic cybersecurity tips and training for employees. This is relevant as it supports the practical implementation and understanding of the framework discussed in the paper. Thank you for your question. and it is developed by us showing our framework 

Line 212: “Secondly”. Where is the first part (“Firstly”)? Also, the discussions in these

two paragraphs mention aspects I don't see in the Figure 1 framework, e.g., GDPR, legal

aspects, MCA, etc. Make sure the discussions map onto what is in Figure 1. Or are you already referring to Figure 2? These uncertainties should be clarified/addressed. See

also the use of MCA in line 255.

Thank you for pointing out the confusion in the text. We acknowledge that the reference should be to Figure 2 and not Figure 1 in the discussion regarding GDPR, legal aspects, and MCA. The necessary correction has been made, and the flow of the argument has been clarified. Furthermore, we have removed "Secondly" to ensure consistency and proper structuring.242

Line 241: You mention the CIA in the Figure 2 framework, but where are the specific

components of CIA in Figure 2?

Thank you for pointing out the issue related to the Cia concept and how it is included within figure 2, plese note that we hve added 2 sentences mentioning clearly where is CIA integrated inside figure 2 please refer to page 10 of the updated manuscript.

Section 5.4. Threat Classification: The naming of this grouping is rather high level.

Instead, you should more explicitly refer to the naming of the components in the

framework, the same way you discussed the Business level and its two subdivisions,

i.e., much more discussion of the Figure 2 framework by explicitly referring to the

framework's components should be conducted.

Thank you for your feedback. We have updated the terminology in Section 5.4 to "Threat-based Risk Assessment" to better reflect the terminology used in Figure 2. This change ensures consistency in the naming conventions and enhances the clarity of our discussion on the framework's components. We have also provided a more detailed discussion of Figure 2 and its components to explicitly align with the updated terminology.

Your numbered list 1 – 11, pages 9 – 10: Did you synthesise these entries yourself, or is

this a standard list in the extant literature? This should be explained. It seems to me that

several references should be added to justify the selection of the individual items in this

list.

We thank the reviewer for their valuable feedback. A mixture of both techniques was used for the URL classification feature selection. Clarification has been added in the corresponding section. 

Section 7.2: I like the idea of using a SWOT analysis to evaluate the utility of an artefact.

Thank you for your positive feedback on the use of the SWOT analysis in Section 7.2. We appreciate your recognition of this approach and believe it provides valuable insights into the utility of the artefact. Your input is greatly valued.

Line 391 – 392: “Upon consultation with an individual with experience in the

cybersecurity field, the following evaluation of the framework was made.” Question:

Did you formally survey this individual? If so, did you have ethical clearance from your

home institution to do so?

Regarding the selection of the external expert, your observation is noted. The phrase 'Upon consultation with an individual with experience in the cybersecurity field' has been revised to ensure greater scientific rigor in the selection process. Additionally, updated data has been included in Section 7.3 (page 16) to enhance the accuracy and reliability of the analysis.

Section 7.4. Usability Testing: Which framework did you show the participants to

“read”? Was it Figure 2? If so, how would they be able to evaluate its usability (UX)

sensibly? Presumably, they also ran the software? So, what is it they were “reading”?

This must be thoroughly explained.

We thank the reviewer for their feedback. The participants were reading the framework shown in Figure 1, the framework itself. They tried the software with different links to see how it was working. This has been made more clear within the section.

Section 7.5. Proposed Pilot Assessment: As I understand, this is, in fact, future work, so

I suggest you move this section into your Future Work section?

We appreciate your feedback and have integrated the suggestions into the revised Future Work section. We acknowledge the need for more quantitative feedback to measure the effectiveness of the framework and tool. To address this, we have included a proposal for a pilot assessment that will be conducted over 6-8 months. This pilot assessment will involve SMEs from various industries, ensuring a diverse dataset that can provide valuable quantitative data on the framework’s performance.

Additionally, we have recognized the importance of obtaining qualitative feedback from SME owners and leaders. Such feedback will help us understand the practical challenges and user experiences associated with the tool, offering deeper insights that quantitative measures alone might not capture. We plan to conduct detailed interviews and surveys with SME stakeholders to gather this valuable information.

These additions aim to enhance the robustness of the framework by incorporating both quantitative and qualitative evaluations, thus improving its overall effectiveness and usability.

Lines 513 – 515: “How[-]ever, with the large number of frameworks available that may

possess better techniques for certain threats, emphasizing a need for periodic review.

”

This seems to be not a full sentence. Verb missing.

Thank you for your valuable feedback. We have revised the sentence to include the missing verb, ensuring clarity in our discussion of the need for periodic review of cybersecurity frameworks. The updated sentence now reads: 

"However, with the large number of frameworks available that may possess better techniques for certain threats, there is an emphasis on the need for periodic review."

We appreciate your attention to this detail and believe this revision enhances the readability and accuracy of our manuscript. 

NB General: The case is made that current frameworks are too complex for smaller

companies like SMEs to use, and complexity and financial constraints are cited as

reasons. However, I would like to see a better motivation for this. I suggest you look at

the characteristics of SMEs, and then map these SME needs onto where the larger

frameworks fall short. This ties up with point [3] above.

Thank you for your comment, we explained what do we mean by complex to be implemented by SMEs refer to page 2 in intoruction section.

Lines 46-47: “A survey conducted 46 by Heikkila et al. (2016)”. Add the square bracket

notation to cite this one. I noticed several other occurrences of this. Please check each

APA/Harvard-like citation in the manuscript

Thank you for pointing out the issue with the citation format. We have corrected the citation for "Heikkila et al. (2016)" and ensured that the square bracket notation is applied where necessary. We have also reviewed the entire manuscript and adjusted all APA/Harvard-like citations to meet the required format. 

We appreciate your attention to detail and believe these corrections improve the overall quality of the manuscript.

Line 101: Replace “useless” with a more academic word.

Thank you for your valuable feedback. We have revised the term "useless" to "compromised" in Line 101 to better reflect the academic tone of the manuscript. We appreciate your suggestion and believe this change enhances the clarity and precision of the text.

We are grateful for your attention to detail.

Line 110: Replace “massive” with (e.g.) major.

Thank you for your insightful comments. I have addressed the points you raised:

Line 101: The term "useless" has been replaced with a more academic word.

Line 110: "Massive" has been replaced with "major."

Line 355: Corrected to "an ML model."

Line 368: Replaced the term "report" with "article."

Line 564: Clarified that some terms listed are acronyms, not just abbreviations.

I appreciate your attention to detail and valuable feedback.

Line 355: “a[n] ML”

Line 368: You are writing an article, not a “report”.

Line 564: Some terms listed are not abbreviations but acronyms, e.g. ML – Machine

Learning.