Open BOK on Software Engineering Educational Context: A Systematic Literature Review
, Jon A. Elorriaga 1, Ana Arruarte 1 and Hironori Washizaki 2Round 1
Reviewer 1 Report
The paper is very well written, it is a survey paper where they dont propose anything new, but they throughly review and summarizes the systematic literature review on open body of knowledge. The authors need to proofread the papers and fix some existing grammar mistakes and spellings.
Author Response
Response to Reviewer 1 Comments
Point 1: The paper is very well written, it is a survey paper where they don’t propose anything new, but they thoroughly review and summarizes the systematic literature review on an Open Body of Knowledge.
Response 1: Dear reviewer thanks for your comments.
The purpose of the review is to be the theoretical basis and guides to create an Open BOK knowledge description model applied in the educational context.
Point 2: The authors need to proofread the papers and fix some existing grammar mistakes and spellings.
Response 2: The existing grammar mistakes and spellings were amended.
Reviewer 2 Report
The work deals with the construction of models for the description of knowledge of areas or fields of knowledge. Specifically, the perspective from which the author approaches it is the construction of open bodies of knowledge (BOK). It is a relevant topic, to which the author contributes through a systematic review of the literature that contributes to providing solid theoretical bases.
The method followed for the review is explained in detail.
The selection of databases would require further justification. On the other hand, it would be interesting to incorporate the search question (or questions) used, since this greatly affects the results obtained.
The inclusion and exclusion criteria require a more operational definition. In some cases the criteria included are quite ambiguous ("Documents related to BOK but not related to the research questions of this study", "Poor arguments. That is, studies with low relevance according to the research were excluded", for example). These criteria should be refined and stated in a more unambiguous way.
The results are very well presented and the conclusions account for the findings of the systematic review of the literature, although the possible limitations and inconveniences that arose in the review are not included.
Carefully review the use of acronyms.
Author Response
Response to Reviewer 2 Comments
Point 1: The work deals with the construction of models for the description of knowledge of areas or fields of knowledge. Specifically, the perspective from which the author approaches it is the construction of open bodies of knowledge (BOK). It is a relevant topic, to which the author contributes through a systematic review of the literature that contributes to providing solid theoretical bases.
Response 1: Dear reviewer, thank you for your comment. The review is correctly structured focusing on the construction of models for the description of knowledge of areas or fields of knowledge. It is a relevant topic since it contributes through a systematic review of the literature that provides solid theoretical bases, as you have very kindly stated.
Point 2: The method followed for the review is explained in detail.
Response 2: Dear reviewer, thank you for your comment.
Point 3: The selection of databases would require further justification.
Response 3: For all scientific areas there is a specific database or at least some multidisciplinary one; therefore, the databases used in the review in relevant information, updating, accurate, proven, and quality in the area of Open BOK on Educational context.
Point 4: On the other hand, it would be interesting to incorporate the search question (or questions) used, since this greatly affects the results obtained.
Response 4: The Research Questions (RQs) were defined:
RQ1: What are the necessary elements needed to describe knowledge in Open BOK on educational context?
RQ2: What is the structure of BOKs to develop a guide of Knowledge?
Point 5: The inclusion and exclusion criteria require a more operational definition. In some cases, the criteria included are quite ambiguous ("Documents related to BOK but not related to the research questions of this study", "Poor arguments. That is, studies with low relevance according to the research were excluded", for example). These criteria should be refined and stated in a more unambiguous way.
Response 5:
The inclusion and exclusion criteria were refined, and the 11 criteria used to evaluate each primary study were added.
Reviewer 3 Report
According to the title and abstract, I, as a researcher in SE education, expect the paper providing a rigor/in-depth investigation and discussion about BOK issues in the domain of SE education, which is indeed scarce in current literature-review studies. However, the statements across the entire paper are all too generic, from the domain problems, methodologies, results, to the conclusions, which are not appropriately described in the context of SE education, nor in SE domain. Searching the term “software engineering” in this paper, it’s found that a signification portion of searching results falls in the Reference section, only few appears in the main body of the paper, for example, it’s only mentioned once in Introduction section, and ‘none’ in the Conclusion! All these make the paper content quite incompatible with the subject. Since the authors have read through the found papers in the domain (being a SLR study), the ‘context’ of research field should be well-addressed, and some fundamental information regarding BOK in SE can be provided at first, i.e., what is the importance of BOK to SE educators or students? what are the specific issues (or weakness) of Knowledge Description in SE (education) in terms of Open BOK? Is there any SE (education) literature or evidence mentioning the weakness?
In terms of methodology, two research questions are given to guide the SLR process, however, without clear discussion of answers in the later sections. The filtering criteria is rigor-less. No keywords or filtering steps mention ‘education’, which leads to the difficulty in measuring the precision of the SLR results. Some raised issues here: What is the number of research papers in stages of the filtering process in terms of inclusion and exclusion criteria, and what are the selected papers entering the final analysis? One major contribution of SLR is to deliver a clear sorting-out literature list according to the defined RQ. In this regard, the filtering process should be scientifically addressed. For example, in line 170, the criteria ‘Poor argument’ is too vague, which could be subjective while making judgment. In addition, three figures (3-5) are provided to support the SLR work, however, without detailed explanation of the purpose and the position regarding SLR.
The academic writing skill should be improved and strengthened with a great effort. The content of each section lacks well-structuralizing. There are too many paragraphs, organized with few sentences, scattered in a single section, Section 1 and 3, for instance. It’s very difficult for readers to reason and synthesize the arguments that the author provides. Furthermore, when siting research papers, instead of giving pure reference numbers, please do not hesitate to mention the authors’ name, for the respect. For example, Professor Kitchenham, should definitely be highlighted while addressing SLR in software engineering, vice versa in knowledge engineering. To mitigate such kinds of writing mistakes, proof-reading is mandatory before submission.
Author Response
Response to Reviewer 3 Comments
Dear reviewer thanks for your comments.
Point 1: According to the title and abstract, I, as a researcher in SE education, expect the paper providing a rigor/in-depth investigation and discussion about BOK issues in the domain of SE education, which is indeed scarce in current literature-review studies. However, the statements across the entire paper are all too generic, from the domain problems, methodologies, results, to the conclusions, which are not appropriately described in the context of SE education, nor in SE domain.
Response 1: Dear Reviewer thank you for your valuable feedback.
Regarding the first point, we would like to say that we have restructured the subject as Open BOK on Educational context: A Systematic Literature Review, where it is addressed in a general approach in the educational context put the bodies of knowledge are adaptable in the different areas of knowledge.
Through the revision was established the theoretical bases as well as the respective guides to describe an Open BOK.
The statements within the document have been improved and given greater scientific support as you suggest.
We have improved the conclusions according to your recommendation.
Point 2: Searching the term “software engineering” in this paper, it’s found that a signification portion of searching results falls in the Reference section, only a few appear in the main body of the paper, for example, it’s only mentioned once in the Introduction section, and ‘none’ in the Conclusion! All these make the paper content quite incompatible with the subject. Since the authors have read through the found papers in the domain (being an SLR study), the ‘context’ of research field should be well-addressed, and some fundamental information regarding BOK in SE can be provided at first, i.e., what is the importance of BOK to SE educators or students? what are the specific issues (or weakness) of Knowledge Description in SE (education) in terms of Open BOK? Is there any SE (education) literature or evidence mentioning the weakness?
Response 2: According to the change indicated in point 1 of the description of an Open BOK no longer focuses specifically on software engineering but in the general educational context; since the idea of the SLR is that it is a methodology replicable to other areas of knowledge in the educational field.
Regarding the importance of Open BOK was established the following:
Open BOK presented in the review is important because defined a set of knowledge, skills, concepts, and behavior that stakeholders and the related disciplines need for the correct consensus. Also, this knowledge description for BOK allowed us to have a validated classification of the boundaries of the disciplines that will support Open BOK.
The Open BOKs described in this thesis showed the hierarchical structure of
the content of each knowledge area.
According to Bourque la weakness in Open BOK lies in the consensus of the areas of knowledge and disciplines realized since each element needs deep analysis and adjustments to each curriculum of different countries.
Developing a general curriculum through Open BOK would allow the harmonization of the containers has been taught in the different areas of knowledge.
The above and supported with scientific references are included in the new version of the article.
Point 3: In terms of methodology, two research questions are given to guide the SLR process, however, without a clear discussion of answers in the later sections. The filtering criteria are rigor-less. No keywords or filtering steps mention ‘education’, which leads to the difficulty in measuring the precision of the SLR results.
Response 3:
The 2 research questions have been focused on the educational context; of which the base reference theoretical framework has been determined to generate the description of the Open BOK, in the results section as in Tables 4 and5 the elements necessary to describe BOK as the structure that would allow the generation of the models for the curricular circuit is addressed.
Added search criteria on line 174: ((“OBOK” OR “Open Body of Knowledge)”) AND ((“SE” OR “Software Engineering”) AND (“Area Breakdown” OR “software engineering body of knowledge” OR “the importance of OPENBOK.” OR “Component”, “OR” Design) ”) AND (“Education”)).
Point 4: What is the number of research papers in stages of the filtering process in terms of inclusion and exclusion criteria, and what are the selected papers entering the final analysis? One major contribution of SLR is to deliver a clear sorting-out literature list according to the defined RQ. In this regard, the filtering process should be scientifically addressed. For example, in line 170, the criteria ‘Poor argument’ is too vague, which could be subjective while making a judgment. In addition, three figures (3-5) are provided to support the SLR work, however, without a detailed explanation of the purpose and the position regarding SLR.
Response 4: In table 1 the number of research documents in stages of the filtering process is shown.
In table 3, the rubric to evaluate each article is shown. The snowball technique was used to perfect the filter respectively.
In the supplementary add-ons is located the documents analyzed and with the use of atlas TI 8.0 software was able to perform an analysis of each of the documents so inspired by the criteria of inclusion and exclusion of the methodology developed.
On line 188 is extended the explanatory of that exclusion criterion.
Figures 3,4,5 try to give an idea of the methodological process of SLR, a description of these processes is added.
Point 5: Academic writing skills should be improved and strengthened with great effort.
Response 5: In the new version the academic aspect and formats were considered in order to improve the review.
Point 6: The content of each section lacks well-structuralizing. There are too many paragraphs, organized with few sentences, scattered in a single section, Section 1 and 3, for instance. It’s very difficult for readers to reason and synthesize the arguments that the author provides. Furthermore, when siting research papers, instead of giving pure reference numbers, please do not hesitate to mention the authors’ name, for respect. For example, Professor Kitchenham should definitely be highlighted while addressing SLR in software engineering, vice versa in knowledge engineering. To mitigate such kinds of writing mistakes, proof-reading is mandatory before submission.
Response 6: The sections are restructured and connected to each other in order for the reader to have a clear view of the revision proposal.
The authors within the methodological aspects have been included within the text as suggested.
Dear reviewer, we are grateful for the suggestions submitted which have made our proposal significantly improved.
Round 2
Reviewer 3 Report
This paper presents a structured SLR study to investigate how knowledge can be organized properly in Open BOK. The author provides detailed descriptions with supplementary materials regarding how SLR was carried out. This paper should be interesting for the community.
Several suggestions to improve the quality of the paper are:
- The importance of BOK in "education" context should be well-addressed in Introduction.
- May provide a brief about the software tools used (Mendeley, Atlas TI), for example, in footnote, so that readers do not need to dig into the reference to understand their usage.
- The academic writing skill should be improved and strengthened. There are too many paragraphs, organized with few sentences, scattered in a single section. The information is fragmented into small pieces. Please do proof-editing (or academic editing) before final submission.
Author Response
Response to Reviewer 3 Comments
This paper presents a structured SLR study to investigate how knowledge can be organized properly in Open BOK. The author provides detailed descriptions with supplementary materials regarding how SLR was carried out. This paper should be interesting for the community.
Dear Reviewer, thank you for your valuable feedback.
Point 1: The importance of BOK in the "education" context should be well-addressed in the Introduction.
Response 1: In line 48 the importance of BOK in education was introduced.
The open description of Bodies of Knowledge in the educational context provides the basis for curriculum development, professional development, and current and future certification schemas. Open BOK promotes integration and connections with related disciplines in the educational context. Educational professional communities have created and used Open BOKs to consolidate their discipline, standardize practices, improve processes, and warehouse community knowledge.
Open BOK has been used across different types of disciplines. The Open BOK could also be used by individuals for extending their skills and for career development. Researchers may find it useful for identifying technology applicable to their research and to help define the skills required for a research team.
Point 2: May provide a brief about the software tools used (Mendeley, Atlas TI), for example, in a footnote, so that readers do not need to dig into the reference to understand their usage.
Response 2: In line 232 provides a brief review of the software tools used.
With the aim of addressing the research questions posed in the review, electronic sheets allowed us to organize the relevant information of each one of the primary studies and evaluate their contribution.
Mendeley allowed us to organize the bibliographic references that are the scientific basis of this review.
In this review, Atlas TI allowed associating codes or labels with fragments of text [83] from the primary studies and creating the concept networks that will be the basis of the description model. Furthermore, Atlas TI allowed us to search for pattern codes, classify them to establish the elements to describe an Open BOK in the educational context.
Point 3: The academic writing skill should be improved and strengthened. There are too many paragraphs, organized with few sentences, scattered in a single section. The information is fragmented into small pieces. Please do proof-editing (or academic editing) before the final submission.
Response 3: The writing was improved. The sections were restructured and connected to each other in order for the reader to have a clear view of the review proposal.
Dear reviewer, we are grateful for the suggestions submitted which have made our proposal significantly improved.
