Evaluation of Spatial Thinking Ability Based on Exposure to Geographical Information Systems (GIS) Concepts in the Context of Higher Education

Round 1

Reviewer 1 Report

First off, I'd like to thank the authors for a well-conducted study. The methods are sound, and I appreciate the use of non-parametric methods as necessitated by the violated assumptions of other statistical tests. One of the main questions in this piece -- whether or not spatial thinking as measured by the STAT was significant in two CUs -- was significant. Several of the more minor relationships (on the role of gender, for instance) were not significant, yet I appreciate the authors reporting these results anyway. Negative results are often devalued, but I believe they can, at times, be just as informative as significant results. I think this is study is one such case. It seems obvious to me that the authors did not "fish" for significance by modifying their research question or tacking on questions tangential to the study at hand. This is commendable aside from the important results of the study.

My comments below are more on the writing, structure, and presentation of the material rather than methodological suggestions. I believe improving these items will make the paper a bit easier to read and increase the chances that the paper is cited.

• I'd suggest modifying the first sentence of the abstract and introduction. As of now, it seems to suggest to the reader that the paper will focus on the connection between spatial thinking and day-to-day life. I'd rephrase this sentence a bit to better characterize what the paper will be about.
• Some paragraphs are relatively short (2-3 sentences). Either expand these into full paragraphs or lump them in with others.
• The use of "for instance" becomes a bit repetitive, especially in lines 45-53 (p. 2). Use others, e.g., "for example", "like", and "such as".
• I think the definition of geospatial thinking (lines 54-55, p. 2) ought to be modified slightly. There should be more emphasis on how it is spatial thinking which is tied to the earth's surface. The use of the term "geographical knowledge" makes the definition ambiguous.
• Make "geographic information system" plural in line 71 (p. 2) as this is referring to a broad class of software rather than a specific program.
• Similarly, make global positioning system (line 88, p. 2) plural.
• Is "Measure of spatial thinking abilities" a section heading? (line 97, p. 2)
• Lump lines 102-105 (p. 3) into the previous paragraph.
• Change "to measure" to "in measuring" (line 123, p. 3)
• Remove "the improvement of" (line 133 p. 3)
• Lines 166-169 can be eliminated (p. 4). It's not necessary to hash out everything that your study did not cover.
• The term geomatics (line 177, p. 4) needs a definition (or another term entirely) based on the way it's introduced here.
• Remove the last "the" appearing in line 202 (p. 4).
• Remove the first "the" appearing in line 209 (p. 5).
• Lump lines 226-229 (p. 5) into the previous paragraph.
• The term "Free and Open Source Software" with the commonly accepted FOSS acronym would be more appropriate in line 232 (p. 5).
• I would prefer bulleted lists for lines 240-260 (p. 5-6) as these would make them easier to read.
• Fix the superscripts in Table 1 (lines 331-333, p. 7)
• Does the material in lines 337-349 add important information over the content in Table 2? If so, clarify this and make the list of items bulleted.
• Change "Visually" to "Visual" in Table 2.
• Left justify the text in all tables (this may happen when the article is typeset anyway).
• Lump together the two paragraphs in lines 364-370 (p. 9).
• Are "Portuguese translation and validation of the STAT" a and "Pilot study" section headings? (line 371, p. 9 and line 382, p. 9, respectively)
• Remove the phrase "These procedures were performed in a room, and" (line 389, p. 9).
• Change "mentally" to "mental" (line. 420, p. 10).
• I don't think line graphs are appropriate for the type of data you have. Line graphs imply time series data. A stacked bar chart would be more appropriate (Figures 1 and 2).
• Is it safe to go so far as to say that the contact with GIS concepts and software "likely" contributed to an increase in students' overall spatial thinking? Rather than just "might have"? I'd think so.
• I much prefer violin plots to box plots due to the issues elucidated in "Same Stats, Different Graphs". See Figure 8 here: https://www.autodesk.com/research/publications/same-stats-different-graphs. This is not a hill I will die on, however, and your sample size may preclude you from using a violin plot anyway.
• In lines 580-584 (p. 15) it'd be worth clarifying this statement a bit further. While I'm admittedly not immersed in the research on gender and spatial thinking, I'd think that the proliferation in the use in smartphones and navigation apps is not simply helping women overcome some inherent spatial thinking inferiority; rather, women have always been just as capable, but these tools are helping reduce the historic social barriers which have led to lower spatial thinking scores in the past. The way this paragraph is worded, it sounds like Collins piece suggests the former.
• Italicize the "p" in p-value.
• In the conclusion it'd be worth hashing out the uniqueness of this study again, as done on p. 4 (lines 199-201).

 

Author Response

We thank the valuable Reviewer#1 suggestions/comments and corrected the paper according with the improvements suggested. The answer to each comment is given below (in red). For a better understanding, the manuscript changes are also in red.

Open Review

( ) I would not like to sign my review report
(x) I would like to sign my review report

 

English language and style

( ) Extensive editing of English language and style required
(x) Moderate English changes required
( ) English language and style are fine/minor spell check required
( ) I don't feel qualified to judge about the English language and style

 

 

	Yes	Can be improved	Must be improved	Not applicable
Does the introduction provide sufficient background and include all relevant references?	( )	( )	(x)	( )
Are all the cited references relevant to the research?	( )	(x)	( )	( )
Is the research design appropriate?	(x)	( )	( )	( )
Are the methods adequately described?	(x)	( )	( )	( )
Are the results clearly presented?	( )	(x)	( )	( )
Are the conclusions supported by the results?	(x)	( )	( )	( )

 

 

 

 

 

Comments and Suggestions for Authors

First off, I'd like to thank the authors for a well-conducted study. The methods are sound, and I appreciate the use of non-parametric methods as necessitated by the violated assumptions of other statistical tests. One of the main questions in this piece -- whether or not spatial thinking as measured by the STAT was significant in two CUs -- was significant. Several of the more minor relationships (on the role of gender, for instance) were not significant, yet I appreciate the authors reporting these results anyway. Negative results are often devalued, but I believe they can, at times, be just as informative as significant results. I think this is study is one such case. It seems obvious to me that the authors did not "fish" for significance by modifying their research question or tacking on questions tangential to the study at hand. This is commendable aside from the important results of the study.

My comments below are more on the writing, structure, and presentation of the material rather than methodological suggestions. I believe improving these items will make the paper a bit easier to read and increase the chances that the paper is cited.

We thank for the reviewer comment. We have also considered all the detailed issues in the revision. All the modifications are presented in red in the revised manuscript.

 

• I'd suggest modifying the first sentence of the abstract and introduction. As of now, it seems to suggest to the reader that the paper will focus on the connection between spatial thinking and day-to-day life. I'd rephrase this sentence a bit to better characterize what the paper will be about.

We agree with the Reviewer and have rephrased the sentences.

• Some paragraphs are relatively short (2-3 sentences). Either expand these into full paragraphs or lump them in with others.

We agree with the Reviewer. Some short paragraphs were lumped in with others.

• The use of "for instance" becomes a bit repetitive, especially in lines 45-53 (p. 2). Use others, e.g., "for example", "like", and "such as".

The all manuscript were revised and the use of the term “for instance” was replaced.

• I think the definition of geospatial thinking (lines 54-55, p. 2) ought to be modified slightly. There should be more emphasis on how it is spatial thinking which is tied to the earth's surface. The use of the term "geographical knowledge" makes the definition ambiguous.

We agree with the Reviewer. The definition of geospatial thinking was reformulated as: “Geospatial thinking is a subset of spatial thinking in the field of Earth science and its representation, .i.e., it consists of making use of geographical knowledge of the Earth´s surface to improve our spatial cognitive skills [9].”

• Make "geographic information system" plural in line 71 (p. 2) as this is referring to a broad class of software rather than a specific program.

We agree with the Reviewer. The term was reformulated in the plural. Similarly, make global positioning system (line 88, p. 2) plural.

Thanks for the comment. The term “global positioning systems” was replaced for the correct term “global navigation satellite systems (GNSS)”.

• Is "Measure of spatial thinking abilities" a section heading? (line 97, p. 2)

Yes, it is. It is defined as 3^rd level of headings, according to the MDPI rules. However, in order to simplify we changed to 2^nd level of headings.

• Lump lines 102-105 (p. 3) into the previous paragraph.

The sentence was lumped to the previous paragraph.

• Change "to measure" to "in measuring" (line 123, p. 3)

The word was modified.

• Remove "the improvement of" (line 133 p. 3)

The term was removed.

• Lines 166-169 can be eliminated (p. 4). It's not necessary to hash out everything that your study did not cover.

Lines 166-169 were removed as suggested.

• The term geomatics (line 177, p. 4) needs a definition (or another term entirely) based on the way it's introduced here.

An introduction to geomatics term was introduced in the sentence.

• Remove the last "the" appearing in line 202 (p. 4).
• Remove the first "the" appearing in line 209 (p. 5).

Both terms “the” were removed as suggested.

• Lump lines 226-229 (p. 5) into the previous paragraph.

The lines 226-229 were lumped into the previous paragraph.

• The term "Free and Open Source Software" with the commonly accepted FOSS acronym would be more appropriate in line 232 (p. 5).

The term "Free and Open Source Software" (FOSS) was introduced in the sentence, as suggested.

• I would prefer bulleted lists for lines 240-260 (p. 5-6) as these would make them easier to read.

The lists were changed to bulleted lists.

• Fix the superscripts in Table 1 (lines 331-333, p. 7)

The superscripts were fixed.

• Does the material in lines 337-349 add important information over the content in Table 2? If so, clarify this and make the list of items bulleted.

We agree with the Reviewer, so we removed those sentences.

• Change "Visually" to "Visual" in Table 2.

The word was replaced.

• Left justify the text in all tables (this may happen when the article is typeset anyway).

All tables were justified to the left.

• Lump together the two paragraphs in lines 364-370 (p. 9).

The two paragraphs were lumped.

• Are "Portuguese translation and validation of the STAT" a and "Pilot study" section headings? (line 371, p. 9 and line 382, p. 9, respectively)

Yes, there are both 3^rd  levels according to MDPI rules. In this case, we think that is better to kept as 3^rd level. However, we can change if the Reviewer finds it important.

• Remove the phrase "These procedures were performed in a room, and" (line 389, p. 9).

The sentence was removed.

• Change "mentally" to "mental" (line. 420, p. 10).

The word was corrected.

• I don't think line graphs are appropriate for the type of data you have. Line graphs imply time series data. A stacked bar chart would be more appropriate (Figures 1 and 2).

We agree that line graphs are commonly used with time in the horizontal axis. However, it is our opinion that the present format is the most adequate to readily compare the overall performance between CUs, courses or gender. Nevertheless, we would appreciate if the Reviewer would give us an alternative suggestion.

• Is it safe to go so far as to say that the contact with GIS concepts and software "likely" contributed to an increase in students' overall spatial thinking? Rather than just "might have"? I'd think so.

We agree with the Reviewer and replace “might have” for “likely” to reinforce our results.

• I much prefer violin plots to box plots due to the issues elucidated in "Same Stats, Different Graphs". See Figure 8 here: https://www.autodesk.com/research/publications/same-stats-different-graphs. This is not a hill I will die on, however, and your sample size may preclude you from using a violin plot anyway.

Thank you for the suggestion of using violin plots. In fact, they are more informative than box plots, since they provide a good visualization of the points concentration differences. However, as noted, with the exception of Figure 3 where all cases are used, in the other 4 figures with boxplots, the resultant subgroups might be too small to make it worth using violin plots. Therefore, to provide a consistent presentation, we would prefer to maintain the box plot in figure 3, but we can change if the Reviewer finds it important.

• In lines 580-584 (p. 15) it'd be worth clarifying this statement a bit further. While I'm admittedly not immersed in the research on gender and spatial thinking, I'd think that the proliferation in the use in smartphones and navigation apps is not simply helping women overcome some inherent spatial thinking inferiority; rather, women have always been just as capable, but these tools are helping reduce the historic social barriers which have led to lower spatial thinking scores in the past. The way this paragraph is worded, it sounds like Collins piece suggests the former.

The sentence was reformulated according to the suggestion of the Reviewer.

• Italicize the "p" in p-value.

The p value was italicized.

• In the conclusion it'd be worth hashing out the uniqueness of this study again, as done on p. 4 (lines 199-201).

In conclusions section, besides the advantages of the study, it was reinforced the uniqueness of this study regarding its importance in higher education context.

 

 

 

 

Author Response File: Author Response.pdf

Reviewer 2 Report

1. The specific or creative technical methods used in this manuscript are not mentioned in the abstract，and I suggest to add more key words.
2. The introduction of the related work is not well organized and without productive analysis. And authors should cite those related work with analyzing methods and conclusions in these studies in detail，rather than simply introducing who did them.

Such as in Page 3, it said “ii) it was based on Learning to Think Spatially [6] and Gersmehl and Gersmehl’s [5] studies;” what kinds of factors are in Gersmehl and Gersmehl’s [5] studies?

And Such as it is said “Liu et al. [23] conducted a standardized STAT to examine spatial thinking abilities of a group of Chinese undergraduates with focus on their spatial reasoning. In 2013, Kim and Bednarz [24] applied the Spatial Concepts And Skills Test (SCAST) to a group of students, measuring the students’ spatial concepts and thinking skills (as in the STAT). It is composed of questions from previously validated spatial thinking tests [21,25-26].”   What are the conclusions and methods in these studies and how did they contribute to your research?

3. In “Curricular units and course background”, are these courses selected representative? such as “Geospatial Engineering” “Land-scape Architecture”.   Or did the authors have a discipline system to refer to when choosing course examples?
4. In “Pilot study”, there are only 14 (4 females and 10 males) students for study. Is it sufficient proof of validity for the research? And in the section “Results”, authors didn’t introduce the experimental data. In the abstract, it said the STAT is given to 83 students. The number of student is a little bit small.
5. In “Data scoring and statistical analysis”, the methods mentioned are simple.
6. In section “Results”, the conclusions drawn seems not definite. So what are contributions of this research?

Such as it said “The statistical values obtained in our study demonstrate that the exposure to GIS software slightly increased the skills and abilities of spatial thinking in the students, since there is a slight increase in the results from pre- to post-test. Therefore, we suggest that exposure to GIS software may increase the students’ thinking and spatial abilities and skills in spa-tial problem-solving.”

7. We can see from this manuscript it often said “none of the differences were statistically significant”. If so, what is the significance of statistical research in this manuscript? And more, the author lists the opposing viewpoints of different scholars in the follow-up. What are the definite or reliable conclusion authors reached?

Author Response

We thank the valuable Reviewer#2 suggestions/comments and corrected the paper according with the improvements suggested. The answer to each comment is given below (in green). For a better understanding, the manuscript changes are also in green.

Open Review

(x) I would not like to sign my review report
( ) I would like to sign my review report

 

English language and style

( ) Extensive editing of English language and style required
( ) Moderate English changes required
(x) English language and style are fine/minor spell check required
( ) I don't feel qualified to judge about the English language and style

 

	Yes	Can be improved	Must be improved	Not applicable
Does the introduction provide sufficient background and include all relevant references?	( )	( )	(x)	( )
Are all the cited references relevant to the research?	( )	(x)	( )	( )
Is the research design appropriate?	( )	( )	(x)	( )
Are the methods adequately described?	( )	( )	(x)	( )
Are the results clearly presented?	( )	( )	(x)	( )
Are the conclusions supported by the results?	( )	( )	(x)	( )

 

 

Comments and Suggestions for Authors

 

We thank to the reviewer comments, and we improve the manuscript according to reviewer suggestions.

 

1. The specific or creative technical methods used in this manuscript are not mentioned in the abstract, and I suggest to add more key words.

 

As the abstract has a limit of 250 words, we cannot introduce all the methods used, so the specific methods used were added to the abstract. Also, three more keywords were added.

 

1. The introduction of the related work is not well organized and without productive analysis. And authors should cite those related work with analyzing methods and conclusions in these studies in detail rather than simply introducing who did them.

 

We understand and add some more information about the studies referred in the introduction section.

Such as in Page 3, it said “ii) it was based on Learning to Think Spatially [6] and Gersmehl and Gersmehl’s [5] studies;” what kinds of factors are in Gersmehl and Gersmehl’s [5] studies?

In the introduction section, we try to contextualize the spatial thinking and the ways of evaluating the spatial thinking skills. The spatial thinking can be evaluated and defined in several contexts. As the theme is very vague, we try to use a specific definition of spatial thinking in earth science context. However, we agree that we do not specify some of the studies referred. New literature information was added to the introduction section.

And Such as it is said “Liu et al. [23] conducted a standardized STAT to examine spatial thinking abilities of a group of Chinese undergraduates with focus on their spatial reasoning. In 2013, Kim and Bednarz [24] applied the Spatial Concepts And Skills Test (SCAST) to a group of students, measuring the students’ spatial concepts and thinking skills (as in the STAT). It is composed of questions from previously validated spatial thinking tests [21,25-26].”   What are the conclusions and methods in these studies and how did they contribute to your research?

In the referred paragraph we try to explain that several studies already applied STAT test as a tool to measure the spatial thinking skills/capacities. This is relevant to validate the STAT as an efficient tool and to emphasize and understand how the GIS learning can be used as a tool to enhance the spatial thinking skills of the students. It proves the potential benefits for future students, specially under higher education. We added some more information to the introduction section, regarding the referred studies.

1. In “Curricular units and course background”, are these courses selected representative? such as “Geospatial Engineering” “Land-scape Architecture”.   Or did the authors have a discipline system to refer to when choosing course examples?

 

The presented courses, “Geospatial Engineering” and “Landscape Architecture”, are the only courses that have GIS and GIS Applied to Natural Sciences in faculty of Sciences University of Porto.

 

1. In “Pilot study”, there are only 14 (4 females and 10 males) students for study. Is it sufficient proof of validity for the research?

 

The number of sample size of a pilot study depends on the context. There are several studies that explain how to define the number of sample size for pilot studies. We consulted two literature references (Connelly, 2008) and (Hertzog, 2008). Connelly (2008) suggests that a pilot study sample should be 10% of the sample projected for the larger parent study. In our case, based on this study, if we have 83 participants, at least 8 participants are enough to the pilot study. Based on Hertzog (2008), samples ranging from 10 to 40 per group are evaluated for them

adequacy in providing estimates precise enough to meet a variety of possible

aims, so it depends on several factors. In our study, we used a class of 14 elements as a representative sample of the participants of the study.

Yes, we agree with the reviewer that the total number of students it’s not enough and may led to the lack of statistical significance. As we understand that, we introduce the limitations of this work in conclusion section.

 

Connelly, L. M. (2008). Pilot studies. Medsurg Nursing, 17(6), 411-2.

Hertzog, M.A. (2008). Considerations in determining sample size for pilot studies. Research in Nursing & Health, 31,180-191.

 

And in the section “Results”, authors didn’t introduce the experimental data.

 

We did not present the results of the pilot study in the “Results” section, because the pilot study was used to verify if the language, the interpretation of the questions, the content of the text and other aspects such as the content validity, discrimination ability, and difficulty level, were correct. Then we applied to the main sample of the study. Also, the students that performed the STAT pilot study were not students that frequents the GIS and GIS Applied to Natural Sciences curricular units. However, if the Reviewer think it is important, we can present the results.

 

In the abstract, it said the STAT is given to 83 students. The number of student is a little bit small.

 

The pilot study had 14 participants, but the study itself was composed by 83 students as mentioned in the abstract.

 

 

1. In “Data scoring and statistical analysis”, the methods mentioned are simple.

 

We agree that the mentioned statistical methods are simple, but it is our opinion they are adequate given the relatively small size of the sample. The use of more complex statistical methods, such as multivariable regression, would be a problem for a sample with n=83 and variables with 5 categories, but it might be an interesting approach to consider with larger sample sizes.

 

1. In section “Results”, the conclusions drawn seems not definite. So what are contributions of this research?

Such as it said “The statistical values obtained in our study demonstrate that the exposure to GIS software slightly increased the skills and abilities of spatial thinking in the students, since there is a slight increase in the results from pre- to post-test. Therefore, we suggest that exposure to GIS software may increase the students’ thinking and spatial abilities and skills in spatial problem-solving.”

Negative results are often undervalued; however, they are just as informative as significant results. We think that our results are important for GIS teaching in higher education. We are aware of the limitations of our study (described in the discussion section), but we also think that our study can bring some contributions to the improvement of spatial thinking with GIS learning, as stated also on discussion section:

 

“This study allowed us to analyze the differences between students in the different areas as well as to analyze the evolution of students from the beginning to the end of the semester, to understand if there was a progression in the students’ spatial thinking. Thus, this study presents some contributions towards the improvement of spatial thinking with GIS learning: i) it contributes to an improvement in the pedagogical practices that have been adopted, given the number of the students and the different basic training; ii) it contributes to improving the learning of students in these areas; iii) it can help professors in a more effective, interdisciplinary, and targeted manner through the management of knowledge within the scope of the CUs; iv) it can guide the CU subjects and didactics for learning that must be guaranteed to all students; and v) it facilitates the development of new learning methodologies that are always appropriated to the different areas of student training. Over the years, the professors of these CUs noticed differences between students from different courses. Hence, application of the STAT helps us to better understand this difference and thus create new pedagogical strategies to improve the teaching of these CUs such that all students learn to solve the type of problems that are posed to them and that involve spatial thinking.”

1. We can see from this manuscript it often said “none of the differences were statistically significant”. If so, what is the significance of statistical research in this manuscript?

 

With respect to the lack of statistical significance, please refer to the answer given to comment #6. of statistical research. Additionally, the relatively small size of the sample is acknowledged as a limitation of the present work in section 4.

 

And more, the author lists the opposing viewpoints of different scholars in the follow-up. What are the definite or reliable conclusion authors reached?

 

In fact, there are opposite viewpoints in some issues. We hope the present study triggers the development of other studies, with larger samples, in order to clarify such observed different conclusions.

Author Response File: Author Response.pdf

Reviewer 3 Report

I appreciate the authors conducted this study to report different group of college students’ STAT scores to evaluate their GIS concept understanding. Particularly, I was impressed by the thoroughness of literature review the authors presented in this manuscript. I also liked the way they presented the results and implications of the study. It was easy to understand and follow.

I have a couple of minor comments and questions.

In Abstract, the authors mentioned that this study was followed by “larger samples studies.” I’m not sure whether 83 students make this study as a larger sample study. Similar studies using STAT had similar or more numbers of participants.

In the Objective of the work section, I’m a bit confused about differences of two groups. Did you divide students into different course work (GIS vs. GIS in NS)? Or their degree level (undergraduate vs. graduate) Or combine both criteria? Please clearly state two groups’ demographics.

I’m also wondering whether masters’ students had prior GIS background and knowledge. If so, how does it affect your study? How did you control this variable?

Were these two courses taught by the same instructor? If not, don’t you think their teaching effectiveness, pedagogical strategies, may have impact on students’ GIS concept understanding? How can you control this variable?

Author Response

We thank the valuable Reviewer#3 suggestions/comments and corrected the paper according with the improvements suggested. The answer to each comment is given below (in blue). For a better understanding, the manuscript changes are also in blue.

Open Review

(x) I would not like to sign my review report
( ) I would like to sign my review report

 

English language and style

( ) Extensive editing of English language and style required
( ) Moderate English changes required
(x) English language and style are fine/minor spell check required
( ) I don't feel qualified to judge about the English language and style

 

	Yes	Can be improved	Must be improved	Not applicable
Does the introduction provide sufficient background and include all relevant references?	(x)	( )	( )	( )
Are all the cited references relevant to the research?	(x)	( )	( )	( )
Is the research design appropriate?	(x)	( )	( )	( )
Are the methods adequately described?	( )	(x)	( )	( )
Are the results clearly presented?	(x)	( )	( )	( )
Are the conclusions supported by the results?	(x)	( )	( )	( )

 

 

Comments and Suggestions for Authors

I appreciate the authors conducted this study to report different group of college students’ STAT scores to evaluate their GIS concept understanding. Particularly, I was impressed by the thoroughness of literature review the authors presented in this manuscript. I also liked the way they presented the results and implications of the study. It was easy to understand and follow.

We thank for the reviewer comment. We also consider all the detailed issues in the revision. All the modifications are presented in blue in the revised manuscript.

I have a couple of minor comments and questions.

In Abstract, the authors mentioned that this study was followed by “larger samples studies.” I’m not sure whether 83 students make this study as a larger sample study. Similar studies using STAT had similar or more numbers of participants.

We apologize for the lack of clarity in the mentioned sentence. We would like to mention the need to for the development of larger studies, as already acknowledged in the “Limitations”. We therefore removed this part of the sentence from the abstract.

In the Objective of the work section, I’m a bit confused about differences of two groups. Did you divide students into different course work (GIS vs. GIS in NS)? Or their degree level (undergraduate vs. graduate) Or combine both criteria? Please clearly state two groups’ demographics.

The students were from two different courses, GIS and GIS Applied to Natural Sciences, meaning that they were divided in two classes, one from GIS and another from GIS Applied to Natural Sciences. The students from GIS are from first degree and the students from GIS Applied to Natural Sciences are from master’s degree. This means that the students have different backgrounds.

I’m also wondering whether masters’ students had prior GIS background and knowledge. If so, how does it affect your study? How did you control this variable?

Master’s students had some GIS background since in some curricular units GIS software is introduced. Lines 251-283 explains in detail both curricular and characterize the students that frequent them. The background of the students of the different courses was effectively a variable to consider. However, in STAT that variable is not evaluated. We interpret the results considering that variable and, in fact, we concluded that, as we expected, the results between the courses was different because the background of the students are characterized by different contents. Also, we concluded that new strategies must be carefully introduced in the curricular units, always taking into account the background of each student and their spatial thinking skills, as explained in the conclusion section.

Were these two courses taught by the same instructor? If not, don’t you think their teaching effectiveness, pedagogical strategies, may have impact on students’ GIS concept understanding? How can you control this variable?

The two courses presented in the study were taught by the same team, composed by two professors (authors of the paper). All the discussion and final conclusions were discussed between the instructors of the two courses, who have a strong and close relationship with the students. In fact, we also discussed that the strategies should be well thoughted, as well as the pedagogical content should be prepared according to the different contexts to which students are exposed.

 

Author Response File: Author Response.pdf

Reviewer 4 Report

The study aims to present the “Evaluation of Spatial Thinking Ability based on exposure to GIS concepts in the context of higher education.” The paper is well organized, but there are some major comments to address before being published. Then, I would like to suggest a major revision.

1. The abstract is very weak. Need to revise the abstract according to the formats of the journal
2. No need subsection to the introduction
3. The introduction is too long
4. The methodology is not clear. Better to explain it clearly
5. Link of lines, 314-317 is not working
6. The result, Discussion, and conclusion are fine.

 

Author Response

We thank the valuable Reviewer#4 suggestions/comments and corrected the paper according with the improvements suggested. The answer to each comment is given below (in orange). For a better understanding, the manuscript changes are also in orange.

Open Review

( ) I would not like to sign my review report
(x) I would like to sign my review report

 

English language and style

( ) Extensive editing of English language and style required
( ) Moderate English changes required
(x) English language and style are fine/minor spell check required
( ) I don't feel qualified to judge about the English language and style

 

	Yes	Can be improved	Must be improved	Not applicable
Does the introduction provide sufficient background and include all relevant references?	( )	( )	(x)	( )
Are all the cited references relevant to the research?	( )	(x)	( )	( )
Is the research design appropriate?	( )	( )	(x)	( )
Are the methods adequately described?	( )	( )	(x)	( )
Are the results clearly presented?	( )	(x)	( )	( )
Are the conclusions supported by the results?	( )	(x)	( )	( )

 

 

Comments and Suggestions for Authors

The study aims to present the “Evaluation of Spatial Thinking Ability based on exposure to GIS concepts in the context of higher education.” The paper is well organized, but there are some major comments to address before being published. Then, I would like to suggest a major revision.

We thank for the reviewer comment. We also consider all the detailed issues in the revision. All the modifications are presented in orange in the revised manuscript.

1. The abstract is very weak. Need to revise the abstract according to the formats of the journal

 

The abstract was modified according to all Reviewers’ suggestions, namely its content was improved and structured according to the journal rules. We hope that the modifications have improved the abstract.

 

1. No need subsection to the introduction

 

We structured the introduction section in sub-sections to organize the information, since the state of the art is a little long and otherwise could be more difficult to the reader to follow it. Therefore, if the Reviewer agree, we would prefer to maintain the division in sub-sections in order to facilitate the reading of the paper.

 

1. The introduction is too long

We agree with the reviewer that the introduction section is too long. However, it was necessary to contextualize the study to clarify our objectives. We think that removing part of the introduction would harm the objective of the study and since the remain Reviewers appreciated the “thoroughness of literature review” (referred by Reviewer#3), we think that if we change it, it would be contradictory with the other revisions.

 

1. The methodology is not clear. Better to explain it clearly

 

We have tried to organize the materials and methods section in a structure easy to read. First, we explain how the curricular units and credits are defined in Portugal higher education, to better understanding of the description of curricular units and courses. Then we describe the participants, followed by the STAT test as an instrument and the application of the STAT measure tool, followed by the statistical treatment of the data. Therefore, we kindly ask the Reviewer to give us more specific indications regarding a possible improvement of the methodology section.

 

1. Link of lines, 314-317 is not working

 

We thank for the comment, but we have checked they are working. Perhaps it might have been some connection failure or the fact that the first number used as superscript was also copied as part of the address. Please let us know if the problem persists.

 

1. The result, Discussion, and conclusion are fine.

 

We thank for the reviewer comment, hoping that the new version, considering all the suggestions, has improved the manuscript overall.

 

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

1.      The research methods in this manuscript are too simple, and the scope of study objects and experimental data are limited, resulting in the contributions of research not significant.

2.      Most of the criteria and references in the research are literatures from 5 years or even more than 10 years ago. In recent years, the spatial thinking abilities of students, research methods and experimental means have been greatly improved, so the methods of research and experiment in this manuscript should be improved a lot.

Author Response

We thank the valuable Reviewer#2 suggestions/comments and corrected the paper according with the improvements suggested. The answer to each comment is given below (in red). For a better understanding, the manuscript changes are also in red.

Comments and Suggestions for Authors

1. The research methods in this manuscript are too simple, and the scope of study objects and experimental data are limited, resulting in the contributions of research not significant.

 

As we mentioned in the first revision, we recognize the limitations of our work, but we also think that our study provides contributions to the improvement of spatial thinking with GIS learning, as stated also in the Discussion section. Despite the relative simplicity of the employed methods, namely in terms of statistical analysis, it is our opinion that the present work based on the application of the STAT test will be of value for the assessment of the impact of GIS learning in higher education students. Nevertheless, despite the lack of statistical power due to the relatively small size of the dataset, as already mentioned in our answer to the previous comment #5, in order to enrich this paper and following the Reviewer suggestion, we have performed a multivariable analysis using multiple linear regression, aiming to assess the separate effects of gender and curricular unit on the global STAT score, in the pre- and post-test moments, as well as on their difference. We did not consider the categorical variable corresponding to the courses since it comprises 5 categories and the number of cases might be considered insufficient on the light of some rules-of-thumb in regression modelling. For each of the three different outcomes and corresponding models, the Omnibus Test was not statistically significant (p-value of 0.182, 0.734 and 0.564, for the pre-test, post-test and difference between pre- and post-test, respectively) - as well as each of the regression coefficients separately (considering a significance level of 5%) - which means that these models do not significantly explain the variation of the outcome (global STAT score), presumably due to the already mentioned lack of statistical power. It is our opinion that these results would not bring additional value to the manuscript, and thus we decided not to include them in the paper. Nevertheless, if the Reviewer finds them relevant, we can add them to the paper.

2. Most of the criteria and references in the research are literatures from 5 years or even more than 10 years ago. In recent years, the spatial thinking abilities of students, research methods and experimental means have been greatly improved, so the methods of research and experiment in this manuscript should be improved a lot.

 

We thank for the comment. We have searched in depth for studies involving research using the STAT test in the context of GIS learning, which is the topic of the present paper. Although there are several papers not so recent, it has deserved in fact a considerable attention to the recent years. Given the larger number of older papers, we have not included all of these, but we have introduced all of those amongst the more recent ones that were found in our literature search. Actually, nearly half of the references of the present paper are from the latest 5 years, and most of them from 2020 and 2021. Nevertheless, we have performed an additional literature search on this paper topic, and we have found one more paper related to STAT, which is the method adopted by us. This reference was added to the Introduction section and to the references list:

R W Romadlon, Y Yusuf, Sarwono. Effect of learning project model-based learning on GIS spatial thinking skills students. In IOP Conf. Series: Earth and Environmental Science 683 (2021) 012045.

“Romadlon and Sarwono (2021) applied STAT to 45 students of 12th grades of MAS AL ISLAM in academic year 2018/2019, in pretest and posttest to analyze how GIS material affects the spatial thinking abilities. This evaluation was performed with the creation of a project-based learning model which is a model that uses the project / activity as a learning process to achieve competence attitudes, knowledge and skills. The results were analyzed by SPSS software through observation, documentation, and testing STAT (pretest and post-test) with quantitative data analysis techniques. It was concluded that the project-based learning model with the help of digital maps and thematic maps significant effect on students' spatial thinking skills and also that taking advantage of technological and geospatial technologies is a great advantage to improve the spatial skills. This work however was focused only on 12th grade students, and thus did not include higher education students.”

 

We understand, as mentioned in our paper, that there are other methods to analyze the influence of GIS learning in increasing the spatial abilities in students. However, our objective was to test the STAT and analyze the results to understand how the professors can deal with different students, with different backgrounds and within different contexts. Nevertheless, we would appreciate if the Reviewer finds it suitable to indicate us additional references that would fit within the scope of our paper.

 

Thanks again for your comments and suggestions.

 

Author Response File: Author Response.pdf

Reviewer 4 Report

Thank you very much for incorporating my comments. Now the paper can be accepted. 

Author Response

Reviewer#4

 

 

Comments and Suggestions for Authors

Thank you very much for incorporating my comments. Now the paper can be accepted. 

Thanks for your positive comments.

Thank you again for your time.

Author Response File: Author Response.pdf