3D Landform Modeling to Enhance Geospatial Thinking
Round 1
Reviewer 1 Report
Regarding section 3.3 authors should provide more details about the workshops: length, number of sessions, number of teachers assisting students …
Also, it could be of great help, to provide access through an open repository to the didactic content described in the paper, to support the reproducibility of work done.
Regarding 3.4 authors explain previous use of TMA test. Authors should clarify if students participating in the workshop had previous exposure to this test.
Regarding last section, 5 Discussion and Conclusions, authors should improve it, as this section in its current form is more a paper summary than a discussion and conclusion section. Really there is only a discussion of results in lines 369-380.
Author Response
Reviewer 1
First of all I want to highlight the great revision work done. A thorough revision of the manuscript has been made. All the proposed recommendations make sense and undoubtedly add value to the article. I can only appreciate the great work done, thanks to which I have learned a lot.
All the changes made in the manuscript are in blue color to facilitate its location.
Regarding section 3.3 authors should provide more details about the workshops: length, number of sessions, number of teachers assisting students …
Now detailed at the beginning of 3.3 point: “The timing of the two workshops was in four phases. Phase 1: specific training on landforms (1 hour). Phase 2: completion of the Topographic Map Assessment Pre-test (30 minutes). Phase 3: installation of Sketch Up Make and / or Autodesk 123D Make applications (30 minutes). Phase 4: realization of the landforms (3 hours for the Sketch Up Make workshop, and 3+2 hours for the Autodesk 123D Make workshop). Phase 5: completion of the Topographic Map Assessment Post-test (30 minutes). Phases 1, 2 and 3 were carried out in a 2-hour session. The following week, phases 4 and 5 were carried out in another session of 3 and half hours, for the Sketch Up Make workshop. For the Autodesk 123D Make workshop, 2 sessions for phases 4 and 5 were needed: 3 hours in one day for phase 4, two hours and 30 min to complete phase 4 and a post test (phase 5) the next day. There were 2 teachers assigned to the students for each of the phases.“
Also, it could be of great help, to provide access through an open repository to the didactic content described in the paper, to support the reproducibility of work done.
No specific repository was used for the contents of the workshops. For the explanation of the concept of the contour line, we worked with the annexed material of the contour description provided by the Topographic Map Assessment Test, whose download address is in reference 28. (To reinforce the possibility of download, I have added in 3.4: “The complete test and its instructions can be downloaded from the Spatial Intelligence and Learning Center, available at http://www.spatiallearning.org/index.php/resources/testsainstruments.”
For the purpose of supporting the reproduction of work done, we have included the web addresses of downloading the applications Sketch Up Make and Autodesk 123D Make.
The links for download the applications Sketch Up Make and Autodesk 123D Make are in the 3.2 heading. I have added, in the description of the TMA (at the end of the 2nd paragraph of 3.4 pint): “The Topographic Map Assessment Test contains an annex on the cartographic representation of the terrain in which there is a detailed description of the contour line concept.”
Regarding 3.4 authors explain previous use of TMA test. Authors should clarify if students participating in the workshop had previous exposure to this test.
Effectively, it is an important point that I had forgotten to include in the article and I thank you. That check was made for all participants in the workshops. It has been included, in point 3.1: “Students participating in the workshops did not have previous exposure to the test that was used in the workshops to measure their geospatial thinking.”
Regarding last section, 5 Discussion and Conclusions, authors should improve it, as this section in its current form is more a paper summary than a discussion and conclusion section. Really there is only a discussion of results in lines 369-380.
I agree. I have added in the 2nd paragraph of Discussion and Conclusions point:
“Compared with previous research: strategies based on the joint use of augmented reality, 3D mesh processing (information represented in a 3D view on a planar surface like a cell phone screen), and 3D digital terrain model (3D printed model) obtained a gain of 4.56 points on the TMA test [31]. In that research there was a control group working with traditional 2D maps, and their gain was 1.55 points on the TMA test. Strategies based on the joint use of augmented reality and traditional 2D maps obtained a gain of 5.41 points on the TMA test [31] (in that case, the gain using only 2D strategies was 2.22 points on the TMA test). The gain obtained in the present research is lower (3.54, SD=2.65) than the previous researches mentioned (although superior to the gains obtained with 2D maps in those previous investigations). In this regard, it is necessary to take into consideration that they are two different tasks: the generation of landforms used in this research and the interpretation of landforms, used in the other two previous researches.
In relation to gender, the results obtained show that there are no significant differences between men and women in the improvement of geospatial thinking after training, which coincides with the results on gender obtained in previous research [29, 31].”
The key findings and contributions of this research are at the 9nd paragraph of the Discussion and Conclusions (later I start with future works)
“This research presents a strategy of first contact with concepts related to basic landforms, and provides a teaching approach based on easy-to-use technologies (Sketch Up and Autodesk123D Make) that develop the geospatial thinking of the students. It serves as a first contact with 3D rendering technologies and contributes, in turn, to the necessary geographic literacy in the early stages of the Digital Earth education, in accordance with the stipulations of the Beijing Declaration on the Digital Earth [24]…. “
I have also added future work to be done with other approaches / technologies:
“At a more advanced level, there are other advanced applications that aim at digital terrain modeling, such as AR Sandbox, Rhino, Vue, Terragen, Maya, 3dsMax, and World Machine, which could be used to enrich the learning process. As a future work, it would be interesting to carry out a workshop, in which the impact of these advanced technologies on the development of participants' geospatial thinking would be measured (using the same measurement tool used in this research).
On the other hand, the role of 3D GIS methods in teaching learning processes is crucial, although they require a great amount of teacher training. In this sense, another proposal to use easy-to-use technologies in the GIS field related to the recognition of landforms could be a workshop based on the geomorphons approach [5] (since it is developed for the open source GIS Software GRASS GIS), measuring its impact on the development of geospatial thinking. It would be interesting, in turn, to extend this analysis to these technologies in the field of the development of geospatial thinking and establish comparisons with CAD systems and / or with tangible interfaces for landscape analysis such as AR Sandbox and Illuminating Clay.”
Reviewer 2 Report
Regarding lines 60-62, the classification of landforms need not be subjective - there are automated, unsupervised classification techniques such as the Geomorphons methods by Jasiewicza and Stepinski. Given uninformed statements like this, I feel that the literature review should be expanded. What other methods are used for studying geospatial thinking with 3D landforms? What about the ARSandbox for example? What about participatory 3D modeling? If there are automated methods for landform classification could these be used to enrich the learning process? How about 3D modeling in Rhino with real-time feedback from automated geospatial analytics by Grasshopper's Bison plugin? There are many powerful tools for simultaneously 3D modeling and analyzing terrain that this study does not even cite, much less consider as methods. A thorough literature review would lead to better methods.
Since these workshops / teaching methods are standard in landscape architecture curricula at least - albeit with a wider range of software (including Rhino, Vue, Terragen, Maya, 3dsMax, and WorldMachine) - the educational aspect of this paper in not very novel. Assessing its effectiveness with the TMA is, however, an important contribution. My concern here is the comparison between two very similar 3D CAD platforms. While the post-test demonstrates learning, there is not a control group who learn by manual / analog methods (which might be better) to create a baseline for comparison. A comparison between analog and digital methods would be more interesting. A comparison between 3D CAD (SketchUp and/or Make123) and GIS would have been interesting too, especially since it was discussed in the paper. Are there learning gains from teaching 3D landform modeling with mixed digital, 3D CAD, and 3D GIS methods? What about a comparison of 3D CAD and tangible interfaces (ARSandbox, Illuminating Clay, etc)?
The sentence structure is awkward, making this paper hard to read. Please revise.
Author Response
Reviewer 2
First of all I want to highlight the great revision work done. A thorough revision of the manuscript has been made. All the proposed recommendations make sense and undoubtedly add value to the article. I can only appreciate the great work done, thanks to which I have learned a lot.
All the changes made in the manuscript are in blue color to facilitate its location.
Moderate English changes
The MDPI English Editing Service has revised the manuscript (type of service language editing and layout). English editing invoice ID: english-6815.
Regarding lines 60-62, the classification of landforms need not be subjective - there are automated, unsupervised classification techniques such as the Geomorphons methods by Jasiewicza and Stepinski. Given uninformed statements like this, I feel that the literature review should be expanded.
Following your recommendations, I have expanded the literature review in the introduction (references 1-9) and specifically on automated methods for landform classification:
“In the process of the identification and classification of landforms, the accuracy of the derived landscape map will depend on the subjective skills of the surveyor / analyst [4], although there are automated unsupervised classification techniques, such as the geomorphons approach [5]. This approach is based on the creation of a variety of possible types of morphological landscape generated using the elevation differences of a given environment, although there are more approaches to address the recognition of landforms. On landform classification methods there are interesting works like the one of Kramm et al. [6] in which they compare different approaches. There are specific applications in the field of automated geospatial analytics such as the Grasshopper's Bison plugin; features tools for terrain mesh creation, analysis, editing, and annotation. 3D modeling and analyzing terrain software such as Rhino, Vue, Terragen, Maya, 3dsMax, and World Machine are used in landscape architecture curricula for building, rendering, and animating realistic natural environments.
In any case, the present research is not specifically focused on the recognition of landforms. The aim is to initiate the students in 3D modeling of basic forms of terrain and to verify their impact on the development of geospatial thinking. For this, a geovisualization environment based on easy-to-use 3D modeling technologies is used.”
What other methods are used for studying geospatial thinking with 3D landforms?
Indeed, there was a lack of information in this regard. I have added, in the introduction (3rd paragraph):
“Research in cognitive studies of the geovisualization and development of spatial skills can have a great role in the growth of geotechnologies and 3D geovisual software environments. There are researches [10,11] in the field of geovisualization that study the improvement of geospatial thinking with 3D landforms, in which students had their first contact with landforms. In these researches, digital technologies (Augmented Reality) and tangible models (digital terrain models printed in 3D) have been used for tasks based on the interpretation of basic landforms. These studies worked with the interpretation of the cartographic relief, but the present research proposes from a different approach: the generation of basic landforms using easy-to-use 3D three-dimensional modeling tools. “
What about the ARSandbox for example? What about participatory 3D modeling? If there are automated methods for landform classification could these be used to enrich the learning process?
The present research is not specifically focused on the recognition of landforms. The aim is to initiate the students in 3D modeling of basic forms of terrain and to verify their impact on the development of geospatial thinking. Describes a first contact of the students with 3d Modeling and tasks related to basic landforms. To reinforce this approach, I have included in 2.1 point:
“However, the learning of the geomorphological processes is complex for students, partly because they lack basic knowledge, and partly because of the difficulty that students have in cartographic interpretation [31-33]. Therefore, in a first contact of the students with tasks related to the visualization and interpretation of the landforms it is necessary to familiarize students with the elementary landforms such as plains, elevations, depressions, ridges, valleys, hills, and cols (also called mountain passes or saddle points) through easy-to-use technological tools that facilitate their first understanding”
In any case, It would be very interesting to conduct research with these technologies (In fact ARSandbox is going to focus my next research). I have added (and Rhino, Vue, Terragen, Maya, 3dsMax, and WorldMachine technologies) it in the Introduction and in future work:
“There are specific applications in the field of automated geospatial analytics such as the Grasshopper's Bison plugin; features tools for terrain mesh creation, analysis, editing, and annotation. 3D modeling and analyzing terrain software such as Rhino, Vue, Terragen, Maya, 3dsMax, and World Machine are used in landscape architecture curricula for building, rendering, and animating realistic natural environments.”
“At a more advanced level, there are other advanced applications that aim at digital terrain modeling, such as AR Sandbox, Rhino, Vue, Terragen, Maya, 3dsMax, and World Machine, which could be used to enrich the learning process. As a future work, it would be interesting to carry out a workshop, in which the impact of these advanced technologies on the development of participants' geospatial thinking would be measured (using the same measurement tool used in this research).”
How about 3D modeling in Rhino with real-time feedback from automated geospatial analytics by Grasshopper's Bison plugin? There are many powerful tools for simultaneously 3D modeling and analyzing terrain that this study does not even cite, much less consider as methods. A thorough literature review would lead to better methods. Since these workshops / teaching methods are standard in landscape architecture curricula at least - albeit with a wider range of software (including Rhino, Vue, Terragen, Maya, 3dsMax, and WorldMachine) - the educational aspect of this paper in not very novel.
Effectively, these applications offer great possibilities for the development of geospatial thinking. The research presents in my manuscript offers a strategy of first contact with basic landforms 3D Modeling and its impact. The use of these advanced technologies (Rhino, Vue, Terragen, Maya, 3dsMax, and WorldMachine) for the development of geospatial thinking has been added as a possible future work at the end of Discussion and Conclusions point. Also, these technologies are now cited in the introduction.
Assessing its effectiveness with the TMA is, however, an important contribution. My concern here is the comparison between two very similar 3D CAD platforms. While the post-test demonstrates learning, there is not a control group who learn by manual / analog methods (which might be better) to create a baseline for comparison. A comparison between analog and digital methods would be more interesting.
In this regard, in the discussion and conclusions I have included results of the TMA test on previous works with analogical methods (2D maps).
“Compared with previous research: the strategies based on the joint use of augmented reality, 3D mesh processing (information represented in a 3D view on a planar surface like a cell phone screen), and 3D digital terrain model (3D printed model) obtained a gain of 4.56 points on the TMA test [11]. In that research there was a control group working with traditional 2D maps, and their gain was 1.55 points on the TMA test. Strategies based on the joint use of augmented reality and traditional 2D maps obtained a gain of 5.41 points on the TMA test [10] (in that case, the gain using only 2D strategies was 2.22 points on the TMA test). The gain obtained in the present research is lower (3.54, SD=2.65) than the previous researches mentioned (although superior to the gains obtained with 2D maps in those previous researches). In this regard, it is necessary to take into consideration that they are two different tasks: the generation of landforms used in this research and the interpretation of landforms, used in the two other two previous researches.”
A comparison between 3D CAD (Sketch Up and/or Make123) and GIS would have been interesting too, especially since it was discussed in the paper. Are there learning gains from teaching 3D landform modeling with mixed digital, 3D CAD, and 3D GIS methods? What about a comparison of 3D CAD and tangible interfaces (ARSandbox, Illuminating Clay, etc)?
Indeed, the comparison you propose is very interesting. I have included it as future work
“On the other hand, the role of 3D GIS methods in teaching learning processes is crucial, although they require a great amount of teacher training. In this sense, another proposal to use easy-to-use technologies in the GIS field related to the recognition of landforms could be a workshop based on the geomorphons approach [5] (since it is developed for the open source GIS Software GRASS GIS), measuring its impact on the development of geospatial thinking. It would be interesting, in turn, to extend this analysis to these technologies in the field of the development of geospatial thinking and establish comparisons with CAD systems and / or with tangible interfaces for landscape analysis such as AR Sandbox and Illuminating Clay. “
The sentence structure is awkward, making this paper hard to read. Please revise.
The MDPI English Editing Service has revised the manuscript (type of service language editing and layout). English editing invoice ID: english-6815.
Reviewer 3 Report
The goal of this paper is to evaluate approaches to foster students’ understanding of certain characteristics of the topography, which is called „geospatial thinking“. The paper presents a study comprising the evaluation of two courses in which different software packages were implemented to fulfil certain tasks on a topography. The suitability of the software packages and the chosen approach for fostering the understanding of the topography was measured by providing points for the tasks fulfilled and by a questionnaire answered by the students. A statistical analysis was conducted to compare the two software packages and approaches. Overall, the content of the paper seems to have some merit for practical teaching purposes. However, particularly in the theoretical part there is a lack of clarity, which needs to be enhanced. Furthermore, the presentation of the statistical analysis has some flaws and needs revision. Therefore, I suggest major changes. In the following I provide my detailed comments.
Abstract:
The abstract should briefly present the problem addressed in the study, the goal of the study, the methods implemented, the major results as well as the implications of the results for research and (teaching) practice. The current abstract does not provide all this information. Furthermore, some parts are not understandable without reading the whole paper. Therefore, it should be revised.
What is the STEM domain? – Pease avoid abbreviations in the abstract.
Lines 19 - 23: For an abstract this is too much detail. Please shorten.
What are „landform workshops“?
What is „the Topographic Map Assessment test“? Please provide a short description.
What are your results? Please add 1- 2 sentences.
What are your conclusions / implications for teaching? Please add one sentence.
Introduction:
The introduction in the topic and the problem is a bit repetitive and not very clear. What is the problem with fostering student‘s understanding of different landforms? What are the goals and the research questions of the conducted study?
Line 38: Please provide an example for „landforms“.
Lines 43-44: DTM and DEM are the same. Do you mean “Digital Surface Model”?
Line 44: What do you mean with „image LiDAR“? LiDAR data are point clouds. For example, airborne laser scanning data (ALS) provide the basis for a digital surface model.
Geovisualization training
The definition of spatial / geospatial thinking is very fuzzy. Please make it more clear.
Please make it more clear what geospatial thinking is aimed at.
Please point out what the actual problem in teaching is with regard to the suitability of tools supporting geospatial thinking.
Line 148: How is „geographic literacy“ defined? How can it be measured?
Material and Methods
3.1 Participants:
Did the students differ in the workshops or were there also students visiting both workshops?
Were the students undergraduates or graduates?
3.3. Procedure
It is not clear whether the training on landforms (lines 191 - 194) were the pre-test.
Line 193: Please provide an example for the „specific contents related to the forms of relief“.
Line 212: In which format did the students deliver their results? Did they export their model?
Lines 234-235: Please provide the country where the Temple University and the Northwestern University are.
Line 254: The questions are given in Table 3. Please correct.
Results
Line 263: How were the items grouped for the MANOVA? Please provide the information that the ANOVA assumptions are fulfilled.
Line 265: Why did you analyze the effect of gender? What was your hypothesis?
Lines 275 - 288: The data is provided in Table 2. Therefore, you can remove it in the text. Just the major results should be pointed out.
Line 290 - 295: As there is no significant difference between the workshops, Figure 3 might not be necessary. The data is provided in the text. - What is exactly the „Moment“? Please define.
Figure 4: The absolute differences do not show much because the tasks comprise different amounts of items and points. Please show the relative gains (%).
Lines 327-333: Cronbach‘s alpha measures whether the questions belonging to one group measured the same thing. Therefore, this is no suitable analysis between the Sketchup and the Autodesk workshop. However, the quality of the questionnaire can be evaluated with Cronbach‘s alpha. Did the questions in one group measure all the same thing (i.e., application; improvement; understanding; implications in the learning-teaching environment)?
Discussion and Conclusions
Line 388: What is a “DE education”? Please be more specific and provide actual fields of study.
Author Response
Reviewer 3
First of all I want to highlight the great revision work done. A thorough revision of the manuscript has been made. All the proposed recommendations make sense and undoubtedly add value to the article. I can only appreciate the great work done, thanks to which I have learned a lot.
All the changes made in the manuscript are in blue color to facilitate its location.
Moderate English changes
The MDPI English Editing Service has revised the manuscript (type of service language editing and layout). English editing invoice ID: english-6815.
The goal of this paper is to evaluate approaches to foster students’ understanding of certain characteristics of the topography, which is called „geospatial thinking“. The paper presents a study comprising the evaluation of two courses in which different software packages were implemented to fulfil certain tasks on a topography. The suitability of the software packages and the chosen approach for fostering the understanding of the topography was measured by providing points for the tasks fulfilled and by a questionnaire answered by the students. A statistical analysis was conducted to compare the two software packages and approaches. Overall, the content of the paper seems to have some merit for practical teaching purposes. However, particularly in the theoretical part there is a lack of clarity, which needs to be enhanced. Furthermore, the presentation of the statistical analysis has some flaws and needs revision. Therefore, I suggest major changes. In the following I provide my detailed comments.
Abstract:
The abstract should briefly present the problem addressed in the study, the goal of the study, the methods implemented, the major results as well as the implications of the results for research and (teaching) practice. The current abstract does not provide all this information. Furthermore, some parts are not understandable without reading the whole paper. Therefore, it should be revised.
What is the STEM domain? – Pease avoid abbreviations in the abstract.
I have substituted that term for Higher Education
Lines 19 - 23: For an abstract this is too much detail. Please shorten.
Effectively, it was a phrase too long. I have rewritten it eliminating the final part.
What are „landform workshops“?
Deleted landform, just “Wokrshops”
What is „the Topographic Map Assessment test“? Please provide a short description.
Short description provided: “a battery of seven tasks related to relief interpretation integrated in 18 exercises”
What are your results? Please add 1- 2 sentences. What are your conclusions / implications for teaching? Please add one sentence.
“The results showed mean gains between 10.7% and 12.6% of the highest score for the TMA. This, together with the results of the questionnaire, confirms the usefulness of this teaching approach using easy-to-use 3D technologies for developing geospatial thinking.”
Introduction:
The introduction in the topic and the problem is a bit repetitive and not very clear. What is the problem with fostering student‘s understanding of different landforms? What are the goals and the research questions of the conducted study?
Effectively, the introduction was a bit confusing. I have tried to rewrite the entire introduction introducing new references to develop a clear line of research approach: Landforms / Visuospatial displays / geovisualization / software / Implications in education-spatial skills development / present my research.
Line 38: Please provide an example for „landforms“.
Done in the first phrase of the intro. “In the tasks of topographic visualization for the characterization of landforms (ridges, valleys, hills, etc) traditional cartographic…”
Lines 43-44: DTM and DEM are the same. Do you mean “Digital Surface Model”?
It was a mistake. I used the DEM acronym to avoid confusions.
Line 44: What do you mean with „image LiDAR“? LiDAR data are point clouds. For example, airborne laser scanning data (ALS) provide the basis for a digital surface model.
I added the LiDAR description: LiDAR (Laser Imaging Detection and Ranging),
Geovisualization training
The definition of spatial / geospatial thinking is very fuzzy. Please make it more clear.
In order to clarify the definition of spatial thinking I have restructured in point 2 the first 2 paragraphs, and I have included a more complete definition.
“Spatial thinking is the ability to visualize and solve problems spatially [12]. It includes the understanding of spatial information, the methods to represent geospatial information and the processes of spatial reasoning, according to the National Research Council (NRC) report, “Learning to Think Spatially” [13]. The complete definition of the concept according to the NRC, is “thinking that finds meaning in the shape, size, orientation, location, direction or trajectory of objects, processes or phenomena, or the relative positions in space of multiple objects, processes or phenomena. Spatial thinking uses the properties of space as a vehicle for structuring problems, for finding answers, and for expressing solutions”. This report has highlighted that map-reading and geotechnologies can support the process of geographical literacy and the development of spatial thinking. Spatial thinking is essential for success in university studies related to the STEM (Scientific, Technic, Engineering and Mathematics) domain [14-19]. Institutions, such as the National Science Foundation (NSF), the United States Geological Survey (USGS), the National Aeronautics and Space Administration (NASA) and the National Geographic Survey (NGS), identified spatial thinking as a crucial competence.
A subset of spatial thinking is the geospatial-thinking concept. Authors such as Huynh and Shape [20], defined geospatial thinking as a subset of spatial thinking in the context of the Earth's surface and its representations. These representations can be traditional maps or digital versions of representation of the terrestrial surface in 2 and 3 dimensions. Geospatial thinking therefore needed when using visuospatial displays geotechnologies such as GIS or Virtual Globes, to name the most common ones. Geospatial thinking is therefore aimed at the part of spatial thinking related to the ability to visualize and solve problems using maps, cartographies and geospatial information.”
Please make it more clear what geospatial thinking is aimed at.
I understand that now, with the changes made in the first two paragraphs of point 2, the concepts spatial thinking and geospatial thinking are better understood. To reinforce the geospatial thinking concept, I included at the end of the second paragraph: “Geospatial thinking is therefore aimed at the part of spatial thinking related to the ability to visualize and solve problems using maps, cartographies and geospatial information.”
Please point out what the actual problem in teaching is with regard to the suitability of tools supporting geospatial thinking.
To illustrate this I have added, in the second paragraph of point 2:
“…Geospatial thinking is therefore aimed at the part of spatial thinking related to the ability to visualize and solve problems using maps, cartographies and geospatial information. Students have difficulties in interpreting the cartographic relief, which limits the development of their geospatial thinking. These difficulties can cause problems of frustration among students who are pursuing careers in which the use of geospatial information is frequent, such as geographers, engineers and / or architects, to name a few [21]. Carbonell, Jaeger and Shipley [21] showed that the use of 3D rendering and geovisualization technologies are effective, along with the use of traditional 2D maps with contour lines, to increase student motivation in tasks related to the interpretation of landforms. The procedures for acquiring geospatial thinking through different forms of 2D and 3D representation is therefore still an active field of research addressed in recent works carried out by Carbonell [21], Collins [22] and Eynard and Bernhard [23] among others. “
Line 148: How is „geographic literacy“ defined? How can it be measured?
I added at the end of point 2: “the competence of individuals to recognize geographic space, as well as the ability to understand, process, and utilize the basic skills of geography.” Since 2010, the use of geovisualization tools in an electronic environment has been proposed to measure geographic literacy [36].”
Material and Methods
3.1 Participants:
Did the students differ in the workshops or were there also students visiting both workshops?
They were different. Explained in 3.1 point:
“These were two groups of different students, each one from a different academic year. Those from the Autodesk Workshop belonged to the 2016-17 course, and those from the Skecth Up Workshop, to the 2017-18 academic year.”
Were the students undergraduates or graduates?
Undergraduates. Indicated in the beginning of the 3.1 point.
3.3. Procedure
It is not clear whether the training on landforms (lines 191 - 194) were the pre-test.
To avoid possible confusion, I added a temporization at the end of point 3.3:
“The timing of the two workshops was in four phases. Phase 1: specific training on landforms (1 hour). Phase 2: completion of the Topographic Map Assessment Pre-test (30 minutes). Phase 3: installation of Sketch Up Make and / or Autodesk 123D Make applications (30 minutes). Phase 4: realization of the landforms (3 hours for the Sketch Up Make workshop, and 3+2 hours for the Autodesk 123D Make workshop). Phase 5: completion of the Topographic Map Assessment Post-test (30 minutes). Phases 1, 2 and 3 were carried out in a 2-hour session. The following week, phases 4 and 5 were carried out in another session of 3 and half hours, for the Sketch Up Make workshop. For the Autodesk 123D Make workshop, 2 sessions for phases 4 and 5 were needed: 3 hours in one day for phase 4, two hours and 30 min to complete phase 4 and a post test (phase 5) the next day. There were 2 teachers assigned to the students for each of the phases.”
Line 193: Please provide an example for the „specific contents related to the forms of relief“.
Now in 3.3:
“In both workshops, first participants first received specific training on the concepts of landforms and contour lines. Each of of the seven elementary landforms with which the participants were going to work in the workshop was defined: plains, elevations, depressions, ridges, valleys, hills, and cols (also called mountain passes or saddle points). These definitions were illustrated with examples visualized in 2D and 3D. The Topography and Cartography subject has specific contents related to the forms of relief. For example, to illustrate the concept of ridge, participants were shown, along with their definition, a representation with contour lines in 2D and 3D and 3D Digital Elevation Model images (Figure 1).”
Figure 1. Example of the definition of a ridge in the specific training phase on landforms. (Left) contour lines map in 2D. (Center) contour lines map in 2D. (Right) LiDAR Digital Elevation Model.
New figure 1 is added in the manuscript.
Line 212: In which format did the students deliver their results? Did they export their model?
They were asked to export their models in PDF format. Included in the paragraph.
Lines 234-235: Please provide the country where the Temple University and the Northwestern University are.
I forgot to indicate it. Thanks. Provided: USA.
Line 254: The questions are given in Table 3. Please correct.
It was a mistake. Thanks. Corrected.
Results
Line 263: How were the items grouped for the MANOVA? Please provide the information that the ANOVA assumptions are fulfilled.
Now in 4.1:
“In the SketchUp workshop, as a first approach, a MANOVA 2x2 was carried out (Male vs. Female and Pre-test vs. Post-test) for the 7 tasks (summed scores for Path, Stream/Water-flow, Slope, Visibility, Elevation Points, Photointerpretation of Relief, Profile and the Final Score). The assumptions of the General Linear model were fulfilled; in addition the analysis is robust due to equal sample sizes. The effect…”
Line 265: Why did you analyze the effect of gender? What was your hypothesis?
“In previous research on the development of the geospatial thinking capacity, the gender component has been studied. For this reason, it is included in the present analysis [29,31]. Included in the 1st paragraph of 4.1 point.
Lines 275 - 288: The data is provided in Table 2. Therefore, you can remove it in the text. Just the major results should be pointed out.
We reduced details in the text, leaving only the most important.
Line 290 - 295: As there is no significant difference between the workshops, Figure 3 might not be necessary. The data is provided in the text. - What is exactly the „Moment“? Please define.
Regarding the term “Moment” we changed it to “Pre-test vs Post-test”.
We consider that Figure 3 (now Figure 4) is important to clarify that there is a significant difference between the total Scores in both moments: Pretest and Posttest. We preferred to display this analysis by groups although there isn’t a significant difference between groups.
Figure 4: The absolute differences do not show much because the tasks comprise different amounts of items and points. Please show the relative gains (%).
Figure 4 (now Figure 5) now shows the e relative gains (%)
Lines 327-333: Cronbach‘s alpha measures whether the questions belonging to one group measured the same thing. Therefore, this is no suitable analysis between the Sketch Up and the Autodesk workshop. However, the quality of the questionnaire can be evaluated with Cronbach‘s alpha. Did the questions in one group measure all the same thing (i.e., application; improvement; understanding; implications in the learning-teaching environment)?
We repeat the analysis joining both groups (n=56). Due to the small sample size, we could not obtain adequate alphas for all subgroups of items, but for a unidimensional scale (supposing that all Items measure the same) we got an acceptable value. So we can suppose that the scale as a whole can inform us about student’s satisfaction with the tool they have used.
Discussion and Conclusions
Line 388: What is a “DE education”? Please be more specific and provide actual fields of study.
I have changed the paragraph and also I added a reference:
“This research presents a strategy of first contact with concepts related to basic landforms, and provides a teaching approach based on easy-to-use technologies (Sketch Up and Autodesk123D Make) that develop the geospatial thinking of the students. It serves as a first contact with 3D rendering technologies and contributes, in turn, to the necessary geographic literacy in the early stages of the Digital Earth education, in accordance with the stipulations of the Beijing Declaration on the Digital Earth [24]…”
Reviewer 4 Report
Comments to Authors
Although authors are aware of statistical analysis and have done nice analyses, the whole paper is vague. It is not clear what exactly authors want to measure and why. Similarly, the conclusion section doesn’t guide reader of what exactly they achieved. I have made some other comments for each section:
Abstract:
Abstract of the paper is not clear. Authors mentioned a challenge in STEM; however, the reader need to guess if the challenge is related to geospatial thinking of students or something else. I suggest clarifying it and explaining it in a sentence or 2.
No findings in the abstract: although it might be the case for some disciplines, I was expecting to read at least a couple of main points about the findings in the abstract. I suggest dedicating a couple of sentences to the findings and contribution.
Introduction
Line 40: Re-write the sentence “The potentiality of visuospatial displays using georeferenced …”
The introduction takes a rather selective and disjointed view of the relevant literature and still fails to relate the study adequately to relevant work, in my view.
2. Geovisualization training
Explains why geospatial thinking and training is necessary for STEM and in the last paragraph there is link to the paper.
Line 127: “Therefore, in the first formative …” is it what authors concluded from literature? If so, need to explain further and link it better with their research.
3.3 Procedure
I didn’t quit understand the task. So students in SketchUp session, were asked to build a terrain model. Was this terrain from a specific area or a generic terrain? Are you assessing the accuracy of terrain between the groups? (i.e. those who used CAD versus SketchUp)? What is reason and rational behind this task?
3.4 Measurement of geospatial thinking improvement
Line 230 and 231: I’m not sure if you’ve explained the pre-test and post -test before. Need to explain it in the procedure section
Table1: you’ve explained the TMA and you derived some insights from this test; however, you need to clearly explain why you are choosing this test, and also need to explain the scoring method in table 1 (Item numbers); Did you modify this scoring from the original test, what people used this test for? (although you’ve mentioned a couple of references, you need to give a structure and after reviewing the related literature justify why you’ve selected it and link it to your work)
Line 252- Please bring the questionnaire list to the appendix to help reader understand the survey better. You can also categorize questionnaire survey into 3-4 sections and explain in a sentence or 2 for each section that why you were asking these specific questions and what you were aiming to achieve from the survey.
4.1 Geospatial thinking
I am not still sure what exactly you want to measure. Please explain at the beginning of this section or under section 4-results
Table 2- need to explain the results in table 2. For example, what does it mean that Path mean was 0.88 in SketchUp group and 0.92 in Autodesk? What does gain means? Are you comparing each condition between the 2 groups?
Same for Figures 3 and 4. They need more explanations.
4.2 3D modelling questionnaire
Good that the questions are mentioned here. You can disregard my previous comment to add them to the appendix.
Line 331: re-write: “But comparing means of Items the 331 difference for Q8 is remarkable.”
5. Discussion and Conclusions
There are some serious issues with the interpretation of the results. It might be because they did not explain the procedure properly and the reader is wandering around with different numbers trying to link them to the aim of the research.
Authors need to answer such questions: whether the results was in line with previous similar studies, what are the key findings and contributions of this research.
Author Response
Reviewer 4
First of all I want to highlight the great revision work done. A thorough revision of the manuscript has been made. All the proposed recommendations make sense and undoubtedly add value to the article. I can only appreciate the great work done, thanks to which I have learned a lot.
All the changes made in the manuscript are in blue color to facilitate its location.
English language and style are fine/minor spell check required
The MDPI English Editing Service has revised the manuscript (type of service language editing and layout). English editing invoice ID: english-6815.
Comments to Authors
Although authors are aware of statistical analysis and have done nice analyses, the whole paper is vague. It is not clear what exactly authors want to measure and why. Similarly, the conclusion section doesn’t guide reader of what exactly they achieved. I have made some other comments for each section:
Abstract:
Abstract of the paper is not clear. Authors mentioned a challenge in STEM; however, the reader need to guess if the challenge is related to geospatial thinking of students or something else. I suggest clarifying it and explaining it in a sentence or 2.
No findings in the abstract: although it might be the case for some disciplines, I was expecting to read at least a couple of main points about the findings in the abstract. I suggest dedicating a couple of sentences to the findings and contribution.
Done. I have changed the abstract to focus on the development of geospatial thinking of students in higher education. I have included findings:
“Geospatial thinking is involved in the visualization-interpretation processes of three-dimensional geographic information. The design of strategies for the interpretation of the Earth´s surface, which allow for the development of students’ geospatial thinking, poses a challenge in Higher Education. In geospatial education, there is a tendency towards a lower number of necessary lectures in favor of a more practical approach, which involves specific training in GIS and/or geotechnologies. However, in the first stages of geospatial education, geographic literacy and geospatial thinking processes can be made more effective and easy to implement through easy-to-use technologies. The present research shows the results of two workshops performed with engineering students, using visuospatial displays in an easy-to-use 3D software environment. This teaching approach developed student’s geospatial thinking, measured using the Topographic Map Assessment TMA test, a battery of seven tasks related to relief interpretation, along with 18 exercises. Participants also completed a questionnaire relating to the following usability topics: operation (application), improvement, implications for education and understanding of the concepts related to relief interpretation. The results showed mean gains between 10.7% and 12.6% of the highest score for the TMA. This, together with the results of the questionnaire, confirms the usefulness of this teaching approach using easy-to-use 3D technologies for developing geospatial thinking.”
Introduction
Line 40: Re-write the sentence “The potentiality of visuospatial displays using georeferenced …”
Yes, it was too long. I have re-written: “The potentiality of visuospatial displays using georeferenced information processing applications (Geographical Information Systems, Spatial Data Infrastructures and Virtual Globes) facilitates the geovisualization of terrain shapes in three dimensions. These applications use different displays, such as Digital Elevation Models (DEM), image LiDAR (Laser Imaging Detection and Ranging), Slope Maps, and Hillshades, to name the most commonly used ones.”
The introduction takes a rather selective and disjointed view of the relevant literature and still fails to relate the study adequately to relevant work, in my view.
Effectively, the introduction was a bit confusing. I have tried to rewrite the entire introduction introducing new references to develop a clear line of research approach: Landforms / Visuospatial displays / geovisualization / software / Implications in education-spatial skills development / present my research.
2. Geovisualization training
Explains why geospatial thinking and training is necessary for STEM and in the last paragraph there is link to the paper. Line 127: “Therefore, in the first formative …” is it what authors concluded from literature? If so, need to explain further and link it better with their research.
Effectively, the idea was not well developed. I have rewritten that part again in order to clarify it. I have added, in addition, two references on the subject.
“…This practical approach could be carried out with GIS, which is a very powerful tool for the teaching and treatment of geospatial information and decision-making. However it would involve a great effort on the part of the teaching community, which would need specific training in the field of desktop GIS. In turn, great effort is needed to provide specific software for the classrooms [12]. Taking into account these difficulties, posed by technologies such as GIS, for the first formative stages in geospatial training, it is possible to consider the use of low cost geospatial technologies that are easier to implement. A similar approach, with the objective of training that develops spatial skills, was employed by Manson et al. [27], who affirm that GIS requires more time to learn the numerous commands of the application, neglecting the development of geospatial thinking. In this sense, researches carried out for the development of spatial planning with low-cost and easy-to-implement digital geotechnologies, have obtained good results [10,11]. Thus, in the present research, students performed the 3D design of elementary landforms using the proposed low-cost applications that are easy to use and implement, such as Skecth Up and Autodesk 123D Make applications.”
3.3 Procedure
I didn’t quit understand the task. So students in Sketch Up session, were asked to build a terrain model. Was this terrain from a specific area or a generic terrain? Are you assessing the accuracy of terrain between the groups? (i.e. those who used CAD versus Sketch Up)? What is reason and rational behind this task?
In order to clarify these issues I have rewritten the procedure point. In the first paragraph I included the timing of the sessions. In the second paragraph I clarified the term "specific training", adding a new figure to illustrate it (Figure 1). In the third paragraph I tried to answer your questions.
The new Procedure:
“The timing of the two workshops was in four phases. Phase 1: specific training on landforms (1 hour). Phase 2: completion of the Topographic Map Assessment Pre-test (30 minutes). Phase 3: installation of Sketch Up Make and / or Autodesk 123D Make applications (30 minutes). Phase 4: realization of the landforms (3 hours for the Sketch Up Make workshop, and 3+2 hours for the Autodesk 123D Make workshop). Phase 5: completion of the Topographic Map Assessment Post-test (30 minutes). Phases 1, 2 and 3 were carried out in a 2-hour session. The following week, phases 4 and 5 were carried out in another session of 3 and half hours, for the Sketch Up Make workshop. For the Autodesk 123D Make workshop, 2 sessions for phases 4 and 5 were needed: 3 hours in one day for phase 4, two hours and 30 min to complete phase 4 and a post test (phase 5) the next day. There were 2 teachers assigned to the students for each of the phases.
In both workshops, first participants first received specific training on the concepts of landforms and contour lines. Each of of the seven elementary landforms with which the participants were going to work in the workshop was defined: plains, elevations, depressions, ridges, valleys, hills, and cols (also called mountain passes or saddle points). These definitions were illustrated with examples visualized in 2D and 3D. The Topography and Cartography subject has specific contents related to the forms of relief. For example, to illustrate the concept of ridge, participants were shown, along with their definition, a representation with contour lines in 2D and 3D and 3D Digital Elevation Model images (Figure 1).”
Figure 1. Example of the definition of a ridge in the specific training phase on landforms. (Left) contour lines map in 2D. (Center) contour lines map in 2D. (Right) LiDAR Digital Elevation Model.
“After the training phase on landforms and their representation, students installed the Skecth Up Make 2017 and Autodesk 123D Make applications on their laptops. Once the participants had installed the applications, the instructor invited them to construct the landforms described in the training phase. The task, therefore, is the realization of seven landforms (plains, elevations, depressions, ridges, valleys, hills, and cols) using the Sketch Up workshop and Autodesk workshop. Students are not asked for a piece of land in a specific area: it is about generating a generic land. Before and after each workshop, participants took the Topographic Map Assessment test and responded to a 3D modeling questionnaire. This research does not try to evaluate the accuracy of terrain between the groups, but to verify which of the two technologies (Sketch Up Maker or Autodesk 123D Make) has a greater effect on the participants' geospatial thinking. In this way, analyzing the pre-test and post-test results of the Topographic Map Assessment test, it can be checked which technology offers a better result for the improvement of geospatial thinking. In addition, the results of the 3D modeling questionnaire will show the preferences of the students in each of the 4 analyzed aspects of the technologies used.”
3.4 Measurement of geospatial thinking improvement
Line 230 and 231: I’m not sure if you’ve explained the pre-test and post -test before. Need to explain it in the procedure section.
It is true. Done in procedure section.
Table1: you’ve explained the TMA and you derived some insights from this test; however, you need to clearly explain why you are choosing this test, and also need to explain the scoring method in table 1 (Item numbers); Did you modify this scoring from the original test, what people used this test for? (although you’ve mentioned a couple of references, you need to give a structure and after reviewing the related literature justify why you’ve selected it and link it to your work)
In order to give a structure to justify the choice of this test I have redrafted the paragraph introducing new references as well:
“There are numerous psychometric tests for the measurement of spatial skills, such as, for example, the Mental Rotation Test [39], which measures spatial rotation, or the Perspective Taking Spatial Orientation Test [40], which measures spatial orientation. For the measurement of geospatial thinking, the TMA examines a great variety of parameters related to the topographical representation of the terrain and its interpretation, and therefore, it is suitable fir this type of research, in which work on terrain representations is conducted. The TMA has been used in the subject of Cartography at the University of La Laguna since the 2014-15 academic year and has been used in other researches related to geospatial thinking and the interpretation and visualization of landforms using 2D and 3D technologies [10,11].”
I did not modify the scoring from the original test.
Line 252- Please bring the questionnaire list to the appendix to help reader understand the survey better. You can also categorize questionnaire survey into 3-4 sections and explain in a sentence or 2 for each section that why you were asking these specific questions and what you were aiming to achieve from the survey.
Ok according with your posterior comment in 4.2.
4.1 Geospatial thinking
I am not still sure what exactly you want to measure. Please explain at the beginning of this section or under section 4-results
I have added under 4-results:
“The present research offers measures on the possible improvement of the geospatial thinking skill of the participants, for which the results of the TMA (post-test minus pre-test) are analyzed. Additionally, a survey offers the opinions of the participants in the workshops.”
Table 2- need to explain the results in table 2. For example, what does it mean that Path mean was 0.88 in Sketch Up group and 0.92 in Autodesk? What does gain means? Are you comparing each condition between the 2 groups?
To clarify this I have expanded the description of table 2 explaining each of its components:
“The means and standard deviations of the different tasks (adding corresponding items) in the Pre-test and the Post-test for each group, as well as the total gains and the percentages they represented for each highest possible score, are shown in Table 2. It can be seen that both workshops produced an improvement for all tasks, except for Path. (Pre: the Pre-test score, before the workshops; Post: the Post-test score, after the workshops; gain: the Post-test score, minus the Pre-test score, for each group in each item; gain%: the % of the highest possible score). “
Table 2. TMA Means and Standard deviations by Group (SketchUp and Autodesk123) in the Pre- and Post-tests, along with the gains.
Same for Figures 3 and 4. They need more explanations.
We add more explanation for both Figures (now Figure 4 and 5) emphasizing remarkable outcomes.
4.2 3D modelling questionnaire
Good that the questions are mentioned here. You can disregard my previous comment to add them to the appendix.
Thank you
Line 331: re-write: “But comparing means of Items the 331 difference for Q8 is remarkable.”
Done (the difference is 3,14).
5. Discussion and Conclusions
There are some serious issues with the interpretation of the results. It might be because they did not explain the procedure properly and the reader is wandering around with different numbers trying to link them to the aim of the research.
Effectively, before the review was not very clear the purpose of the investigation, which hindered the interpretation of the Discussion and Conclusions point. I hope that with all the changes suggested by you and the rest of the reviewers in points 1,2 and 3, reading is easier now.
Authors need to answer such questions: whether the results was in line with previous similar studies, what are the key findings and contributions of this research.
Done. I have added in the 2nd paragraph of Discussion and Conclusions point:
“Compared with previous research: the strategies based on the joint use of augmented reality, 3D mesh processing (information represented in a 3D view on a planar surface like a cell phone screen), and 3D digital terrain model (3D printed model) obtained a gain of 4.56 points on the TMA test [11]. In that research there was a control group working with traditional 2D maps, and their gain was 1.55 points on the TMA test. Strategies based on the joint use of augmented reality and traditional 2D maps obtained a gain of 5.41 points on the TMA test [10] (in that case, the gain using only 2D strategies was 2.22 points on the TMA test). The gain obtained in the present research is lower (3.54, SD=2.65) than the previous researches mentioned (although superior to the gains obtained with 2D maps in those previous researches). In this regard, it is necessary to take into consideration that they are two different tasks: the generation of landforms used in this research and the interpretation of landforms, used in the two other two previous researches.
In relation to gender, the results obtained show that there are no significant differences between men and women regarding the improvement of geospatial thinking after training, which coincides with the results on gender obtained in previous research [29, 31].”
The key findings and contributions of this research are at the end:
“This research presents a strategy of first contact with concepts related to basic landforms, and provides a teaching approach based on easy-to-use technologies (Sketch Up and Autodesk123D Make) that develop the geospatial thinking of the students. It serves as a first contact with 3D rendering technologies and contributes, in turn, to the necessary…”
Round 2
Reviewer 3 Report
The author's revision increased the readability and clarity of the manuscript. All points that I mentioned were addressed adequately. However, the manuscript still contains some minor language errors and wrong terms, which need to be corrected.
line 48: "image LiDAR (Laser Imaging Detection and Ranging)" - Please replace this with the following: "LiDAR (Light Detection and Ranging)"
[For your further information on LiDAR you may consult the following webpage: https://oceanservice.noaa.gov/facts/lidar.html]
line 58: "[...] Grasshopper's Bison plugin; features tools for [...]" - Please provide the reference for "Grasshopper's Bison plugin". Please add "it" after the semicolon.
line 60: Please provide the references for the software packages Rhino, Vue, Terragen, ....
line 87/88: "3D three-dimensional" - This is a doubling. Please remove either 3D or three-dimensional.
line 115: Please add "is" in "Geospatial thinking [is] therefore needed [..].
line 163 and throughout the text: "Skecth Up". - The name of the software package is misspelled. Please check carefully and correct it. Also the URL contains this error (line 208).
line 243: Word doubling. Remove one "first".
line 244: Word doubling. Remove one "of".
line 323: Correct "fir" to "for".
lines 349-350: Move these two sentences to line 347 before the sentence "The effect of gender was not significant [...]".
line 437: Word doubling: Remove one "two".
line 488: Please provide references for "AR Sandbox" and "Illuminating Clay".
Author Response
The author's revision increased the readability and clarity of the manuscript. All points that I mentioned were addressed adequately. However, the manuscript still contains some minor language errors and wrong terms, which need to be corrected.
I am very grateful for your work in reviewing the article.
line 48: "image LiDAR (Laser Imaging Detection and Ranging)" - Please replace this with the following: "LiDAR (Light Detection and Ranging)"
[For your further information on LiDAR you may consult the following webpage: https://oceanservice.noaa.gov/facts/lidar.html]
Done and thak you for the link. Very interesting.
line 58: "[...] Grasshopper's Bison plugin; features tools for [...]" - Please provide the reference for "Grasshopper's Bison plugin". Please add "it" after the semicolon.
Done
line 60: Please provide the references for the software packages Rhino, Vue, Terragen, ....
line 87/88: "3D three-dimensional" - This is a doubling. Please remove either 3D or three-dimensional.
Done
line 115: Please add "is" in "Geospatial thinking [is] therefore needed [..].
Done
line 163 and throughout the text: "Skecth Up". - The name of the software package is misspelled. Please check carefully and correct it. Also the URL contains this error (line 208).
Done
line 243: Word doubling. Remove one "first".
Done
line 244: Word doubling. Remove one "of".
Done
line 323: Correct "fir" to "for".
Done
lines 349-350: Move these two sentences to line 347 before the sentence "The effect of gender was not significant [...]".
Done
line 437: Word doubling: Remove one "two".
Done
line 488: Please provide references for "AR Sandbox" and "Illuminating Clay".
Done
Reviewer 4 Report
I found a couple of minor typos- please check for typos before submitting:
Line 323: "....it is suitable fir this type of..."
Need to add references for each equipment/software that mentioned in line 475 such as AR sandbox, Maya,.. if they have really been used to enrich learning process. This can be one example for using AR sandbox, for instance: https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-4/5/2018/
Line 475-477:
"At a more advanced level, there are other advanced applications that aim at digital terrain modeling, such as AR Sandbox, ..."
Author Response
I am very grateful for your work in reviewing the article. I have corrected line 323 and have entered all the references you asked me for.
