A Project-Based Instruction Approach to Improving Student Lunar Phases Learning Outcomes: A Quantitative Inquiry

Abstract

We investigated how students’ lunar phases learning outcomes were affected by student and teacher demographic characteristics (gender, race/ethnicity, spatial thinking ability, and content knowledge). The study identified moderately strong correlations both between students’ spatial thinking ability and understanding of lunar phases, as well as between the pre-to-post-intervention scores of the two measures. Multilevel modeling showed significant predictors of learning outcomes from both student and teacher variables. This study furthers works on establishing a connection between student learning outcomes and the content knowledge and spatial ability for themselves as well as their teachers, and shows promise for a project-based instruction approach in aiding in lunar phases understanding.

1. Introduction

The Next Generation Science Standards (NGSS) [1] emphasize the need for students to use celestial objects (e.g., the Sun, Moon, and stars) to describe predictable patterns, use observations to describe patterns in the natural world, understand how the motion of the Earth and Moon with respect to the Sun is the cause for observable patterns, and develop and use models to describe, test, and predict phenomena. Research has shown that spatial thinking ability (e.g., ability to mentally rotate, revolve, and visualize objects from multiple frames of reference) is needed to understand lunar phases as well as other STEM content [2,3,4,5]. Despite its importance, developing spatial ability along with content is undervalued and rarely seen as an explicit goal in K-12 classroom instruction.

1.1. Lunar Phases Misconceptions

Though it is a phenomenon we can easily engage with and observe on a daily basis, lunar phases are rife with misconceptions that persist even into adulthood [6,7,8,9]. Three common misconceptions about the cause of lunar phases are prevalent in the literature: (1) a blocking notion where clouds, some other object, or the blackness of the sky block our view of part of the moon, causing the moon to appear different (e.g., lunar phases) from day to day; (2) an Earth’s shadow misconception, where people think the unlit portion of the moon is caused by the Earth creating a shadow on the moon, causing the visible lunar phases, and (3) a Sun’s shadow misconception, where people think the Sun itself is the cause of a shadow that results in some parts of the moon appearing lit while others are unlit [9]. Black [10] showed pre-service teachers have difficulty in explaining lunar phases, partly due to the difficulty in understanding them as observations that are conducted from a fixed earth-based perspective. Trumper [11] similarly found that in-service teachers struggled with understanding lunar phases, and Wilhelm et al., [9] further documented this problem, identifying that the middle school teachers they studied had similar misconceptions as their students; the teachers struggled with understanding the motions, scale, and geometry of the Earth/Moon/Sun system as a cause of lunar phases. These misconceptions and/or the teachers’ spatial thinking ability could impact what their students learn. Arslan and Durikan [12] found that teachers’ science backgrounds did not predict the accuracy of their mental models of lunar phases or other astronomical concepts compared to teachers with non science backgrounds; all teachers in the study possessed similar misconceptions.

1.2. Spatial Thinking in Astronomy

Research has shown students need to possess a variety of skills, such as being able to mentally visualize the rotations and revolutions associated with the celestial movement of bodies within an Earth/Moon/Sun System, to fully understand the cause of lunar phases and other astronomical phenomena [2,13]. Plummer [14] argued students’ development of accurate models for astronomical phenomena and the celestial motion of objects can actually be defined by an increasing sophistication of spatial knowledge and reasoning. Further, Plummer et al. [15] investigated 15 children between the ages of seven and nine with regards to how they made connections between frames of reference and perspective-taking skills as they related to the apparent motion of the Sun, Earth, and Moon. It was found that students with a higher ability of spatial perspective-taking skills were able to make more explicit connections between frames of reference and celestial movement.

Wilhelm [16] identified quantitative evidence supporting the notion that students’ spatial ability is related to their understanding of lunar phases, and classified four spatial-mathematical domains students need to master in order to successfully understand the phases of the Moon: Geometric Spatial Visualization (GSV), visualizing the geometric features of a system from above/below/within the system’s plane; Spatial Projection (SP), projecting to a different location and visualizing from that perspective; Cardinal Directions (CD), distinguishing directions (N, S, E, W) to document an object’s vector position in space; Periodic Patterns (PP), recognizing occurrences at regular intervals of time and/or space. Cole et al., [17] further showed that using Moon observation journals is successful in helping students create accurate mental models and improve related spatial thinking necessary for understanding the cause of lunar phases.

While there is a convincing body of evidence linking spatial thinking to performance in STEM [18,19], additional work needs to be done on ways to foster spatial reasoning for students, so that they develop necessary spatial thinking skills, thereby opening the gated STEM community to participation by a broader range of students [20].

1.3. Project-Based Instruction

Project-based instruction (PBI) is an open-ended, inquiry-based method of teaching that provides rich opportunities for creative thinking to address real-world, authentic problems while also learning the content required by the standards [21]. PBI provides opportunities for the teacher to tailor instruction to the students, providing student-centered, culturally-relevant, contextualized learning opportunities [21]. The presence of these features is dependent on having a well-designed PBI unit, as well as the pedagogical content knowledge of the teachers and the fidelity of implementation of the unit. A project is the most obvious, but not the only essential feature of a project-based unit. A driving question (DQ) is also essential; the DQ provides the focus of the unit and is also a starting point from which students can develop their own questions to pursue in their projects [21,22]. The projects should be student-driven and related to real-world, relevant phenomena. Teachers also lead benchmark lessons throughout the unit in order to address standards, as well as just-in-time lessons required by the students’ projects. Often, students can also lead these just-in-time lessons in order to foster a learning community in the classroom. The PBI unit should also include milestones [21,23], where the students not only check in with the teacher in their project work, but also share with the class, continuing to contribute to the community of learners in the classroom.

Research has shown PBI can increase the development of content knowledge and skills, as well as encourage collaboration in group-work [24]. As PBI is a student-centered approach that allows for opportunities to reflect on students’ self-learning while constructing their own projects, it has also been shown to increase motivation and self-efficacy for learning [25]. Thus, a project-based unit centered on lunar phases was developed. Related professional development for teachers in the current study interested in learning more about designing and conducting project-based instruction was conducted. The unit was later implemented by teachers recruited into this study which took advantage of a specific earth/space curriculum to serve their diverse students (see

Table 1. Astronomy unit used by teachers in this study [21].

Lesson 1	Can I see the Moon every night and why does it appear to change shape?—Students listen to the story, “Many Moons” and discuss the size, distance, and composition of the Moon as a group.
	Moon Journals—Students keep daily Moon observation journals for 5 weeks. Each day, students record the position (azimuth and altitude angle) of the Moon, sketch the shape of the Moon, and look for patterns in the appearance and position of the Moon.
Lesson 2	How do I measure the distance between objects in the sky?—Students learn to measure the distance between objects in the sky using their fists. They also use this method for estimating the position of the Moon in the sky.
Stellarium	Students observe the apparent motion of the Moon over the course of a day and compare this motion for the Northern and Southern hemispheres.
Lesson 3	How can I say where I am on the Earth?—Students explore the concepts of latitude and longitude, including discussing where these angles come from and how our position on Earth affects where we see the Sun in the sky.
Lesson 4	How can I locate things in the sky?—Students use a sky map to locate stars, planets, and constellations in the sky. They draw each of these as they see them, then students measure the angular distance between stars in the sky.
Lesson 5	Why do we have Seasons?—Students model the Seasons and discover the reasons the Earth has seasons.
Lesson 6	What can we learn by examining the Moon’s surface?—Students compare photos of the highlands and the maria to determine the relative age of each, crater density in each, and to make an inference about the early Solar System.
Lesson 7	What affects a crater’s size?—Students brainstorm variables that affect a crater’s size and then investigate one of these variables by making craters.
Lesson 8	The scaling Earth/Moon/Mars NASA Activity—Students use ratio and proportion concepts to better comprehend the size of the Universe by building a scale model of the Earth, Moon, and Mars.
Lesson 9	Moon Finale -Students use foam balls and a light to discover the Earth/Moon/Sun geometries necessary to produce the phases of the Moon. Students are asked to refer to their Moon Observation Journals to check whether their geometry matches what was observed in nature.

Reprinted with permission from Ref. [21]. Copyright 2019, Springer Nature.

). The curriculum provides students with context-based, situated learning classroom experiences and uses certain activities to help them develop and strengthen their spatial ability as well as understanding of lunar phases. Previous research has shown that PBI can aid students in successfully improving their understanding of lunar phases while also developing their spatial thinking ability [26,27]. The current study extends this prior work to model which student and teacher characteristics may be most important in predicting related student learning outcomes [28].

2. Materials and Methods

In this study, teachers were trained to implement a PBI lunar phases unit, providing students with spatially-rich, context-based, situated learning classroom experiences. The PBI unit has been designed to provide experiences with spatial activities where lunar phases can be explored through direct observations, journaling, illustrations, 2D and 3D modeling tasks, and classroom discussions. All teachers received professional development pertaining to the content and skills necessary to implement the PBI unit, and were provided classroom supplies necessary for implementing the unit.

2.1. Subjects

Research subjects in the study were sixth-grade students and their teachers from five suburban middle schools located in the south-east-central region of the US. Student demographics are summarized in

Table 2. Demographic Student Breakdown of Race/Ethnicity and Gender.

Race/Ethnicity	Boys	Girls	Total
White, Non-Hispanic	98	147	245
African American	10	20	30
African (Not American)	1	3	4
Hispanic American	9	13	22
Asian American	5	8	13
Asian (Not American)	6	8	14
Native American	3	3	6
Other	14	21	35
Declined to Answer *	16	10	26
Missing	2	2	4
Total	164	235	399

*: Treated as missing data together with other physically missing data.

. The 399 students were enrolled within one of eight, White Non-Hispanic, female teachers’ classes; the teachers implemented a seven-week PBI unit emphasizing content pertaining to earth/space Science. The unit was designed to give students experiences with spatial geometric activities where lunar phases can be explored through direct observations, journaling, illustrations, 2D and 3D modeling tasks, and classroom discussions. All teachers received professional training to implement the PBI unit.

Student demographic characteristics were considered to determine if an association exists between these characteristics and student learning outcome on their content knowledge of lunar phases at the conclusion of the PBI unit. Two instruments were used for assessment in the study: (1) Lunar Phases Concepts Inventory (LPCI) [29] for assessing content knowledge on lunar phases and (2) Purdue Spatial Visualization Test: Rot (PSVT-Rot) [30] for measuring the spatial ability to mentally rotate irregular geometric objects or shapes around multiple axes. The LPCI is a 20-question multiple choice assessment that addresses both content knowledge about lunar phases, as well as dimensions of spatial thinking in a lunar context. This test was chosen because the questions can be mapped to the GSV (Geometric Spatial Visualization), SP (Spatial Projection), CD (Cardinal Directions), and PP (Periodic Patterns) spatial domains as described in [16]. The PSVT-Rot is a 20-question multiple choice test that assesses mental rotation ability in a non-science context. Blocks are rotated about one or two axes and students are asked to identify what a block would look like if rotated in the same way as an example block. Mental rotation is a component or type of spatial thinking. This test was chosen as mental rotation is necessary for understanding the constantly moving Earth, Moon, Sun system.

The two instruments were administered to both students and their teachers before and after the implementation of the PBI. In this study, students’ LPCI performance as assessed by their LPCI total score (excluding item 8 because it is not classified in a domain) served as a measure of student learning outcomes on their content knowledge of lunar phases. Students’ PSVT-Rot total score measuring spatial ability was used as a potential predictor of that outcome. Next, also used as potential predictors were teacher domain-specific scores of LPCI (excluding the CD domain due to its low reliability as evidenced in [28]), teacher PSVT-Rot total score, as well as student demographic characteristics. While neither assessment was originally designed for use with middle school students, previously published research has shown acceptable reliability data for similar populations as in this study [28,31].

2.2. Research Questions and Measures

This study investigates how the student learning outcome on the content knowledge of lunar phases as measured by their total LPCI score is associated with their demographic characteristics, using the spatial ability of rotating irregular objects as measured by their PSVT-Rot total score, and contextual factors from their teachers. To that end, four research questions (RQs) are proposed and addressed.

RQ1: At the conclusion of the PBI, did a student with a higher LPCI total score also tend to have a higher PSVT-Rot total score?

RQ2: Before and after the implementation of the PBI, was an increase in the LPCI total score of a student associated with an increase in his or her PSVT-Rot total score, and vice versa?

RQ3: Was the PBI unit effective in improving student learning outcome of their content knowledge of lunar phases?

RQ4: Among measures of student demographic characteristics, their spatial ability of rotating irregular objects, and contextual factors from their teachers, what were those that significantly predicted their content knowledge of lunar phases?

2.3. Analysis

To address RQ1 and RQ2, a correlation analysis for repeated observations, a special multilevel model per Bakdash and Marusich [32] (p. 7), was conducted. Specifically, between- and a within-subjects correlations were computed, respectively [33,34]. A between-subjects correlation investigates whether research subjects with high LPCI total scores also tend to have high PSVT-Rot total scores, and vice versa. By contrast, a within-subjects correlation examines whether an increase in LPCI total score in one subject is associated also with an increase in PSVT-Rot total score in that same subject, and vice versa.

To address RQ3 and RQ4, a multilevel modeling analysis was conducted in a hierarchical approach where multiple multilevel models were specified and estimated sequentially to predict student LPCI performance. Specifically, a total of five models were fitted: (1) Model 1 containing only one indicator of time (Pre-PBI vs. Post-PBI), (2) Model 2 where two student demographic variables were added: Gender and Race/Ethnicity, (3) Model 3 where student PSVT-Rot total scores were entered, (4) Model 4 where the scores of three of the four spatial-mathematical domains of the teachers were added, and (5) Model 5 where the PSVT-Rot total scores of the teachers were added.

3. Results

Regarding the correlation analyses for RQ1 and RQ2, the between- and within-subjects correlations were r_between = 0.3839, p < 0.001 and r_between = 0.2617, p < 0.001. Both correlations are moderately strong [35], suggesting that at the conclusion of PBI, a student with a higher LPCI total score tended to have a higher PSVT-Rot total score, and vice versa; before and after the implementation of the PBI, an increase in a student’s LPCI total score was associated with an increase in his or her PSVT-Rot total score, and vice versa.

Table 3. A Hierarchical Approach to Multilevel Modeling of LPCI Total Score.

Predictors		Model 5	Model 4	Model 3	Model 2	Model 1
Intercept		11.3709 *	9.7803	16.6347 **	24.1146 **	22.6423 **
Time	Post-PBI	14.5466 **	14.8436 **	14.7417 **	15.2846 **	15.2846 **
	Pre-PBI	NA	NA	NA	NA	NA
Gender	Female	0.7759	0.6784	0.4418	−0.8023
	Male	NA	NA	NA	NA
Race/Ethnicity	African, Not American	2.6753	3.0506	4.2648	1.9698
	African, American	−2.9454	−2.7736	−3.6381	−4.8054 *
	Asian, Not American	0.8570	0.4685	0.1616	1.0230
	Asian American	0.0618	0.1018	1.0000	0.6599
	Hispanic American	−5.0264 *	−5.0803 *	−4.1904	−5.7146 *
	Native American	0.1980	0.0187	0.5180	−1.3557
	Other	−4.2500 *	−4.0455 *	−3.8747	−3.3469
	White	NA	NA	NA	NA
PS_overall_S		0.1935 **	0.1936 **	0.2068 **
LP_PP_T		−0.0203	−0.0202
LP_GSV_T		0.0881 **	0.0816 **
LP_SP_T		0.0340	0.0306
PS_overall_T		−0.0451
Model Fit	AIC	4891.0	4888.1	4890.6	5810.6	5854.6
	AICC	4891.1	4888.2	4890.6	5810.7	5854.6
	BIC	4898.5	4895.6	4898.1	5818.5	5862.4

**: p < 0.01 and *: p < 0.05.

contains the results of multilevel analysis, and Model 4 was retained for addressing RQ3 and RQ4 which reached the optimal values of all three overall model fit indices: AIC (Akaike information criterion), AICC (Hurvich and Tsai’s Criterion) and BIC (Bayesian information criterion). From Model 1 to Model 4, all three model fit criteria kept decreasing until they all reached the minimum at Model 4 (AIC = 4888.1, AICC = 4888.2 and BIC = 4895.6), suggesting the fit of the model to the data was improving and was optimal at Model 4. After Model 4, the three criteria started to increase, indicating a worsening model-data fit. Besides the overall fit of each model, regarding the significance of individual predictors, Models 1 through 4 were each able to add at least one predictor which was statistically significant, whereas Model 5 failed to add any statistically significant predictors.

Under Model 4, student LPCI total score significantly improved after the PBI by an average of 14.8436 points, $p<0.01$, thus supporting the effectiveness of the PBI. Next, there was no significant difference between female and male students on their LPCI performance. When it comes to race/ethnicity, students from two race/ethnicity groups (Hispanic American and Other) performed statistically significantly worse, $p<0.05$, than the white students. Also, student PSVT-Rot total score is a highly significant predictor of student LPCI total score, $p<0.01$. Finally, one of the contextual factors from teachers, teacher GSV score, provided highly significant explanatory power of student LPCI performance, $p<0.01$.

4. Discussion

The current study agrees with prior research, showing a correlation between spatial thinking ability and understanding of the cause of lunar phases (e.g., [2,31]). However, the multilevel modeling analysis appears to disagree with some prior research. While Wilhelm et al., [28] previously showed that this PBI curriculum showed promise for closing achievement gaps between race/ethnicity groups, the current study saw two race/ethnicity categories (Hispanic American and Other) performed statistically significantly worse than their white classmates. One reason for the difference between the current study and the previous study could simply be that this study disaggregated the race/ethnicity groups, while prior research only grouped students into white students or students of color due to low numbers within the self-reported race/ethnicity categories. We cannot say for sure why the students who identified as Hispanic American or Other were predicted to perform worse than their white classmates. It could be that some of these students were English learners, or had other reasons to have difficulty with the assessments. Or, perhaps other unknown factors caused the difference. As this work was quantitative in nature, we know what the model predicted, but that cannot necessarily explain why these two groups specifically are predicted to score statistically significantly worse than their white classmates. We also acknowledge that there are a small number of students who self-reported their race/ethnicity as Hispanic American (N = 22) or Other (N = 35) compared to the overall sample (N = 399) in this study, so while these two predictors were statistically significant, they may or may not be meaningfully significant. The small numbers of students in some race/ethnicity groups in this sample are a limitation of this study. Having larger groups within each race/ethnicity category besides White would add to the meaningful interpretation of a similar study in the future. Additional work should be done to further examine whether this or similar PBI curricula are helpful in addressing achievement gaps, as well as whether it is helpful for all students or only better for students of certain demographics. Particular attention should be paid to both student and teacher characteristics, as in the current study we saw the teachers’ Geometric Spatial Visualization scores significantly predicted improved learning by their students. This result makes sense, as the cause of lunar phases is the relative geometric positions of the Earth, Moon, and Sun. In order for students to effectively learn this, their teachers need to be able to understand, visualize, and communicate within this domain effectively. While it is not the only relevant spatial thinking domain, it is arguably the most important, as it is a space-based perspective [15] that must be considered separately from our earth-based vantage point. One cannot directly observe the Earth/Moon/Sun geometry directly from our earth-based perspective; rather, the GSV perspective must be developed as mental or physical models in order to make sense of this geometry as the cause of lunar phases.

Student performance on the LPCI significantly improved pre to post PBI unit. This suggests that it was effective in improving student understanding of lunar phases and their cause. This agrees with previous studies (e.g., [2,28,31]) that the unit effectively improves student performance on the LPCI. Similarly, the unit also provided opportunities for students to also improve their performance on the PSVT-Rot, which also agrees with previous research. Neither of these improvements are a surprise, as the unit was explicitly designed to foster the learning of lunar phases-related content as well as the development of spatial thinking skills. PBI units such as this one are also designed to provide opportunities for students to engage with and explore scientific phenomena in an inquiry manner. While we can suggest components of the PBI unit (as designed) that may have been especially helpful, that is an area where further investigation is needed, particularly into the ways in which the teachers understand, adapt, and implement the project-based unit. Knowing that teachers’ spatial thinking also significantly explains student performance on the LPCI post-unit, teacher spatial thinking as well as content knowledge needs to be further investigated and fostered.

5. Conclusions

The findings support the effectiveness of the PBI in improving students’ content knowledge of lunar phases, and demonstrate that there is a statistically significant, positive association between content knowledge and students’ spatial ability in rotating irregular objects. The findings also identify contextual factors from teachers (such as teachers’ content knowledge and spatial thinking ability) which contribute significantly to student content knowledge of lunar phases, thus establishing a connection between teachers’ domain-specific content knowledge and student’s understanding. Further, exploring these characteristics is important as it helps understand how to better prepare teachers to work with middle school students, such as those in this study [36]. PBI units such as the one used by the teachers in this study provide an active, engaging learning environment with challenging, relevant, and purposeful instruction. In addition to the content, teachers who are prepared to teach middle school science content should also be aware of the importance of spatial thinking in understanding astronomy content, such as the cause of lunar phases [2].