Project-Based Unit Development by Middle School Science Teachers: Investigations on Watershed Water Quality

Abstract

This case study explored changes in seven in-service middle school science teachers’ understandings of project-based learning (PBL) environments after participating in a summer institute on PBL. Of particular interest was their participation in the institute as learners in a PBL unit exploring the effect of land use on water quality in the watershed. We investigated how well teachers were able to apply their understanding of PBL as they designed their own units on water quality in their watershed. Research questions focused the study on how participation in a summer teacher institute on PBL prepared middle school teachers to describe key features of project-based learning environments, and how well they were able to incorporate these features in PBL units. Data collection included a qualitative pre/post PBL survey, teachers’ watershed units, and field notes from the institute. Findings from the pre and post survey showed that teachers demonstrated a vague understanding of essential features of PBL environments pre institute and a detailed understanding of PBL post institute. Teachers’ units varied in the degrees to which PBL features were exhibited. Strengths of the units included driving questions and benchmark lessons. Shortcomings included few opportunities for student-directed investigation of sub-driving questions.

1. Introduction

A Framework for K-12 Science Education [1] and the Next Generation Science Standards (NGSS) [2] have sparked renewed emphasis on collaborative student-centered inquiry in K-12 science education. Such experiences place students in investigative roles that organically foster the eight science and engineering practices: (1) asking questions and defining problems, (2) developing and using models, (3) planning and carrying out investigations, (4) analyzing and interpreting data, (5) using mathematics and computational thinking, (6) constructing explanations and designing solutions, (7) engaging in argument from evidence, and (8) obtaining, evaluating, and communicating information [1,2,3]. Inquiry-based pedagogical strategies such as project-based learning (PBL) [4,5,6] are well aligned with this call. PBL creates opportunities for student agency in STEM (science, technology, engineering, and mathematics) learning [7,8]. Developing PBL units around real-world issues relevant to the local community can also motivate students in their learning and doing of science [7,9,10,11].

A review of A Framework for K-12 Science Education [1] illustrates the integral nature of environmental education in K-12 science curricula [12]. Environmental concepts are embedded in several of the core ideas including LS2: Ecosystems: Interactions, Energy, and Dynamics; ESS2: Earth’s Systems; and ESS3: Earth and Human Activity. All these concepts can be studied within the context of the local community, whether that is in urban, suburban, or rural settings [13]. Further, they can be studied within well designed PBL units that engage students in authentic learning experiences about real-world environmental issues in their local communities [10,14,15]. Studying science in the context of the local environment can foster improved attitudes toward science, better ability to apply science to real-world issues, and cultivate a personal connection to the place and issue under study [10,14].

Teachers often need support in learning how to develop and implement PBL units [5,16]. Quality professional development has the potential to guide teachers in creating PBL units that focus on issues within the local community [14,17] while also scaffolding their transition from teacher-centered to student-centered inquiry-based pedagogy [18,19,20,21]. Prior research has explored the impact of such programs on K-12 science teachers’ abilities to construct and implement PBL units [16,22,23,24,25]. Common challenges reported by teachers with less than a year of experience creating PBL units include the tendency to select only the PBL features that fit their current instructional practice and omit a driving question to frame the unit. As a result, their units often are fragmented—integrating disconnected topics and skills, and lacking a clear focus [5,23,26,27,28].

Researchers [5,29,30] have called for future studies to design preservice teacher training and professional development programs where teachers are learning through PBL so that they “can grasp the pedagogical approach required in PBL”, (p. 15) [5]. They further recommend that such studies focus on the essential features of PBL, particularly the features the research literature most often reports to be missing [5] (i.e., driving questions, investigations, purposeful assessments).

The purpose of this paper is to report on findings from a case study [31] that explored how teachers were able to transfer their understandings of PBL to the development of their own PBL units after they participated in a PBL unit as learners. The focus of this study was on teachers’ abilities to construct environmental science units based on their experiences participating in a PBL summer institute. The institute immersed participants in an authentic project-based investigation on the effects of land use on water quality in a region’s watershed. The PBL activities were planned around middle school state science standards and addressed land use issues affecting the communities the teachers represented. In addition, the institute activities included embedded instruction on essential features of PBL in STEM education [6] that teachers were applying as learners in the institute. The explicit instruction on PBL was considered an essential feature in the design of the institute to develop teachers’ understanding of PBL pedagogical practices.

The terms PBL units and PBL environments are used throughout this paper. The former is the planned curriculum teachers create in preparing to implement a PBL experience with students, whereas PBL environments are learning spaces that incorporate student-, knowledge-, and community-centered learning framed around a driving question about a real-world issue relevant to the students [6]. When implemented with effective design and instruction, students have demonstrated greater academic achievement [32], deeper understanding of science concepts [11], and improved critical thinking and questioning skills [33,34].

2. Literature Review

2.1. Effect of Land Use on Water Quality as a PBL Topic

When considering topics that will engage students in learning, topics of great concern to local and national communities offer context for integrating many connections to NGSS, and to students’ life experiences [35]. Water quality is a topic of great concern to many Americans. For example, the Responsive Management National Office conducted a national telephone survey with 1014 respondents on Americans’ Knowledge of and Attitudes Toward Water and Water-related Issues [36]. Survey outcomes indicated the top three concerns Americans share include (1) health care, (2) education, and (3) water quality. More specifically, those sampled expressed concern for the availability of fresh drinking water and the healthiness of ocean environments. Although 75% of the individuals sampled used water from a municipal water system, most did not drink the water directly but used some method to filter the water. Therefore, water quality in the region’s watershed would create a place-based [37] issue that likely is of interest to the region’s students and their families. In addition, framing the project around the region’s watershed creates a shared purpose to better understand and be more informed about water quality issues in the region, as well as potential mitigation practices to create a more sustainable environment [38] that uses and draws water from the Kentucky River.

2.2. Project-Based Learning as a Conceptual Framework

The Next Generation Science Standards [2] call for student-centered instructional strategies such as project-based learning (PBL) to promote the application of scientific knowledge and problem-solving skills to real-world problems. PBL environments have several fundamental features [5,6,39]. Wilhelm et al. [6] identifies that the following key features are instrumental towards the successful design of a PBL environment:

PBL Key Features

• Teacher/researcher selects driving question.
• Students select sub-driving question.
• Students have opportunities to develop their background knowledge and confront or disrupt misconceptions.
• Students, teachers, researchers, “expert” members of the community collaborate.
• Students use benchmark activities and technological tools to scaffold conceptual understanding, and to assist with research, data collection, data analysis, feedback, and communication.
• Students are given ample feedback and time for revisions (via project milestones).
• Students create an end artifact, which relates to their initial sub-driving question.
• Students share their learned experiences to a community of learners which include their parents/guardians.

The most essential of these features are teacher-generated driving questions and student-generated sub-driving questions [6], two features the research literature often reports missing from teachers’ PBL units [5,25]. These two features serve to frame and organize the instructional tasks and activities and connect them to district and/or national standards. The driving question should be derived from national, state, or district standards and should provide the opportunity for students to generate a multitude of investigable sub-driving questions [6]. According to Krajcik and Czerniak [40] and Wilhelm et al. [6], the driving question should also be feasible (i.e., an investigable question), worthwhile, contextual, meaningful, and sustainable (i.e., should not be a simple googleable question).

Benchmark lessons [6,40] are used to develop students’ background knowledge and skills and advance project work, as well as to confront and dispel content misconceptions. Collaborations among students, teachers, researchers, and expert members of the community (including parents/guardians) develop a rich learning community to explore their sub-driving question. Students use benchmark activities and technological tools to build understanding and assist them in research, data collection, data analysis, feedback to peers, and communication. Through the learning process, students create a series of artifacts and products that address the problem or question, and end artifacts that relate to their initial sub-driving question and findings from their own research. Throughout the learning experiences, students are given time for feedback from teachers and peers, revision of artifacts, and time to share their learning experiences with peers via milestone checkpoints [41]. Learning environments created through PBL foster authentic contexts that both motivate and engage learners in an active, authentic scientific investigations [6,42].

Although the teacher should not be expected to know all the content and specific paths to students’ final project outcomes, they should know how to seek the assistance of subject matter experts within the local and/or online community. Subject matter knowledge does have a direct effect on teachers’ pedagogical content knowledge and their abilities to effectively teach science [43] and effectively implement PBL [5]. Subject matter knowledge plays an integral role in what teachers select to teach, how they organize it, and how they present it to their students [19,44,45]. Research has also revealed that teachers’ depth of subject matter knowledge affects the level of questions they pose to students [46], their ability to construct explanations in response to student queries [47], and the level of activities they design to support student learning [47]. By opening the classroom doors to outside experts, teachers will learn more ways to effectively illustrate to students how they also are learners, as well as how to communicate and collaborate with experts in the field, and how to contribute to the scientific community.

2.3. Teacher Professional Development for 21st Century Teaching

Carefully designed professional development programs have been effective in improving teachers’ science content knowledge and pedagogical practices [18,19,20,21]. Six key features of effective professional development have been reported in large national studies. These include (1) immersing participants in inquiry experiences [18,19,20], (2) intensive and sustained activities [18,19,20], (3) engage teachers in concrete, active learning experiences that build on their prior knowledge [18,19,20], (4) have professional development span a longer period (35 or more hours) [18,19,20], (5) address disciplinary content knowledge for teachers that is directly related to district and state standards [18,19,20,21], and (6) provide opportunities for teacher collaboration [18,19,20]. Other effective features include sufficient time for teachers to understand instructional practices so they are able to compare new practices with their existing beliefs [18,20]. Additional effective strategies include teacher support in the classroom, development of partnership collaborations, and extension of longer than one week [48,49].

The purpose of this paper is to report on changes teachers demonstrated in their knowledge and understanding of project-based learning environments and their abilities to create PBL units on water quality that reflect authentic student investigatory experiences. Questions guiding this study were:

1. 

What changes do middle school science teachers demonstrate in their understandings of essential features for creating successful PBL environments after experiencing a summer PBL institute?

2. 

How well are in-service middle school science teachers able to develop their own PBL units on water quality in their region’s watershed after experiencing professional development through a summer institute?

3. Methods

This case study [31] was designed to explore how well teachers were able to apply their understandings of PBL as an instructional model to create their own PBL units where students could investigate water quality issues within their own local watersheds.

The case study method was selected because of the “boundedness” of the system [31] (p. 2) created within the institute model and its participants. The institute members included the designers and instructors, researchers, expert collaborators, and participating teachers. Further, the purposeful use of a major watershed basin within the region also created another component of the system, which served to frame the PBL model incorporated into the institute and the recruitment of teacher participants. The diversity of school locations (rural, suburban, urban) represented by teachers participating in the study and the inclusion of teacher teams and single teachers from different school districts added enriched contexts to explore collaboration that occurred between teachers within a school and across school systems.

3.1. Participants

Seven teachers from four different school districts across the region’s watershed participated in the study. Teachers were self-selected based on the criteria that they (1) taught middle school science (grades 6–8) and (2) were interested in creating PBL units that would explore water quality issues in their local watersheds. In some instances, principals suggested that teachers attend the institute, but even so, teachers that participated had a high interest in learning to create PBL units on concepts related to ecology and/or watershed ecology. The teachers were all female with a range of experience levels of teaching science, from one year to 28 years with a mean of 12 years of experience. All teachers were certified to teach science at the grade level they taught.

Table 1. Teacher Participants, their Backgrounds, and Areas of Certification.

Teacher	School	Years of Experience	Grade Taught	Education	Certification
Catie	A	28	6 (science)	BS, elementary ed.; MA, Elementary Education; Rank I	Elementary (Grades 1–8)
Jeanette	A	11	7 (science)	BA, MS, National Board Certification	Middle School Math & Science (Grades 5–9)
Lori	A	7	8 (science)	BS, Geography, Minor: Environ. Analysis & GIS application; MAT Earth & Space Science Educ.	Secondary Earth & Space Science
Karen	B	9	7 (science)	BA, Middle School Math & Science Education; MA, Teacher Leadership—Gifted	Middle School Math & Science (Grades 5–9)
Jacquie	B	2	7 (science)	BS, Animal Science, Post bac, Middle School Sci. Education	Middle School Science (Grades 5–9)
Wynne	C	11	7 (variable topics class)	BA, Biology, Chemistry Minor, MA, Education	Middle School Math & Science (Grades 5–9)
Lena	D	15	7 (science)	BS, Middle School Math & Science; MS, Library Sci.	Middle School Math & Science (Grades 5–9)

identifies the teachers by a pseudonym, their years of experience, and their areas of certification. Three of the teachers taught at School A (Cathy, Jeanette, and Lori), two teachers taught at School B (Karen and Jacquie), and one teacher each taught at Schools C (Wynne) and D (Lena).

3.2. The Summer Teacher Institute

For this current study, researchers at a large southeastern university developed a professional development summer institute with follow-up Saturday sessions across the academic year to improve middle school science teachers’ abilities to develop and implement engaging project-based learning (PBL) units about water quality in their local watersheds. The summer institute was designed using criteria for highly effective professional development for 21st Century teaching [18,19,20]. The central goal was to improve middle school teachers’ competence and confidence in teaching and applying NGSS [3] and project-based learning in ways that support authentic student investigations of water quality in the region’s watershed. A secondary goal was to introduce teachers to the watershed in the region and to learn how geology, topography, land use, and other human factors can influence water quality. This topic also was selected based on a survey across the region that found many citizens were unaware of watershed issues but were concerned about water quality.

The summer institute was designed around three major features. These included: (1) model a PBL unit on land use issues affecting the large watershed in the region, (2) embed instruction on the essential features of PBL for successful STEM environments [5,7,40], and (3) support teachers in designing and implementing PBL units in their classrooms. The summer institute spanned 40 h across five days with four six-hour follow-up sessions scheduled across the academic year. In total, teachers participated in the institute and follow-up sessions for approximately 64 h with additional hours allotted to preparing and teaching their units. Teachers had completed two of the six follow up sessions at the collection of the units. Teachers participating in the project were all from school districts within the region’s watershed. The PBL environment designed for the institute followed Wilhelm et al.’s [6] and Krajcik and Czerniak’s [42] models of PBL. Key PBL features incorporated into the institute model included (a) a driving question, (b) benchmark lessons, (c) formative assessments referred to as milestones [7], and (d) student-derived sub-driving questions that guided teachers’ own investigations [6,42]. In addition, embedded PBL sessions were integrated throughout the week to introduce the key features of PBL noted above, to scaffold teachers in developing and applying their learning to the activities in the institute, and later, to designing their own units.

The driving question for the PBL model asked, “How does land use affect water quality throughout the Kentucky River Watershed?” Benchmark lessons were created to address watershed ecology and water quality testing and analysis, and to introduce the essential components of PBL for STEM environments [6]. Approximately five hours a day were focused on watershed ecology, types of land use within the watershed and its effects on water quality, and water quality testing experiences in the field. Another two hours each day focused on benchmark lessons on PBL features and application of those features in the institute PBL model. One day of the institute was spent collecting data from several sites in the watershed. Discussions to facilitated teachers’ thinking about data collection and analysis, meaning of analyzed data in the context of the driving question, and teachers’ sub-driving questions composed the remaining time of the institute.

Benchmark lessons designed to build teachers’ content knowledge and skills on watershed ecology and water quality testing and analysis were led by experts from the field. This strategy was created to introduce teachers to potential resources they could use in developing their own units as they learned about the Kentucky River Watershed and sampling and analytic techniques used in water quality testing. The inclusion of the experts also demonstrated ways to incorporate community partners within the PBL unit model [6,42]. Benchmark lessons were sequenced to introduce teachers to a watershed and then develop techniques used to collect and analyze water quality sampling data to assess the quality of specific sites within the watershed. In addition, benchmark lessons guided teachers in extrapolating the water quality in the local and regional watershed. Activities incorporated into the benchmark lessons included an introduction to watersheds through hands-on modeling activities (for a similar example, see https://serc.carleton.edu/eslabs/drought/2a.html accessed on 7 August 2022) and mapping activities using Kentucky Geological survey maps, Google Earth, and ARC GIS (https://www.esri.com/en-us/arcgis/products/arcgis-online/overview accessed on 12 July 2022). Additional activities taught teachers water quality testing techniques and how to identify and catalog macroinvertebrates found at water testing sites as another source for assessing water quality. Field trips to five sites across a 60-mile length of the region’s watershed (spanning 263 miles) created opportunities to practice water quality testing in the field and collect data across the five pre-selected locations within the watershed. Other benchmark lessons included analyzing water quality data, drawing preliminary conclusions, and then comparing water quality data teachers collected to published data that had been collected by the Kentucky Watershed Watch network (https://www.kywater.org/home accessed on 7 August 2022), a non-profit organization that collects water quality data across the state. Additional benchmark lessons addressed the geology and topography of the region and a habitat assessment of a location in the watershed.

Benchmark lessons on essential features of PBL were embedded across four days of the institute. Lessons were designed to introduce a key PBL feature and then have teachers work with data and watershed content to create and analyze the features and discuss application of the features within PBL and within the institute specifically. At the end of the institute teachers created unit outlines including the key features that they continued to develop in the fall during the academic year in follow-up workshop sessions.

An example of a PBL feature teachers organically developed within the summer institute was their own sub-driving questions that they constructed from field experiences. After analyzing data collected from their field work and then comparing the results to previously collected data from the region (Kentucky Water Watch network, https://www.kywater.org/home accessed on 12 July 2022), teachers constructed sub-driving questions about land use and its potential effects on water quality in the Kentucky River watershed. Some of these questions included: How do we measure the quality of water in nature? How do stream ecosystems respond physically, biologically, and chemically to land use, land cover, and geomorphology? How is water quality affected by urbanization? How do stream ecosystems differ in various locations within the Kentucky river watershed?

The summer institute benchmark lessons guided the teachers through creating, modifying, and evaluating sub-driving questions that could be used to guide their own research studies. Working in teams of two and three, teachers selected one sub-driving question to modify to fit their area of interest. Many of the groups pursued questions they could use in their own PBL instructional units, such as, ‘How does water quality in our neighborhood stream compare with a remediated stream in the same watershed?’ They then planned and carried out an investigation guided by their question. They used paper maps from the state geological survey, Google Earth, and ARC GIS (https://www.arcgis.com/index.html accessed 12 July 2022) to study land use in the local region within a 30-min drive from the institute location. They also used Google Earth and ARC GIS to study waterways within the region and to identify testing sites where they later collected data and then posted their results to Google Earth for comparison across the teacher teams.

Assessments in the form of milestones were integrated into the institute to provide feedback to teachers across the entire PBL unit. Milestones were also planned to provide feedback to teacher teams at critical points in planning their investigations. These included short presentations on (a) research questions, (b) research designs, (c) site selection, (d) data collection procedures, (e) preliminary analysis results, and (f) sharing data points added to Google Earth. The institute culminated with teacher teams presenting their study findings to a panel of experts, and time for teachers to collaborate as they planned their own PBL watershed units.

Follow-up work sessions were scheduled during the academic year to support teachers in creating their PBL units. At these meetings, teachers were provided examples of previous PBL units that were not on watershed topics, samples of state level standards for grades 6–12 related land use and water quality, and a review of milestones and assessment techniques for PBL. Teachers also were provided a template to standardize the format of their units. The template included space for the driving question, benchmark lessons, sub-driving questions, student-led investigations, and milestones. In addition, teachers shared their ideas and had time to work collaboratively on unit planning during each session.

3.3. Data Collection

This case study [31] incorporated a pre-post design to investigate changes in teachers’ knowledge and skills about PBL. The units that teachers created were used to characterize their ability to design a PBL watershed unit for their own students. In addition, field notes were used to characterize events in the institute and teachers’ discussions framed around materials they selected to include in their units.

A PBL survey was administered pre and post to gauge teachers’ knowledge of project-based learning. The questions were designed to assess how teachers defined PBL, their perceived use of PBL in instruction, and the type of assessment tasks applicable within a PBL environment. The questions were designed by the second author based on years of experience working with teachers in designing PBL instruction and conducting research on PBL in K-12 schools. The survey was also derived from key PBL features as cited in the literature [6,15,42]. The PBL survey was used to measure changes in teachers’ knowledge after the institute. The unit outlines teachers developed during the institute were collected as artifacts. Their full units were collected early in the fall at the end of the first follow-up PD meeting.

In addition, qualitative data were collected to document the institute and teachers’ abilities to construct PBL units following their experiences in benchmark lessons and their own investigations. Field notes were recorded during the institute to document benchmark lessons, key ideas teachers shared during the institute. The outlines teachers constructed for their units along with any additional materials they created were collected from teachers in early spring the following academic year.

3.4. Data Analysis

A constant comparative method [50] was used to analyze data from the open response to the pre and post PBL survey to demarcate how teachers defined PBL, and what structural elements and assessments they associated with PBL. Pre and post responses were compared to characterize changes in their understandings.

Krajcik and Czerniak’s Project-based Science Unit Assessment Rubric [42] (pp. 367–372) was used to analyze the units (see

Table A1. Comparison of Pre- and Post- Outcomes from PBL Summer Watershed Institute.

Day	PBL Element	Description
0	Pre-testing	Pre-testing completed online prior to institute.
1	Morning: Introduction to PBL, watersheds, and chemical water quality testing techniques Afternoon: Geomorphology of watershed basin	Welcome, presentations and hands-on activities to introduce watersheds and water quality testing techniques Lecture by expert presenter
1 continued	Water sampling field trip Analyzing and making sense of water quality data	Collated data, analyzed and discussed class results
2	Field trip all day Water quality testing, invertebrate sampling End of day discussion	Water quality data collection from two sites (i.e., pH, temperature, dissolved oxygen, conductivity, flow rate, turbidity, heavy metals, riparian zones; invertebrate sampling); short discussion on data collected and regional geology and topography
3	Morning: Debrief field trip Analyze field trip data Compare data collected from multiple sites Discuss findings Field trip 3 — local site data collection Afternoon: Review of NGSS	Analysis compared pH, dissolved oxygen, turbidity, heavy metals, riparian zones, temperature, invertebrate samples NGSS connections to PD PBL unit (PE, DCI, CCC, SEP)
3 cont’d	PBL mini lectures and student-centered application activities Homework: Teacher teams draft research plans for one of the sub-driving questions	Introduction to benchmark lessons (BL); identify BL in institute; use state-wide water quality data to derive sub-driving questions
4	Morning: Field trip: Habitat assessment of local stream Afternoon: Integrated PBL Milestones Teacher teams draft research plan for a sub-driving question	Under direction of field expert, teachers conduct assessment of local stream Milestones for feedback and support for teacher teams’ research studies; conduct studies in the field; analyze data to evaluate stream water quality; compare collected data to Water Watch data collected during year from same streams.
5	Morning: Milestones on data analysis and presentation Teacher team presentations Afternoon: Teachers work on PBL unit outlines Post-test and workshop evaluation survey	Data collection feedback Present studies to panel of experts Teachers work with experts to design PBL unit outline (continue Sept. and Oct. in Saturday follow-up sessions)

). The rubric was selected because of its comprehensive coverage of PBL indicators and the use of Krajcik and Czerniak’s [42] project-based science model that was combined with Wilhelm et al.’s [6] to develop the unit in which teachers participated during the summer institute. The rubric is organized around five key features of PBL [42]: (1) driving question, (2) scientific investigation (benchmark lesson), (3) technology incorporation, (4) collaborative opportunities, and (5) assessment techniques. Further, driving questions was subdivided into six sub-indicators: (a) feasible, (b) worthwhile, (c) contextualized, (d) meaningful, (e) ethical, and (f) sustainable. Likewise, the scientific investigations section was subdivided into two parts that were closely connected to the science and engineering practices in NGSS [2]. Specific indicators are included for each PBL feature to evaluate the level that the unit fits the PBL feature (see

Table 2. Characteristics of Essential PBL Features.

Essential PBL Feature	Characteristics of PBL Features
Driving Question	Characteristics of driving question include: Feasibility: developmentally appropriate, room for students to plan and conduct own investigations Worthwhile: reflects work of scientists, connects to standards and across NGSS dimensions (i.e., disciplinary core ideas, crosscutting concepts, science and engineering practices) Contextualized: anchored in real world issues and consequences Meaningful: phenomena are meaningful and important to learners; connects to learners’ everyday lives, reality, and culture Ethical: Answering the question will not harm organisms or environments Sustainable: allows for pursuit of questions over time and in great detail
Scientific Investigation (Scientific Investigation I in current study)	Unit includes evidence of opportunity for student to: engage with phenomena and explore ideas hypothesize about phenomena make predictions about investigation results find information to guide planning and conducting investigations design investigation procedures carry out and refine procedures develop and revise models based on evidence develop and revise explanations based on evidence and reasoning
Scientific Investigation (Scientific Investigation II in current study)	Unit includes evidence of opportunity for students to: transform and analyze data make claims based upon evidence and reasoning develop scientific explanations using claims, evidence, and reasoning share ideas with others move to the next round of investigation based on things they have learned link explanations with science concepts
Technology Incorporation	Unit includes evidence showing: technology is used in such a way that it transforms teaching practices. technology supports students’ scientific investigation. technology supports students’ exploration of scientific ideas. technology use reflects authentic practice (mirrors work of scientists). use of technology extends the boundaries of the classroom (experiences outside of school). use of technology helps students enhance their understanding of complex, abstract ideas. technology enables students to explore phenomena inaccessible by other means.
Collaborative Opportunities	Unit includes evidence showing opportunities for: students to generate ideas, questions, conjectures intellectual rigor, constructive criticism, challenging of ideas sharing diverse viewpoints student collaborations with more knowledgeable community members (e.g., scientists, industry professionals, etc.)
Assessment Techniques	Unit includes assessment that: measure learning performance expectations. includes final products that are tangible and represent a response to the driving question. is embedded, and reflect a continuous process, and is responsive to context is multidimensional. engages students in the assessment process. encourages clarifying, extending, and measuring students’ understanding of core science ideas, crosscutting concepts, science and engineering practices; and reflection on thinking and building metacognitive skills aligns with current educational goals and accommodates cultural diversity. is consistent with learning theory

Note: PBL features and characteristics were taken from the Project Based Science Unit Assessment Rubric. See Krajick and Czerniak [42] (pp. 367–372) for more details.

for a summary of indicators for each PBL feature). Each indicator is evaluated on a 5-point scale where 1 is not at all reflective of PBL and 5 indicates reflective of PBL to a great extent. The outcomes of the analysis are then compared with the unit as a whole to determine a capsule rating that reflects the fitness of the unit to a PBL instructional model based on a similar five-level scale where the lowest level is “far removed from project-based science” and the highest level is “exemplary PBL”, [42] (p. 373).

Several modifications were made to the rubric for use in the analysis (Table 2). First, the scientific investigations section was divided into two sections: Scientific Investigation I (i.e., phenomena, asking, and research design) and Scientific Investigation II (data analysis, results, and reporting). More specifically, Scientific Investigation I focused on students exploring phenomena, asking and refining questions, constructing hypotheses and predictions, planning and refining investigative procedures, developing and revising models based on evidence, and developing and revising explanations from their investigations. Scientific Investigation II emphasized students engaging in data manipulation, transforming and analyzing data, constructing claims based on evidence and reasoning, linking claims with scientific concepts, and opportunities for iteration of student projects so they can apply what they have learned.

Two other modifications were made to the rubric. One of these was the decision to omit the technology section because the summer institute did not focus on exploring how technology could be used to transform teaching practices (a major focus of indicators in the rubric) but rather implemented technology tools (i.e., pH indicator, GPS cameras, Google Earth, ARC GIS, electronic spreadsheets) seamlessly for data collection, analysis, communicating ideas and results, and collaborating with others.

While these strategies can transform teaching practices and support scientific investigation, the research team decided to focus analysis on the essential features of PBL [6], including driving questions, sub-driving questions, student-led investigations, and artifacts and authentic assessments in teachers’ first attempt to create PBL Units.

The other modification was using a three-point scale rather than the 5-point scale used in the Krajcik and Czerniak [42] rubric. On the 3-point scale, a 1 represented “not at all reflective of PBL features” and 3 represented “reflective of PBL to a great extent”. Researchers used a 0 to denote “no information available in a unit to make a judgement”.

A team of three researchers met to review the rubric features and to collaboratively score a unit. Once they reached consensus on scoring the unit, they scored each unit independently. An iterative process of independent rescoring and meeting to discuss discrepancies continued until the team’s inter-rater reliability score reached a minimum of 80%. Areas of discrepancy were discussed until agreement could be reached. In some cases, agreement was made by negotiating an average of all the ratings.

4. Results

4.1. Pre and Post PBL Survey

Pre and post results from the PBL survey (

Table 3. Pre and Post PBL Survey Questions.

PBL Survey Questions
1. What is project-based learning? What key features are necessary for effective implementation of creating a project-based environment?
2. To what extent do you feel you have utilized a project-based approach in your own classroom practice? Please describe your implementation of any kind of projects with your students.
3. What are the costs–benefits for utilizing a project-based approach within a science classroom?

) responses indicated clear changes in teachers’ definitions of PBL environments and the features inherent to PBL units.

Initial data collected from teachers prior to the beginning of the institute indicated they had a vague understanding of PBL and designing PBL units. In terms of key features of PBL, pre-responses included: students should have authentic experiences to promote hands-on inquiry skills, the teaching should be student-centered, and funding should be provided for materials and field trips. Only one pre-response mentioned the need for a driving question. Some examples of the pre-responses are shown below. Key features teachers identified in the pre-survey are highlighted in bold text.

• The learning is student-centered, rather than teacher-centered.
• It should provide authentic experiences on which the student can hang information that they already have on to new information they acquire to make sense of it. It provides choices for the student to demonstrate their learning in the end. It also provides a variety of experiences to appeal to different learning styles. I can see it as organized chaos.
• Key features necessary for implementation are a prepared teacher, supportive colleagues/administrators, funding for necessary materials and field trips, and students who are willing to actively participate.
• Key features include units developed to promote the hands-on inquiry skills for learning and the use of sufficient materials/tools. The learning is student-centered, rather than teacher-centered.
• A way in which students gain knowledge over a period of time. A driving question or problem, and students are able to make choices about the products that they are going to make.

Post-institute responses were more explicit and detailed in terms of key PBL features. The responses described units that were organized by a driving question, benchmark activities or lessons (to develop student skills and knowledge), and student-developed sub-driving questions for which they collect data to answer in the form of a product or artifact. In addition, four responses specifically named use of milestone assessments and/or formative assessments throughout the process. Representative samples of post survey responses are presented below with essential features of PBL teachers identified highlighted in bold text. In addition,

Table 4. Pre and Post PBL Survey Results of Essential PBL Features Teachers Identified.

Key PBL Features Identified in Survey Responses	Pre Number of Teachers (n)	Pre %	Post Number of Teachers (n)	Post %
Overall guiding/driving question	1	14.3%	7	100%
Benchmark lessons	0	0%	6	85.7%
Sub-driving questions	0	0%	5	71.4%
Assessments aligned to standards	0	0%	4	57.1%
Field Studies/data collecting	1	14.3%	3	42.9%
Presentation of project	1	14.3%	3	42.9%
Teacher experienced in facilitating	0	0%	3	42.9%
Draw conclusions from collected data	0	0%	2	28.6%
Peer feedback (e.g., milestones)	0	0%	2	28.6%
Student driven Inquiry	1	14.3%	2	28.6%
Collaborative Discussions	0	0%	1	14.3%

details pre and post PBL features described in the survey responses and the percentage of teachers identifying each feature.

• Project-based learning is a system developed to get students to take more responsibility for their education and learning. Students generate a question they want to answer and then, with the help of their teacher and expert guides, use a logical approach to answer their question, including the classroom community/access to experts/resources.
• Students learn through participation in which students create projects/draw conclusions from collected data. Key features include an overall guided question, sub-driving questions created by students, benchmark lessons, assessments aligned to standards, field studies/data collecting, final presentations of project.
• Project-based learning involves developing an overarching driving question for the unit/topic to be studied. The question should be broad enough to allow for student choice and variability. The driving question leads to benchmark lessons that all students participate in before they develop their own sub-driving research questions. Once students have developed their own sub-driving questions, they complete the research and present the data in some format.
• PBL is a learning process in which students investigate a question via inquiry, with benchmark lessons and milestones along the way. Key features: driving and sub-driving question, teacher experienced in facilitating, benchmark lessons, milestone assessments, peer feedback, and students’ presentation of a final project to communicate students’ findings and knowledge on the subject.

After experiencing the summer institute, teachers demonstrated changes in their thinking about whether they had previously used PBL in their instruction. By the post survey (Table 4), only two science teachers indicated that they felt they had used project-based learning in their past classrooms (28.6%), while five teachers (71.4%) had either never used project-based learning in the classroom or did not use the full model. Teachers reported having used guided instruction, lab activities, and inquiry lessons with few opportunities for students to choose their own questions to investigate.

When weighing the costs–benefits of utilizing a project-based approach in their science classrooms, teachers generated the following list of benefits: problem-solving applications, real-world skills, student-centered, cooperative skill development, reusable materials, career prep, meaningful and relevant learning, authentic, hands-on, development of critical and creative thinking. The costs of PBL listed by teachers included: takes time, takes materials, can be costly and difficult to manage the class, and takes teacher effort.

Analysis of post-survey data revealed marked changes in teachers’ definitions of PBL units. Further, unit outlines teachers developed at the end of the summer institute illustrated their use of a driving question to frame their units, and selection of benchmark activities to prepare students to pursue their own research questions. These PBL components and other features teachers included in their unit designs are discussed in the next section.

4.2. Teachers’ PBL Unit Designs

Teachers’ units demonstrated some of the PBL features they identified in the PBL post-survey. Of the units submitted by the seven teachers participating in the study, six units were complete, and one was partially complete; the latter was missing the assessment activities.

Table 5. Summary of Outcomes for the Teachers’ Individual Units and all Units Combined.

PBL Indicator Mean	Catie	Jeanette	Lori	Jacquie	Karen	Wynne	Lena	Overall Mean
Driving Question	2.9	2.8	2.5	2.5	2.7	2.6	2.8	2.7
Benchmark Activities linked to Driving Question	3.0	3.0	2.7	2.0	2.5	2.0	2.5	2.5
Scientific Investigation I	2.0	1.3	2.0	1.8	0.5	1.3	0.9	1.4
Scientific Investigation II	2.0	1.6	2.2	1.6	2.4	0.0	0.8	1.6
Collaboration Opportunities	2.0	1.7	1.9	1.6	1.6	1.2	1.4	1.7
Artifacts and Assessments	1.8	1.7	1.7	1.5	1.8	0.0	1.3	1.4
Overall Capsule Evaluation of unit	2.0	2.0	2.0	1.0	2.0	2.0	2.0	1.9

summarizes the outcomes from the analysis of teachers’ units using the Project Based Science Unit Assessment Rubric [42] (p. 367).

All units were designed around a driving question that focused on water quality in the watershed. Teachers created sub-driving questions to frame the unit around the driving question and to guide each lesson. Each sub-driving question connected to one or two lessons that focused on an aspect of watersheds, water quality, or techniques and tests students would learn to perform in preparation for a study of a local stream system. Six of the seven teachers planned to take their students to a nearby stream using funding for field trips provided by the grant project. Authentic artifacts described in some of the units included final in-class presentations where students would report their findings from stream study investigations, letters written to a local farmer about strategies for improving the water quality of the steam running through his farm, a final written report on a water quality issue, a Piktochart (see https://piktochart.com accessed 7 August 2022) describing one water quality issue and what the public can do to mitigate it, and a wall mural of the local stream ecosystem food web developed from student reports on organisms that live there.

The level of student autonomy created within the units varied from teacher-directed assignments to independent student work. The unit Catie, the sixth grade teacher from School A, designed was the most comprehensive. She designed the unit around the driving question, “How does land use impact water quality and aquatic ecosystems around my school?”. Teacher-created sub-driving questions were used to outline the unit and focus students’ attention on specific topics. Activities placed early in the units included locating the school on Google Earth and identifying land use around the school on low-tech state Geological Survey maps. Another activity introduced watersheds using a 3D watershed model in which a watershed is created in a box or pan by crumpling paper to create landforms. The paper is then covered with one sheet of aluminum foil and molded to create peaks, hills, and valleys. Water tinted with food coloring is then dripped onto the highest peak of the watershed. Rivulets of the dyed water flow down the peaks creating streams and pooling in cervices to create lakes or travel down the watershed to create streams and rivers. Catie also planned for students to go outside and draw the local area from each of the cardinal direction points, noting the topography of the land and changes in the riparian zone along the local stream. Later in the unit, a classroom activity investigated the effect of temperature on dissolved oxygen in water. The activity incorporated skills students would later apply as they conducted a field study of a local stream. She followed the stream study with a teacher-led activity on analyzing the data and discussing the results. The final artifact was a letter that students would write to the farmer, on whose farm the field study was to be conducted, to recommend modifications to the riparian zone along the stream to improve the water quality there.

In the following sections, we look more closely at each of essential PBL features identified in the analysis. Essential PBL features presented include: (1) driving questions; (2) sub-driving questions; (3) benchmark lessons; (4) scientific investigations; (5) opportunities for collaboration; and (6) milestones, assessments, and student artifacts.

4.2.1. Driving Questions in PBL Units

Driving questions are the central organizing feature of PBL and set the stage for all the activities and investigations included in a unit [6,39]. Each of the seven units was guided by a driving question. The average evaluation of the driving questions across the seven units was 2.74 on the 3-point scale, a score that indicates the questions were reflective of PBL, since a level of 3 represents “reflective of PBL to a great extent.” The mean scores for the driving question from each individual unit ranged from 2.5 to 2.8. All seven of the units were planned around one or two driving questions that fit many of the indicators addressed in the unit assessment rubric [39]. The questions were feasible in that they were developmentally appropriate; relevant to students; rich enough to lead to other questions; and broad enough to offer opportunities for students to design and conduct investigations using materials identified in the units. The driving questions were worthwhile in that they related to some extent to questions scientists would ask and were anchored to the real-world issue of land use and its impact on water quality locally and regionally. Moreover, the driving questions had the potential to sustain investigations across the year, and year after year. In five of the units, the driving question focused on the effect of land use on water quality in the watershed. The remaining two units addressed how students’ actions might affect water quality in their local stream and how their locale is affected by the Kentucky River watershed. Limitations of the driving questions included incomplete framing of a question and a focus on sub-driving questions teachers constructed rather than questions generated by students. It also should be noted that an intended outcome of the summer institute was that teachers’ units would in some way connect to the topic addressed in the institute PBL unit: land use and its effect on the region’s watershed,

Designing a driving question that addressed the standard they identified for their units was a challenge for several of the teachers. For example, Jacquie, the seventh grade teacher at School B, created two driving questions so that she could focus on the seventh grade standard 07-LS1-5: Construct a scientific explanation based on evidence for how environmental and genetic factors influence the growth of organisms. The two driving questions she created were: “How does land use affect the water quality in the Kentucky River watershed?” and “Why is it that some organisms are more likely to reproduce successfully than another?” Jacquie’s colleague and mentor, Karen, also a seventh grade teacher, experienced similar difficulty in developing the driving question. She combined the two questions Jacquie used into one: “How does land use impact water quality and affect the variety of organisms in the Kentucky River Watershed?” Both teachers shared their discontent with their driving questions during discussions in the institute. Jeanette, another seventh grade teacher from School A, had the same difficulty in constructing a driving question. She decided to create a more general question: “How does the water quality of the Kentucky River watershed affect our county?” Wynne and Lena experienced similar challenges in creating their questions. During the institute, the teachers talked about the difficulty they had in constructing a driving question. They had created essential questions for lessons and smaller units in the past but had not done so for an entire unit. In contrast to their struggles, Lori, the eighth grade teacher from School A, was able to create her driving question with little difficulty. She began with an essential question from a water quality unit she had designed the previous year, expanding the question to encompass a larger regional watershed: “How does land use impact water quality around my school, in my community, & in the Mississippi River basin?”

4.2.2. Sub-Driving Questions in PBL Units

One of the limitations of the units was the lack of opportunity for students to pose their own sub-driving questions. Although many of the units illustrated how the teachers constructed sub-driving questions to guide lesson activities and engage students in writing reflections, there were limited opportunities for students to construct their own sub-driving questions. Likewise, there were few opportunities for students to plan and carry out their own investigations. Of the seven units analyzed, only one attempted to give students an opportunity to ask and pursue their own sub-driving questions on their local watershed. The remaining six units included aspects of students leading investigations in the form of online research projects. For example, Lori, the eighth grade teacher from School A, included a research project for students to identify water quality issues relevant to local, national, or global communities. Similarly, Karen and Jacquie, both from School B, created an assignment for students to research an organism from the Kentucky River Watershed that they would later hang on the wall to create a class-designed end artifact: a wall mural of the local stream food web constructed from students’ reports. These project topics were all designed by the teacher. They incorporated some freedom of choice for students to select a topic of interest but the goals and purposes for conducting the assignments were generated by the teachers.

4.2.3. Benchmark Lessons in PBL Units

The use of benchmark lessons was the third feature of PBL analyzed in the units. The average evaluation of the benchmark activities across the seven units was 2.5 on the 3-point scale with individual unit mean scores ranging from 2.0 to 3.0. As briefly noted earlier, teachers often generated sub-driving questions to create logical and meaningful outlines around the driving question. This was observed in 4 of the 7 (57%) units. These units often incorporated a sub-driving question followed by one to four lesson-level essential questions to guide activities and connect them to the driving question.

Table 6. Driving Questions and Sub-Driving Questions from School A Teachers PBL Units.

Criterion	Teachers
	Catie	Jeanette	Lori
Grade Level	6	7	8
Driving Question	How does land use impact water quality and aquatic eco-systems around my school?	What is the Kentucky River Watershed?	How does land use impact water quality around my school, in my community, and in the Mississippi River basin?
Benchmark Lessons organized by sub-driving question	1. Where is my watershed? a. What is a watershed? b. What is the role of the hydrologic cycle within a watershed c. What streams and/or bodies of water are within my local watershed? d. How does the topography of the land in my community affect the flow of water in my local watershed? 2. How is the land in my watershed used? a. What effect does the way the land is used near a stream have on its overall health? b. How can I show ways the land near the streams around my school are used? 3. What abiotic factors (chemical and physical) determine the health of a stream? a. What is a riparian zone? How does it affect a stream’s health? b. How does dissolved oxygen and temperature of the water affect the health of a stream? c. How do I determine the turbidity of a stream? 4. What biotic factors affect the health of a stream? a. What are macro-invertebrates? b. What types of food chains/webs are associated with our local creek?	1. What is a watershed? 2. What is the Kentucky River watershed area? 3. What is water quality? 4. How can I determine water quality? How does the Kentucky River affect [our] county? 5. What effect does the way the land is used near a stream have on its overall health? 6. What effect do the waterways have on our county? 7. What lives in the Kentucky River watershed and why? 8. What organisms live in the water, especially in [our] County? 9. What are the interactions of organisms in these habitats 10. How does water quality affect organisms in our area? Can I determine the health of the Kentucky River watershed in [our] County? 11. What data should I collect? 12.How do I collect data about the health of a stream? 13. What does this test tell me about the health of the water in our county?	What is a watershed? LT1. Construct a model of a watershed and identify the components that make up a watershed. LT 2: a. Investigate a water quality issues around the world and in our local community. b. Present and conduct the water quality test to educate your peers. LT3. a. Identify macro-invertebrates that live in the water and evaluate their sensitivities to water quality. LT 3b. Use this sensitivity and their presence to analyze the water quality using the Biological Quality Assessment Scale. LT 4: a. Analyze and discuss my water testing results from our creek. LT 5: a. Conduct an investigation on our county’s land use and watershed using a GIS interface. LT 6. a. Research and evaluate an issue pertaining to Water Quality. LT 6b. Evaluate competing design solutions for maintaining biodiversity and ecosystem services. LT 6c. Construct an argument that supports or refutes claims for either explanations or solutions about the natural and designed worlds. [To answer the essential question: How does land use impact Water Quality around my school, in my community and/or in the Mississippi River Basin?]

Note: Wording and spelling were maintained from teachers’ units.

illustrates how Catie and Jeanette organized their units around sub-driving questions with essential questions used to drive learning in individual lessons. Also illustrated in the table is Lori’s unit that used learning targets (LT) instead of sub-driving questions to organize most of her lessons; a practice observed in two other teachers’ units.

The framework employed to organize benchmark lessons in the units varied across teachers even when they collaborated with colleagues from the same school. For example, School A teachers, Catie, Jeanette, and Lori, used two different strategies to organize their benchmark lessons. Lori identified learning targets to sequence the activities in her unit, whereas Catie and Jeanette created sub-driving questions. The learning targets connected the assessments and outcomes to the standards and NGSS dimensions (i.e., disciplinary core ideas, crosscutting concepts, science and engineering practices), which were emphasized throughout Lori’s unit. In comparison, the sub-driving questions Catie and Jeanette used to organize their units created a meaningful sequence across a closely connected group of activities. From the researchers’ perspectives, there were more gaps in the lesson topics across Lori’s unit than in Catie’s or Jeanette’s units, whereas the assessments were more clearly identified and sequenced across Lori’s unit. Lori’s decision to use learning outcomes to organize the benchmark lessons in might have been influenced by her view of student-centered learning. In Lori’s model, student-centered instruction was manifested as students having the freedom to choose topics to study from a list of teacher-derived options. This was observed in the water quality unit Lori had designed the previous year. It consisted of a series of assignments and projects that students worked through independently or in pairs. Using learning targets to sequence the unit allowed her to construct standard-based assignments and projects for the unit. In contrast, Catie’s and Jeanette’s benchmark lessons were designed in sequence to build skills and knowledge in whole class activities while maintaining a focus on the driving question.

Units designed by teachers in School B (Jacquie and Karen) showed similar strategies for organizing benchmark lessons. Like Lori, Jacquie, a seventh-grade teacher with one year of teaching experience, used learning outcomes to frame her unit. She connected the learning outcomes to standards and NGSS dimensions and then selected benchmark activities that would articulate the standard or build knowledge and skills students would need to demonstrate the standard. She also created formative assessments that were closely connected to the standards, activities, and expected student outcomes. In comparison, Karen’s unit outline, also designed for seventh grade, most often used benchmark lesson activities to organize the unit. Although she sequenced the unit activities similarly to Jacquie’s unit, her unit did not make clear connections across standards, NGSS dimensions, lesson activities, and assessments. Karen also created an additional section in her unit outline to list student products for each benchmark lesson, pre- and post-assessments, and milestones.

Wynne and Lena (from Schools C and D, respectively) applied a similar strategy to Karen’s to organize their units. They would identify benchmark lesson activities by a sub-driving question (i.e., What is a watershed? What is water quality?) in one section, and in other cases they simply listed an activity by name (i.e., Watershed PPT, testing different water quality factors) creating a more fragmented unit. They also included only a few resources with their units so it was challenging to determine what students would do and what the teacher would do in the activities. For example, Wynne’s unit outline simply listed water quality followed by a list of topics (i.e., silt/soil, eutrophication) making it difficult to decipher what she planned to do with these topics. She had listed a habitat assessment as a part of the unit materials, but it was not included in the unit outline.

4.2.4. Scientific Investigation in PBL Units

Student-led investigations are a hallmark of PBL but were not a strong component of teachers’ units. The mean score for the Scientific Investigation I was 1.35, with mean scores ranging from 0.05 to 2.0 for individual units. Similarly, the mean score for scientific investigation II was 1.63, with individual unit mean scores ranging from 0 to 2.4. See Table 5 for a summary of teachers’ scores. Units developed by Karen (School B, mean score of 2.4), Lori (School A, mean score of 2.2), and Catie (School A, mean score of 2) had the highest ratings of the cohort.

Two benchmark lessons from the teacher institute appeared in all seven units as exploratory models for students. The first lesson was the same 3D crumpled paper watershed model described earlier in Catie’s unit. Some of the units also added water-based markers, sugar-flavored drink powder, or other water-soluble materials to simulate the effects of natural and man-made pollution from land use and other human activities.

The second benchmark lesson from the institute appearing in the units introduced water quality testing techniques. All the units incorporated water quality testing as an in-class experience to create a space for students to learn and practice conducting the tests. The institute had provided teachers a class set of LaMotte water monitoring test kits (see https://lamotte.com accessed 12 July 2022) for a class of 30 students working in small groups. The 3D crumpled paper watershed model and the water quality testing lessons were used to introduce the phenomenon of watersheds and build water testing skills that students would use later in other investigations.

Another investigation common to all the units was some form of water quality or habitat assessment of the local watershed. This investigation applied skills developed in previous benchmark lessons, such as the watershed model, water quality testing techniques, and identification of macroinvertebrates (the latter activity appearing in all the units). Teachers created worksheets to scaffold students’ data collection and analysis in the field. All the teachers from Schools A, B, and C incorporated a field study of a local stream or other body of water.

The other two teachers also incorporated exploratory activities that they participated in during the institute PBL unit. Wynne, from School C, incorporated a habitat assessment in her unit that used a modified version of a data collection tool she had used in the institute habitat assessment study. Lena’s unit (from School D) did not include a water quality field study, rather she adapted water quality testing activities so that students would be able to perform collect and analyze water quality data, including a macroinvertebrate assessment, to determine the quality of water samples from around the area that she planned to bring into the class. Lena was the only teacher that did not include an outdoor field experience in her unit.

Catie (School A) and Karen (School B) both designed their units to include the identification of organisms within the local watershed and the interactions among them. Karen also added a short internet research project to her unit in which students would create a short report about the lifecycle and food web of an organism in the local watershed. As noted earlier, the reports would then be displayed in a wall mural that illustrated the local watershed food web.

Asking questions and arguing from evidence are two important practices in NGSS [2]. Activities in the units created engaging, investigative experiences for students that focused on content and applied scientific practices. Overall, activities followed a structured or guided inquiry format [34,35,36], often reflective of traditional teacher-directed inquiry rather more open-ended student-led investigation inherent to PBL.

Several of the units incorporated the science and engineering practice of asking questions. For example, Catie (School A) and Karen (School B) incorporated time for students to generate sub-driving questions following a discussion about the outcomes of an investigation. Likewise, Jessica (School A) and Jacquie (School B) included discussions where students would be able to brainstorm ideas and raise questions about watersheds. However, it was only in Karen’s unit where students could work collaboratively in pairs or independently to construct sub-driving questions about data collected from a stream study included earlier in the unit. Students would then select one sub-driving question around which they would plan and carry out an investigation using the stream study data the collected earlier. They would later present their findings to the class.

4.2.5. Collaborative Opportunities in PBL Units

Collaboration was another PBL feature that was represented in the units. The mean score for all the units combined was 1.5. In comparison, individual unit mean scores ranged from 1 to 2.2. The indicator most represented by the units was opportunities for collaboration that encourages students to generate ideas, questions, conjectures, and/or propositions. The units all included small group work where students would explore maps of the watershed, identify macroinvertebrates, practice water quality testing, gather data in the field, and/or analyze data and report results. Moreover, six of the seven teachers (from Schools A, B, and C) included some space for students to share diverse viewpoints, often cultivating these ideas through readings about water quality issues in articles produced in magazines written for middle school students. Written responses and class discussions created space for students to critique each other’s ideas and offer constructive criticism, but in general, such experiences were not well developed in the units.

The units that teachers from Schools A and B created included a field experience similar to one Lori (from School A) had shared with them in the first follow up institute session. The teachers from Schools A and B incorporated the field packet from Lori’s field survey to guide students in conducting the stream field study. The worksheets demarcated group roles and specific tasks for each member to perform, creating scaffolding to support student collaboration.

Several other strategies were incorporated into the units to guide student thinking and reasoning. Questions were provided to guide the reading of articles, conducting online research, and viewing videos on water quality issues. Web links also were included to limit the time needed for students to research a topic and a list of resources was provided for student research assignments. Graphic organizers (e.g., T-charts, tables) were used to scaffold students in recording and analyzing data.

A PBL indicator for student collaboration that was not represented in the units was the opportunity for students to collaborate with community members and/or experts (scientists, industry professionals, etc.) in planning and conducting their studies. One exception was observed in Karen’s unit where a guest speaker from a local non-profit environmental education organization would be invited to class to introduce students to macroinvertebrate identification and how they can be used as indicators of water quality. The other units included reading activities similar to Karen’s unit, and internet research to identify macroinvertebrates or other organisms they found in water sampled in class or that were included in a list in the activity worksheet. However, collaboration with community members was not indicated in these units.

4.2.6. Milestones, Assessments, and Student Artifacts in PBL Units

The last features of PBL analyzed in the units were milestones and assessments, and student products generated from students’ own investigations. Wynne’s (School C) unit was omitted from this part of the analysis because it did not include information to identify assessments she planned to use. Further complicating the analysis were the limited descriptions teachers included to explain the purposes of the assessments and how they were connected to the unit and student work.

The mean score for the six units (all but Wynne’s unit) analyzed was 1.6 with a range from 1.3 to 1.8 for individual unit means. This indicated that some of the assessments shared aspects of PBL, but taken together, the assessments did not portray the reflective purposes of milestone assessments. That is, they were created to provide students formative feedback that could further guide students in constructing sub-driving questions, planning and carrying out of investigations, organizing and analyzing data, and critically thinking about their findings and implications. However, there were very few opportunities for students to provide peer feedback.

The assessments were geared toward confirmation of completed work or the understanding and application of knowledge and skills. Graphic organizers were common across the units and were used to scaffold data collection and analysis. Whole class discussions also were noted across all six units as a medium to review ideas gleaned from readings and videos, share data from investigations, reinforce instructions on conducting data analysis, and share findings from studies and investigations. Opportunities for small group discussions were also prevalent across the units. Scaffolds such as guided questions, data collection instructions, a variety of table displays, and activity procedures created support for students when they negotiated data collection and recording procedures, discussed how to perform water quality tests, and grappled with the meaning of outcomes from the investigation (i.e., level of water quality, effect of low levels of dissolved oxygen and their causes). Even though teachers planned these activities as traditional formative assessments to check for understanding, monitor learning, and guide student work, the activities included in the units created the potential for small group discussions.

There were several units that illustrated some development toward assessments reflective of PBL. For example, units created by Catie and Lori from School A, and Karen from School B incorporated science notebooks, short written responses, or explanations where students could ask questions, construct sub-driving questions, and reflect on their thinking about what they were learning in relation to data collected and analyzed.

5. Discussion, Limitations, and Conclusions

5.1. Discussion

The purpose of this study was twofold. First, it aimed to explore how well a one-week summer teacher institute informed in-service middle school teachers’ knowledge of watersheds and understanding of essential features of project-based learning. Second, the study sought to explore how well the middle school science teachers were able to develop their own PBL units on water quality in their local watersheds following their experiences in the summer institute. The findings will be discussed in relation to the research questions and extant research on supporting in-service science teachers to teach PBL.

5.1.1. Research Question (RQ) 1: Teacher’s Knowledge about PBL

Teachers’ pre responses to the PBL survey demonstrated a primitive understanding of PBL pre institute with teachers holding broad ideas of student-centered classrooms and authentic experiences, but they provided little detail. By the end of the institute, teachers had more detailed awareness of key features of a PBL classroom with all teachers realizing the importance of a driving question and most teachers realizing the need for sub-driving questions and benchmark lessons. The identification of a driving question is an important area of growth for the teachers and a component often omitted from PBL units designed by teachers [5,25,26]. However, knowledge is not enough to support teachers in designing and implementing PBL in their classrooms [26]. Han et al. [26] found that even when their teachers demonstrated knowledge of problem-based learning, they were not able to apply their knowledge to creating problem-based units. Similarly, Mentzer et al. [25] reported that PBL units that teachers developed in the first year of their study also lacked driving questions to frame the units and tended to be fragmented and lacking a central focus. We also analyzed the units that teachers constructed following their participation in the summer institute. We discuss these findings in RQ2.

5.1.2. RQ2: Teacher’s Ability to Construct PBL Units

Results from the unit analysis showed that teachers had incorporated some aspects of PBL in the units, but they did not do so at a level that reflected a fully developed PBL model. The units were guided by a driving question derived from district and national standards. In addition, the units were often organized by sub-driving questions, or by learning targets the teachers constructed. Scientific investigations included in the units mirrored those from the summer institute, such as (a) 3D watershed models students can create to explore water flow in a watershed; (b) study of the local and regional watersheds using Kentucky Geological Survey maps, Google Earth, and ARCGIS (https://www.arcgis.com/index.html accessed on 7 August 2022); (c) water quality testing procedures; (d) macroinvertebrate identification for water quality assessment; and (e) field studies on water quality and/or habitat assessment of local watersheds. The units also demonstrated use of scaffolding in the form of clear instructions and graphic organizers to guide student work, and teacher demonstrations designed to model water quality testing techniques. Class discussions were also included in the plans to address collecting and analyzing data and making sense of the results from in-class and field investigations.

Teacher-directed inquiry investigations were common in the units, but only a few examples portrayed student-directed, open inquiry investigations [7,42] that are a hallmark of PBL [4,5,6,40,42]. In-class investigations were the most frequent type of inquiry represented, and online research assignments were the second most frequently represented inquiry activity. The units that Karen (School B) and Lori (School A) created were the only two that included a form of student-directed inquiry. The student investigations in Karen’s unit were designed around sub-driving questions culminating from outcomes of the field study data. Due to constraints of time and resources, she had planned for students to develop sub-driving questions and student-directed investigations using water quality data they collected from the field study planned earlier in the unit. In comparison, Lori created a culminating internet research report that included online research about a water quality issue in the Mississippi River Basin, the larger watershed basin holding the region’s watershed that was studied in the summer institute.

Authentic student artifacts and student-led investigations were less common across the units. Only Catie from School A and Karen from School B incorporated artifacts students created that could be shared with a larger audience beyond the classroom. The letter to the farmer Catie included would give students an authentic outlet for sharing the findings they learned through their class research and water quality study. Karen’s editorial piece would offer the same opportunity but neither unit indicated students would have opportunities to present to a community audience. In comparison, many of the other teachers concluded their unit with a traditional student lab report on the water quality field study. Wynne was the outlier in the group since she did not provide information on the assessments to be included in her unit. The omission of assessments suggested she needed additional support to think through how to incorporate meaningful student products in her unit. Haatainen and Aksela [51] reported similar findings in their study. Taken together, the units portrayed greater focus and more meaningful sequencing of activities than what has been recently reported in the research literature [5,27,29]. Markula and Aksela [5] reported that more PBL features appeared in units developed by teachers participating in a university-led training program in comparison to the units from teachers that did not have any training. Even so, all the PBL units in their study tended to focus on 21st Century skills rather than on standards-based concepts. Severance and Krajcik [28] reported that teachers in their study tended to conform PBL units to their instruction and frequently omitted science and engineering practices. Similarly, Han et al. [29] found that the teachers that participated in their PBL institute gained knowledge of PBL but designed and implemented PBL units that reflected aspects of their own instruction rather than PBL modeled in the institute. The activities in units from the current study emphasized science content and science and engineering practices [2], thus going beyond a focus on 21st-century skills [5].

Units in our study also reflected the major benchmark lessons integrated into the PBL unit. When designing their units, teachers drew from institute PBL activities to create the early drafts of their units. The units also incorporated science and engineering practices in a similar manner to what was modeled in the institute PBL unit. The practices that were most often represented included (a) planning and carrying out investigations, (b) analyzing and interpreting data, (c) using mathematics, and (d) constructing explanations. Thus, participating in PBL as learners appeared to benefit teachers as they constructed their units.

The prevalence of teacher-directed inquiry in the units rather than student-directed inquiry inherent to PBL might be a factor in learning to construct PBL units. Etmer et al. [52] reported that teachers in their study made small changes to their existing units as they began to implement features of problem-based inquiry in their early units. In later years, their units illustrated more advanced implementation of the problem-based inquiry. Specifically, teachers generally started small, adapting previous units and using a backwards-planning process. Mentzer et al. [25] also reported this incremental development of teachers’ PBL units over a three-year period. The researchers reported that later units had a central focus and reflected more features of PBL compared to the units developed in the first year. These findings suggest that the teachers in the current study may have been applying similar strategies in their attempts to create their first PBL units, thus showing a slow development of pedagogical content knowledge [48] for developing and implementing PBL.

Small changes Lori made to her water quality unit further substantiate the hypothesis that teachers often demonstrate slow incremental development of skills to develop PBL units. Because of her experience designing units that incorporated multiple projects and independent student activities, we expected to observe in Lori’s unit a transition from a teacher-centered to a student-centered approach. Doing so would have required changing the framing of the unit and adding additional instructions to guide students in their independent work. Contrary to our expectations, she made small additions to increase the depth of knowledge in the existing water quality unit. For example, one change she made to the unit was modifying the stream study to focus on two locations along the stream: one section above the sewage treatment plant effluent and one below. This modeled the plan that her group created for their own research study they planned and conducted during the institute. The two data collection sites along the stream that was nearby her school created a comparative study that could lead students to draw conclusions about a possible local water quality issue, such as algal overgrowth from nitrites released in the effluent. Lori also expanded the stream study by adding aspects from the water quality testing techniques and water quality sampling benchmark lessons she participated in during the institute.

A second change Lori made to her unit was modifying the culminating assignment. The original plan had required students to conduct online research on a water quality issue they selected from a list of options given in the assignment. Students were then to write a newspaper article to educate the local community about the issue. She modified this assignment to require students to write a newspaper article in the form of an argument supporting or refuting a natural or designed world solution around the water quality issue they researched. This small change highlighted the science and engineering practice of argumentation [1,2]. Taken together, the small changes made to the unit improved the depth of knowledge developed across the unit but did not change the teacher-centered nature of the unit.

Learning to develop a new pedagogical strategy takes time. Mentzer et al. [25] reported that it took their teachers two to three years beyond their initial participation in the extended PBL professional development program before their units began to reflect the PBL modeled in their program. The strength of the units that teachers designed in our institute was that they incorporated many of the of key PBL features presented in the institute, including the driving questions, benchmark lessons, science investigations, and student collaboration. These findings suggest that through their participation as learners in the institute PBL unit, the teachers developed a knowledge base from which to draw as they began developing units. Thus, this study provides a small sample of evidence to support the assertion posed by other researchers that having teachers learn through PBL will help them understand the pedagogy of PBL [5,31,32].

5.1.3. Collaboration in a Community of Practice

Collegial collaboration was an important factor that supported teachers in designing their units. This was particularly apparent when reviewing units from Schools A and B in comparison to those submitted by the two teachers from Schools C and D. The latter two units lacked the detail and substance observed in units designed by teachers that collaborated with colleagues at their schools. Three teachers from School A and two from School B participated in the institute, whereas teachers from the other two schools, C and D, were the only representatives from their schools. Even though they could collaborate with the other teachers during the institute, they did not have the same collegial support in their schools as teachers at Schools A and B, but they did have the support of school administrators. Another teacher that participated in the summer institute withdrew from the follow-up sessions because she changed schools and the science teachers with whom she worked were not interested in developing PBL units. Other studies have reported similar findings. For example, Toolin [53] identified collegial collaboration—teachers that collaborated with other peers at their schools—as a contributing factor to their implementing PBL. Krajcik et al. [17] reported similar findings in their study where researchers collaborated with teachers to create and implement project-based science units.

Mentoring from more knowledgeable others was another collaborative activity the teachers in the current study sought as they began to construct their units. They viewed Lori as a mentor and role model after learning about the water quality unit she had created the previous year. They were eager for her to present it to the group and asked for access to the activities as they were planning their own units. Their use of Lori’s materials was evident in units from Schools A and B. For example, the end unit artifacts Catie (School A) and Karen (School B) created in their units were drawn from Lori’s unit. Catie’s end unit artifact was a letter to the farmer and Karen created an assignment where students would write an editorial about a water quality issue. Similarly, teachers modeled their selection and use of science articles in their units after the articles Lori had included in her water quality unit. The teachers also incorporated the video Poison Waters [54] to inform students about water quality issues that could be affecting their own communities, which was another activity borrowed from Lori’s water quality unit.

5.1.4. Professional Development Model: Strengths and Areas for Growth

Professional development programs that are longer in duration both by number of hours in which teachers participate and longer in the time span the activities occur can have positive impacts on teachers’ pedagogical content knowledge [21] and reform-based instructional practice [18,19,20]. Moreover, longer (more than 24 h) professional development programs [18,19,20] specifically emphasizing inquiry can create opportunities for teachers to engage in inquiry from multiple perspectives as learners, as scientists, and as teachers [7,18]. Longer immersive professional development experiences have also been shown to lead to changes in teaching practice [7,18,19,20,21,25]. Placing teachers in the role of learners in inquiry experiences that are similar to those they will implement in their classrooms can also create opportunities to learn about the work of scientists and about instructional practices for inquiry [21,22,23,24,55]. In addition, professional development models that incorporate teacher–researcher collaborations also have been shown to cultivate growth in teachers’ knowledge and implementation of units that incorporate the essential features of PBL [5,7].

The professional development model created for this study was designed to support teachers in developing the knowledge and skills needed to construct PBL units. Teachers participated as learners in the institute PBL unit to learn about the region’s watershed and water quality testing techniques they used to sample several areas of the watershed. They assumed the role of scientists as they planned and conducted their own investigations guided by sub-driving questions they posed earlier in the institute as part of the embedded PBL instruction. Their units incorporated many activities taken from these institute PBL units. In addition, their understanding of PBL was supported by their inclusion of many of the essential features of PBL addressed in the institute and that appeared in their units. Further, the inclusion of driving questions, student investigations, field studies, engaging students in analyzing data, constructing explanations, and student artifacts illustrate an understanding of PBL and their developing ability to construct PBL units.

The unit analysis also identified several areas for growth in the institute model. The infrequent use of assessments and milestones in the units suggests that future institute sessions should focus more explicitly on formative assessments that support PBL pedagogy. There were many formative assessments included in the institute, such as data collection sheets, discussions about analytic methods and results from analysis, presentations on field study findings, and discussions that cultivated wonderings and sub-driving questions. Based on teachers’ challenges in planning their first PBL units, the formative assessment activities needed to be made more explicit in situ and during embedded PBL instruction. Further, having examples of assessments used in previous PBL units on similar topics (e.g., watershed ecology, watershed water quality) may also be helpful to teachers.

The lack of milestones appearing in the units also suggests teachers need more support in learning how to incorporate this type of formative assessment into their instruction. The teachers had participated as learners in milestones [5,41] during the last three days of the summer institute. Some of these activities included constructing and critiquing each other’s sub-driving questions, presenting their investigation plans, receiving peer feedback on data they collected and their analysis, and presenting their final presentations to the panel and their peers. While these experiences were made explicit as milestones in the institute, the hesitancy teachers demonstrated as they began planning their units suggests that providing examples of other milestones used in instructional units at their grade band would help close the gap between their experiences as learners and as planners of PBL environments.

The units also identified strengths in the institute design. The geographic frame of the region’s watershed created a shared focus of the teachers’ units. Regardless of where their school was located. They all lived and worked in the same watershed basin. This promoted the sharing of lesson examples and investigation ideas. Planning PBL units on the same topic that was addressed in the institute PBL unit created a model for teachers to refer to and draw from as they worked collaboratively to construct an outline for their units. Even so, without Lori’s middle school unit to use as a guide, the teachers likely would have been less successful in unit planning. They wanted to see not only the institute unit but also a complete middle school unit that had already been implemented. This desire might stem from teachers’ limited pedagogical content knowledge [45] of PBL [26]. If so, it would be expected that their development would progress as they became more familiar with PBL pedagogical practices. This desire to refer existing PBL units on the same topic also might be a scaffold to build into future professional development programs on PBL.

In addition, access to the institute leaders, researchers, and field experts was another asset that supported teachers’ planning. Prior research has illustrated how collaborations between university professors and teachers scaffolds teachers’ development of PBL units [ 17,28]. This was also observed in the current study. In the future, it would also be helpful to not only have field experts available as collaborators as teachers began planning their units, but to also recommend ways in which teachers could incorporate experts in the unit designs. Doing so would foster the student–expert collaborations that were missing from many of the teachers’ units.

5.2. Limitations

There were several limitations important to note in this study. To begin, the findings from this study are not generalizable beyond the seven teachers sampled. Their successes and areas for growth are closely linked to their education, prior experiences, prior learning, and experiences and interactions that occurred in the institute and afterward when researchers were not present. Moreover, teachers’ ideas captured in field notes may not fully represent factors that influenced the planning decisions they made in creating their PBL units. Further, there were diverse differences across the schools represented in the study. Two schools were located in small communities in rural settings, another was located in a suburban area near a mid-size city, and another was located in a small town. Student population is a factor not addressed in the analysis of the teachers’ units, but likely influenced teachers’ planning decisions. However, these factors also create strengths in the unit analysis because of the diversity of teachers, schools, and student populations represented.

5.3. Conclusions and Implications

This study sought to explore how well teachers were able to construct PBL science units of instruction following their participation in a week-long PBL summer institute. The institute was followed by four Saturday sessions scheduled across the academic year. Teachers had completed two of the sessions prior to submitting their units. A major component of the summer institute was the PBL unit investigating the effect of land use on the region’s watershed. The first iterations of the units the teachers designed showed their reliance on the unit modeled in the PBL institute and on a water quality unit one of the teachers had created the year before the institute. The teachers relied on these models as they constructed their own unit plans, often incorporating aspects from each.

The findings from this study add to the research on PBL in terms of professional development models that appear to support teachers in constructing their first PBL units. Recent studies have highlighted the challenges teachers experience in constructing PBL units. Too often, their initial units lack a central focus and omit a driving question [5,25]. Following the recommendations of prior studies [5,31,32] our model immersed teachers in an authentic PBL unit that also served as a model for their own units. Another benefit was focusing the teachers’ first unit around the same topic as that addressed in the institute PBL unit. Doing so created a scaffold to support teachers in creating their units. Another important factor was teachers collaborating with colleagues from their school even when colleagues taught at different grade levels. Future directions for the current study will seek to explore how the teachers implement their units and support networks they design to scaffold their implementation.

Future studies should also focus on how collegial relationships that form within professional development programs can bolster even those teachers that do not have other collaborators in their schools. In the current study, two of the teachers that participated as the solo representative from their schools both implemented PBL units, but their units were not as focused on the driving question as teachers that had collegial collaborators at their schools. However, had they not participated in the summer institute, they would not have had the opportunity to grow as they learned to create PBL units for their students.