Learning in Citizen Science: The Effects of Different Participation Opportunities on Students’ Knowledge and Attitudes

Abstract

Citizen science (CS) projects are becoming increasingly popular in schools. They promise to expand knowledge, promote scientific literacy, as well as improve environmental attitudes and behavior. However, studies on the educational potential of CS projects show varying results. These inconsistent findings can be attributed to differences in the design of CS projects. It is assumed that participants who are involved in more phases of the scientific process show greater learning outcomes than participants who are involved in fewer phases. Various models about participation in CS have been developed but have not been thoroughly empirically tested. Therefore, the research question of this study is as follows: How do different participation opportunities influence the educational outcome of CS projects for students? To answer this question, a CS project was carried out with three experimental groups, whereby the participation opportunities were varied, and 199 students were included. The students’ knowledge, attitudes toward science, as well as their environmental attitudes and behavior were evaluated three times (before, shortly after, and 2–3 months after the intervention). The analyses show only minor or no statistically significant influences of the participation opportunities on participants. These results are consistent with studies that also expressed doubts about whether the level of participation determines the educational potential of CS projects.

1. Introduction

Citizen science (CS) is an approach in which people without institutional ties to science are involved in scientific processes [1]. The participation of these individuals in projects can vary widely—from assisting in data collection to investigating their own questions. For the scientists involved, the CS approach offers the opportunity to collect data over longer periods of time and over a wider geographical area than would be possible for them alone. For the citizens involved, participation in a CS project can be rewarding for several reasons. First, participants may find enjoyment in the work (e.g., birding outdoors). Second, some projects (e.g., eBird, Ornitho), for example, offer to make relevant databases available to their participants. This is so they can decide which birding hotspots to visit next time. Third, all CS projects offer educational potential. For example, CS projects promise to promote participants’ understanding of science through an authentic scientific approach [2,3]. In addition, CS projects that focus particularly on environmental or ecological issues also have a potential impact on environmental constructs such as attitudes toward nature, which plays a crucial role in the education for sustainable development [4,5]. Besides influencing these relevant environmental variables, CS projects have a high additional value in education for sustainable development (especially in terms of ESD 2—learning for sustainable development [6]) due to the (at least potentially) open and participatory approach in CS projects—where authentic, open-ended problems are addressed.

Due to their unique approach and the promise they hold for science education or education for sustainable development, participation in CS projects is increasingly taking place in or through schools [7]. CS projects in the school context differ from other CS projects because participation—if the whole class participates—is not necessarily voluntary for the individual student, and/or it takes place in their free time. While the volunteer participants in a CS project represent a very selective sample in which a high level of interest in the topic and a high level of motivation can be assumed, participating school classes are significantly more heterogeneous regarding these variables, which may impact learning. Nevertheless, CS projects in the school context represent an innovative way through which to integrate authentic science practice in the classroom, which is required by both learning theory [8] and curriculum documents (e.g., [9]).

In view of the potential that CS projects have for the school context, the question arises as to how such CS projects should be designed in order to work well in the school context, which is where the educational potential is especially important. Since there are very different design options in CS projects—especially with regard to the participation opportunities of the participants—and since this results in different educational potentials (see below), we used a quasi-experimental design to analyze what effect differently designed CS projects have on the relevant educational variables among students.

2. Theoretical Background and State of Research

The designs of CS projects are very diverse: there are CS projects in which the participants exclusively collect data, and there are projects in which the participants independently generate and investigate their own research questions. If we want to examine the educational potential of CS projects, we need to distinguish between these different types of CS projects and to link these to learning theory.

2.1. Learning in Different CS Projects

A classification of different designs of CS projects was first described in the model of Bonney and colleagues [10]. This model distinguishes different types of CS projects by taking the level of participation into account: contributory, collaborative, and co-created CS projects. Contributory projects are solely designed by scientists, and the focus is on the data in this type of project. For most projects, this is a traditional top-down approach. As a result, volunteer participation is mostly limited to data collection. In contributory projects, it is assumed that the data quality is high and that the learning effects for the participants are rather low. In collaborative projects, participants are considered assistants to the scientists. In addition to data collection, their tasks sometimes include analyzing the data and contributing to the study design. This is considered to decrease the quality of the data but increase the educational potential. In co-created projects, volunteers and scientists work together as equals. This means that volunteers can participate in all steps of the scientific process, from formulating the research question to discussing the results. The assumption here is that this further reduces the quality of the data collected; however, at the same time, it has the highest learning potential for the participants. This project category often follows a bottom-up process, i.e., the initiative comes from the volunteers.

Table 1. Three-stage typology according to Bonney et al. [10] (p. 17, modified). X—when the public participates in the step, (X)—when the public sometimes participates in the step.

Steps in the Scientific Process	Contributory Projects	Collaborative Projects	Co-Created Projects
Definition of the research question			X
Development of hypotheses			X
Development of the study design		(X)	X
Data collection/sampling	X	X	X
Analysis of samples		X	X
Analysis of the data	(X)	X	X
Interpretation, conclusions		(X)	X
Dissemination of conclusions	(X)	(X)	X
Discussion and outlook			X

shows this typology as based on the steps of the research process.

The typology by Bonney et al. [10] aligns well with the theories on inquiry learning in science education. Chinn and Malhotra [8] distinguish between simple inquiry tasks and authentic scientific investigations for learning in science, and they argue that more cognitively demanding tasks related to inquiry also lead to better learning and understanding of scientific work. Regarding the distinction between simple inquiry tasks and authentic scientific investigations ([8], pp. 180–182), similarities to Bonney’s typology (see Table 1) can be found with regard to the degree of autonomy and involvement in the scientific process. In a few studies on the effect of different designs of CS projects, Bonney’s typology was used to systematically vary the experimental conditions in CS projects [11,12,13]. Here, no effects due to different designs were found, but these were CS projects conducted with citizen volunteers outside the school context.

However, results regarding the learning in regular CS projects are difficult to generalize in the school context. In regular CS projects, people usually participate who already have a high motivation and a high interest in the topic of the CS project. As a result, the participant base tends to be composed of highly educated, highly interested, mostly high-earning, older, white individuals (e.g., [14]). School classes, on the other hand, represent—as already described in the introduction—heterogeneous groups in terms of prior knowledge, interest, motivation, and attitude. Not every student in a class would participate in a CS project of their own accord. Therefore, when carrying out CS projects in a school context, one can assume that the group of participants will be more diverse (at least if whole classes participate in the project). Additionally, with respect to the general goals of school and science education, participating in a CS project should have positive effects on student learning. Therefore, special attention needs to be paid to the question of what impact CS projects have on students, especially when participation in a CS project occurs during school time. When investigating the impact of CS projects on the participants (in particular, the effectiveness regarding the learning of the participants), the question that comes into play is understanding which variables one wants to observe in the participants. In order to show the degree of engagement in CS projects, Bonney and colleagues [2] named various aspects that can be used for this purpose, such as the amount of time spent participating in the project or the number of visits to the project’s website. However, these variables may not be meaningful when examining CS in the school context because, after all, participation in the project takes place in the context of classroom instruction. Moreover, CS projects in the school context focus particularly on educational potential. In order to investigate the extent to which CS projects contribute to the participants’ fundamental understanding of science and thus to the development of scientific literacy [15], variables such as subject knowledge, attitudes toward science and/or the environment, and environmental behavior (especially in the case of environmental or ecological projects), etc., are frequently used in the literature, although these variables might not always be collected simultaneously. Regarding subject knowledge as an outcome of CS projects, we can also distinguish between different types of knowledge: content knowledge, which relates very specifically to the context of the CS project (e.g., knowledge about insects), and the understanding of science or knowledge about science, which is more of an epistemological perspective and is often operationalized as the nature of science (e.g., [16,17]).

2.2. State of Research: Educational Potential of CS Projects

Since the aforementioned variables such as subject knowledge, attitudes toward science, environmental attitudes, and environmental behavior are especially relevant to assess the educational potential of CS projects, we will focus on them in our study and report on the state of research in the following. Furthermore, since there are only a few studies that focus on the school context, we will also draw on the results of regular CS projects here.

Subject knowledge (content knowledge and the understanding of science): Almost all studies on CS projects assess the participants’ knowledge. However, the methodological approach to assessing knowledge is problematic in many studies, e.g., because only self-reports are used (which are not valid in terms of actually capturing knowledge) or only one measurement point is included (and thus no conclusions can be made about the impact of the CS project) [18]. Due to this, the research on the impact of CS projects on knowledge is not consistent. We have found studies that both show that participants’ knowledge increases and studies that show the CS project had no effect on knowledge. When knowledge is divided more precisely into content knowledge on the one hand, and understanding of science on the other, an inconsistent pattern also emerges. Almost all studies on CS projects argue for an increase in the content knowledge of participants (e.g., knowledge about insects) (including [19,20,21,22,23,24,25]). Additionally, several intervention studies can demonstrate an increase in participants’ content knowledge [3,4,26,27,28,29]. Regarding understanding of science, Trumbull et al. [30] used an analysis of 750 letters and feedback to show that 78% of participants thought and acted scientifically; thus, they concluded that CS projects promoted science literacy. Evans et al. [20] conducted 45 interviews and written surveys of participants in a monitoring project on birds and did not find any advancement on understanding of the scientific method. They were nevertheless able to measure an increase in content knowledge. Similar results were found in an online survey with self-reported questions [31], showing a slight increase in content knowledge but no change in understanding of science. Studies using a pre-post-test design do not reach a consistent conclusion: three out of six intervention studies found an increase in the participants’ understanding of science [27,32,33].

Attitudes toward science: Regarding attitudes toward science, different studies show different results regarding the influence of CS projects. More positive attitudes toward science were found by Krach et al. and Price and Lee [27,33], whereas Brossard et al. and Crall et al. [3,28] did not find changes in participants’ attitudes. Haywood et al. [19] found, through a written survey and interviews, that the value of science to respondents increased as a result of their participation in CS projects. For Krach et al. [27], the interest in science careers increased 50% for students, among others. For Vitone et al. [34], however, CS projects on insects did not increase confidence in, nor interest in, science among college students.

Environmental attitudes: Studies investigating environmental attitudes also show inconsistent results [35]. For example, two of the intervention studies show an improvement in the environmental attitudes of participants [26,27], whereas two of the studies could not detect any changes in attitudes [3,28]. Furthermore, environmental attitudes increased in 50% of the participants in eight different CS projects [5]. Druschke and Seltzer [22], on the other hand, found no change in attitudes when evaluating a CS project on bees.

Environmental behavior: Results on environmental behavior show a nearly consistent picture across intervention studies. Participants report behaving as more environmentally aware of the context of, for example, invasive plant species, after participation in the CS project [26,27,28]. Cosquer et al. [21] also showed in their interview study that participants behave more environmentally aware. Druschke and Seltzer [22] were unable to detect any change in participants’ environmental behavior.

Studies in the school context show comparable results to the studies in the informal education context. Ballard et al. [36], in their study of two CS projects (LiMPETS (coastal protection in California) and in EBAYS (water and air pollution)), showed that students’ environmental awareness and knowledge increased. Both Kelemen-Finan et al. [7] and Poppe et al. [29] found that students enjoyed direct nature experiences and fieldwork the most as they found them the most interesting, thus their motivation was the highest there. The study by Kountoupes and Oberhauser [37] also confirmed that the children and adolescents had the most fun and enjoyment working outside, and that they shied away from work such as data entry on a PC. This is also consistent with statements from other CS projects, where participants prefer the hands-on work and prefer to leave the theoretical work to the scientists [38].

Overall, many of the studies on CS projects have some limitations: Research on CS is still very focused on single cases without experimental designs [3,25,39,40,41]. This limits the interpretability of their results and conclusions for project design. An exception here are the studies by Brossard and colleagues [3], which used a control group, and Crall et al. and Cronje et al. [28,32], both of which had a control group and implemented an experimental setting on the effect of training in the context of CS. There is also a trend in recent studies to turn more to experimental setups [11,12,13,42]. Furthermore, many measurement instruments were developed by the research team specifically for the CS project in question (including [27,34]). Accordingly, the validity of the measurement instruments is often unclear. Furthermore, the school context has also been mostly neglected to date [43].

In summary, studies show an inconsistent picture regarding the educational potential for the participants of CS projects. For educational potential, there are studies that both empirically show a promotion in the respective potential through CS and studies that do not support this promotion. The reasons for these inconsistent results may lie in the different design of each CS project (what kind of project leads to a further development of competencies among participants?), but also in different methodologies or in different methods, e.g., how the relevant variables are captured in different ways, which makes it difficult to compare results across different studies.

3. Research Questions

The studies on the effects of CS projects on participants show diverse and sometimes contradictory results. These diverse results are often justified by the different design of CS projects (e.g., according to Bonney’s typology, see Table 1). However, empirical confirmation of this relationship has been lacking, with initial empirical findings finding no evidence for this relationship [11,12] and a few articles contradicting the model’s proposition without empirical findings [43,44]. However, these initial empirical results were related exclusively to CS projects outside of the school context. Therefore, this work aims to answer the following guiding question within a quasi-experimental design: How do different participation opportunities in a CS project influence the educational outcomes for students in a school context? This question was divided into the following research questions:

• Which level of participation opportunity has the greatest influence on the content knowledge and understanding of science?
• Which level of participation opportunity has the greatest influence on attitudes toward science?
• Which level of participation opportunity has the greatest influence on environmental attitudes?
• Which level of participation opportunity has the greatest influence on environmental behavior?

In line with the model by Bonney and colleagues, as well as Jordan et al. [10,45], it is hypothesized that educational potential is higher in a high-participation project than in a low-participation project.

H1: 

Content knowledge M_Contributory < M_{Collaborative} < M_Co-created.

H2: 

Understanding of science M_Contributory < M_{Collaborative} < M_Co-created.

H3: 

Attitudes toward science M_Contributory < M_{Collaborative} < M_Co-created.

H4: 

Environmental attitudes M_Contributory < M_{Collaborative} < M_Co-created.

H5: 

Environmental behavior M_Contributory < M_{Collaborative} < M_Co-created.

In addition, different methodological approaches (e.g., the assessment of relevant variables) make it difficult to compare the results of different studies. Therefore, in our study, we rely on well-known instruments that have already been validated in other contexts.

4. Materials and Methods

A quasi-experimental design with three experimental groups (Contributory, Collaboration, and Co-created) and three measurement times (pre-, post-, and follow-up test). The intervention was implemented within the CS project QueichNET. Within QueichNET, the students from secondary schools in Germany studied the water quality of the Queich (a river that runs through the Rhineland–Palatinate region).

4.1. Research Design

The three experimental groups (EG) are distinguished by the different involvement of the participants in the scientific process, and were designed according to the participation levels described in [10,46] (see Table 1). The intervention consisted of seven modules each, which were carried out by the teachers, and one workshop, which was carried out by the first author (see

Table 2. The design of the modules for the three experimental groups (EGs).

Contributory	Collaboration	Co-Created
Module 1: Introduction to the topic of flowing water ecology and the Queich. Aim: To arouse the interest of the students (Ss) in the Queich, to familiarize them with the question “What is the condition of the Queich?”, and to familiarize the Ss with the problem of water pollution. Duration: 45–90 min.
Module 2: Introduction to the topic “Scientific Work” Objective: The Ss explain how scientific knowledge is generated with the help of the black box investigation. Duration: 45–90 min
Workshop A Objective: The Ss learn theoretical and practical methods to determine the water status of the Queich. Important: No hypotheses Duration: 90 min.	Workshop B Objective: The Ss learn theoretical and practical methods to determine the water status of the Queich. Important: Hypotheses are given Duration: 90 min.	Module 3c: Formulating own research questions Objective: The Ss transfer their learned knowledge and formulate their own research questions and hypotheses. Duration: 90 min.
Module 3a: Data recording Objective: The Ss record data according to the specifications. Duration: 90 min.	Module 3b: Creation of the study design Objective: The Ss transfer their learned knowledge, plan the data collection, and design the recording sheets. Duration: 90 min.	Workshop C Objective: The Ss learn theoretical and practical methods to determine the water status of the Queich. Important: Hypotheses are made by the Ss Duration: 90 min.
Module 4a: Data recording Objective: The Ss record data according to the specifications. Duration: 90 min.	Module 4b: Data recording Objective: The Ss record their data according to their study design. Duration: 90 min.	Module 4c: Creation of the study design Objective: Ss combine their research questions with the methods and create their own research design. Duration: 90 min.
Module 5a: Data recording Objective: The Ss record data according to the specifications. Duration: 90 min.	Module 5b: Data recording Objective: The Ss record their data according to their study design. Duration: 90 min.	Module 5c: Data recording Objective: The Ss record their data according to their study design Duration: 90 min.
Module 7a: Presentation of results Objective: The Ss create a poster for their research work. Duration: 45–90 min.	Module 6b: Evaluation and interpretation Objective: The Ss apply mathematical tools to their data and perform an evaluation of the data. They interpret the data and refer to the research question. Duration: 45–90 min.	Module 6c: Evaluation, interpretation, and discussion Objective: The Ss apply mathematical tools to their data and perform an evaluation of the data. They interpret and discuss the data, and refer to their questions. Duration: 45–90 min.
Module 7a: Presentation of results Objective: The Ss create a poster for their research work. Duration: 45–90 min	Module 7b: Presentation of results Objective: The Ss learn about the different forms of scientific presentation and create a poster for their research work. Duration: 45–90 min.	Module 7c: Presentation of results Objective: The Ss learn about the different forms of scientific presentation and create a presentation or poster for their research work. Duration: 45–90 min.

). Each teacher was provided with a module manual—which included procedures, worksheets, and an extended class book—to ensure the independent implementation of each module and to document any changes and incidents during the intervention. The teachers did not make use of the documentation in the extended class book. However, after the project was completed, the teachers confirmed that they had carried out the modules as intended and that no deviations had occurred. Within the workshop led by the first author, the students learned to determine water quality theoretically and practically.

Within the project, the EG “Contributory” focused intensely on data collection. The students collected data on water quality at various locations along the Queich. Participants in this EG only descriptively evaluated their own research sites, which is where they had collected water quality data. The students worked without hypotheses. The EG “Collaboration” participated in the development of the study design (3b), data collection (4b and 5b), the analysis and interpretation of the data (6b), and the dissemination of the results (7b). The students additionally learned about the hypotheses that were to be tested. In the EG “Co-created”, the students collaborated in all steps of the scientific process from the research question to discussion. The students came up with their own hypotheses about the water quality of the Queich.

Due to the organizational aspects that were present in cooperation with the schools, the students could not be randomly assigned to the experimental groups individually; instead, whole classes were assigned to the experimental groups. Accordingly, the internal validity of this study is limited. However, a high external and ecological validity can be assumed since the study takes place in the field.

4.2. Instruments

Directly before the start of the project, as well as directly at the end of the project and two to three months after the end of the project, the students filled out a questionnaire. The questionnaire contained scales on the variables of interest in this study (see

Table 3. Overview of the instruments used, with references and sample items.

Variable	Instrument/Source	Item Example
Content knowledge	Knowledge about freshwater ecology and water quality [47], 14 MC items	What are the disadvantages of biological water analysis?
Understanding of science	Nature of Science [48], 7 subscales with 44 items—rating scale of “not true at all (1)”–“is absolutely true (5)”
	Certainty of scientific knowledge	Knowledge in science is true for all time. (-)
	Sources of scientific knowledge	Beginners are not yet able to observe natural phenomena. (-)
	Development of scientific knowledge	The ideas in science textbooks sometimes change.
	Justification of scientific knowledge	An experiment is a good way to find out if something is true.
	Simplicity of scientific knowledge	Scientific theories are often more complicated than they need to be (-).
	Purpose of science	The goal of scientific theories is to explain natural processes.
	Creativity of scientists	Scientific knowledge is also a result of human creativity.
Attitudes toward science	Attitudes toward science [49], 3 subscales, 14 items, rating scale of “I disagree (1)”–“I agree (4)”
	Enjoyment and interest in science (5 items)	I like to read about science.
	Value of science (5 items)	Science is valuable to society.
	Future-oriented science-related motivation (4 items)	I would like to work on science projects as an adult.
Environmental attitudes	2 major environmental values [50], 2 subscales, 20 items, rating scale of “not true at all (1)”–“is absolutely true (5)”
	Utilization of nature (10 items)	We should only protect useful animals and plants.
	Preservation of nature (10 items)	I feel good in the silence of nature.
Environmental behavior	General ecological behavior [51], 6 subscales, 33 items, rating scale of “never” (1)–“very often” (5)
	Energy conservation	As the last person to leave a room, I switch off the lights.
	Mobility	I am driven around by car (-).
	Waste avoidance	I buy procuts in refillable packages.
	Recycling	I separate waste.
	Consumerism	When shopping, I prefer products with eco-labels.
	Vicarious behaviors toward conservation	I am a member of an environmental organization.
Motivation (only post-test)	Short scale on intrinsic motivation [52], 4 subscales, 12 items, rating scale of “not true at all (1)”–“is absolutely true (5)”
	Interest/enjoyment	I enjoyed the activity in the project.
	Perceived competence	I am satisfied with my performance in the project.
	Perceived autonomy	I was able to control the activity in the project myself.
	Pressure/strain	When working on the project, I felt under pressure.

Note: (-) Item must be recoded as it is worded negatively in relation to the scale.

). We also included a short scale on motivational aspects to capture the motivational effects of project participation (but only in the post-test). Since our project is a CS project in a school context, the voluntary nature of participation in the project cannot be directly compared to other CS projects (where participants are highly motivated on their own).

The validity and reliability of the used scales have already been established in other studies, which is why we assumed that they are also sufficiently valid and reliable in our study. Reliability was checked again in our study and is given for each scale in the results section. To operationalize the students’ understanding of science, we used an instrument to assesses the students’ views on the nature of science, as pointed out in Section 2.1.

4.3. Statistical Analyses and Sample

The student data were analyzed using SPSS and an analysis of variance across three measurement time points (with a significance level of α = 0.05). We calculated the sum scores for the content knowledge measure and means on each scale of the questionnaire. To obtain a comprehensive picture of the variables regarding the understanding of science (operationalized as the nature of science) and self-reported environmental behavior, analyses were conducted across all items rather than separately in subscales. We planned the sample size with gPower, assumed a usual significance level of p = 0.05, and a test power of 0.80 for (rm)ANOVA, as well as a small effect size. On this basis, the sample should include 204 students.

The sample consisted of 199 students (53.8% female) who participated in 2018 and 2019, which were split across 12 classes (grade 7–grade 12) and 6 schools. For the calculation of the (rm)ANOVA, however, the sample was reduced because some students did not participate at all measurement time points, and a few did not answer all questions (but this was seldom the case). The presence of missing values could be attributed to absent students (e.g., due to illness); as such, the missing values were completely at random and not related to the variables of interest. We therefore decided to exclude the students in question from the analyses. Between 150 to 156 subjects could be included in the analyses. The students were distributed across the three experimental groups as follows: contributory n = 52 (19 females—grade 7 28.8%, grade 9 30.8%, and grade 12 40.4%), collaboration n = 61 (38 females—grade 7 21.3%, grade 9 27.9%, grade 10 13.1%, and grade 12 37.7%), and co-created n = 43 (31 females—grade 9 44.2% and grade 12 55.8%).

5. Results

The analyses of the influence of participation opportunities on subject knowledge (content knowledge and understanding of science, i.e., NOS), attitudes toward science, environmental attitudes, and environmental behavior are reported below.

5.1. Subject Knowledge

The results for the students’ content knowledge and understanding of science (i.e., NOS) are shown in

Table 4. Descriptive results for the variables of content knowledge and understanding of science. The minimum and maximum for the scales are given, as well as the internal consistency in the pre-, post-, and follow-up tests (Cronbach’s α).

	EG Contributory		EG Collaboration		EG Co-Created
	M	SD	M	SD	M	SD
Content knowledge on water quality (14 items, min-max 0–14, Cronbach’s α = 0.44–0.62)
Pre-test	5.00	2.20	4.21	1.93	5.21	2.14
Post-test	6.94	2.95	6.56	2.16	7.35	2.52
Follow-up test	6.92	2.85	6.08	2.40	6.86	2.64
Understanding of science (i.e., NOS, 44 items, min-max 1–5, Cronbach’s α = 0.89–0.94)
Pre-test	3.75	0.51	3.79	0.31	3.86	0.37
Post-test	3.85	0.53	3.91	0.34	3.89	0.43
Follow-up test	3.83	0.55	3.88	0.42	3.96	0.48

. The students’ content knowledge on water quality increased significantly over the project period (main effect of time, Huyn-Feldt: F(1.96, 300.20) = 77.73, p < 0.001, partial η² = 0.34, indicating a large effect). The interaction effect did not reach statistical significance (interaction effect, Huyn-Feldt: F(3.92, 300.30) = 0.44, p = 0.78, partial η² = 0.01), indicating that all of the students learned about water quality regardless of which experimental group they were assigned to. When turning to the students’ understanding of science (i.e., NOS), similarly, we do not find a significant interaction between the experimental group and measurement time (Huyn-Feldt: F(3.75, 283.13) = 3.02, p = 0.41, partial η² = 0.01). However, we did find a significant small-to-medium main effect of time (F(1.88, 283.13) = 8.42, p < 0.001, partial ƞ² = 0.05)—this meant that all students learned about NOS regardless of the experimental group.

5.2. Attitudes toward Science

The attitudes toward science were measured with three subscales: enjoyment and interest in science, the general value of science as seen by the students, and future-oriented science-related motivation (see

Table 5. Descriptive results for the variables attitudes toward science. The minimum and maximum of the scales are given, as well as the internal consistency in the pre-, post-, and follow-up tests (Cronbach’s α).

	EG Contributory		EG Collaboration		EG Co-Created
	M	SD	M	SD	M	SD
Enjoyment and interest in science (5 items, 1–4, Cronbach’s α = 0.91–0.93)
Pre-test	2.87	0.61	2.96	0.65	2.91	0.72
Post-test	2.83	0.65	3.00	0.67	2.67	0.93
Follow-up test	2.73	0.77	2.88	0.80	2.82	0.86
Value of science (5 items, 1–4, Cronbach’s α = 0.62–0.74)
Pre-test	3.00	0.46	3.15	0.42	3.29	0.37
Post-test	2.97	0.50	3.06	0.46	3.10	0.58
Follow-up test	2.99	0.45	3.07	0.51	3.10	0.46
Future-oriented science-related motivation (4 items, 1–4, Cronbach’s α = 0.91–0.92)
Pre-test	1.98	0.83	2.29	0.83	2.10	0.97
Post-test	1.95	0.79	2.31	0.83	2.19	0.98
Follow-up test	2.05	0.83	2.34	0.87	2.05	1.00

).

In the EG Collaboration, enjoyment and interest in science increased slightly from pre- to post-test, while in the EG Contributory and EG Co-created, enjoyment and interest decreased over the project period (interaction effect: F(4, 300) = 2.54, p = 0.004, partial η² = 0.03). This influence of the EG was only statistically significant in the short term from pre-test to post-test (F(2, 150) = 4.08, p = 0.02, and partial η² = 0.05), and not in the long run.

The general value students place on science significantly decreases statistically across the measurement time points (main effect of time, F(2, 298) = 3.84, p = 0.02, partial η² = 0.03). Both the attitude change from pre- to post-test (F(1, 149) = 6.27) and from pre- to follow-up test (F(1, 149) = 5.51) were significant (p < 0.05). We found no interaction effect (F(4, 298) = 0.88, p = 0.48, partial η² = 0.01), thus indicating that the experimental group did not affect the students differently.

The students’ future-oriented science-related motivation did not change significantly over the project period (main effect of time, Huyn-Feldt F(1.97, 290.08) = 0.17, p = 0.84, partial η² = 0.00), and the EGs also had no significant effect on this variable (interaction effect, Huyn-Feldt: F(3.95, 290.08) = 0.77, p = 0.55, partial η² = 0.01).

5.3. Environmental Attitudes

Results on the students’ environmental attitudes in terms of the utilization of nature or the preservation of nature are presented in

Table 6. Descriptive results for the variable of environmental attitudes. The minimum and maximum of the scales are given, as well as the internal consistency in the pre-, post-, and follow-up tests (Cronbach’s α).

	EG Contributory		EG Collaboration		EG Co-Created
	M	SD	M	SD	M	SD
Utilization of nature (10 items, 1–5, Cronbach’s α = 0.77–0.82)
Pre-test	1.89	0.63	1.82	0.54	1.71	0.58
Post-test	1.79	0.58	1.82	0.57	1.56	0.51
Follow-up test	1.92	0.74	1.79	0.64	1.56	0.42
Preservation of nature (10 items, 1–5, Cronbach’s α = 0.79–0.81)
Pre-test	3.17	0.58	3.25	0.56	3.30	0.62
Post-test	3.26	0.63	3.18	0.63	3.19	0.70
Follow-up test	3.26	0.59	3.17	0.65	3.24	0.56

. They did not change significantly depending on the experimental groups (utilization, interaction effect, Huyn-Feldt: F(3.18, 241.73) = 1.66, p = 0.17, partial η² = 0.02; preservation, interaction effect, Huyn-Feldt: F(3.91, 297.42) = 1.74, p = 0.14, partial η² = 0.02). There was also no statistical evidence of an effect of time (utilization, Huyn-Feldt: F(1.59, 241.73) = 2.53, p = 0.09, partial η² = 0.02; preservation, Huyn-Feldt: F(1.96, 297.42) = 0.34, p = 0.71, partial η² = 0.00). All in all, the students’ environmental attitudes remained relatively constant.

5.4. Environmental Behavior

Results on the students’ environmental behavior are shown in

Table 7. Descriptive results for the variable general environmental behavior. The minimum and maximum of the scales are given, as well as the internal consistency in the pre-, post-, and follow-up tests (Cronbach’s α).

	EG Contributory		EG Collaboration		EG Co-Created
	M	SD	M	SD	M	SD
General environmental behavior (33 items, 1–5, Cronbach’s α = 0.81–0.85)
Pre-test	3.11	0.50	3.22	0.48	3.33	0.38
Post-test	3.20	0.51	3.22	0.51	3.34	0.44
Follow-up test	3.25	0.48	3.27	0.51	3.46	0.35

. It was not affected differently by the experimental groups that the students were assigned to (F(4, 304) = 1.27, p = 0.28, partial η² = 0.02). However, the students’ environmental behavior significantly improved over the project period (F(2, 304) = 11.20, p < 0.00, partial η² = 0.07). The change from pre-test to post-test was not significant, but the change from pre-test to follow-up test was (F(1, 152) = 19.55, p = 0.00, partial η² = 0.11).

5.5. Motivation

The results on the students’ motivation during the project, which we only assessed in the post-test, are shown in

Table 8. Descriptive results for motivational aspects, which we only assessed in the post-test. The minimum and maximum of the scales are given, as well as the internal consistency (Cronbach’s α).

	EG Contributory		EG Collaboration		EG Co-Created
	M	SD	M	SD	M	SD
Motivation (12 items, 1–5, Cronbach’s α = 0.65–0.91)
Motivation total	3.59	0.65	3.66	0.68	3.53	0.55
Interest/enjoyment	3.82	0.93	3.82	1.07	3.52	0.84
Perceived competence	3.70	0.79	3.56	0.92	3.48	0.82
Perceived autonomy	3.17	0.97	3.20	0.78	3.42	0.81
Pressure/strain	2.34	0.84	1.96	0.80	2.31	0.85

. We did find a statistically significant difference in how the participants perceived pressure or strain during the project (F(2, 164) = 3.52, p = 0.03, partial η² = 0.04). In the EG Collaboration, the perceived pressure was significantly lower than in the EG Contributory and EG Co-created. In all other motivational aspects, there were no differences between the groups.

6. Discussion

We hypothesized, in line with the model by Bonney et al. [10] and with the theories on learning during inquiry tasks in science [8], that a higher level of participation opportunities in a CS project would lead to higher educational benefits for the participants. Our empirical findings do, however, not support this assumption. Of all the variables we examined, only the subscale enjoyment and interest in science (attitudes toward science) was influenced by the experimental groups. However, contrary to our hypotheses that higher levels of participation are associated with higher educational potential, the participants in the group with an intermediate level of participation opportunities benefited the most (in the sense that they reported higher enjoyment and interest in science).

With regard to the effects of different levels of participation opportunities, our results are in line with other studies [11,12,13,38,43,44,53], which also found no differences in the educational potential between the different forms of participation that correspond with the model by Bonney et al. [10]. These authors suggest that the levelsof participation are not necessarily essential for the educational potential of CS projects, but that other variables and factors play a more decisive role, such as the motivation of the participants and the consistency of participation in the project. Since the CS project studied here was situated in a formal educational context, i.e., a school, further investigation of the motivational factors seems particularly promising. In fact, for this reason, we collected motivational data regarding the work in the project. In general, there were almost no differences in the reported motivations between the different groups. Only the subscale of perceived pressure varied significantly between the experimental groups. The EG Collaboration showed a lower perceived pressure than the other two experimental groups. This means that, when regarding motivational aspects, the group with an intermediate level of participation opportunities shows the most favorable characteristics for learning. Nevertheless, further analysis is needed, particularly with regard to the initial motivation for participating in a project (i.e., motivation or interest before the project begins) and how motivation changes as a result of the work and tasks involved in the project. In this regard, there are some studies that report that the students particularly enjoyed working in the field [11,13,29,36]. Philipps et al. [53] also indicated that participants prefer to take on easier tasks and leave the more difficult tasks to the scientists. Taken together, this could indicate that the students perceived the practical work in the field, in particular, as formative and that the other work was less enjoyable for them, such that their learning was also influenced by this. Thus, the middle level of participation opportunities (EG Collaboration) can represent the middle ground, which is neither over demanding nor under demanding for the participants (see also [54,55]). However, this needs to be explored in more detail in further studies, which may include additional interviews with participants to validate their perceptions of the different steps and tasks of the CS project.

Regardless of the participation opportunities, all participants in the CS project have acquired content knowledge and knowledge about science (i.e., NOS) over time. This finding is in line with other studies [27,32,33]. In addition, the students’ general environmental behavior also improved at the follow-up test 2–3 months later, and this was similar to the results of other studies (e.g., [21,26,27,28]). This fact means that we did find positive changes that occurred over time of the CS project.

However, when turning to the attitudes of participants, the picture is not quite so clear. In terms of the general value that participants attribute to science, we found a negative effect, although it is not particularly large. Druschke and Selzer [22] also report negativeeffects on students’ attitudes, although they were not statistically significant. We can only speculate about the reason why we found a negative effect in our project. A possible explanation would be that the students may have had an inaccurate idea of scientific work before the project and that they adjusted their attitude somewhat due to the insight into scientific work during the project. Additionally, the effects seem to be strongest, although not significant, in the Co-created group, so the students may have been overwhelmed with the more difficult tasks in this group. Nevertheless, this result is contrary to the intention of CS projects. Although the limitations of the study (see below) make the interpretation difficult, it may be valuable to approach this phenomenon more closely as negative effects have not yet been reported so frequently in the CS project literature. The participants’ environmental attitudes were not influenced by the CS project and remained relatively constant. In other studies, we also did not find effects of CS projects on the participants’ attitudes (e.g., [3,28]). This could be due to the fact that attitudes are relatively stable constructs [56], and the interventions may not address them properly.

Research on CS is a young and rapidly developing field of research. However, the educational potential of CS, in particular, is often neglected. There is a lack of systematic and comparative studies [40], which we have tried to address with our study. In order to arrive at a more comprehensive research picture of CS, CS projects should therefore be examined in terms of which factors are decisive in determining educational potential. Based on empirical comparative studies, the factors and structures that condition and promote successful learning in CS can be identified. This is especially important for CS projects that are implemented in formal educational settings, such as schools. For CS project organizers, the inclusion of formal educational institutions offers the opportunity to diversify the participant base and attract a broader audience that may wish to volunteer in the CS project. Within this study, integrated secondary schools, high schools, and private schools were all attracted to the CS project, and many students participated in the CS project who had never had contact with universities or science before. The latter is particularly relevant with regard to education for sustainable development [14], as the integration of CS in schools can reach many people who would otherwise not participate in a CS project on ecological or environmental issues in their free time.

7. Conclusions

In summary, we can conclude—by drawing on the results of other studies as well as the results in this study—that the level of participation opportunities in citizen science projects is probably not the decisive factor that influences educational potential. Other factors seem to be more decisive, e.g., the duration of participation or the motivation of the participants. For both informal and formal education, it is relevant how the different science tasks within a CS project are structured and adapted to the target group in order to avoid over and under challenging the participants. In the school context, this is particularly enabled by integrating the project in the classroom by linking it to the science curriculum, and by providing targeted preparation and follow-up.

8. Limitation

The validity of our study is affected by some limitations, which we would like to address in the following section. The first limitation is low test power in the hypothesis testing. A total of 199 students participated in the intervention, and this corresponded with the previously determined sample size that was obtained via gPower. However, experimental mortality was high at 25%; thus, only 156 students could be included in the analyses, thus resulting in low test power. This must be considered when interpreting the findings. In addition, the students could not be assigned to the three experimental groups in a completely randomized manner due to organizational conditions. A randomized assignment of whole-class groups was carried out, which—however—resulted in a cluster sample. The instruments used in this study were already used in other studies with students, and their validity was shown but not specifically in the CS context. Nevertheless, we assume that our study can make a significant contribution to the clarification of the conditional determinants for the educational potential of CS projects.