Incentivizing Student Participation in QAS Questionnaires: An Evaluation of a Guaranteed Prize System at the University of Malaga

Abstract

This paper investigates the effects of a guaranteed prize incentive, in the form of an extra score, on student engagement in the quality assurance system (QAS) questionnaires employed for evaluating teaching performance at the University of Malaga. The incentive system aims to counteract declining participation rates and mitigate potential survey fatigue among students. Employing a comprehensive dataset spanning multiple academic years and subjects, the study utilized statistical analyses to evaluate the incentive’s effectiveness, considering its potential impact on both final grades and QAS questionnaire outcomes. The results demonstrate a substantial increase in participation rates, with over 85% of students acknowledging the motivating influence of the incentive. However, concerns regarding the compromise of anonymity arose among 40% of students, possibly linked to the physical presence of teachers during the verification process of the QAS questionnaire submission. The statistical analyses raise questions about the incentive’s influence on students’ final grades while indicating that the incentive system does not significantly affect the results of the QAS questionnaires. The study contributes valuable insights into the complexities of incentivizing student participation in teaching assessments within the higher education landscape. To the best of our knowledge, there are few publications that investigate the use of an additional score as an incentive for students’ participation in QAS questionnaires.

1. Introduction

The adaptation of studies to the European higher education area (EHEA) has led Spanish universities to develop systematic evaluation mechanisms for the teaching–learning process in terms of methodology and teaching competencies [1]. To enhance quality in line with the standards outlined by the European Association for Quality Assurance in Higher Education (ENQA), universities have established internal and external systematic mechanisms for evaluating the quality of their activities. The official regulations in Spain [2,3,4] mandate that university programs must incorporate an internal quality assurance system encompassing, among other aspects, evaluation and improvement procedures for teaching and staff. These programs are required to adhere to the criteria and guidelines for quality assurance outlined in the EHEA (Standards and Guidelines for Quality Assurance in the European Higher Education Area, ESG).

In this context, the evaluation of teaching activities has become a significant aspect of this assessment. Consequently, the “DOCENTIA” programs were initiated in Spanish universities in 2007 [5], aligning with standards established by internationally recognized organizations for teacher evaluation. These standards, particularly those outlined in “The Personnel Evaluation Standards” by The Joint Committee of Standards for Educational Evaluation [6], serve as a reference for designing, developing, and evaluating teacher assessments. DOCENTIA programs provide a procedure that allows addressing the evaluation of teaching activity in all areas of action of university teaching staff by analyzing four basic dimensions: (1) teaching planning, (2) teaching development, (3) results, and (4) innovation and improvement. The various stakeholders engaged in the learning process, such as department heads, deans, the teaching innovation service, teachers, and students, participate in assessing these four dimensions. Notably, the students’ perspective currently holds a significant influence on the appraisal of teaching quality [5,7]. Moreover, in the context of evaluation and improvement procedures, the established protocol includes deliberate mechanisms linking teacher evaluation to their training, acknowledgment, and promotion, as outlined in the support guide of the verification program [4].

In this sense, Spanish universities have developed questionnaires that incorporate the assessment of various items related to the teaching–learning process for each subject and for all the teachers involved in that subject. These questionaries are normally filled out voluntarily and anonymously by students at the end of the semester.

Table 1. Questionnaire for students at the University of Malaga.

No.	Questions
Teaching Planning
Teaching and Learning Planning
Design of the programs/teaching guides/subject guide
1	The teacher informs about the different aspects of the teaching guide or program of the subject (objectives, activities, syllabus contents, methodology, bibliography, evaluation procedure, etc.).	1	2	3	4	5	NR/DK
Teaching Development
Instructional Development
Adherence to the planned activities
2	The teacher complies with the planning established in the teaching guide/program of the subject (objectives, evaluation systems, bibliography, and other recommended sources of information).	1	2	3	4	5	NR/DK
3	The planned theoretical and practical activities have been coordinated.	1	2	3	4	5	NR/DK
Teaching methodology
4	The teacher organizes the activities conducted in class well.	1	2	3	4	5	NR/DK
5	Utilizes teaching resources (blackboard, transparencies, audiovisual media, and support materials on the virtual network) to facilitate learning.	1	2	3	4	5	NR/DK
Teaching competencies developed by the teacher
6	Explains clearly and confidently and highlights important content.	1	2	3	4	5	NR/DK
7	The teacher is concerned about the level of comprehension of their explanations.	1	2	3	4	5	NR/DK
8	Resolves any questions or concerns that may arise.	1	2	3	4	5	NR/DK
9	Encourages students to develop an interest in the subject (fluent and spontaneous communication).	1	2	3	4	5	NR/DK
10	Is respectful in dealing with students.	1	2	3	4	5	NR/DK
Learning Evaluation
Evaluation system
11	I have a clear understanding of what I need to learn to pass this subject.	1	2	3	4	5	NR/DK
12	I consider the established evaluation criteria and systems adequate to assess my learning.	1	2	3	4	5	NR/DK
Results
Effectiveness
13	The activities undertaken (theoretical sessions, practical exercises, individual work, group projects, etc.) contribute to the achievement of the subject’s objectives.	1	2	3	4	5	NR/DK
Student satisfaction
14	I am satisfied with the teaching performance of this teacher.	1	2	3	4	5	NR/DK

shows the questionary used at the University of Malaga, indicating the three dimensions to which the students’ evaluations contribute. The items are assessed using a Likert scale with a scoring range from 1 (completely disagree) to 5 (completely agree). The results of these questionnaires contribute 28% to the overall evaluation of teaching activity, while the remaining evaluation is contributed by teaching managers (30%), the innovation service (30%), and teachers (12%).

Teachers undergoing the accreditation process for promotion require the assessment, which is expressed numerically on a scale from 0 to 100, with the following categorization: unfavorable (<50), favorable (50–69), very favorable (70–89), and excellent (>90).

Questionnaires are a valuable tool in the assessment of teaching quality, enabling the collection of feedback from students and the generation of easily analyzable quantitative data [8,9]. The anonymity provided by questionnaires encourages students to express their opinions freely. However, the widespread use of surveys, driven by the desire to ensure teaching quality, has resulted in potential survey fatigue among students [10,11]. This saturation may affect the quality of responses, as individuals can become overwhelmed by the number of surveys they encounter. Furthermore, student participation in surveys is typically voluntary, and this fatigue can result in low participation, leading to potential underrepresentation and biased feedback.

Making survey participation mandatory could raise ethical concerns, as it infringes upon students’ autonomy and may generate coerced or insincere responses. The imposition of mandatory participation can lead to student resentment and compromise the integrity of the feedback. In such a scenario, students may provide responses just to fulfill the requirement, potentially undermining the purpose of collecting meaningful feedback. Incentivizing students to engage in surveys emerges as a potential solution [12,13]. While incentives can enhance participation rates, the choice of incentives is crucial for ensuring the integrity of feedback.

This study focused on the use of incentives to encourage student participation in the questionnaires for evaluating teaching quality. It is worth noting that we detected a lack of publications in the existing literature exploring the implementation of a supplementary evaluation score as a motivational tool for student participation in QAS questionnaires. In this context, the proposed incentive system provides an extra score equivalent to 5% of the overall attainable grade for the student. The QAS questionnaires are provided by the university’s quality service and distributed by the teacher. The responses, which serve as the primary data source in this study, were collected from the student participants through an online platform. It is crucial to emphasize that this online format ensures the anonymity of the student responses. The research was conducted across a total of five subjects, including diverse undergraduate and master’s courses from two different degree programs of the University of Malaga, to guarantee the representativeness of the study results. With the aim of assessing the impact of the incentive system, the results were compared with those from previous academic years where the incentive system was not employed. The proposed questions that served as a guide for this research are the following:

• How does the proposed guaranteed prize incentive system influence student participation in quality assessment questionnaires?
• Is there any significant relationship between the incentive system and students’ final grades?
• How does the incentive system influence the student’s perception regarding the anonymity of their responses to the survey?

The assessment of the impact of the proposed prize incentive system is a critical aspect of this research. Understanding how incentives influence student engagement in the evaluation process is essential for improving the effectiveness of questionnaires as a tool for assessing teaching quality. The proposed methodology aims to assess the motivational impact of the incentive system. Studying the potential relationship between the incentive system and the students’ final grades is crucial to understand its implications on academic outcomes. Finally, this paper explores students’ perceptions regarding the anonymity of their questionnaire responses, focusing on understanding their concerns regarding the confidentiality of their feedback.

2. Materials and Methods

The study was conducted as part of an educational innovation project at the University of Malaga (see funding section), one of whose objectives was to evaluate the use of incentives in assessing teaching quality.

2.1. Participants

The initiative was implemented in five subjects across various undergraduate and master’s programs, as detailed in

Table 2. Characteristics of subjects under study.

Level	Program	Subject	Acronym	Course	Character
Undergraduate	Chemical Engineering	Simulation and Optimization of Chemical Process	SOCP	4	Mandatory
Undergraduate	Chemical Engineering	Integrated Management Systems	IMS	3	Mandatory
Undergraduate	Chemical Engineering	Treatment of Contaminated Soil	TCS	4	Optional
Master	Chemical Engineering	Mass Exchange Network for Pollution Prevention	MENPP	1	Optional
Undergraduate	English Studies	Psycholinguistics Applied to the Study of the English Language	PASEL	3	Optional

. Students participating in this study were enrolled in at least one of these five subjects. The age range of the participants is 20 to 24 years, comprising 46% men and 54% women.

In all the programs where the study was conducted, each semester comprises six subjects, and it is common for each of these subjects to be taught by more than one teacher. Hence, the number of questionnaires to which the students are subjected ranges between six and twelve per semester. This substantial volume of surveys has led to a decline in student participation in this evaluation process. Specifically, the teachers of the subjects analyzed in the present study observed the beginning of this decline between 2016 and 2018. Within the extensive data published in the public information section of the quality assurance system (QAS) of the University of Malaga, information on the student response rate to the QAS questionnaire (IN_45) is available only from the 2019–2020 academic year [14]. The values of this response rate for the entire university are alarmingly low: 5.20%, 14.69%, and 12.40% for the academic years from 2019–2020 to 2021–2022.

Based on these data, the necessity for implementing a student incentive system to encourage questionnaire completion is evident. Moreover, the design of this incentive should be compatible with the official procedure employed for collecting responses to these questionnaires.

2.2. Instruments

Questionnaires serve as the primary tool for gathering data from participants. This study employed two types of questionnaires: the official QAS questionnaire of the University of Malaga (Table 1) and a concise, self-developed questionnaire designed to gather information regarding the implementation of the new incentive system.

Furthermore, the distribution of students’ final grades in the academic years under investigation was obtained while maintaining the anonymity of the students. This information was sourced from the professors responsible for each of the subjects.

2.2.1. Official QAS Questionnaire

The official QAS questionnaire model is imposed by the quality service of the University of Malaga. Its structure is common for all programs offered at the university, which prevents any modification. Logically, the participants’ responses to these questionnaires are anonymous.

The current model presented in Table 1 has been in use since the 2017–2018 academic year. It represents the culmination of an evolutionary process from previous questionnaires; it initially comprised 35 questions (before 2007–2008), was then reduced to 22 questions (until 2016–2017), and then reached the current version with 14 questions. This reduction in the number of questions is designed to mitigate survey fatigue among participants.

2.2.2. Questionnaire about the New Incentive System

This questionnaire (

Table 3. Questionnaire about the new incentive system.

No.	Questions
1	Has the additional score motivated you to participate in the official QAS questionnaire?	YES	NO
2	Do you believe that the procedure followed for awarding this extra score has compromised anonymity?	YES	NO
3	Please indicate approximately the number of official QAS questionnaires that you have completed during the current semester.

) is intentionally concise to avoid overwhelming students with an excessive number of questions. The participants’ responses to this questionnaire are anonymous. Its aim is to gather information regarding the following aspects:

• Evaluating students’ satisfaction with the incentive system;
• Determining the potential compromise of anonymity resulting from the teacher’s presence during the QAS questionnaire;
• Verifying if participation in official QAS questionnaires across all subjects taught in that semester remained low.

2.3. Procedure

2.3.1. Official QAS Questionnaire

Initially, prior to the 2018–2019 academic year, the compilation of the questionnaires was entrusted to a group of students who received a scholarship to carry out this task (1–2 students per center). These students assumed the role of surveyors, actively engaging with their peers in class to conduct the surveys. During the survey process, the teacher temporarily left the classroom for approximately 15 min, allowing the surveyors to explain the significance of gathering students’ opinions for the evaluation of teaching quality. The students manually completed the questionnaires on paper, assuring their anonymity. This collection system offered the advantage of raising a sense of peer-to-peer encouragement: A fellow student, familiar to their classmates, served as a motivating force, encouraging active participation in the questionnaire process.

In the 2018–2019 academic year, the system for collecting surveys changed to the current system. The main change involved transitioning from in-person surveys to online surveys, eliminating the role of the student surveyor and placing the responsibility of survey administration on the teaching staff. The current process at the University of Malaga is outlined below, following the guidelines from the quality service of the university. The questionnaires are filled out online, and there are two possible mechanisms for distributing them to students:

1.

On-line distribution: The access code to the questionnaire is sent to students through forums, email, etc. The teacher specifies the period during which the questionnaire will be available, typically spanning several days.

2.

In-class distribution: At the beginning of a class, the teacher provides the access code to the students present in the class. Students can then complete the questionnaire using their mobile devices, requiring the teacher to briefly leave the class for approximately 15 min.

In both procedures, students are required to identify themselves to access the questionnaire. The teacher must communicate to students that the quality service of the University ensures the anonymity of the survey. The identification process is implemented to prevent errors and duplications and to ensure that each teacher and corresponding subject are accurately evaluated. In cases of duplicate assessments, the last survey completed by the student is used.

Of the two options for distributing the questionnaires, only the second one allows us to reliably determine whether the student has participated, and thus, it enables us to encourage that participation.

The incentive employed involves providing an extra score equivalent to 5% of the total achievable grade for the student. In the case in which two teachers are involved in teaching a subject, as is the case of the IMS subject, the extra score is extended to 10%: 5% for each questionnaire associated with each teacher. The grading system in Spain ranges from 0 to 10 points. In theory, this supplementary score could enable the student to surpass the maximum grade of 10. If a student attains the maximum grade (10), the additional score would not be applicable. While one might consider that this detail may not be motivating enough for students who consistently achieve high grades, it is reasonable to assume that such students would not refrain from participating in any activity that could enhance their final grade. Naturally, teachers communicate the incentive mechanics clearly to students, ensuring a thorough understanding of the potential impact on their overall grades.

As previously mentioned, the surveys are administered in the face-to-face class by the teacher. However, this incentive system deviates from the directive from the university’s quality service, which specifies that the teacher should leave the classroom during the survey. In this innovative procedure, the teacher remains apart from the students in a corner of the classroom. When a student completes and submits the online questionnaire, they request the teacher to approach and view the acknowledgment screen presented by the online application upon survey submission. Importantly, the student’s answers to the questionnaire are not visible on the device screen during this verification. After confirming the student’s participation, the teacher records the student’s name to assign the corresponding extra score. To maintain transparency and motivation, the list of participating students along with their respective additional scores is published on the subject webpage the following day, providing the timely feedback. The results of the questionnaires from one semester are provided to the professor in the subsequent semester once the evaluation for the preceding semester has concluded.

2.3.2. Questionnaire about the New Incentive System

During the final week of classes, the teacher distributes a printed version of this short questionnaire in class. Students anonymously complete the questionnaire within approximately 10 min. Subsequently, a student volunteer collects the responses from their classmates and delivers them collectively to the teacher.

The new incentive system was initially implemented in two of the subjects involved in this study (SOPC and IMS). Following the confirmed success of the incentive in encouraging questionnaire participation, the system was subsequently extended to the remaining subjects.

In a university context, assessing the effectiveness of an incentive system for enhancing student participation in questionnaires goes beyond its success in increasing that engagement. It requires an examination of whether the extra score significantly affects a student’s final grade and an analysis of its impact on the outcomes of the QAS questionnaires. The students’ final grade should predominantly reflect the assessment of acquired skills related with the subject. To ascertain these impacts, a statistical comparison was conducted, analyzing the grades from academic years implementing the incentive system against those from previous courses where this incentive was not employed. Similarly, the outcomes of the QAS questionnaires were also compared for those same academic years. These comparative analyses serve as a comprehensive approach to gauge the system’s influence on participation, academic outcomes, and feedback from questionnaires, ensuring a detailed assessment of its overall effectiveness.

3. Results

Figure 1 shows the evolution of student participation in QAS questionnaires for the subjects SOCP and IMS. The modifications in the system for collecting questionnaire responses are also highlighted, along with the period of COVID-19 confinement in Spain. The gaps in the trends result from the fact that none of the authors held the teaching responsibility for the subject during those periods. Consequently, the feedback from those questionnaires remains private to safeguard the personal assessments of the instructors who taught the subject in those academic years.

As can be seen, participation in the questionnaires started declining in both subjects from 2016. The shift to an online survey administration did not stop this trend. The slight increase in participation observed during the COVID-19 confinement could probably be attributed to the exceptional circumstances of the confinement and the widespread imperative for interpersonal engagement. Nevertheless, participation in the academic year 2020–2021 remained consistently low. It was only with the implementation of the incentive system that a significant surge in participation became evident.

The significant enhancement in participation observed in those two subjects during the 2021–2022 academic year led to the extension of the incentive system to the remaining subjects included in this study for the 2022–2023 courses.

Table 4. Students enrolled in the subjects and their participation in the QAS questionnaires.

Subject	Enrollment			Participation in Questionnaires			Participation-to-Enrollment Ratio
Academic Year	20–21	21–22	22–23	20–21	21–22	22–23	20–21	21–22	22–23
SOCP	59	42	36	4	29 *	29 *	7%	69%	81%
IMS	40	48	41	2	33 *	31 *	5%	69%	76%
TCS	16	7	18	10	6	18 *	63%	86%	100%
MENPP	10	15	12	0	0	7 *	0%	0%	58%
PASEL	84	88	96	0	6	24 *	0%	7%	25%

* Shaded cells indicate that the incentive system was applied.

presents the results obtained in all five subjects over the three academic years under investigation.

Table 5. Results of the questionnaire about the new incentive system.

No.	Questions	YES	NO	NR/DK
1	Has the additional score motivated you to participate in the official QAS questionnaire?	121	14	2
2	Do you believe that the procedure followed for awarding this extra score has compromised anonymity?	52	84	1
3	Please indicate approximately the number of official QAS questionnaires that you have completed during the current semester.	Mean = 3.8 Standard deviation = 2.8

shows the responses of students to the short questionnaire regarding the new incentive system. The overall participation rate of students in this questionnaire was 70%.

The advantage of the incentive used in this study is that it imposes no monetary cost on the institution; it involves an extra score added to the student’s final grade. However, as mentioned earlier, it is essential to verify that the impact of this additional score does not significantly affect the overall final grades of students, ensuring that it does not unduly influence the academic assessment process. The box-and-whisker diagram in the Figure 2 compares the grades achieved by students over the three academic years under examination. As the same teacher taught the SOCP subject for over 5 years, its results span two additional academic years, taking advantage of the data availability.

At first glance, there seems to be no substantial impact of the incentive on the final grades of the two mandatory subjects, SOPC and IMS. Furthermore, if any effect were present in IMS, it would apparently go against what was expected. In the TSC subject, the upward trend in the average grade seems to precede the introduction of the incentive. The average grade trends in MENPP and PASEL subjects appear to exhibit more randomness.

To quantify whether the incentive significantly affects final grades, a one-way analysis of variance (ANOVA) [15] was employed to examine whether the differences between the average grades across academic years are statistically significant.

Table 6. ANOVA results for all subjects at a significance level of α = 0.05.

	Source of Variation
	Between Groups		Within Groups
Subject	Sum of Squares	Degree of Freedom	Sum of Squares	Degree of Freedom	F	F Critical	Significant
SOPC	47.4	4	913.2	182	2.362	2.421	No
IMS	21.1	2	133.9	95	7.500	3.092	Yes
TCS	18.8	2	31.5	36	10.756	3.259	Yes
MENPP	27.7	2	46.9	30	8.862	3.316	Yes
PASEL	30.4	2	267.3	139	7.902	3.061	Yes

summarizes the ANOVA results for all subjects at a significance level of 0.05, treating academic years as distinct groups.

Furthermore, we can take a step further and examine whether specific groups differ significantly from one another in the four subjects where ANOVA results indicate significant differences in average grades across the three academic years under study.

Table 7. Tukey’s tests at a significance level of α = 0.05.

		Differences in Average	Standard Error	q Critical	Difference in Average Critical	Significant
IMS	20–21 and 21–22	0.92	0.212	3.367	0.71	Yes
	21–22 and 22–23	0.20	0.201	3.367	0.68	No
	20–21 and 22–23	1.11	0.214	3.367	0.72	Yes
TCS	20–21 and 21–22	1.03	0.319	3.457	1.10	No
	21–22 and 22–23	0.48	0.312	3.457	1.08	No
	20–21 and 22–23	1.51	0.231	3.457	0.80	Yes
MENPP	20–21 and 21–22	1.41	0.384	3.486	1.34	Yes
	21–22 and 22–23	2.11	0.362	3.486	1.26	Yes
	20–21 and 22–23	0.70	0.398	3.486	1.39	No
PASEL	20–21 and 21–22	0.27	0.212	3.350	0.71	No
	21–22 and 22–23	1.05	0.198	3.350	0.66	Yes
	20–21 and 22–23	0.79	0.200	3.350	0.67	Yes

The academic years in which the incentive system was applied are marked in bold.

presents the results of pairwise multiple comparison tests, highlighting the academic years with incentive in bold.

Finally, the outcomes of the QAS questionnaires were also compared for the same academic years across all subjects, except for MENPP, as it has questionnaire results available only for the 2022–2023 academic year. These comparisons are presented in

Table 8. ANOVA results for QAS questionnaires in the SOCP subject (α = 0.05).

	Source of Variation
	Between Groups		Within Groups
Question No.	Sum of Squares	Degree of Freedom	Sum of Squares	Degree of Freedom	F	F Critical	Significant
1	0.327	2	22.03	58	0.431	3.156	No
2	0.222	2	24.62	59	0.266	3.153	No
3	0.290	2	20.76	59	0.412	3.153	No
4	0.684	2	19.73	60	1.040	3.150	No
5	0.180	2	22.21	59	0.239	3.153	No
6	1.447	2	19.63	60	2.211	3.150	No
7	0.343	2	11.93	59	0.848	3.153	No
8	0.018	2	0.97	59	0.561	3.153	No
9	1.019	2	29.45	59	1.021	3.153	No
10	0.171	2	16.43	60	0.312	3.150	No
11	0.107	2	9.59	59	0.330	3.153	No
12	0.370	2	25.06	60	0.443	3.150	No
13	0.313	2	22.00	60	0.427	3.150	No
14	0.140	2	18.83	59	0.220	3.153	No

,

Table 9. ANOVA results for QAS questionnaires in the IMS subject (α = 0.05).

	Source of Variation
	Between Groups		Within Groups
Question No.	Sum of Squares	Degree of Freedom	Sum of Squares	Degree of Freedom	F	F Critical	Significant
1	0.093	2	17.44	63	0.168	3.143	No
2	0.117	2	13.73	64	0.272	3.140	No
3	0.295	2	13.30	63	0.698	3.143	No
4	1.074	2	14.99	58	2.077	3.156	No
5	0.027	2	32.09	63	0.027	3.143	No
6	0.596	2	36.15	64	0.527	3.140	No
7	0.747	2	56.24	63	0.418	3.143	No
8	1.023	2	18.05	64	1.813	3.140	No
9	2.104	2	35.74	62	1.825	3.145	No
10	0.073	2	7.39	64	0.314	3.140	No
11	0.539	2	47.88	64	0.360	3.140	No
12	0.843	2	27.28	63	0.973	3.143	No
13	0.047	2	22.40	64	0.067	3.140	No
14	0.919	2	17.40	63	1.664	3.143	No

,

Table 10. ANOVA results for QAS questionnaires in the TCS subject (α = 0.05).

	Source of Variation
	Between Groups		Within Groups
Question No.	Sum of Squares	Degree of Freedom	Sum of Squares	Degree of Freedom	F	F Critical	Significant
1	1.705	2	32.68	31	0.809	3.305	No
2	0.502	2	27.73	31	0.281	3.305	No
3	0.021	2	21.62	30	0.014	3.316	No
4	1.626	2	26.84	31	0.939	3.305	No
5	0.763	2	20.68	31	0.572	3.305	No
6	1.932	2	27.70	30	1.046	3.316	No
7	0.391	2	25.85	30	0.227	3.316	No
8	0.920	2	20.84	31	0.684	3.305	No
9	2.252	2	31.93	30	1.058	3.316	No
10	0.602	2	19.46	30	0.464	3.316	No
11	2.771	2	31.61	31	1.359	3.305	No
12	1.391	2	26.84	31	0.803	3.305	No
13	1.129	2	20.90	31	0.838	3.305	No
14	1.705	2	24.68	31	1.071	3.305	No

and

Table 11. ANOVA results for QAS questionnaires in the PASEL subject (α = 0.05).

	Source of Variation
	Between Groups		Within Groups
Question No.	Sum of Squares	Degree of Freedom	Sum of Squares	Degree of Freedom	F	F Critical	Significant
1	0.411	1	15.59	30	0.792	4.171	No
2	0.001	1	17.87	30	0.001	4.171	No
3	1.260	1	24.24	30	1.559	4.171	No
4	1.383	1	32.62	30	1.272	4.171	No
5	2.658	1	14.22	30	5.608	4.171	Yes
6	1.750	1	34.47	30	1.523	4.171	No
7	3.159	1	20.56	30	4.609	4.171	Yes
8	0.244	1	21.47	30	0.342	4.171	No
9	8.880	1	29.59	30	9.004	4.171	Yes
10	0.394	1	11.07	30	1.069	4.171	No
11	1.143	1	16.86	30	2.034	4.171	No
12	0.582	1	12.77	29	1.320	4.183	No
13	5.855	1	25.50	29	6.658	4.183	Yes
14	1.200	1	30.00	28	1.120	4.196	No

.

4. Discussion

Survey fatigue represents a facet of respondent burden, commonly defined as the time and effort required to participate in a survey [16]. Survey fatigue can arise in both scenarios: surveys with a substantial number of questions and the administration of consecutive surveys. In the former, it typically manifests as a decline in response rates towards the end of the survey. Conversely, in the latter, it is characterized by a reduction in participant engagement [10].

In our study, we observed fatigue primarily due to the consecutive administration of questionnaires to students within the same semester. This is evidenced by the decline in questionnaire participation over time, as shown in Figure 1. Interestingly, we did not observe fatigue attributable to the number of questions within the survey, as the response rates remained consistent across all 14 questions included in the quality assurance survey (QAS) questionnaire. This finding aligns with previous research indicating that the length of the survey itself may not be the sole determinant of survey fatigue [10]. Instead, our results suggest that the timing and frequency of survey administration play a significant role in exacerbating respondent burden and subsequent survey fatigue among participants.

As can be seen in the results presented in Figure 1 and Table 4, the incentive system consistently leads to a significant increase in student participation in QAS questionnaires across all subjects. The effectiveness of incentivized surveys in increasing participation has been demonstrated in various non-educational environments [13,17,18]. Incentives such as gift vouchers, participation in raffles, and lotteries are commonly employed. However, it appears that the educational environment is not an exception, and a guaranteed prize [19,20,21] is more likely to be successful, as observed in this experience.

The following theories provide frameworks for understanding how incentives influence response rates in surveys by considering factors such as perceived value, costs, and rewards [22,23]:

1.

Social exchange theory: This theory posits that individuals weigh the costs and benefits of participating in an activity. In the context of surveys, respondents evaluate the effort required to complete the survey against the perceived rewards or incentives offered. If the perceived benefits outweigh the costs, respondents are more likely to participate;

2.

Leverage saliency theory: This theory suggests that respondents are more likely to participate in surveys when they perceive the incentives offered as valuable and relevant. In other words, incentives that are salient and meaningful to respondents are more likely to motivate participation;

3.

Benefit–cost theory: This theory emphasizes the comparison between the benefits gained from participating in a survey (such as incentives or rewards) and the costs associated with participation (such as time and effort). If the perceived benefits exceed the perceived costs, respondents are more likely to participate.

The results from the short questionnaire administered in the last week of class indicate that students are very satisfied with the incentive system (Table 5). Overall, 88% of students acknowledged that the incentive in the form of an extra score played a significant role in motivating their participation in the QAS questionnaires. This positive response suggests that the extra score is an effective incentive, aligning with the idea discussed above that a guaranteed prize is a favorable motivating factor. Our opinion aligns with the literature findings [13,22,23,24], indicating that using a guaranteed prize to incentivize questionnaire participation is consistent with leverage saliency theory. The extra score seems to exert more influence over students’ decisions to complete the survey than the time and effort required.

However, a potential concern arises: Approximately 40% of students expressed concern about the compromise of anonymity within the incentive system. This concern appears to arise from the physical presence of the teacher during the verification process of the questionnaire submission, even though the device screen only displays an acknowledgment upon survey submission. Addressing this concern could enhance the perceived confidentiality of the process. A solution to the anonymity issue could be that the quality service of the university takes on the responsibility of verifying students’ submissions. This could be achieved through an email notification sent to the teacher with a list of students who have successfully submitted the QAS questionnaire, eliminating the need for direct teacher–student interaction during that verification process. Finally, students reported completing an average of 3.8 questionnaires in the current semester, which is significantly lower than the expected range of 6 to 12 questionnaires. This suggests a notable decline in participation, reinforcing the notion of survey fatigue among students, in line with the consecutive administration of surveys [10]. Addressing this issue is crucial for maintaining the effectiveness of the evaluation process and ensuring a representative response rate.

Regarding whether the incentive significantly influences the academic assessment process, the ANOVA results summarized in Table 6 indicates that the average grades across the five academic years did not differ significantly in the SOPC subject. This indicates that the incentive system did not have a significant impact on the academic assessment in that subject. In contrast, for the remaining four subjects, the F values exceed their critical values, suggesting that the observed differences are unlikely to be attributed to random chance. This indicates that there is some factor affecting the average grade, but it does not directly identify the incentive as the factor causing the observed effect.

The results of pairwise multiple comparison tests for these four subjects (Table 7) are not conclusive. For IMS, Tukey’s test [15] suggested no significant difference for the last two academic years (with incentive); nevertheless, as shown in Figure 2, the decrease in average grade from 6.8 (2020–2021) to 5.8 (2021–2023) is not coherent with an extra score. For TCS, the pairwise multiple comparison test also indicated no significant difference for the last two academic years, one without incentive and the other with it, suggesting no significant impact of the incentive system on the average final grade. Similar discrepancies were found in the Tukey’s comparison tests across academic years for MENPP and PASEL subjects. In conclusion, the non-significance in some of the Tukey’s post hoc tests do not definitively negate the global significance found in the ANOVAs for those four subjects. It suggests that there may not be clear evidence of specific pairwise differences. The lack of clarity on the incentive’s effect on the final grade could be attributed to factors such as sample size, effect size, heterogeneity within academic years, etc. However, this does not necessarily imply that the incentive has no effect; rather, it highlights the complexity of interpreting results in the context of various statistical considerations. Therefore, the influence of the incentive on the final grade remains uncertain or inconclusive based on the current analysis. This emphasizes the need for further investigation or consideration of additional factors to draw more definitive conclusions.

Regarding whether the incentive significantly affects the outcomes of the official QAS questionnaires (Table 8, Table 9, Table 10 and Table 11), no significant impact on the results of any questions within the QAS questionnaire was detected for the SOPC, IMS, and TCS subjects at a confidence level of 0.05. Regarding the PASEL subject, the questionnaire results, available for only the last two academic years, were analyzed, with notably low participation (7%; six student responses) in the 2021–2022 academic year. This could potentially explain why questions 5, 7, 9, and 13 appeared to be influenced by the incentive system, exhibiting an increase in their assessment when the incentive was applied.

Finally, when examining the crucial question, “14. I am satisfied with the teaching performance of this teacher”, it becomes evident that the incentive system does not impact students’ satisfaction with the teaching performance across all subjects. This finding aligns with the results of other authors who did not find significant differences in response distributions among groups that received incentives or not [13,25,26,27], although it should be highlighted that those works are not in the education area.

5. Conclusions

The guaranteed prize incentive consisting of an extra score has proven to be highly effective in significantly boosting student participation rates in the QAS questionnaires used for teaching performance assessment at the University of Malaga. The notably positive feedback from students, with over 85% acknowledging that the incentive motivates their engagement, demonstrates its success in encouraging active participation. However, the incentive system should address a noteworthy concern raised by 40% of students regarding the compromise of anonymity in the QAS questionnaire due to the implementation of the incentive. This concern seems to arise from the physical presence of teachers during the verification process of the questionnaire submission. We suggest that a third-party entity, such as the quality service of the University, should take responsibility for verifying student submissions.

The emergence of survey fatigue is evident in the low number of QAS questionnaires that students reported completing per semester, aligning with the official participation rates published by the university. This highlights the importance of addressing this issue to maintain the effectiveness of the assessment process and ensure a representative response rate.

The analysis of the influence of the extra score incentive on final grades remains inconclusive. While no significant differences were found in the final grades for one subject across academic years both with and without incentives, the results for the remaining subjects did not provide a clear indication of the incentive’s impact. This underscores the complexity of interpreting the influence of incentives on academic outcomes and highlights the need for further investigation or consideration of additional factors for more definitive conclusions.

Regarding the QAS questionnaire results, the study indicates that the incentive system does not significantly affect the results for all the subjects studied.