Prediction of Students’ Use and Acceptance of Clickers by Learning Approaches: A Cross-Sectional Observational Study

Abstract

The student response system (a.k.a clickers) had been widely used in classrooms for various pedagogical purposes these years. However, few of the studies examine students learning approaches toward both technology and engagement. The present study adopted a cross-sectional study method to investigate the relationship between students’ user acceptance of clickers, learning approaches, and general engagement in the clicker classes. A group of 3371 university students were investigated by an online questionnaire that contained with Unified Theory of Use and Acceptance of Technology, Study Process Questionnaire, and National Survey of Student Engagement across a two-semester span in 2015 and 2016. A regression analysis had been adopted to examine the relationship between those variables. Results indicated that a deep learning approach significantly predicted all user acceptance domains towards using clickers and significantly predicted several engagement domains such as collaborative learning and reflective and integrative learning. We concluded that deep learners tend to share a constructive attitude toward using clickers, especially when their peers are also using the clickers. While deep learners prefer integration of knowledge and skills from various sources and experiences, we hypothesize that their willingness to integrate clicker activities in their learning process stems from seeing clickers as a medium for consolidation in the learning process. Future research is, therefore, necessary to provide more detailed evidence of the characteristic of deep learners on the qualitative arm or in a way of mixed research method.

1. Background

A paradigm shift towards an individualized [1] and constructivist approach [2] to learning have driven flourishing implementation and research [3,4,5] on the use of students’ response systems in classroom settings. This simple technology replaces the traditional ways to collect student feedback and makes the classroom become a more interactive and engaging learning environment [6]. To promote student engagement, there is increased use of this effective tool in the classroom. Due to the reforming higher education system in Hong Kong, it is not difficult to discover the increasing large classes in various Hong Kong higher education institutions. The use of students’ response system (SRS, a.k.a, clickers) in large classrooms has been established as an effective approach for facilitating students’ engagement in active learning [7,8]. However, students have diverse motivations and different learning approaches—such as surface versus deep learning motives—that influence their learning attitudes and behaviors [9,10] in terms of need for cognition further learning exploration and enjoyment of learning in attitude towards literacy [11]. Those diverse learning motivations from students may result as a difference on their academic achievements [11]. It is suggested that those diverse motivations may possibly affect the effectiveness of using clickers in class. Previous studies had suggested that learners adopting a deep approach had a positive attitude toward the eLearning platforms such as learning management systems [12,13]. Nevertheless, there are few existing pieces of evidence to prove the relationship between students’ learning motivation, attitudes toward clickers, and learning outcomes. The purpose of this study was to determine whether students’ learning approaches would predict their acceptance of clickers and their class performance in classes which use clickers.

A review of the literature suggested the benefits of implementing clickers in classrooms such as improving large classes’ student engagement and academic performance [14,15,16]. Most of the current studies focus on the effectiveness of clickers [17,18,19] on summative results and students’ engagement. Others are identifying the factors for using the audience response system among teachers and students [20,21,22,23]. Recent studies suggested that introducing clickers in class must include the spirit of collaborative learning to facilitate students’ active learning [8,24]. However, previous studies usually focused on the comparison of academic performance, learning outcomes, and student engagement between classes which used clickers and those that did not. Few of the studies focused on the learners themselves, particularly in the interaction effect of inquisitive learning attitude together with positive acceptance and use of technology and level of learners’ engagement. To accommodate the relationship between them, the objective of the current study was to determine if students’ learning approach would improve student general engagement and user acceptances of clickers in class. In answering the above research questions, we hope to conduct a study to gain a better understanding of the nature of clicker classrooms. While research on these questions is still at a beginning stage, findings will have broad implications in several areas of pedagogy. This examination was guided by the following method and hypothesis.

2. Materials and Methods

2.1. Design and Participants

A cross-section observational design was adopted in this study. This study employed a convenience sampling method. Participants were 3371 undergraduate university students from 89 courses in The Hong Kong Polytechnic University. Students were surveyed on their experiences in using clickers towards active and constructive learning activities, including formative assessment questions related to course content, mini-quizzes, peer assessment, revision, and peer instruction towards collaborative learning [25]. They provided responses via the student response system voluntarily after the teachers posted the clicker questions during class time. Online questionnaires were sent to the students who enrolled in the clicker classes and invited them to spend about 15 to 20 min. The data collection was conducted in the 2015–2016 academic year across a two-semester span. Informed consent was sought from all respondents in the survey collection process.

2.2. Procedures

Ratings on students’ learning approaches and attitude towards clickers were collected with an online questionnaire including items from the Unified theory of acceptance and use of technology (UTAUT), the revised two-factor study process questionnaire (R-SPQ-2F), and National Survey of Student Engagement (NSSE). Survey returns were collected over three periods of four-week windows towards the end of each semester to capture responses to clicker experience among students during the semesters.

2.3. Instruments

Unified Theory of Use and Acceptance of Technology (UTAUT): Students’ acceptance use of clickers was assessed by the UTAUT [26]. This scale comprises 5 domains and 20 items on a 5-point Likert scale. The overall internal consistency (Cronbach’s Alpha) of UTAUT in this study ranges from 0.84 to 0.90. This instrument proposed how a user accept the using of technology from a perspective of attitudes toward behavioral intention. Those constructs are: performance expectancy (PE): the perceived advantages of using the technology for a specific task; effort expectancy (EE): the effort of using in adopting the technology; social influence (SI): the degree of how peers influence the user’s actual use of the technology; facilitating conditions (FC): the perceived technical support and surrounding hemisphere for the users to adopt the technology.

The revised two-factor study process questionnaire (R-SPQ-2F): The 20-item instrument aims to assess the learning approaches and motivation of the students in the clicker classes [9]. The main factors are deep and surface approaches. Each domain comprises two subscales, strategies and motive for illustrating students toward their learning. The deep learning approach refers to the intrinsic motivation of a student toward his/her learning process. The deep learners are willing to understand the concepts by examining the course materials carefully and extend their learning outcomes to daily life. In contrast, students who adopt the surface learning approach tend to meet only the basic requirements of a course or just simply pass the course assessment. One of the examples of surface learners could be related to rote learning, memorizing the course materials without further understanding. In short, these learning approaches represent the qualities of students’ learning outcomes [27]. The R-SPQ-2F exhibited good internal consistency with Cronbach’s alpha coefficient in the present study between 0.87 and 0.88.

Selected learners-oriented engagement domains from the National Survey of Student Engagement (NSSE) [28,29,30]: Students’ engagement, collaborative learning and learning outcome in the clicker classes were measured by four domains. Each domain contains four to eight items. The NSSE in the present study estimate a generally good internal consistency (Cronbach’s Alpha), ranges from 0.78 to 0.89. Subscales are the following: collaborative learning (CL): students cooperate with others in grasping difficult material by assisting others, explaining course content to each other and working as a group; higher order thinking (HO): a study process demonstrates how students first remembered the information they learnt from class and applied to their real-life situation or experience, students also evaluated the information and forming innovative ideas from the sources after they master in an application level; learning strategies (LS): how often students adopted basic approaches for academic achievements, such as finding and refreshing the main gist in course content after class; reflective and integrative learning (RI): how often of the students link up existing knowledge with other sources such as information from another class, various perspectives from others and reflect their own opinions.

2.4. Data Analysis

All data was analyzed with the IBM SPSS Statistics 22 software [31]. A simple linear regression model was adopted for predicting the acceptance and use of technology and students’ learning outcome based on the deep learning and surface learning approach.

2.5. Research Questions/Hypotheses

In view of the preceding research purpose, two hypotheses to be addressed in this study are as follows:

• The deep learning approaches predict students’ acceptance behavior of using clickers;
• The deep learning approaches predict students’ engagement in the clicker classes.

3. Results

Figure 1 illustrated the demographics of the study’s participants. The mean age of the participants is 19.78 years old, with 63.5% students being freshman at the university (See

Table 1. Demographic characteristic of the students.

	N	%
Age
Median	19
Mean	19.78
Gender
Female	2005	59.5
Male	1336	40.5
Year of Study
First year	2139	63.45
Second year	522	15.49
Third year	381	11.30
Fourth year	313	9.29
Fifth year or above	16	0.47
Disciplines
Business	842	24.98
Health & Social Sciences	975	28.92
Humanities & Design	370	10.98
Sciences, Technology & Engineering	838	24.86
Tourism & Hospitality	346	10.26

). Regarding to the clicker classes, a balanced mix of academic disciplines with health & social sciences and humanities & design accounting for 63% (See Figure 1). A preliminary analysis of the demographic data had indicated that there were no significantly mean differences on gender in the learning approaches and engagement domains (data not shown).

Results from simple linear regression revealed that a deep learning approach significantly predicted all user acceptance domains towards using clickers, particularly the social influence domain (β = 0.50, p < 0.001) and performance expectancy (β = 0.50, p < 0.001) (See

Table 2. Learning approaches and UTAUT.

	B	Std. Error	Beta	t	Sig.
Deep learning approach
Performance expectancy	0.61	0.02	0.50	32.4	0.00
Effort expectance	0.50	0.02	0.43	26.43	0.00
Social influences	0.56	0.02	0.50	33.21	0.00
Facilitating condition	0.50	0.02	0.46	28.75	0.00
Behavior intention	0.55	0.02	0.45	28.51	0.00
Surface learning approach
Performance expectancy	0.03	0.02	0.03	1.94	0.05
Effort expectance	−0.04	0.02	−0.04	−2.42	0.02
Social influences	0.11	0.02	0.11	7.47	0.00
Facilitating condition	0.01	0.02	0.01	0.68	0.50
Behavior intention	0.02	0.02	0.02	1.23	0.22

). The deep learning approach also explained a significant proportion of variance in social influence scores, (R² = 0.29, F (1, 3368) = 689.42, p < 0.001). The surface approach failed to predict acceptance factors such as effort expectancy (β = −0.04, p < 0.005). Apart from attitudes toward clickers, the deep learning approach significantly predicted all learning and engagement realms such as higher order thinking (see

Table 3. Learning approaches and NSSE.

	B	Std. Error	Beta	t	Sig.
Deep learning approach
Collaborative learning	0.29	0.02	0.28	16.63	0.00
Higher order thinking	0.40	0.02	0.37	22.04	0.00
Learning strategies	0.48	0.02	0.45	27.94	0.00
Reflective and integrative learning	0.41	0.02	0.42	26.18	0.00
Surface learning approach
Collaborative learning	0.05	0.02	0.05	3.22	0.00
Higher order thinking	−0.06	0.02	−0.07	−4.00	0.00
Learning strategies	−0.11	0.02	−0.12	−7.24	0.00
Reflective and integrative learning	−0.02	0.01	−0.02	26.18	0.00

). Notably, deep learning approach was significantly associated with reflective and integrative learning (β = 0.42, p < 0.001).

4. Discussion and Implications

In this paper, we present the results of the prediction of students’ use and acceptance of clickers by learning approaches. Both Hypothesis 1 and 2 were significantly confirmed with the present results.

Regarding the Hypothesis 1, the deep learning approach had significantly predicted all domains from the UTAUT. The effects were particularly strong in the performance expectancy and social influence. It is hypothesized that deep learners tend to share a constructive attitude toward using clickers, especially when perceived advantages on using the clickers in their learning and their peers are also using the clickers. It seems like social influence is one of the factors that contribute a lot in students’ active learning in clicker classes. Previous studies had also shown that students’ intention to use clickers tended to be affected a lot by peer influence and collaborative learning [24,32,33].

Regarding the Hypothesis 2, deep learning approaches significantly predict students’ engagement domains and particularly strong in learning strategies and reflective and integrative learning. While deep learners prefer integration of knowledge and skills from various sources and experiences, we hypothesize that their willingness to integrate clicker activities in their learning process stems from seeing clickers as a medium for consolidation in the learning process. As Blasco-Arcas [24] noted, using clickers must include collaborative learning elements. The findings of deep learning predicted higher order thinking and collaborative learning are in line with previous studies that clickers adoption had triggered and lead students to have some cognitive outcomes [6,15,34,35,36]. As mentioned above, collaborative learning is defined as “students cooperate with others in grasping difficult material by assisting others, explaining course content to each other and working as a group” [28,29,30], general clickers activities in the present studies such as peer instruction [25] and simple revisions could stimulate the students to work collaboratively in an interactive setting [33].

Promoting interactive activity in class could lead to more effective learning outcome [37,38]. Having effective learning outcomes is consistent with the learning goal of deep learners. Such consistency may lead to an interesting discovery that deep learners seek effective learning and willing to participate actively in the context of collaborative learning. We hypothesized that the deep learners consider that cooperating and assisting others to examine the course materials (answering clicker questions) precisely in the clicker activities may benefit their learning outcome. These findings align with previous findings that students reached an active and collaborative learning experience via their positive attitudes toward student response systems [6]. Reflective and integrative learning correlated with performance expectancy of using clickers. As reflective and integrative learning refers to students linked up to various sources and information to their learning [28], we hypothesized that the learners first recognized the advantages of using clickers in the class and then integrate clickers into their learning toward a deep learning approach.

Lastly, those findings from these data suggest that the social learning elements such as social influence and collaborative learning in clicker classes. As social influence had indicated a rather strong effect compared with other domains, it is not surprising that collaborative learning could also be formed when the deep learners started to use clickers in the classroom. These results pointed out the importance of social interaction toward active learning in a clicker classroom.

Based on the current results, effect of using clickers on collaborative learning was not particularly strong when compared with other domains. A possible explanation would be the mixed usages of clickers in the classrooms in terms of frequency, duration, and pedagogical orientation. However, it is astonishing that even controlling for pedagogical orientations for using clickers, the results still illustrated a noteworthy relationship between clickers and social learning in a higher education context. These results highlighted the importance of social interaction toward active learning in a clicker classroom.

4.1. Limitation of the Current Study

Despite the large sample, the design of the present cross-sectional study is not without limitations. Having acknowledged the limitations of the clicker class observation data, we can nevertheless confirm that teachers seemed to have a separate set of pedagogical features because we did not control the usage of the clicker activities from the various disciplines taught sessions. Teachers may use clickers to serve numerous pedagogical purposes and the present research only illustrated the primary impression of all kinds of clicker activities toward collaborative learning in a large sample scale. It could also explain that the collaborative learning construct was not connecting strongly in the study. As the present study focuses more on the prediction of learning attitude toward acceptance of clickers, future work may continue to compare the learning outcomes among the different clicker activities.

The second limitations could be related to the self-report bias from the survey. The present scales—UTAUT and R-SPQ-2F—are five-point scales, bias such as social desirability would be the major conundrum because participants tend to select the middle options [39,40]. Hence, we suggested removing the “Neutral (3)” option in the future studies that could make the results more noticeable. We are hopeful that future research will provide more detailed evidence of deep learners on the qualitative arm or in a mixed research method.

4.2. Implications for the Teaching and Learning & Conclusions

Several pedagogical implications can be drawn from the present study. The present findings suggest some benefits of using clickers in class. It clearly supports the notion that some study motives and strategies of deep learners. Deep learners tend to have the characteristic of learning collaboratively and willing to integrate other sources into their learning. The present finding echoes the current findings of how deep learners learn better compared with other forms of learners. It confirmed that deep learners have a positive attitude toward those teaching aids which let them have advanced academic achievement [12]. As Webster [41] proposed, a positive learning environment could facilitate students to adopt a deep learning approach.

Their positive attitude toward the clickers may bring advantages for them in the current blended learning model. It must be noted that teachers should try to integrate other sources within their teaching to facilitate the quality of student learning. Adopting clickers in the class seems to be an effective way to facilitate positive collaborative learning due to the interaction between peers and teachers, which differ from the traditional passive teaching approach [14,24,38,42]. The present findings suggested that clickers provided a new way of learning and collaborative rooms for students. This environment aids learners and promotes positive peer interaction for expanding their course knowledge in a non-traditional way. As mentioned before, the attitude of students toward clickers should be the most necessary factor to show the effectiveness of clickers. This single little act of pressing a button in the class encourages students to learn in some much more interesting ways. The student response system motivates their will to learn, to reflect, and to seek other forms of learning experience which may crystalize their attitude toward their future learning.

In conclusion, this study illustrated university students’ user acceptance of clickers, learning approaches, and general engagement in clicker classes through a cross-sectional method. Results indicated that students’ deep learning approach would improve student general engagement and user acceptance of clickers in classes. Deep learners tend to share a constructive and positive attitude toward using clickers, especially when their peers are also using the clickers. The present finding supports how deep learners learn better compared with other forms of learners and benefit in a collaborative learning context. While deep learners prefer integration of knowledge and skills from various sources and experiences, we hypothesize that their willingness to integrate clicker activities in their learning process stems from seeing clickers as a medium for consolidation in the learning process.