Students’ Perception of Using Digital Badges in Blended Learning Classrooms

Abstract

A digital badge system with a leaderboard ranking was implemented in an 18-week course offered by a Chinese mainland university during the 2018 Spring Semester. Fifty-four junior college students participated in this study, and the learning activities were based on a combination of online and offline methods using the Moodle system. The purpose of the study was to explore these college students’ perception of digital badges in blended learning classrooms as well as their perception of digital badges in general. Q methodology was used in a single case study of the students’ subjective experience of digital badges, and thirty-six Q-statements were selected after a comprehensive reflection of their views of digital badges. The participants were asked to sort the statements to model their opinion in a modified rank-order procedure. Twenty-two valid sorts were entered for the analysis, and three factors were identified to represent groups of participants with similar views. The results are discussed based on the views of students at a local university in mainland China and the way their demographic characteristics related to their perception of digital badges.

1. Introduction

With the large-scale open curriculum development of Massive Open Online Courses (MOOCs and the rise of the use of flipped classrooms, there is an urgent need to reform the educational modes in higher education. According to Liu, Huang, and Wosinski [1], “the fusion and development of information technology eliminates the barriers preventing the sharing of information and knowledge, as well as the learning boundary”. Blended learning is now adopted on a widespread scale, and there has been a significant increase in the number of blended courses in higher education in the past decade [2]. Blended learning (BL) combines the advantages of traditional face-to-face instruction with e-learning [3]. The online learning environment can provide students with information on various aspects of the curriculum, can compensate for the lack of learning in class, and can provide online learning resources for them. In view of the convenience of online resources, it is becoming increasingly important to evaluate students’ online learning outside of class and to measure their academic performance based on formal and informal learning. Many instructors believe that learning cannot be recognized without monitoring the learning process and worry that the effectiveness of learning cannot be guaranteed outside class. This has become a barrier to the promotion of blended learning.

However, blended learning and traditional face-to-face learning can be connected well by using digital badges. Mehta, Hull, Young, and Stoller [4] indicate that digital badges, as a form of microscopic certification, could provide a more detailed framework to assess students’ knowledge and skills, to supply evidence of their learning, and to demonstrate their ability to substantially change. Digital badges capture the learning performance in a way that is not recognized by the formal academic accreditation processes in higher education [5]. Even more, open badging can be an easy way to capture formal and informal learning experiences [6].

Previous research on digital badges is somewhat limited. Firstly, it is mainly based on k12 students rather than those in higher education [7,8,9]. Secondly, digital badges are usually given by institutions in programs such as teachers’ professional development [10] but fewer in college students’ learning activities. Thirdly, authors of small studies give their opinion of higher education students’ perception of digital badges. Finally, it is challenging for college students and teachers to accept digital badges as an assessment tool, and it needs to be continuously practiced in higher education to prove its effectiveness.

A digital badges system was used in an undergraduate obligatory course of multimedia animation design offered to junior students by the Department of Journalism. The purpose of the study was to explore these college students’ perception of digital badges in blended learning classrooms as well as their perception of digital badges in general. The structure of the study is as follows: Section 2 contains a review of the literature related to digital badges in education; the Q methodology and practical background, data sources, participants, and implementation process of the research curriculum are explained in Section 3, while Section 4 contains an analysis of the results. The conclusion and some suggestions for further research are presented in Section 5.

2. Theoretical Background

Digital badges (hereinafter badges) are a digital display that acknowledges, recognizes, and validates the achievement of skills and competencies through explicit evidence [11]. Abramovich [12] utilized digital badges in videogames and recommended their use as not only a credentialing mechanism but also an assessment tool in higher education. Cameron, Pierce, Banko, and Gear [13] found that achievement-based rewards during learning or testing increased participants’ intrinsic motivation. Rewards and incentives are used to improve students’ performance and motivation in educational settings. This section contains a review of the literature related to the reward mechanism in education and the use of digital badges in blended learning, as well as digital badges as reward mechanics on a Moodle learning platform.

2.1. Reward Mechanisms and Digital Badges in Education

Badges with points and leader boards can be a gamification element that enables learners to compete with themselves or others [14]. Some researchers have indicated that people have positive, negative, or mediocre perceptions of badges, and some participants have found badges to be less prestigious than certificates of completion [15]. Jones, Hope, and Adams [10] indicate that teachers have a favorable view of digital badges but that they have insufficient understanding of their purpose. Teachers who work in lower grades place greater value on badges received during the experience [10]. Digital badges can affect learners’ motivation [16,17].

However, Hanus and Fox [18] suggest that badges and leaderboards may harm educational outcomes in a competitive context and that students tend to be less intrinsically motivated in a gamified course, which is likely to cause lower final exam scores. Meanwhile, badges have no effect on students with a stronger intrinsic motivation. Some are alarmed that these external rewards represent the ultimate risk by shifting the focus from learning to badge-collecting, thereby “increasing reliance on extrinsic incentives for learning” [19].

Abramovich and Wardrip [20] consider badges as powerful, positive tools to motivate students, and Filsecker and Hickey [7] indicate that external rewards did not undermine fifth graders’ motivation and interest by the introduction of badges. Informing students of the types of digital badges before the learning activities can also increase their motivation [21]. Badges can affect learners’ motivation based on the goal achievement theory.

Badges and external reward mechanisms are used to positively reinforce and stimulate students’ behavior and self-efficacy by measuring their learning progress and learning feedback. Hew, Huang, Chu, and Chiu [22] introduced game competitive mechanisms (badges, points, and leader boards) based on the motivation theory and pointed out that positively motivated students engage with more difficult tasks. Grant [23] classified learners into three types according to their purpose of earning badges as “badge hunters, sharers, and dodgers”. Hunters care about quantity, while sharers care about displaying badges and quality participation, and dodgers are not interested in badges. Biles and Plass [24] found that competition and collaboration elicit a greater situational interest than individual play. They identified five different types of players based on different motivations: socializer, achiever, stimulation seekers, explorers, and collectors. Socializers were motivated by badges that revolved around a common platform for sharing across games. Achievers desired the badge to be linked to some sort of reward. Stimulation seekers and explorers were mainly focused on the game story or environment and considered badges to be a secondary motivation. Collectors focused on challenges and achievements. Since different types of students have different motivations, as well as different and perceptions of badges, it is necessary to explore the relationship between their cognition of badges and their motivation in order to match badges rewards with students’ motivation.

2.2. Blended Learning in Higher Education

This study used a blended learning curriculum, which included lectures to guide learning, face-to-face communication in the classroom, a general discussion of the subject of the course, exercise questions for supervising and checking students’ learning level, and activities that students can perform collaboratively outside the class [2,3,21]. By applying learning theories, Carman [25] concluded that live events, online content, collaboration, assessment, and reference materials were five important elements of a blended learning process. Riffell and Sibley [26] indicate that students in the hybrid course would have a better and more frequent interaction with the instructor in group studies than in a traditional course. Bonk [27] introduced the TEC-VARIETY Model (Tone, Encouragement, Curiosity, Variety, Autonomy, Relevance, Interactive, Engagement, Tension, and Yields Products) for online and blended learning practices to motivate and retain online learners. Based on the cognitive theory, learners’ cognitive beliefs, technological environment, and social environment have the most impact on students’ satisfaction with learning [2]. The definition of BL is as a mix of classroom instruction (i.e., face-to-face) and out-of-class online learning where online work is substituted for class time in this study [28].

As an innovative form of education certification, a digital badge is an online assessment and accreditation mechanism [23]. Digital badges testify learners’ ability more comprehensively than a traditional degree and certification. They can represent a visual record of achievement and meet learners’ needs based on the elasticity of the online learning environment. They can be created, awarded, displayed, and stored online, as well as shared on social media and professional social networking sites [15].

2.3. Digital Badges as Game Mechanics in Moodle Learning Platform

Moodle is a free and open-source learning management system (LMS) used for blended learning, distance education, flipped classroom, and other e-learning projects in schools, universities, workplaces, and other sectors [29]. As a famous learning LMS, Moodle is a diversified technical tool that supports an evaluation of blended learning. Hew, Huang, Chu, and Chiu [22] used Moodle to report the effects of game mechanics to foster users’ engagement in blended learning and found that the digital implementation of game mechanics automatically kept track of students’ activities in Moodle.

Points, badges, and leaderboards are the most-used game elements in gamification to improve students’ attitude, engagement, and performance [30]. In addition, students’ gender affects their experience of and relationship with badges. Students of different genders who obtained badges had a different incentive to learn [23], and anonymity and badge-sharing behavior were related to students’ cognitive type. The mentality of their willingness to share badges is worth exploring [31]. Even more, Yang, Benazir, and Chen [32] found that leaderboard ranking did not affect learners’ English learning outcomes because of their different attitude toward competitions. Numerous researchers have generally used interviews, focus groups, quantitative and qualitative surveys, questionnaires, and experiments to collect data to determine the effect of digital badges in blended learning [7,9,16,22].

3. Materials and Methods

3.1. Q Methodology for Analyzing Students’ Perception

The present paper used the Q methodology, which is an effective method to study people’s subjective perception [33]. The Q method is useful for systematically studying human subjectivity, especially the subjective aspects of people’s views, beliefs, attitudes, opinions, motives, needs, values, and emotions [34]. Watts and Stenner [35] indicate that this method enables the use of “a by-person factor analysis order to identify groups of participants who make sense of (and hence Q ‘sort’) a pool of items in a comparable way”. Hurley, Wehby, and Feurer [36] used the Q-sort technique to find participants who held three distinctive viewpoints about valued social competence behavioral goals. Q methodology combines the advantages of qualitative research with a quantitative analysis [37]. A small sample was used in view of the limited literature on the research method [38]. The Q methodology is used to explain the main viewpoints favored by specific participants, regardless of the number [35]. It is used to study the subjective viewpoint of the participant more effectively [39] (pp. 23–25). Wheeler and Montgomery [33] explored the views of students on learning mathematics and used the Q methodology to analyze their experience of education and their beliefs. Chen, Chen, and Chen [40] used the Q methodology to discuss students’ perception of collaboration in flipped classrooms. Ramlo [41] used it to study campus stakeholders’ different perspectives of free speech. Walker, Lin, and McCline [42] made the Q methodology more accessible by using the Q-Perspectives^® Online tool in face-to-face, flipped, and online classrooms.

The Q methodology is composed of five stages in this study [39] (pp. 27–28).

Step 1: An issue domain (concourse of communication), linguistic source was the sample in this study (i.e., Q-sort items), which was collected from written narratives of students who enrolled in this course management system.

Step 2: Sample participants (i.e., person samples or P-set) were selected for the Q sorting to reflect the students’ opinion of digital badges in blended learning using a content analysis approach. Q statements marked as unclear were rephrased with the assistance of an expert cooperative until all items were rated as clear.

Step 3: The participants’ subjectivity was expressed by modeling their viewpoints through the operational medium of a Q sort. The conditions of instruction requested for agreement (+4) and disagreement (−4) were used for rank-ordering and were employed to operationalize hypothetical constructs and categories [35]. Positive scores were placed to the right of the 0 in ascending order and negative scores to the left, and items that were neutral, ambivalent, or uncertain for participants were placed in the middle.

Step 4: The Q-sort data consisting of the intercorrelation of Q sorts as variables was analyzed, and a factor analysis was conducted on the correlation matrix based on Stephenson’s original formula.

Step 5: Finally, the factors were interpreted “in terms of factor scores rather than in terms of factor loadings (as is typical in R methodology)” based on the Q methodology [43].

3.2. Context

The study took place in the context of an obligatory undergraduate course offered to junior students by the Department of Journalism in a Chinese mainland university. The course contained fifty-four class periods, including thirty-six periods of theoretical teaching and eighteen periods of computer experiments twice a week (three periods per week of one time). The duration of the course was almost 18 weeks. The plan was to arrange six subjects and five homework assignments, using a mixed online and offline teaching model. The Moodle course management platform (see Figure 1) was designed to support self-directed and collaborative learning activities and to provide a participatory platform for teachers and students to submit assignments; to grade and award badges; and to contribute, share, and give feedback. The researcher utilized some badges ranking plug-ins that Moodle supports to build a competitive mechanism.

When Moodle had been set up, each student was given an account number (i.e., ID). Teachers created criteria with modules of digital badges and uploaded their class resources, such as videos or audio flips, images, and other types of course files to the platform. The process of awarding digital badges and the blended-learning for this study consisted of three steps: Firstly, the teachers introduced the online learning system to confirm the course tasks and rules, and students were given a Welcome Badge when they first entered the system; others were awarded depending on their achievement during the course. Secondly, face-to-face lectures, discussions, group reports, and other activities were organized. The difficulty of the task was gradually increased, and the time allotted to complete them was adjusted accordingly. Thirdly, the ranking scores were published (see Figure 2).

Students who achieved a great cumulative score for thematic activities were awarded a Participation and Completion badge, and finally, all the participants submitted group and individual assignments, completed a peer review (see Figure 3), and shared their experience. Assessment badges, Knowledge talent badges, Grade badges (Novice, Adept, Apprentice, and Professor), Independent learning, Great team badges, and Communication talent badges were issued, until the end of the six topics.

The operation and badge rules were explained and posted on the website at the beginning of the class in order to eliminate the impact of technical barriers on the participants’ initial exposure to the badge concept. The badge design criteria was provided in accordance with the goal achievement theory and Malone’s theory of intrinsic motivation in educational games [44] (See

Table 1. A list of the badge criteria in the course.

Badges	Name	Criteria and Rules	I/G	F	Q	Issuer	A
	Great team Badge	Complete teamwork: Two people in a group must submit Flash homework; total score of 80 or above.	G	1	More than 5 badges in total	Ir	No
	Completion Badge	Submit work on time; complete the course activities.	I	3	3 badges at most	Ir	No
	Welcome Badges	Log into the system, register, and provide the required information.	I	Irgr	1 badge per person	Ir	No
	Participation Badge	Issued to top 20 in course activities ranking. Rules: 5 points for log-in and 20 for each specified activity completed (including replies and comments in forum, evaluation, peer-assessment, feedback, download course resources, and other learning activities).	I	2	20 badges per time	Ir	No
	Independent learning badges	Complete independent learning activities on time, including viewing course materials before class; complete pre-class test; and complete after-school test.	I	1	1 badge per person	P	No
	Communication talent Badge	Issued to students who receive a peer review medal	I	1	1 badge per person	P	S
	Knowledge talent Badge	Issued to students who achieve more than 80 points in knowledge test.	I	1	1 badge per person	Ir	No
	Assessment Badge	Complete peer review.	I	1	1 badge per person	Ir	S
	Grade Badge: Novice	Final examination badges. Novice Grade Badge awarded for less than 70%. Adept Grade Badge awarded for between 71% and 79%. Apprentice Grade Badge awarded for between 80% and 89%. Professor Grade Badge awarded for more than 90%.	I	Once at the end	1 badge per person	Ir	No
	Grade Badge: Adept
	Grade Badge: Apprentice
	Grade Badge: Professor

* I: Individuals; G: Groups; F: Frequency; Irgr: Irregular; Q: Quantity; Ir: Instructor; P: Peers; A: Anonymous; S: Self-Selection.

), and the badge icons system was developed based on Piha’s badge component map [45]. The Mozilla Open Badges platform (https://www.openbadges.me/designer.html) was used to design the badge icon and the Credly (https://credly.com/badge-builder#badge_text) platform was used to issue and earn digital badges for social media.

3.3. Participants

The subjects in this study were 26 male and 28 female students (n = 54), who participated in blended learning activities. Based on previous enrollment information, they were expected to have experience with multimedia animation design and production. Students’ information literacy was one of the factors that influenced the effect of the badges. Since demographic variables were important factors to consider in this study, the participants were asked to provide them in the registration system.

When users have a clear goal, their performance improves [46]; hence, it was important to clarify the task content in the course planning and curriculum design. Ranking scores (see Figure 2), a badge ladder (see Figure 4), and a plug-in that Moodle supports were utilized to generate a competitive mechanism in this course to motivate the participants to take more responsibility.

Both teachers and students could see each other’s ranking on the Badge Ladder during the whole term, either the number of students owning a badge or the details of badges owned by each student (see Figure 5).

3.4. Instrument

Q statements (i.e., a Q set) form the sample in a Q methodology. The teachers collected the statements of the students who participated in the blended learning activities and selected sentences as representative of the use of various strategies in the form of a Q sample. They sought permission to collect the students’ views of the use of digital badges in this course by asking questions and establishing feedback questionnaires on the course’s online platform. The open question was: How do you feel about the use of digital badges in this course? Please state the advantages and disadvantages in 8–10 sentences.

Forty-nine feedback statements were collected, and 36 Q statements were extracted from them to ensure that the number of availing and unavailing statements was balanced. The final Q-statements were selected from a comprehensive reflection of students’ views on digital badges and divided into four themes (i.e., teaching and learning, learning evaluation, communication, and technological issues) with the 36 statements representing the main ideas of each theme as the research instrument. The participants were then asked to model their opinions with these items in a modified rank-ordering procedure (i.e., Q sort). The Q method involves Q sorting, a method of data collection and a factor analysis to assess subjective (qualitative) information [47].

3.5. Procedure

The teachers were given permission to collect the data through scheduled class meetings. One researcher briefly explained the rights of the participants, the purpose of the study, and the instructions and steps in the Q-sorting process, and any questions were answered. Learners who agreed to participate were given a set of Q cards, a score sheet (see Figure 6), and a request for demographic information including name, age, gender, grade, college, and department. They were asked to sort the statements on a nine-point forced distribution for each Q sort. They indicated their answer on a continuum from most agree (+4) to most disagree (−4) to the question, what do you think about digital badges?

3.6. Data Analysis

The PQMethod 2.35 program was downloaded from http://schmolck.org/qmethod and used to facilitate Q-sort data entry, correlation and factor analyses, and theoretical rotation of factors and to calculate the factor scores. Twenty-two valid sorts were entered for analysis. A correlation matrix was generated to represent the relationship between each pair of Q sorts before being submitted to a centroid factor analysis and judgmental (or manual) rotation for interpretation.

4. Results

A combination of auto- and hand flagging resulted in twenty-two valid sorts, and a three-factor solution was finally chosen. These factors represented three related views of using digital badges in blended learning classrooms. As can be seen from

Table 2. The factor loading and demographic variables for each type of learner *.

Person No.	Factor 1	Factor 2	Factor 3	Gender
13	69			Male
20	66			Male
14	63			Female
9	63			Female
1	58			Female
3	58			Male
7	54			Male
22	47			Male
15		−64		Female
5		−58		Male
10		−85		Male
17		49		Female
19		80		Male
21		37		Male
16		83		Male
6			−48	Female
11			87	Female
12			87	Female
18			39	Male
2			76	Male
8			40	Female
4			53	Female

* Only significant loadings shown (p 0.01); decimals omitted.

, the factor analysis indicated three broad classes of Q sorts, factors 1, 2, and 3; however, the second was bipolar [35,43], which indicated four different understandings of digital badges. Factor 1 contained 8 sorts, factor 2a (the negative pole of the factor) contained the views of Persons 15, 5, and 10; in contrast, factor 2b (the positive pole of the same factor) contained the views of Persons 17, 19, 21, 16, and 7 sorts on the third factor contained diametrically opposed views of Person 6 advocating an opposing viewpoint (factor loading of −0.48). Neutral learners, Extreme learners, and Skeptical learners represented each factor (see Table 2). The Q methodology primarily utilizes statements to define a theoretical factor for interpretation [43]. The statements that were ranked from most agree (+4) to most disagree (−4) helped to explain the characteristics of each factor with factor scores (see

Table 3. Students’ statement scores by factor types.

Factor Statements		Factor	Rankings (*)
		1	2	3
Factor 1 Neutral learners
33	Digital badges have a clear objective for learning outcomes; I can learn better.	4	−1	−1
35	Digital badges do not represent a person’s true ability.	3	−1	0
28	Digital badges and the websites have insufficient attention.	3	−3	0
1	Digital badges make me actively participate in blended learning.	1	3	−2
24	The network is not convenient, and user experience is not good.	1	−3	4
2	Digital badges are less satisfying than physical rewards	0	−4	−2
31	Digital badges add to the fun of interaction and communication.	0	2	−1
21	Badges represent acquired skills and achievements, which encourages me.	0	4	−4
10	Obtaining badges is conducive to communication with classmates	−1	3	2
19	Digital badges improve my learning.	−2	2	1
6	Digital badges are novel and unique with a bright design.	−3	3	−1
25	Ranking of badges has a negative effect on me.	−4	−1	−1
Factor 2 Extreme learners
21	Badges represent my achievements and stimulate me.	0	4	−4
15	Badges help to foster a sense of competition and improve my shortcomings.	−1	4	−3
1	Digital badges make me actively participate in blended learning.	1	3	−2
6	Digital badges are novel and unique with a bright design.	−3	3	−1
31	Digital badges add fun to interaction and communication.	0	2	−1
36	Blended learning tends to be compared to traditional education.	4	2	3
27	Digital badges improve interaction and teaching efficiency.	−2	1	−1
29	Digital badges refer to grades.	−3	0	−4
3	There is unfair phenomenon of low scoring when students evaluate each other.	0	−1	0
20	The evaluation results of digital badges do not reflect the true level.	3	−2	3
30	Digital badges are not famous or widely used.	1	−2	0
14	Badges are not motivating unless teachers emphasize them in class.	2	−2	2
28	Digital badges and the website have insufficient attention.	3	−3	0
24	The network is not convenient, and user experience is not good.	1	−3	4
2	Digital badges are less satisfying than physical rewards	0	−4	−2
Factor 3 Skeptical learners
24	The network is not convenient, and user experience is not good.	1	−3	4
5	The display position of a personal badge is not obvious enough	−2	−1	4
16	Acquiring a badge rather than acquiring knowledge is not conducive to an intrinsic motivation to learn.	−2	0	2
18	Computer study is troublesome. I don’t like learning on the internet.	−4	−3	1
22	Digital badge icons are drab and imperfect.	−1	−2	1
28	Digital badges and the website have insufficient attention.	3	−3	0
6	Digital badges are novel and unique with a bright design.	−3	3	−1
31	Digital badges add fun to interaction and communication.	0	2	−1
1	Digital badges make me actively participate in blended learning.	1	3	−2
2	Digital badges are less satisfying than physical rewards.	0	−4	−2
13	Digital badges give timely feedback on learning.	2	0	−2
17	Online learning is convenient and conducive to independent learning.	1	1	−3
21	Badges represent acquired skills and achievements, which encourages me.	0	4	−4

* Item rankings: +4, most agree; 0, neutral or uncertain; −4, most disagree in this sample.

), while the negative pole of the bipolar factors advocated an opposing viewpoint. All groups identified the mutual evaluation in the group as an effective instructional strategy in blended learning.

Factor 1 Neutral learners (five males and three females) contained 8 sorts, 36% of the 22 students’ Q sorts. They acquired 2 Professor Badges, 4 Apprentice Badges, and 2 Novice Badges during their learning performance. This group generally believed that evaluating results with badges did not represent their true ability (+3), but they thought that digital badges had a clear objective for learning outcomes, namely, to learn better (+4). One observed that “Digital badges can better assess learners’ skills, provide an evaluation and certification at the same time as clarifying the learning process.” However, they disagreed that the ranking of badges had a negative effect (−4); rather, they described them as an innovative and future development trend compared to traditional classroom education and the credit system. One of this type of learners said “This is an innovative method of teaching that should be encouraged and promoted.”

Factor 2 Extreme learners (five males and two females) had both positive and negative significant loadings, which was a “bipolar” factor, when Factor 2a (the negative pole of the factor) contained 3 sorts (14% of the 22 students’ Q sorts) and factor 2b (the positive pole of the same factor) contained 4 sorts (18%). They acquired 2 Professor Badges, 3 Adept Badges, and one of each of the other badges during their learning performance. As can be seen from Table 2, extreme learners with a negative pole (Factor 2a) who disagreed with the item rankings had the most negative perception of badges. Extreme learners with a positive pole (Factor 2b) had the most positive perception that badges encouraged and cultivated a sense of competition more actively in blended learning. One student in this group responded to the open question, “Digital badges can well evaluate online learners’ skills and achievements, so that they can have a clearer understanding of their strengths and weaknesses.” At the same time, the mutual evaluation function provided by digital badges can make it more convenient for learners to know about the homework of other learners. These students appear to possess the characteristic of a Hunter based on their strong agreement with the statement [24], “badges are a recognition of skills and encouragement (+4) which helps to raise their competitive spirit and improve their inadequacy (+4)”. Therefore, Factor 2b agreed that badges cultivate a sense of competition and more active blended learning (+3), while the negative group, Factor 2a, agreed with the converse configuration.

Factor 3 Skeptical learners (two males and five females) were composed of 7 sorts (32% of all 22 students’ Q sorts). They acquired 1 Adept Badge and 2 of each of the other badges during their learning performance. They were deep and active participants in blended learning, although they thought that the networking and user experience were inconvenient (+4) and that the display position of personal badges was not obvious enough (+4). Further evidence could be found in the responses to the open question:

“Website page layout is not clear, and user experience is not good, although it is easy to download information. As a new attempt, digital badges are worth promoting.”

They thought that neither the convenience of online learning (−3) nor the fact that digital badges represented acquired skills and achievements were encouraged enough (−4). They also disagreed that digital badges were less satisfying than physical rewards (−2). In other words, they thought that digital badges were at least as satisfying as physical rewards. In addition, Person 6 advocated an opposing viewpoint when all the item rankings in the same configuration were reversed.

The results showed that the three groups equally agreed or disagreed with several statements. Although all three groups had different opinions of the digital badges in blended learning, there was a consensus on Statements 4, 7, 11, 12, and 34 (see

Table 4. The statements on factor types that achieved a consensus.

Factor Statements		Factor		Rankings (*)
		1	2	3
4	Peer assessment increases the chances of learning from each other.	2	3	3
7	The ranking rules of digital badges in blended learning activities are reasonable and clear.	0	0	0
11	Digital badges, like Red Flowers for primary school students, are not suitable for college students.	−3	−4	−3
12	Digital badges are easy to use, but they reduce learning supplies.	1	1	1
34	I am not interested in the interactive activities of obtaining digital badges and have little enthusiasm for them.	−3	−3	−3

* Item rankings: +4, most agree; 0, neutral or uncertain; −4, most disagree in this sample.

).

5. Discussion and Conclusions

The Q methodology is an approach that can be employed to understand students’ perspective of the use of digital badges in a blended learning course in higher education. The Q methodology was used for the data collection and analysis in this study to explore the perspective of students with different types of characteristics of the use of digital badges in blended learning. A survey was used to collect the students’ opinions of the functions of digital badges so that they can be improved to motivate students’ learning by focusing on an instructional strategy.

5.1. Discussion

5.1.1. Student’s Perception of the Digital Badges Mechanism

The participants’ opinion of the performance of digital badges in blended learning was demonstrated using a Q-sort survey. The students were divided into groups of Neutral learners, Extreme learners, and Skeptical learners and the results indicated that they all identified the mutual evaluation of the group as an effective instructional strategy in blended learning, even though some students had a negative perception of it. All the groups equally agreed that peer assessments increased the chances of learning, which supports the observation of Riffell and Sibley [26]. Furthermore, digital badges encourage Extreme learners to actively participate in blended learning, but these learners did not all feel that digital badges were less satisfying than physical prizes, as Skeptical learners did, which supports the opinion of Abramovich and Wardrip [20] that badges are a positive tool to motivate students. Moreover, Skeptical learners thought that digital badges added fun to the interaction and communication, although they felt that the display position of personal badges was not obvious enough (+4). It is confirmed in this study that digital badges have a positive effect on blended learning, which is similar to the idea of Andrioni [48] that combining different forms of traditional and modern online learning leads to new ways of involvement and active participation in learning.

Another interesting viewpoint highlighted by the Q methodology was involved the Factor 1 and Factor 3 groups of leaners. Both groups agreed that the evaluation results of digital badges do not reflect the true level (+3). They also agreed that badges were not energizing unless teachers emphasized them in class (+2). The Q sort demonstrated the individuality of the factors. Factor 1 felt neutral or uncertain (0) about the encouragement of badges that represent acquired skills and achievements, while the Factor 3 felt dissatisfied to the degree of grading it a −4. The factor 2b groups expressed the opposite view with +4 satisfactory. The result of this part of the study is consistent with Hanus and Fox’s [18] findings, which suggest that a competitive context or other digital badge mechanisms may harm educational outcomes. This viewpoint is inconsistent, since it simultaneously provides useful information for higher education administrators on where to focus improvements to the teaching incentives.

In addition, all the groups had a positive opinion of the use of digital badges in blended learning. The factor 1 group believed that digital badges have a clear objective of enabling them to learn better. The factor 2 group was neutral about the ranking rules of digital badges in blended learning activities, and felt that digital badges were novel and unique with a bright design. For the latter, badges represent their achievements and stimulate them. The factor 3 group agreed that digital badges are easy to use but that they reduce learning supplies (+1).

5.1.2. Effect of a Digital Badges Mechanism on Blended Learning

A further analysis of the consensus statements between factor types showed that all the groups agreed that peer assessment increases the chances of learning from each other, and they were all interested in the interactive activities to obtain digital badges, which increased their enthusiasm. This result is consistent with Riffell and Sibley’s findings [26]. It was also found that the mechanism of digital badges enhanced participants’ motivation and that rewards based on achievements increased their intrinsic motivation to learn. Factor 3 expressed dissatisfaction with the online learning environment, and one of them pointed out his dislike of computer learning, but the application of digital badges in education requires teachers and students to have basic knowledge of computer technology. Therefore, it is proposed that university educators need to optimize the availability of the university network and technical support. Furthermore, almost all the participants described digital badges as authentic and innovative compared to traditional classroom education and credit systems, although some had a negative or mediocre perception of them, stating that they were less prestigious than a certificate of completion. However, digital badges, as a form of microscopic certification, could provide a more detailed framework to assess college students’ knowledge and skills, could provide evidence of their learning, and could demonstrate their ability to respond to significant change. Digital badges help to capture a learning performance that is not recognized by formal academic accreditation processes. At the same time, they enable students to obtain overall information in a more timely fashion that a final exam by being an informal way to receive feedback.

These digital badges can be a more innovative and authentic means of instruction and assessment in blended learning classrooms. Their functions are determined through this course practice, and some suggestions for teaching are proposed. The researchers also argue that the results derived from these participants in blended learning practice may be influenced by external factors in field settings [49], such as the Moodle learning platform.

5.2. Conclusions

A digital badge mechanism was designed in this study and implemented in a blended learning environment in order to determine college students’ perception of digital badges. A research framework was proposed to investigate the objective of digital badges, and a leaderboard ranking was used to examine the participants’ motivation to learn. Three groups of Neutral learners, Extreme learners, and Skeptical learners gave their opinion of the use of digital badges. Digital badges make Factor 2b groups actively participate in blended learning because they thought that badges represented their achievements stimulated them and helped to foster a sense of competition and to improve their shortcomings. The teachers of this group need to pay attention to reforming traditional approaches and blended learning classroom settings.

Almost all the participants though that digital badges were authentic and innovative, although some participants (Factor 3) had the negative perception that they were mediocre and less prestigious than a certificate of completion. Factor 3 participants suggested that the display position of personal badges was not obvious enough and that the network should be more convenient for a good user experience, especially in blended learning.

Furthermore, focusing on the design of criterion rather than on the competition based on badge ranking can be effective in motivating individuals to do their best. According to Factor 3, the acquisition of knowledge is more conducive than the acquisition of a badge to increasing their intrinsic motivation to learn.

5.3. Limitations and Future Work

One limitation of this study involves the structure of the sample Q statements, although various strategies were used in the form of a Q sample. Another limitation was that the use of Q sort forces students to express their feelings and the choice of statements that represent their real opinions. Although the Q method lacked generalizability based on its complex research process, it still provided the instructors with an abbreviated opinion of the college students through the transferability of the Q methodology. Furthermore, since these results were only based on the perspective of students at a local mainland China university, the researchers wish to consider widening the educational area in the future.