Next Article in Journal
Connection to Nature Boosts Adolescents’ Mental Well-Being during the COVID-19 Pandemic
Next Article in Special Issue
Theory and Practice of VR/AR in K-12 Science Education—A Systematic Review
Previous Article in Journal
The Giant Leap for Smart Cities: Scaling Up Smart City Artificial Intelligence of Things (AIoT) Initiatives
Previous Article in Special Issue
Towards Figurative Expression Enhancement: Effects of the SVVR-Supported Worked Example Approach on the Descriptive Writing of Highly Engaged Students
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Applying the ARCS Motivation Theory for the Assessment of AR Digital Media Design Learning Effectiveness

Department of Information Management, National Chin-Yi University of Technology, Taichung 411030, Taiwan
Sustainability 2021, 13(21), 12296; https://doi.org/10.3390/su132112296
Submission received: 8 October 2021 / Revised: 20 October 2021 / Accepted: 28 October 2021 / Published: 7 November 2021
(This article belongs to the Special Issue Augmented Reality and Virtual Reality-supported Sustainable Education)

Abstract

:
This research adopts Keller’s ARCS motivation theory as a method to create a teaching experiment by integrating augmented reality (AR) into teaching in order to enhance learning interest and learning effectiveness in a digital media design course. The purpose of this research is to examine the application of AR in quarantine during the COVID-19 pandemic, whereby students can enhance their learning interest, learning satisfaction, and learning performance. Augmented reality acts as a tool for this research, wherein it is applied with the course of a 3D model-based interface and built-in learning contexts for the “digital media design” of the learning topics. The learning performance of the test group students was examined through the aspects of attention, relevance, confidence, and satisfaction, according to the ARCS motivation theory. According to the results of quantitative validation, experimental teaching with AR is more effective than traditional teaching methods. The learning feedback of the test group students obtained a positive result by using the AR. This research concludes with three results: firstly, integrating AR into teaching can improve students’ concentration with respect to digital media design practice; secondly, video teaching with an AR interface can help increase students’ confidence in digital media design learning; lastly, applying AR during the learning process can enhance the digital media visual effects that effectively enhance students’ self-learning abilities.

1. Introduction

Alongside the advancement of digital technology, the integration of digital media into daily life has become deeper and more thorough. Integrating AR technology into mobile devices or other digital media forms has already made a huge impact in contemporary businesses. It has also been applied in the field of education, entertainment, shopping, medicine, and military, as well as in museum guides. In order to keep pace with the rapid advancement of the world’s digital economy market trend, higher education teaching also needs a breakthrough via digital media technology. Various innovative digital media tools have had an increasing and beneficial influence in the education and learning process, especially for teaching methods, learning methods, the evaluation of learning effectiveness, etc. The main issue here concerns the method by which one cultivates the talents of proficient digital content producers in order to trigger further developments in digital industries. This is the main issue in education, and will be considered in higher education levels at this stage. Digital media design education is a field of innovative cross-disciplinary studies, integrating information, design, media, art, etc. It is an innovative field that combines tradition with modern technology, in which aesthetic skills, professional skills, and teamwork are the critical factors that can help students keep up with the rapid advancement of modern digital trends. In addition to theory and knowledge learning, the core abilities resulting from qualified digital media design education include software and hardware training, design practices, and creative thinking. A digital media design teacher could use AR technology as an auxiliary tool for an innovative teaching method in order to enhance overall learning effectiveness.
Methods for innovative learning environments that combine traditional teaching methods with digital media technology have been popularized in instructional design [1,2,3,4] and include augmented reality (AR), which has been commonly used in education by providing an interactive learning environment. AR combines teaching materials with augmented reality and has become more and more popular in recent years [3,5]. AR technology can connect images and other various elements of the virtual world to the real world by producing an overlapping action of animated images (as well as utilizing stunning interactive features). Integrating AR technology into smart mobile phones has the potential to create a better ambience and enhance the entertainment of students by combining the virtual world and the real world in the field of education. This phenomenon has been academically proven. AR can enhance students’ attention and create an overall more pleasant learning experience and enhance the value of their education [6].
The result of integrating AR technology into teaching can improve most students’ learning effectiveness. However, key questions: what is the crucial factor that helps trigger this learning effect? Is it by learning motivation, learning interest, learning attitude, or instructional design that the AR approach is praised by learners? This research adopted Keller’s ARCS motivation theory [7] to carry out a teaching experiment in order to verify whether integrating AR technology into teaching can enhance “digital media design” learning efficiency and improve overall learning performance. ARCS motivation theory refers to a process of piquing students’ attention and interest through a novel teaching program thar helps students comprehend that the things they learned are closely related to their own lives, and, as a result, help them build self-confidence to handle new situations. Consequently, students would feel a significant sense of accomplishment after learning.
This research takes mobile devices as the platform to develop 3D AR-based interfaces as a teaching experimental tool by which the “digital media design” learning materials can be instantly presented during the COVID-19 pandemic in 2021. The teaching program, which is integrated using AR into digital media design learning content, would be a valuable instrument for helping student learning. This research primarily uses this teaching experiment to conduct an examination of student learning performance. The number of test group students in the AR teaching program was 46 (Class 1, n = 46), and the number of control group students learning with traditional teaching methods was 42 (Class 2, n = 42). The quantitative empirical statistics method was applied for the examination of the difference of learning efficiency between the two groups’ students (i.e., test group students and control group students). The ARCS motivation theory was used to find the influence factor of learning performance for the test groups. There are two main purposes of this research. The first is to compare the difference in learning performance and learning interest between the two groups of students after the experimental teaching. The second is to examine if AR-based teaching can improve the test students’ concentration, learning interest, confidence, satisfaction, and learning performance by means of the ARCS motivation theory.

2. Literature Review

2.1. Digital Media Technology Applied in Self-Guided Learning

The wave of global digitalization has rapidly changed operations in the fields of business, education, industry, design, etc. Technology and networks have been popularized for 20 years and have deeply changed aspects of peoples’ lives [8]. Higher education is facing challenges, and the need for change is substantial. As a result, digital media proficiency in teaching has become a core component of modern innovative education [9]. In Taiwan, integrating digital technology into innovative teaching has become mainstream in educational reformation. The innovative education perspectives such as digital globalization trend and global visions, as well as development capability, have been widely promoted since 2013 in order to reach the goal of global higher education. The Ministry of Education in Taiwan has proposed that Taiwan’s education in the future will be connected with the latest learning technology, media, creative learning methods and environment, as well as a variety of learning programs to construct an innovative method for cultivating talents [10].
Nowadays, students who were born in 1982–2002 are called Generation Y. They have been growing up in a digital age with globalization and unprecedented access to tools of information, self-exploration, and entertainment. Therefore, their learning style mostly focuses on visual, aesthetic, and multimedia interaction. Compared to the traditional teaching methods in classrooms regarding the learning environment, they are more likely to be willing to have an interactive learning environment where students are actively engaged in the learning process [11,12]. Due to the rapid advancement of digital media technology, industrial society has evolved into the digital era, a society full of digital information that leads and influences the economy and culture. For Generation Y students, they experience the innovative learning trend with the feature of visual thinking, digital narratives, and high-speed networks, which shape their entertainment-oriented learning style and also enhance their ability to operate digital devices and encounter large amounts of information.
Digital media technology includes animation, games, interactive multimedia, AR/VR, etc. These can enhance the Generation Y students’ learning performance and learning efficiency, as has been verified by related studies [4,6,13,14,15,16,17]. Through online learning platforms with digital media technology support, education and knowledge can be broadcasted to users from different areas. Users can enjoy flexibility with this online learning platform depending on their situation and adjust the learning speed. The critical features of AR- or VR-based learning are not static but dynamic, and with more than two dimensions that can be wisely seen in most AR teaching materials in the field of education [13,18]. Through AR- or VR-assisted animation, games, or other interactive digital media, a unique learning experience can be provided for students to enjoy unique learning benefits, both physically and emotionally [19]. Regarding the development of AR, Kerr and Lawson mentioned the prototype of AR, Master of Time, which is applied for the education of first-grade students and used for the foundational principles of landscape architecture design. In their research, they found positive results from learning by experiencing AR-based digital media courses in situated experiences [13]. The teaching of landscape architecture education has been conducted studio classroom settings for a long time in university. AR-based landscape architecture education has become an innovative method for landscape architecture teaching. Kerr and Lawson suggested that innovation in education cannot be achieved without a deep understanding of AR digital media technology among educators.
In terms of the teaching materials, teachers can use digital media technology to design more interesting content in order to increase students’ interest. Digital media technology has been proved to enhance students’ skills, learning effectiveness, learning motivation, self-learning capability, and satisfaction [20,21,22]. Previous researchers have studied the integration of digital media technology into teaching in various contexts, for example, in interior design [4], or in English teaching with digital media [23] and an AR-based classroom [24], and even in cultural preservation and education [25]. Moreover, digital media technology application can support traditional teaching methods, such as with video podcasts or exotic ecosystems that can help enhance students’ environmental education [26].
Based on the above-mentioned literature, we can conclude that digital media technology can help Generation Y students understand the contents of their learning programs. Digital media also can help teachers to create innovative programs for students. Thus, the present research takes an AR-based teaching program with digital media as an assisted learning tool for students to learn and experience a new form of education.

2.2. Augmented Reality Realizes Innovative Teaching

In 1997, the concept of augmented reality had already been proposed by several scholars, such as Azuma with the aspect of systematization [27]. The theory of augmented reality has been practically and positively applied in various countries.
Innovative services based on virtual integrated reality have been widely applied in the field of medicine, and in art exhibitions, electronic commerce, trade show design, industrial product design, tour guidance, online learning, sports training, and physical product digitalization [4]. Through AR technology, virtual 3D images can be simulated as a real-world object and clearly seen from any perspective or angle. Therefore, the user can perceive the stunning visual effects of the combination of the virtual world and real world [28,29,30,31]. The AR 3D objects in the real world can be seen from a mobile phone device that allows a virtual experience for users.
Since the advancement of digital technology applications, real object emulation has become more popular in learning. Real object emulation can include digital technology, 3D models, animated images, as well as situations that can be repeatedly tested [32]. In the realistic virtual world, the environment can allow a user to perceive an impressive experience and also avoid the actual dangers of a real-world situation. Due to the above characteristics, integrating AR technology, 3D, animation, or games into education can be effective for learners to explore the world of knowledge [33,34]. As mobile devices are common, AR application offers more innovative learning content for students, such as real-time interactions with 3D graphics, visualization of 3D models, interactive UI, audio/video content, academic keywords tagging, and infographics [4].
AR allows people to experience a real-world learning environment. AR-assisted innovative applications have more applied value beyond the 2D-based interface with linear video/audio modes in digital learning. By integrating AR technology into a head-mounted display or other display devices, such as an Oculus headset, the digital information can be presented in the user’s own perspective and the objects can be perceived in a situation where the real world is combined with the digital world [35]. AR technology enables students to experience various unique interactions as well as to strengthen their self-learning capabilities.
According to above literature review, AR technology may be considered as an effective method to enhance learning interest. Through smart mobile phones or tablets, a student can learn in a simulated situation [36]. Hsu’s research results confirmed that the application of AR technology can effectively help students to experience contextual topics and achieve learning goals [3]. Hsu developed two AR-based game systems for learning English in a contextual environment, finding that AR technology can create a ubiquitous learning environment for students to experience. Furthermore, students’ learning motivation is also improved by real-time feedback factors as well as the sense of participation. AR technology enable students to experience a ubiquitous learning program and enhance their perception of linguistics, and the result proves that students are more likely to actively learn English through AR technology [37]. Khan et al. [38] claimed their results show that the AR system can improve students’ learning motivation and skills. They proposed the ARToolKit-based Interactive Writing Board (IWB) as a learning device that integrates AR technology. IWB can be used to teach Arabic or Urdu to elementary school students.
Based on related studies and literature reviews, this research uses AR technology as a tool to construct a digital multimedia instructional program by creating a ubiquitous contextual learning environment for students in order to examine if applying AR into teaching programs can enhance students’ learning performance and learning efficiency.

2.3. The Learning Motivation Theory

ARCS motivation theory was proposed by John Keller, an American professor who suggests a systematized motivation model based on the motivation theory and other related theories in order to stimulate students’ learning motivation [39]. Keller states that learning performance cannot be enhanced if the instructional program design cannot arouse a student’s learning interest. Keller expected that educators can take the ARCS motivation theory as a method to confirm and design instructional strategies according to students’ requirements in order to arouse learning motivation and effectively enhance learning performance. In 1987, Keller put forward four factors in the theory of learning motivation that can effectively help teachers stimulate students’ learning motivation and interest in learning [7]. The four factors are “attention”, “relevance”, “confidence”, and “satisfaction”, comprising the crucial elements of the ARCS motivation theory. ARCS motivation theory is used to improve teaching programs and help instructional design. The four elements of the learning motivation theory must be taken into account for instructional design in order to stimulate students’ learning motivation [7].
After integrating several motivation theories, ARCS motivation theory has been proposed to strengthen systematized instructional design, which serves not only to provide practical applications and conceptual organization but also to stimulate student interaction and participation in learning activities. The four factors of the ARCS motivation theory are closely related to each other where the educator’s teaching performance would be directly affected. The teacher should apply the ARCS motivation theory during the teaching process to facilitate student learning in a positive learning environment. The achievement of the instructional program would be directly affected if one of the four factors of the learning motivation theory is missing during teaching periods. John Keller emphasized that ARCS also has the function of “diagnosing” problems and helping to improve learning ability. This means that if learners are struggling to learn, there are ways to overcome these problems through the ARCS model. In other words, teachers must use more adaptive teaching methods to design textbooks for such students [40].
The four elements of the ARCS motivation theory as well as its definitions are clarified in Table 1. Based on Keller’s ARCS motivation theory, the definitions of the four elements are connected to the application purpose, which is divided into four parts: learning attention, learning interest, learning behavior, and learning satisfaction [4]. Together, they help teachers to recognize students’ learning motivation and also to create an interesting and effective teaching program for students to learn with their full attention during class.
The crucial function of the ARCS motivation theory is to help teachers generate an interesting instructional program that can attract learner attention and enliven the teaching materials and also escalate student interest in learning and offer students a positive learning experience. The variable of the “relevance” factor has been defined as “learning interest”, which refers to when a teacher creates an appropriate instructional program in order to amplify students’ learning interest during class, and meanwhile, during the learning process, students can recognize the knowledge system and realize the learning value. The learning performance and efficiency, however, will be directly affected by attention and learning interest. As a matter of fact, any single element of the learning motivation theory directly influences the others. Learning interest is considered as the core value of pleasant learning [41].
ARCS motivation theory has been widely applied in experimental teaching research. Chang et al. [4] used the ARCS motivation theory and integrated AR technology into interior design to create a real-time visual effects feature and enable students to recognize the problems that may be happening during the interior design and to find the best solution through teamwork. Zheng et al. [22] pointed out that with the ARCS motivation theory, instructional programs can be improved and students’ learning motivation can be strictly and objectively evaluated. Chang, Chen, and Liao [24] applied ARCS motivation theory to design an AR-assisted situational classroom for English teaching, and found that it enhanced EFL students’ learning satisfaction during the experimental teaching.
Based on above-mentioned statements and literature reviews, this research uses ARCS motivation theory as a tool to validate the effectiveness of experimental teaching. The aspects of the validation include students’ learning attention (concentration), learning interest, learning confidence, and AR-based learning satisfaction in digital multimedia design teaching class. I also endeavor to verify if integrating AR technology into experimental instructional programs can help enhance students’ learning performance.

3. AR-Based Teaching Tool in Digital Media Design Course

3.1. Course Instructional Design

In digital multimedia design courses, many students do not understand how to produce a complete multimedia project. Thus, this research takes an interactive treehouse as the learning theme for students to practice. At first, the teacher offers a draft of a treehouse with an interactive design for students, as seen in Figure 1. After the instruction of the concept of digital media design, the students undertake the phase of learning by doing. The learning topics include:
  • Topic setting: The teacher takes a treehouse based on the fairytale of Hansel and Gretel as the digital media topic in which the various interactive buttons have been designed for users to explore. To illustrate, a giraffe appears after pressing the first button. After pressing the sixth button, the users can access the inner space of the treehouse, where the scene of the Hansel and Gretel fairytale is presented in the form of a digital media interaction, as seen in Figure 2.
  • Program design: Programming capability as well as coding for the interactive treehouse design is carried out by C# script that will be taught by the teacher in the class for students to create various buttons for the variety of interactive effects.
  • Platform practice: In order to develop an interactive digital media treehouse project, students must be familiar with operating the Unity platform, taught by teachers, since the digital media project was developed by Unity platform in this course.
  • Content produce: Users’ experience in the interaction with digital media systems will be affected by the perception of the content interaction. The project content includes images, animation, sound effect, character dialogue in the story, navigation guides and instructions for users, UI design, and so forth.
  • Post-production effects: Various interactive effect settings for the treehouse in post-production as the feedback of user behavior includes visual effects, vocal effects, environmental sound effects, etc.
This workflow for this study’s test group students who are learning through the AR-based teaching program is shown in Figure 3.

3.2. Software Development

To execute this study, I applied the AR technology to create the aid-learning system suited for smartphones and mobile tablets so that the students could enjoy this interactive experience. I used the C# language on the Unity platform to develop the AR system and provided it to the test group students and to validate their learning experience with digital media design. The overall system architecture is similar to other Unity-based solutions, as shown in Figure 4. I also compared traditional learning methods with the AR-based textbook, which is described in Table 2. The application system-related specifications that were utilized are listed below:
  • Unity 2018.4.10f1 (64-bit): The developed platform built-in Vuforia Kits and C# script, as illustrated in Figure 4.
  • SDK Tools for Android: It was necessary to install Android SDK to produce an Android application package (APK) requiring Kits of Unity.
  • Java SE Development Kit 8: In addition to SDK Kits, Unity also needs to install the Java SE software development Kit to build APK.
  • The 3D model creation software for AR, e.g., 3D MAX.
  • The mobile device with an Android operating system for the AR application executing.
From a user-oriented perspective of digital media, this research developed an innovative instructional approach for students to learn that is aligned with the traditional learning steps, taking the example of the buttons’ design on the devices that can provide interactive effects for users, as seen in Table 2.

4. Research Methods

4.1. Research Design and Approach

Bearing the principle of scientific and objective evaluation in mind, this research was carried out the experimental teaching mechanism (illustrated in Figure 5) to examine the instructional program and learning performance and to validate students’ learning performance. This research adopted a “nonequivalent pre-test and post-test” design as the method to examine how learning performance is affected by the experimental teaching program. The independent variable, dependent variable, control variables, and experimental phases for this research are clarified as follows. Firstly, the independent variable in this research is the groups of students. One group is the test group students who had the AR-based digital multimedia instructional program, and the other group is the control group students who were taught by traditional teaching methods. Both groups were examined by “digital multimedia design” pre-tests and post-tests in order to verify the difference in learning performance between the two groups of students. Secondly, the results of the pre- and post-tests of the two groups are used as variables to evaluate if there is a significant difference in learning performance, via the t-test. Thirdly, there are three control variables: the first is that the teacher who instructs the two groups of students is the same, in order to have reliable research findings; the second is that the two groups of students will be examined through the pre- and post-tests conducted by a researcher; and the third is that the researcher is also the instructor carrying out the examination of student learning performance via the knowledge tests of multimedia design.
The experimental phases can be divided into three. In the first phase, the “digital multimedia design” pre-test and digital media learning interest scale test (Appendix A) were completed in the first week of teaching. In the second phase, the two groups each had a 50-minute AR-assisted digital multimedia design class three times a week and the teaching experiment lasted for 18 weeks. In the third phase, the post-test was conducted after the 18-week instructional program. The two groups were examined through “digital multimedia design” tests (Table 1) in order to validate the difference in learning performance between the two groups. The research subjects were 88 students at the National Chin-Yi University of Technology (NCUT) management department; the number of test group students was 46 (n = 46) and the number of control group students was 42 (n = 42). The post-test on the digital media learning interest scale for the two groups of students was also conducted during the process.
In the process of learning by doing, the motivation element of learning digital multimedia design courses among the students of the two groups was examined and analyzed by the pre- and post-tests; the factors were evaluated and validated in each learning topic for digital media design. The interest scale test was conducted for both groups to determine if applying this AR-assisted instructional experiment can effectively enhance learning interest.

4.2. Test Questionnaire Design

Based on the ARCS motivation theory, the research questionnaire was designed to evaluate the experimental teaching method. Before and after the experimental teaching, the questionnaires were distributed to validate the two groups’ concepts of digital multimedia design and examine their learning performance and learning interest (Appendix A). The questionnaire was distributed via Google Forms and then collected and statistically sorted with the SPSS tool. An independent sample t-test was conducted on the pre-test questionnaire in order to analyze the difference between the two groups’ learning performance and also to identify the homogeneity between the two groups of students. The result of the t-test showed that the analysis was non-significant, indicating the homogeneity between the two groups. The post-test after the experimental teaching examined the learning performance and to obtain further verification of whether integrating AR technology into this digital multimedia design instructional program can improve learning effectiveness.
Regarding the two groups’ knowledge of digital multimedia design, as seen in Table 3, the learning performance pre- and post-tests included three parts: design capability, learning topic, and project creativity. The pre-test only focused on the first and second parts of Table 3, since the students had no experience of developing the project. After the experimental courses, three parts of the evaluation were carried out by teachers based on the post-tests.
The data of the pre-test included a preliminary test and an interest scale test. The examination of the difference of the two groups was performed by means of an independent sample t-test to verify the homogeneity between two groups. If the result was non-significant, this means homogeneity existed in the two groups and that this research can take the result of the post-test to examine whether integrating AR technology into digital multimedia design instructional programs can enhance students’ learning interest and improve learning performance.

4.3. Research Participant

The research subjects in this research are sophomore undergraduate students who are taking the “Digital Media Design” course in the department of information management, National Chin-Yi University of Technology (NCUT) in Taiwan. Considering research ethics, the experimental teaching groups were not divided among in the same class. The 42 students of the digital media design course in 2020 were used as the control group, while the 46 students of the digital media design course in 2021 were used as the test group. In 2020, COVID-19 in Taiwan was well controlled, and students were learning in schools. The period of experimental teaching for the control group was from 2 February 2020 to 30 June 2020; the test group was from 22 February 2021 to 25 June 2021. In 2021, COVID-19 started to spread in Taiwan and all the learning activities in Taiwan moved online. The researched AR-based teaching program can allow students to learn at home. The two groups of students filled out the interest scale on the test questionnaire. The preliminary test was conducted before the experimental teaching. The start date for the control group was 3 February 2020, and that for the test group was 23 February 2021; the final test was conducted after 18 weeks of the experimental teaching program. The control group’s final test was on 30 June 2020 and the test group’s final test was on 22 June 2021. There were 46 test group questionnaires and 42 control group questionnaires collected; valid questionnaires totaled 88 for the examination. The ARCS questionnaires were sent to the test group students by using Google Forms at the end of June 2021. Then, SPSS was used to statistically analyze the data. The first analysis is the result of the average value and standard deviation for two groups’ pre-test and interest scale test by t-test, and the second is based on the pre-test results to conduct a single factor covariate analysis (one-way ANOVA) to see if there is a significant difference between the two groups to identify the homogeneity or lack thereof. The third analysis is the post-test result of two groups, examined by an independent sample t-test in order to verify whether integrating augmented reality into experimental teaching can improve learning performance. The fourth analysis is the ARCS motivation theory model correlation analysis, conducted for test group students. Finally, the last analysis is the hypotheses verification by regression coefficient analysis.

5. Experimental Result

5.1. The Teaching Effective Validation of t-Test Analysis for Independent Sample

The results of the experimental teaching pre- and post-tests are shown in Table 4 (Knowledge Test for Multimedia Design). According to the results of the pre-test, it can be seen that the values between two groups are quite close, whereas the test group’s post-test values are relatively higher than the control group. In the analysis of the post-test, there is a significant difference between groups. As for the aspect of learning interest, it can be found that the pre-test results of the groups had no significant difference, while in the post-test results, the test group values are relatively higher than the control group. There is a significant difference between groups, which indicates that the experimental teaching was successful in producing improved results compared to the traditional teaching way.

5.2. Test of Homogeneity of Test and Control Groups

In this study, I used the Knowledge Test for Multimedia Design as well as the Interest Scale Test in order to make comparisons between the test group and the control group regarding students’ learning capability. The tests were conducted before the experimental teaching in order to verify whether there was a similar learning capability for these two groups. Based on the results of pre-test and the interest scale test, regression coefficient analysis was employed to perform the test of homogeneity of regression between the two groups, to confirm whether the regression line was parallel or not within the group and if the difference between the two groups by p value reached a statistically significant level. The one-way ANOVA was then used to examine homogeneity by verifying whether the average values between groups were close enough to confirm the learning capability homogeneity. The level of significance of the statistical analysis was 0.05 (95% confidence level). The ANOVA result shows that there are no significant differences between two groups in the “Knowledge Test for Multimedia Design” and “Interest Scale Test”, which means the two groups of students had the same learning capability (seen in Table 5). Table 5 verifies whether the two sets of learning abilities are the same for comparison. This test was performed before the experimental teaching to check if the two groups were different, or if the two groups showed similar learning abilities. The results show that the two groups are homogeneous.

5.3. ANOVA for Post-Test

This research took the post-test of two groups as the examination of learning efficiency. After 18 weeks of the experimental instructional program, the “Knowledge Test for Multimedia Design” examined the two groups’ learning performances. As mentioned in Section 4.2, the final test results of groups were evaluated in three parts: design capability, learning topic, and project creativity. The post-test of “Learning Interest Scale” (Appendix A) was also conducted after 18 weeks of the experimental teaching program. The ANOVA statistical analysis was used for evaluation. The level of significance of the statistical analysis was 0.05 (95% confidence interval). The result of ANOVA, in Table 6, shows that in the Knowledge Test for Multimedia post-test, there was a significant difference between the test group and control group (F = 32.907, p < 0.001). Regarding learning performance, the analysis result indicated that the test group students who experienced the AR-based teaching program fared better than the control group students who were taught by the traditional teaching way. The difference in learning efficiency between the two groups was validated. As for the part of interest scale test, this research considers the post-test of the groups’ learning interest as the dependent variable to carry out the ANOVA. The level of significance of the statistical analysis was 0.05 (95% confidence interval). As can be seen in Table 6, the ANOVA shows that on the interest scale, post-test results had a significant difference between the two groups (F = 18.895, p < 0.001). It is clear that integrating AR technology into instructional programs can be more effective than traditional teaching methods. Thus, the first aim of this research has been achieved.

6. ARCS Motivation Theory Model for the Examination of Learning Motivation

6.1. The Definition of the ARCS Motivation Theory Model Constructs

Based on Keller’s ARCS motivation theory elements and its definition [7], this work designed an approach for evaluation that includes four constructs, the first being attention. Students’ attention was evaluated. The second construct is relevance. Students’ personal recognition was developed by the AR-assisted teaching approach as well as past learning experience; students’ learning interest can be aroused and evaluated by the method of questionnaire. The third construct is confidence. After this experimental teaching program using AR, students built up their self-learning capabilities and their confidence was be evaluated. The fourth construct is satisfaction. Students’ satisfaction with the AR-based teaching program was evaluated. Based on ARCS theory, I designed twenty questions divided into four parts (Appendix B). The definitions of the ARCS theory’s four elements are clarified as follows:
  • Attention. The role of attention in learning is important in that the capacity for information processing is limited. From this perspective, immersed experiences are regarded as when one’s attention is focused on an object for longer periods of time to allow cognitive processing [42]. Based on the evaluation approaches of Csikszentmihalyi [43] and Chambers [44], “immersed experiences” are classified into the part of “concentration” for this study’s questionnaire design. Individuals will devote more attention, energy, and time to certain things they are interested in. Thus, concentration is considered as a sort of “immersed experience”, which often takes place during the process of learning; that is to say, when students are attracted to situational contexts during learning, they tend to focus better on the program and immerse themselves in it [42].
  • Relevance. An AR textbook design and its learning topics can help to enhance student learning interest and make students immerse themselves in learning activities, and if they forget about the time—these are evaluated by using the construct of relevance. Learning interest can help cultivate self-learning capability and enhance pleasant learning experiences [6]. Based on the definition of relevance, the aspect of learning interest was designed for this research questionnaire in order to evaluate students’ interest in AR textbooks for digital multimedia design instructional programs and its learning topics.
  • Confidence. Confidence can be seen as a sort of learning style. Does an AR-based teaching program enhance students’ self-confidence? When a student recognizes that learning efficiency is enhanced by AR teaching, their confidence will be improved, as well. Chang et al. [4] stated that by observing students’ learning behavior during the AR-based class, students’ confidence can be examined. Moreover, Zheng et al. [22] also proposed that confidence refers to a kind of mental state; to illustrate, in a student’s internal mental state, the expectation of success is aroused while experiencing an interesting and enjoyable learning program.
  • Satisfaction. Satisfaction is evaluated during the experimental teaching to verify if integrating AR into teaching can enhance student satisfaction. Zheng et al. [22] stated that students will be satisfied with AR-based experimental teaching programs if they feel that the courses are helpful.
This research took Likert’s five-point scale as the method to carry out the survey; the questionnaire was designed for students to rate the questions so the research constructs can be validated for examination. Regarding the questionnaire, numbers 1 to 5 are used for students to examine their learning performance; 5 indicates “strongly agree”, 4 is “agree”, 3 is “neutral”, 2 is “disagree”, and 1 indicates “strongly disagree” (see Appendix A). The ARCS model and research hypotheses are illustrated in Figure 6.
This study’s four hypotheses are listed below:
Hypothesis 1. 
The factor of learning interest in the relevance construct positively affects students’ attention.
Hypothesis 2. 
The factor of learning interest in the relevance construct positively affects students’ confidence.
Hypothesis 3. 
The factor of learning interest in the relevance construct positively affects students’ satisfaction.
Hypothesis 4. 
The factor of learning interest in the relevance construct positively affects students’ learning performance.
The research hypotheses were tested using reliability analysis, efficiency analysis, correlation coefficient analysis, as well as the research hypothesis tested approach.
  • Reliability Analysis: Cronbach alpha values are the indicator for the reliability of each construct’s analysis data.
  • Validity Analysis: Confirmatory factor analyses are applied for the examination of construct validity degree; data reduction is conducted by the confirmation factor analysis and the factor is extracted by the principal component analysis. Finally, analysis data are verified by means of the Varimax Method of Kaiser–Meyer–Olkin and Bartlett’s Spherical Verification.
  • Correlation Coefficient Analysis: Pearson’s correlation coefficient analysis is applied for the examination of this relationship between factors. The correlation coefficient matrix between two variables is analyzed according to the analysis results and can verify if they are positively related to each factor in the motivation theory model (significant existence).
  • Research Hypothesis Tested Approach: This research takes the principle of the regression analysis and the assumption of model constructs as the approaches to create the regression functions to examine the research hypotheses. The relationship between the cause and effect of the two variables can be testified, and the variables are directly or indirectly affected by each other and can be validated. The analysis finding R2 as the dependent variable in the variables can be explained by independent variables and the percentage that was produced by the regression analysis model. This research uses SPSS 12.0 to carry out the regression statistical analysis in order to examine how the dependent variable is affected by one single independent variable or several variables, and the β coefficient is obtained based on the significance. Finally, the explanatory power of the model is based on the adjusted R2 value.

6.2. Reliability Analysis and Validity Analysis

According to the results of descriptive statistics on the test group survey from an ARCS questionnaire in Appendix B, the data on the test’s groups learning performance were collected by the Knowledge Test for Multimedia Design post-test and converted through the Likert 5-point scale. In Table 7, the mean of each element in the ARCS motivation theory shows that the test group students positively view this experimental teaching program and that they are satisfied with AR learning (average value of each element > 3). The test group students’ feedback on the AR-based experimental teaching is positive. Learning attention, learning interest in AR teaching, learning confidence, and learning satisfaction were positively affected by the experimental instructional program.
Cronbach’s alpha values are taken as indicators of the examination of reliability degree; values under 0.35 show a low degree of reliability (α < 0.35), values between 0.35 and 0.70 indicate a medium reliability degree (0.35 < α < 0.70), and there is a high degree of reliability if the value is over 0.7 (α > 0.7). After statistical analysis of the constructs, the alpha of “learning attention” is 0.9, “learning interest in AR teaching” is 0.824, “learning confidence” is 0.890, and “learning satisfaction” is 0.849. Overall, the questionnaire’s reliability degree is 0.7, which means there is a high degree of reliability. The data of the survey show consistency in the acceptable range for this research.
Since the units of the variable measurements are the same in this study, based on a five-point Likert scale, the reliability can be measured by Cronbach’s α value. If Cronbach’s α value is less than 0.35, it has low confidence, whereas medium confidence is 0.35 < α < 0.70 and an α value greater than 0.7 shows that the reliability is high and acceptable. The reliability validation for the questions in this questionnaire shows a high reliability for the standard of Nunnally [4,45].
For the examination of the validity degree, I used SPSS to carry out the confirmatory factor analysis (ES). Factors were extracted by principal component analysis with the Varimax method of KMO and Bartlett’s Spherical Verification to further determine how suitable the data are for the factors. Through factor analysis, the KMO analysis value was 0.668, and spherical verification was 815.680; the values are significant, which means the questionnaire questions are applicable for a factor analysis in this research.
By means of principal component analysis (PCA) and the Varimax method, four factors have been extracted: learning attention, learning confidence, learning satisfaction, and learning interest, and rotation square and the loading show that the eigenvalues are 4.108, 3.937, 3.328, and 3.301, respectively. It is clear that all the eigenvalues are greater than 1, indicating a high degree of validity for the factors, as the rotation square sum is up to 72.482, as seen in Table 8.

6.3. Correlation Coefficients Analysis

In this section, I use Pearson’s correlation analysis to explore the linear relationship between two variables and determine whether the two variables are positively correlated. Based on the analysis results, the correlation coefficient matrix between the two variables obtained was analyzed (see Table 9). According to the correlation verification results of this study, it was found that:
  • Regarding the test group learning performance analysis, the “learning attention” and “learning interest in AR teaching” have a significant positive correlation, 0.432 (**); “learning attention” and “learning confidence” are significantly and positively correlated, 0.409 (**); “learning attention” and “learning satisfaction” are significantly and positively correlated, 0.413 (**); and “learning attention” and “learning effectiveness” are significantly and positively correlated, 0.501 (**).
  • In the test group students, “learning interest in AR teaching” and “confidence of learning behavior” are positively correlated and the correlation coefficient is 0.545 (**); “learning interest in AR teaching” and “learning satisfaction” are positively correlated and the correlation coefficient is 0.497 (**); and “learning interest in AR teaching” and “learning effectiveness” are positively correlated and the correlation coefficient is 0.732 (**).
  • There is a significant positive correlation between “learning confidence” and “learning satisfaction” in the test group students and the correlation coefficient is 0.712 (**); “learning confidence” and “learning effectiveness” are positively correlated and the correlation coefficient is 0.658 (**).
  • “Learning satisfaction” and “learning effectiveness” have a significant positive correlation; the correlation coefficient is 0.714 (**).
Based on the Pearson correlation coefficient analysis findings, the data regarding the correlation of the ARCS motivation theory model and learning performance are shown in Table 9.

6.4. Research Hypothesis Test

This research used path analysis as a tool to statistically examine the overall statistic model explanatory power. The linear regression analysis was conducted to validate four hypotheses concerning whether students’ (1) learning attention, (2) learning confidence, (3) learning satisfaction, and (4) learning performance can be significantly enhanced by AR-assisted instructional programs. The causal relationships between two variables and how they directly or indirectly affected each other were analyzed. The R2 value is the indicator for the percentage of total variance calculated by the independent variables and the regression analysis model. This research used SPSS to carry out the statistical regression analysis.
In the following functions, one can see how the independent variable directly or indirectly affect the others by the method of linear regression analysis; values are obtained by the path analysis approach to show how one variable is affected by the other. Furthermore, the adjusted R2 value is taken as the explanatory power indicator for the dependent variable. By means of the path analysis, the four research hypotheses are explained as follows:
  • Function 1: Learning attention = f {Learning interest}. According to Keller’s ARCS theory, the factors that affect learning interest are external environmental factors such as teaching methods and textbook design; in this research, this factor is the AR-based teaching. The F statistic value is 10.110 and the significant level is p = 0.003 < 0.05. Linear regression analysis has a reliable predictive capability for the verification of the hypothesis. Based on the analysis, “learning attention” is positively affected by the factor of “learning interest in AR teaching” in the test group students (β = 0.432, p = 0.003 < 0.05, R2 = 0.168). The explanatory power is 16.8%, which is significant. Thus, H1 is valid. Based on the linear regression analysis model, it was proved that AR teaching can effectively improve students’ self-confidence during learning.
  • Function 2: Learning behavior = f {Learning interest}. Linear regression analysis results show that “learning behavior” is obviously affected by the factor of “learning interest”; the value is significant (β = 0.641, p = 0.000 < 0.001 R2 = 0.281). The linear regression statistics analysis model has a reliable predictive capability for the examination of hypothesis. Learning behavior is positively affected by the factor of “learning interest” and the correlated value is significant. Thus, H2 is valid. It was proved that the test group students’ learning behavior is affected by the AR teaching and that the learning confidence can be improved.
  • Function 3: Learning satisfaction = f {Learning interest}. According to the linear regression analysis results, “learning satisfaction” is positively affected by the factor of learning interest AR teaching and the correlation value is significant (β = 0.481, p = 0.001 < 0.001 R2 = 0.272). The linear regression analysis model has a reliable predictive capability for the verification of the hypothesis. It was proved that “learning satisfaction” is positively affected by the factor of learning interest in AR teaching. Thus, H3 is valid.
  • Function 4: Learning effectiveness = f {Learning interest}. According to the linear regression analysis, learning effectiveness is positively affected by the factor of learning interest and the correlated value is significant (β = 0.538, p = 0.000 < 0.001 R2 = 0.526). The linear regression analysis model has a reliable predictive capability for the verification of the hypothesis. The explanatory power is up to 52.6. Thus, it was proved that “learning effectiveness” is positively affected by the factor of learning interest; therefore, H4 is valid.
The overall linear regression analysis findings are shown in Figure 7 in which arrows appear between the factors showing the different facets and the asterisk (*) shows that the coefficient correlated value is significant. The confidence interval is 95%, which means there is 95% credibility coverage of the true value. This result is the same as the judgment result of the 95% level p-value of statistical verification.
The following four research hypotheses have been validated:
  • Students’ learning interest is aroused by AR textbooks. The teaching result proved this research aim. Thus, H1 is verified (as mentioned in Section 6.3). Learning interest in AR teaching can positively affect learning attention (attitude). The learning topics for digital multimedia design were designed for the contingency of the COVID-19 pandemic during the experimental period; however, it is crucial that the teaching design is compatible with studying in isolation. Hypothesis 1, which makes the AR teaching program possible for such a contingency, has been validated (circle 1 in Figure 8).
  • Some research related to AR-based teaching took gameplay as the theme to carry out the examination [33,34]. Nevertheless, the factor of learning interest for AR-based teaching videos was taken here as the independent variable to conduct the examination. In the verification of H2, the impact of the AR textbook on learning confidence reached 64.1%. The research found that the AR with teaching videos can help students practice homework exercises successfully according to their individual abilities, and enhance the confidence of students to continue learning. A student’s performance obviously depends on his or her individual abilities. The AR textbook enlivens the learning materials by which student learning motivation and interest can be enhanced, and then students voluntarily put in more effort. According to the research findings and the linear regression analysis, mentioned in Section 6.4, self-learning capability and learning confidence can be improved by AR-assisted instructional programs. This hypothesis, H2, has been validated (circle 2 in Figure 8).
  • By using the AR textbook-based teaching method, self-study motivation can enhance students’ learning satisfaction. Students’ learning satisfaction can be enhanced while they are immersing themselves in the AR textbook during learning. From the perspective of equity, students can achieve high-quality learning value and commitment to the learning process through long-term self-study with AR textbooks. This hypothesis, H3, is validated (circle 3 in Figure 8).
  • Through an AR-based instructional program, students can self-study repeatedly; meanwhile, learning motivation can also be enhanced, and the learning performance would be improved. This research hypothesis, H4, has been validated. The explanatory power R2 that learning interest for AR textbooks affecting learning performance is up to 52.6%. This means that learning performance is significantly affected by the factor of learning interest. Hypothesis 4 has proved that AR learning influences students’ motivation, encouraging students to put in more effort into learning and thus improving their learning performance (circle 4 in Figure 8).
In Keller’s psychological research related to motivation theory [40], Keller divided teaching experience and learning experience into output and input constructs. The construct of input includes individual and environmental factors; the construct of output includes effort, attitude, and learning performance. Individual factors include learning motivation, learning interest, individual learning capability, knowledge and skills, and so forth. Environmental factors are the learning situation in the classroom, instructional design, and teaching approach innovation. Students’ learning attention is affected by the teaching topics, attractiveness of learning materials, learning mood, and external environment (see Figure 8). Thus, instructional program design is a critical component of environmental factors; students will be attracted by the interesting and useful learning materials, and lively teaching lets students enjoy learning [41]. During the experimental teaching, students’ learning experiences are relative to their own learning interest, which is of positive emotional value. In the process of learning, students gradually recognize the meaning of the knowledge system and its value which in turn affects their learning attention, learning interest, and learning performance, and even one affects one another. Learning interest is considered as the core value for enjoyable learning [39]. For creating a positive cycle of learning effectiveness, individual factors that encourage students make more efforts are the main concern in ARCS motivation theory. The critical point for a positive cycle of learning effectiveness is to arouse learning interest through instructional program design—in other words, to enable students’ learning interest to be stimulated by a comfortable and interesting learning environment. An excellent instructional design would use an interactive approach to increase students’ learning interest by which the learning confidence is enhanced, and in turn, they will show initiative in learning. Students who put in more effort can achieve better learning performance, which is the learning value that the ARCS motivation theory emphasizes.
This research used regression analysis to verify that students’ learning attention, learning confidence, learning satisfaction, and learning effectiveness (the explanatory power is 52.6%) can be enhanced by an AR textbook design during the COVID-19 pandemic. The independent variable of the regression analysis is the learning interest for the AR textbook design. We all know that there are many factors that affect students’ learning effectiveness. For example, learning materials, instructional program design, interactions with peers, individual learning capability and style, etc. By means of the regression analysis, I examined the four factors of Keller’s ARCS motivation theory model, “attention”, “relevance”, “confidence”, and “satisfaction”; the learning effectiveness of test group students was also enhanced by using an AR textbook design.

7. Conclusions and Limitations

In this work, I designed an AR textbook for a mobile smartphone platform in which digital media design teaching materials can be visualized. AR technology can present real-time digital multimedia design instructional topics and teaching materials as well as an operation guide, which is an effective way to encourage students’ self-learning. Students can view or preview the learning content at any time, even in lockdown period during the COVID-19 pandemic. An AR textbook-based experimental teaching program has been evaluated in this research. For the semester-long digital multimedia design courses, the pre- and post-test data were collected for the examination of student learning performance. In order to verify if an AR textbook can stimulate students to learn actively, and if students’ learning interest as well as learning effectiveness can be enhanced, the feedback regarding the aspects of learning interest, instructional program design, learning confidence, and learning satisfaction from students are the critical data for examination.
In Section 5, the learning performances of the two groups of students were statistically analyzed by independent sampling t-tests. The results showed that after this experimental instructional program, the learning performance of the test group was better than the control group. In Section 6, Keller’s ARCS motivation theory model was used to examine whether applying AR into experimental teaching programs affects learning motivation, learning interest, or learning performance. By means of verified reliability, validity, correction coefficient analysis, and regression analysis, the second purpose of the research was examined. This research concluded that integrating AR technology into digital multimedia design instructional programs can effectively enhance learning efficiency and teaching efficiency.
In my previous research [4], an interior design course teaching was used as a research field. The students used an AR device as an aid for viewing 3D model prototypes of furniture placement and verified the layout design practice results intuitively. In this research, AR instructional videos were used to teach students how to develop digital media projects, involving the Unity platform operation and program coding technology learning. The students of the test group needed to learn all the topics set in the AR course completely before they could establish the practical technology for project development. In the past, the practical technology in digital media was taught and demonstrated by teachers in physical classrooms. Due to the COVID-19 pandemic in Taiwan, all schools were closed at the time of study, and students could only build their self-learning ability through the eLearning system. This is the most obvious difference in the experimental teaching design of these two studies. The test group students of this research needed to continue learning and completing their practice homework while relying on AR textbooks.
The contribution of this research is proving that AR teaching materials are suitable for normal learning programs during the COVID-19 pandemic. At present, it appears as though online learning and self-study at home will become the norm for the students all over the world. Digital media design was the research field, and the coding program and the project development as professional domain knowledge of the Unity platform were found suitable for AR teaching materials for online learning solutions. According to the correction coefficient analysis in Section 6.3 and the regression analysis in Section 6.4, the AR textbooks (teaching videos) provided by this research can improve students’ self-learning ability. The self-learning ability of students—that is, the correlation between learning interest, learning attention, learning confidence, learning satisfaction, and learning performance of the ARCS model—is significant, and the four hypotheses of this research were verified. The research results confirm that the use of AR teaching materials promoted the positive performance of the test group students in terms of their learning motivation, which means that the students in the experimental teaching program established their self-learning ability.
However, the limitations of the research are that for successful AR teaching experiences, teachers must devote substantial additional time to work on AR-based textbooks, in contrast to other teaching models, which represents a major impediment for educators. As Kerawalla et al. [16] and Rasimah et al. [17] noted, limitations occur due to a lack of a foundational knowledge and “conceptual framework” for teachers to apply the digital media technology [13]. Only when teachers can use digital media technology and continue to invest in the production of teaching materials can they provide students with novel and rich educational textbooks.

Funding

This research was funded by MOE Teaching Practice Research Program, Taiwan, grant number PHA1090407 and the APC was also funded by PHA1090407.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to data privacy of the tool and students.

Acknowledgments

The author would like to thank the MOE Teaching Practice Research Program of Taiwan for the funding support given to this research.

Conflicts of Interest

The author declares no conflict of interest.

Appendix A

Interest scale test for test and control groups
  • I think that learning digital media design is interesting in the planning practice.
  • I think that learning digital media design is interesting in the coding practice.
  • I think that learning digital media design is interesting in the developed platform practice.
  • I think that learning digital media design is interesting in the content design practice.
  • I think that learning digiatl media design is interesting in post effect and integrated practice.

Appendix B

ARCS questionnaire for test group
A: Learning attitude for digital media design
  • I can immerse in the digital media design learning for the planning practice.
  • I can immerse in the digital media design learning for the coding practice.
  • I can immerse in the digital media design learning for the developed platform practice.
  • I can immerse in the digital media design learning for the content design practice.
  • I can immerse in the digital media design learning for the post effect and integrated practice.
R: Learning interest for digital media design by AR materials
  • I think that learning digital media design by using AR is interesting in the planning practice.
  • I think that learning digital media design by using AR is interesting in the coding practice.
  • I think that learning digital media design by using AR is interesting in the developed platform practice.
  • I think that learning digital media design by using AR is interesting in the content design practice.
  • I think that learning digital media design by using AR is interesting in post effect and integrated practice.
C: Learning behavioral for digital media design
  • I have confidence in the digital media design learning for the planning practice.
  • I have confidence in the digital media design learning for the coding practice.
  • I have confidence in the digital media design learning for the developed platform practice.
  • I have confidence in the digital media design learning for the content design practice.
  • I have confidence in the digital media design learning for the post effect and integrated practice.
S: Learning satisfaction
  • I am satisfied with the digital media design learning for the planning practice.
  • I am satisfied with the digital media design learning for the coding practice.
  • I am satisfied with the digital media design learning for the developed platform practice.
  • I am satisfied with the digital media design learning for the content design practice.
  • I am satisfied with the digital media design learning for the post effect and integrated practice.

References

  1. Clark, R.E. Reconsidering research on learning from media. Rev. Educ. Res. 1983, 53, 445–459. [Google Scholar] [CrossRef]
  2. Catrambone, R.; Seay, A.F. Using animations to help students learn computer algorithms. Hum. Factors 2002, 44, 495–511. [Google Scholar] [CrossRef] [PubMed]
  3. Hsu, T.C. Learning English with augmented reality: Do learning styles matter? Comput. Educ. 2017, 106, 137–149. [Google Scholar] [CrossRef]
  4. Chang, Y.-S.; Hu, K.-J.; Chiang, C.-W.; Lugmayr, A. Applying Mobile Augmented Reality (AR) to Teach Interior Design Students in Layout Plans: Evaluation of Learning Effectiveness Based on the ARCS Model of Learning Motivation Theory. Sensors 2020, 20, 105. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Bier, E.A.; Stone, M.C.; Pier, K.; Buxton, W.; DeRose, T.D. Toolglass and Magic Lenses: The See-through Interface. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques, Anaheim, CA, USA, 2–6 August 1993; ACM: New York, NY, USA, 1993; pp. 73–80. [Google Scholar]
  6. Chang, Y.-S.; Chen, Y.S.; Chiang, C.W. The differences in pleasing value and learning performance among different groups using mobile augmented reality system for cultural environment learning. Multimed. Tools Appl. 2018, 78, 4965–4986. [Google Scholar] [CrossRef]
  7. Keller, J.M. Development and use of the ARCS model of instructional design. J. Instr. Dev. 1987, 10, 2–10. [Google Scholar] [CrossRef]
  8. Kozma, R.B.; Isaacs, S. The Technology, Economic, and Social Contexts for Educational ICT Policy. In Transforming Education: The Power of ICT Policies, 1st ed.; UNESCO: Paris, France, 2011; Available online: https://books.google.com.tw/books?id=eqR2iOMtUVgC&lpg=PP5&ots=uSKTtFmfms&lr&hl=zh-TW&pg=PP5#v=onepage&q&f=false (accessed on 12 July 2021).
  9. Anderson, J.; Weert, T.V.; Duchâteau, C. Information and Communication Technology in Education: A Curriculum for Schools and Programme of Teacher Development; UNESCO: Paris, France, 2002; Available online: https://researchportal.unamur.be/en/publications/information-and-communication-technology-in-education-a-curriculu (accessed on 13 July 2021).
  10. Futures Imagination and Creativity in Education; Advisory Office of the Ministry of Education: Taiwan, 2021; Available online: https://sites.google.com/site/futurepoforall/home (accessed on 12 July 2021).
  11. Pendergast, D. Generational theory and home economics: Future proofing the profession. Fam. Consum. Sci. Res. J. 2009, 37, 504–522. [Google Scholar] [CrossRef] [Green Version]
  12. Gobé, M.; Zyman, S. Emotional Branding: The New Paradigm for Connecting Brands to People; Allworth Press: New York, NY, USA, 2001; p. 27. [Google Scholar]
  13. Jeremy, K.; Lawson, G. Augmented Reality in Design Education: Landscape Architecture Studies as AR Experience. Int. J. Art Des. Educ. 2019, 39, 6–21. [Google Scholar] [CrossRef] [Green Version]
  14. Abarghoui, M.; Taki, S. Measuring the Effectiveness of Using “Memrise” on High School Students’ Perceptions of Learning EFL. Theory Pract. Lang. Stud. 2018, 8, 1758. [Google Scholar] [CrossRef] [Green Version]
  15. Kopcha, T.J.; Neumann, K.L.; Ottenbreit-Leftwich, A.; Pitman, E. Process over product: The next evolution of our quest for technology integration. Educ. Technol. Res. Dev. 2020, 68, 729–749. [Google Scholar] [CrossRef]
  16. Kerawalla, L.; Luckin, R.; Seljeflot, S.; Woolard, A. ‘Making it real’: Exploring the potential of augmented reality for teaching primary school science. Virtual Real. 2006, 10, 163–174. [Google Scholar] [CrossRef]
  17. Rasimah, C.; Ahmad, A.; Zaman, H. Evaluation of user acceptance of mixed reality technology. Australas. J. Educ. Technol. 2011, 27, 1369–1387. [Google Scholar] [CrossRef]
  18. Kesim, M.; Ozarslan, Y. Augmented reality in education: Current technologies and the potential for education. Procedia Soc. Behav. Sci. 2012, 47, 297–302. [Google Scholar] [CrossRef] [Green Version]
  19. Radu, I. Augmented reality in education: A meta-review and cross-media analysis. Pers. Ubiquitous Comput. 2014, 18, 1533–1543. [Google Scholar] [CrossRef]
  20. Campbell, T.; Wang, S.K.; Hsu, H.-Y.; Duffy, A.M.; Wolf, P.G. Learning with web tools, simulations, and other technologies in science classrooms. J. Sci. Educ. Technol. 2010, 19, 505–511. [Google Scholar] [CrossRef]
  21. Tella, A.; Adu, E.O. Information communication technology (ict) and curriculum development: The challenges for education for sustainable development. Ind. J. Sci. Technol. 2009, 2, 55–59. [Google Scholar] [CrossRef]
  22. Zheng, H.-W.; Chen, S.-T.; Fan, G.-G. Study on the Evaluation of Students’ Learning Movtivation Indicators from Using Facebook Communities—Talking the Culture Thinking and Creation of Digital Photography as an Example. Int. J. Digit. Media Des. 2019, 11, 17–32. [Google Scholar]
  23. Baecher, L.; Rorimer, S.; Smith, L. Video-mediated teacher collaborative inquiry: Focus on English language learners. High Sch. J. 2012, 95, 49–61. [Google Scholar] [CrossRef]
  24. Chang, Y.-S.; Chen, C.-N.; Liao, C.-L. Enhancing English-Learning Performance through a Simulation Classroom for EFL Students Using Augmented Reality—A Junior High School Case Study. Appl. Sci. 2020, 10, 7854. [Google Scholar] [CrossRef]
  25. Chang, Y.-S.; Hu, Y.-J.R.; Chen, H.-W. Learning Performance Assessment for Culture Environment Learning and Custom Experience with an AR Navigation System. Sustainability 2019, 11, 4759. [Google Scholar] [CrossRef] [Green Version]
  26. Hill, J.L.; Nelson, A. New technology, new pedagogy? Employing video podcasts in learning and teaching about exotic ecosystems. Environ. Educ. Res. 2011, 17, 393–408. [Google Scholar] [CrossRef]
  27. Azuma, R.T. A Survey of Augmented Reality. In Presence: Teleoperators and Virtual Environments; MIT Press: Cambridge, MA, USA, 1997; pp. 355–385. [Google Scholar]
  28. Höllerer, T.; Feiner, S. Mobile augmented reality. In Telegeoinformatics: Location-Based Computing and Services; Karimi, H., Hammad, A., Eds.; Taylor & Francis Books: London, UK, 2004. [Google Scholar]
  29. Wagner, D.; Schmalstieg, D.; Billinghurst, M. Handheld AR for collaborative edutainment. In Advances in Artificial Reality and Tele-Existence; Pan, Z., Cheok, A., Haller, M., Lau, R.W.H., Saito, H., Liang, R., Eds.; Springer: Berlin/Heidelberg, Germany, 2006; pp. 85–96. [Google Scholar]
  30. Mackay, W.E. Augmenting Reality: A New Paradigm for Interacting with Computers. La Recherche1. 1996. Available online: http://www-ihm.lri.fr/~mackay/pdffiles/LaRecherche.English.pdf (accessed on 13 July 2021).
  31. Shadiev, R.; Yang, M. Review of Studies on Technology-Enhanced Language Learning and Teaching. Sustainability 2020, 12, 524. [Google Scholar] [CrossRef] [Green Version]
  32. MIT. Introducing TEAL (Technology-Enabled Active Learning). 2005. Available online: http://icampus.mit.edu/teal/ (accessed on 13 July 2021).
  33. Schrier, K. Using augmented reality games to teach 21st century skills. In ACM SIGGRAPH 2006 Educators Program; Association for Computing Machinery: New York, NY, USA, 2006. [Google Scholar]
  34. Cai, S.; Wang, X.; Chiang, F.-K. Acase study of augmented reality simulation system application in a chemistry course. Comput. Hum. Behav. 2014, 37, 31–40. [Google Scholar] [CrossRef] [Green Version]
  35. Milgram, P.; Kishino, F. A taxonomy of mixed reality visual display. IEICE Trans. Inf. Syst. 1994, 77, 1321–1329. [Google Scholar]
  36. Crescente, M.L.; Lee, D. Critical issues of m-learning: Design models, adoption processes, and future trends. J. Chin. Inst. Ind. Eng. 2011, 28, 111–123. [Google Scholar] [CrossRef]
  37. Ho, S.C.; Hsieh, S.W.; Sun, P.C.; Chen, C.M. To activate English learning: Listen and speak in real life context with an AR featured u-learning system. J. Educ. Technol. Soc. 2017, 20, 176–187. [Google Scholar]
  38. Khan, D.; Rehman, I.; Ullah, S.; Ahmad, W.; Cheng, Z.; Jabeen, G.; Kato, H. A Low-Cost InteractiveWriting Board for Primary Education using Distinct Augmented Reality Markers. Sustainability 2019, 11, 5720. [Google Scholar] [CrossRef] [Green Version]
  39. Keller, J.M. Motivational design of instruction. In Instructional Design Theories and Models: An Overview of Their Current Status; Reigeluth, C.M., Ed.; Lawrence Erlbaum Associates: Hillsdale, NJ, USA, 1983; pp. 386–434. [Google Scholar]
  40. Keller, J.M. Five fundamental requirements for motivation and volition in technology-assisted distributed learning environments. Rev. Interação 2010, 35, 305–322. [Google Scholar] [CrossRef]
  41. Kintsch, W. Learning from text, levels of comprehension, or: Why anyone would read a story anyway. Poetics 1980, 9, 87–98. [Google Scholar] [CrossRef]
  42. Csikszentmihalyi, M. Beyond Boredom and Anxiety; Jossey-Bass: San Francisco, CA, USA, 1975. [Google Scholar]
  43. Chambers, A. The Reading Environment; Thimble Press: Stroud, UK, 1991. [Google Scholar]
  44. Krapp, A.; Hidi, S.; Renninger, K. Interest, learning, and development. In The Role of Interest in Learning and Development; Psychology Press: East Sussex, UK, 1992; pp. 3–25. [Google Scholar]
  45. Nunnally, J.C. Psychometric Theory. In McGraw-Hill Series in Psychology; McGraw-Hill: New York, NY, USA, 1978; ISBN 9780070474659. [Google Scholar]
Figure 1. Digital media design blueprint instructed by teachers for students to practice.
Figure 1. Digital media design blueprint instructed by teachers for students to practice.
Sustainability 13 12296 g001
Figure 2. Interactive buttons were designed before learning topic was set. In the case of this interactive treehouse, the users can access the inner space of the treehouse to experience the story of Hansel and Gretel in an interactive mode by clicking the button on the door of the treehouse.
Figure 2. Interactive buttons were designed before learning topic was set. In the case of this interactive treehouse, the users can access the inner space of the treehouse to experience the story of Hansel and Gretel in an interactive mode by clicking the button on the door of the treehouse.
Sustainability 13 12296 g002
Figure 3. The workflow of the AR tool to support students in the learning process of digital media design.
Figure 3. The workflow of the AR tool to support students in the learning process of digital media design.
Sustainability 13 12296 g003
Figure 4. The project settings for the AR application on the Unity platform.
Figure 4. The project settings for the AR application on the Unity platform.
Sustainability 13 12296 g004
Figure 5. The three stages (phases) of the experiment.
Figure 5. The three stages (phases) of the experiment.
Sustainability 13 12296 g005
Figure 6. ARCS model and hypotheses.
Figure 6. ARCS model and hypotheses.
Sustainability 13 12296 g006
Figure 7. Regression analysis. ** p < 0.01, *** p < 0.001.
Figure 7. Regression analysis. ** p < 0.01, *** p < 0.001.
Sustainability 13 12296 g007
Figure 8. The findings of this research (proven) correspond to Keller’s motivational theory (1983).
Figure 8. The findings of this research (proven) correspond to Keller’s motivational theory (1983).
Sustainability 13 12296 g008
Table 1. The definitions of ARCS model.
Table 1. The definitions of ARCS model.
ElementsDefinitionVariablePurpose
A
Attention
Arouse students’ learning interest to stimulate their curiosity in order to maintain learning attention [4].Learning attention
  • Confirm that students are immersed in AR to practice multimedia design learning.
  • Observe student enthusiasm about the subject matter, and concentration increase in learning.
R
Relevance
Students’ relevant personal recognition is to be developed by the innovative learning materials [4].Learning interest
  • Take an AR-based instructional textbook as a teaching approach to incite students’ interest.
  • Make students feel that this course is worthwhile so as to motivate their interest in learning.
C
Confidence
Arouse students’ expectations of success and help them to build positive attitudes so as to help increase their self-confidence [4].Learning behavior
  • AR technology can be useful and practical for students’ learning.
  • The confidence and concentration in learning of students can be improved via AR-based teaching activities.
S
Satisfaction
Students’ satisfaction can be effectively enhanced during learning by the improvement of self-learning effectiveness [4].Learning satisfaction
  • The AR-assisted learning program can help student self-learning by which their satisfaction, sense of accomplishment, and learning motivation will be effectively enhanced.
Table 2. Descriptions of experimental and traditional teaching methods.
Table 2. Descriptions of experimental and traditional teaching methods.
Traditional Teaching MethodsExperimental Teaching Approach
Topic for interactive button design
Teacher:
  • Homework exercise as well as the Explanation of Design interactive button: Unity button design + design concept guide.
  • The teacher explains and demonstrates how to design buttons on the Unity platform and C# interactive coding of buttons in the classroom.
Students:
  • Button material preparation or drawing from the button sketch.
  • Design button on Unity platform + C# interactive coding for button.
Teacher:
  • Comment on and revise the button design practice for the user interface.

Teacher:
  • Teacher assigns the exercise and explains the interactive button design: Unity button design + UI concept guide.
  • The teaching topic is the same as traditional teaching in the classroom, yet students can self-practice with AR-assisted interactive button design.
  • Interactive button design and practice study with AR.
Students:
  • Self-study how to design an interactive button and learn the C# coding on the Unity platform with AR and the teaching video.
  • Students who were quarantined during the COVID-19 pandemic can still keep up with related exercises and programming for digital media design project development and use AR textbooks.
Table 3. Learning performance evaluation (knowledge test for digital media design).
Table 3. Learning performance evaluation (knowledge test for digital media design).
TopicFinal Term ProjectID
Evaluation Date:
Work Title Evaluator
Learning Performance EvaluationEvaluation Element and ExplanationGrade
Design capability1. Design methodsProject design procedure and method
2. Learning experienceProject design learning experience is learning by doing with instructor’s guidance
3. Project practiceProject practice, guidance, problem solving, problem-solving skills
4. Project capabilityProject planning and management, challenge, integration
Learning topic1. TopicTopic setting (theme communication and interaction form) for digital media
2. Program designProgramming capability and language coding for digital media
3. Platform practiceOperation proficiency for Unity platform
4. Content produceContent collection, planning, information organization, and presentation
5. Post-production effectsPost-production practice for digital media
Project Creativity1. ThemeWhether the theme is clearly conveyed for final project work
2. Visual effect designDigital media visual effect design for final project work
3. ApplicationThe value of digital multimedia design application for final project work
4. TechniqueDigital media design techniques for final project work
5. InnovationInnovative digital media design for final project work
Table 4. The t-tests of pre- and post-tests for the two groups.
Table 4. The t-tests of pre- and post-tests for the two groups.
Test ItemsGroupStudents NumberPre-Test AveragePre-Test Standard DeviationPost-Test AveragePost-Test Standard Deviation
Knowledge Test for Multimedia DesignTest group4669.5659.061980.4753.4213
Control group4265.90510.719975.4294.7727
Interest Scale TestTest group463.6960.4364.2480.396
Control group423.6670.5353.9230.289
Table 5. The pre-test results for the two groups.
Table 5. The pre-test results for the two groups.
Test ItemsResourceSSdfMSF ValueVariability Homogeneity Test Significance
Pre-test: Knowledge Test for Multimedia DesignSSB294.1681294.1683.0090.086
SSw8406.9238697.755
SST8701.09187
Pre-test: Interest Scale TestSSB0.1810.0180.0780.78
SSw20.272860.236
SST20.29187
p > 0.05. In Table 5, the data show that the pre-test results of the groups are non-significant (F = 3.009, p = 0.086 > 0.05; F = 0.078, p = 0.78 > 0.05) and that the values meet the principle of paralleled standard, which means homogeneity existed between the two groups and thus the issue that the different learning capability may affect this experimental teaching research can be removed. Therefore, this research can further take ANOVA to examine the post-test of the two groups.
Table 6. ANOVA analysis for learning performance.
Table 6. ANOVA analysis for learning performance.
Test ItemsResourceSSdfMSF ValueVariability Homogeneity Test Significance
Post-test: Knowledge Test for Multimedia DesignSSB558.9021558.90232.9070.000 ***
SSw1460.6618616.984
SST2019.56387
Post-test: Interest Scale TestSSB2.30512.30518.8950.000 ***
SSw10.491860.122
SST12.79687
*** p < 0.001.
Table 7. Descriptive statistics of the ARCS model elements for the examination of the test group.
Table 7. Descriptive statistics of the ARCS model elements for the examination of the test group.
ElementsNumberMeanStandard Deviation
Learning attention464.32611.26797
Learning interest for AR teaching464.16960.30395
Learning confidence464.21740.35734
Learning satisfaction464.15510.31408
Valid N (excluded completely)46
Table 8. Total variance explained of the factor loadings.
Table 8. Total variance explained of the factor loadings.
IngredientsInitial EigenvalueRotation Square Sum of Factor Loadings
SumVariance%Cumulative%SumVariance%Cumulative%
Learning confidence9.23746.18746.1874.43522.17522.175
Learning satisfaction2.57412.87259.0594.15520.77542.950
Learning interest1.3576.78365.8423.40016.99959.950
Learning attention1.3286.64072.4822.50612.53272.482
Note: Extraction method: principal component analysis.
Table 9. Correlation analysis.
Table 9. Correlation analysis.
Learning AttentionLearning InterestLearning ConfidenceLearning SatisfactionLearning Performance
Learning attention
Pearson Correlation
10.432 (**)0.409 (**)0.413 (**)0.501 (**)
Significance (two-tailed) 0.0030.0050.0040.037
Learning interest
Pearson Correlation
0.432 (**)10.545 (**)0.497 (**)0.732 (*)
Significance (two-tailed)0.003 0.0000.0000.003
Learning confidence
Pearson Correlation
0.409 (**)0.545 (**)10.712 (**)0.658 (**)
Significance (two-tailed)0.0050.000 0.0000.000
Learning satisfaction
Pearson Correlation
0.413 (**)0.497 (**)0.712 (**)10.714 (**)
Significance (two-tailed)0.0040.0000.000 0.0000
Learning effectiveness
Pearson Correlation
0.501 (**)0.732 (*)0.658 (**)0.714 (**)1
Significance (two-tailed)0.0000.0030.0000.0000
* p < 0.05 (two-tailed), ** p < 0.001 (two-tailed), n = 46.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Chang, Y.-S. Applying the ARCS Motivation Theory for the Assessment of AR Digital Media Design Learning Effectiveness. Sustainability 2021, 13, 12296. https://doi.org/10.3390/su132112296

AMA Style

Chang Y-S. Applying the ARCS Motivation Theory for the Assessment of AR Digital Media Design Learning Effectiveness. Sustainability. 2021; 13(21):12296. https://doi.org/10.3390/su132112296

Chicago/Turabian Style

Chang, Yuh-Shihng. 2021. "Applying the ARCS Motivation Theory for the Assessment of AR Digital Media Design Learning Effectiveness" Sustainability 13, no. 21: 12296. https://doi.org/10.3390/su132112296

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop